This is basically a description of pre-1.0 refactoring of wicked

Wicked depends on providing implicitly expected behavior through a set of matching algorithm; predicting user expectations is sensitive work, and matching is sensitive work. our integration test prove that the old system work, but made new work and fine tuning difficult.

The basic goals of this were to break filter and wicked into smaller more manageable testable pieces as well as starting to decouple wicked's behavior from the systems underneath it(the source of the slow test and partially responsible brittle nature of the code).

First steps: Decomposition of dependencies

Currently 'filter' is a closely coupled extension of the Archetypes field object. AT schemas are composed of fields; the field acts as the configuration object for data typing, storage(getting & setting), presentation.  wicked, by proxy, also depends heavily on the field and the archetypes content instance whose schema contains the field.   And so on: AT is  dependent on the CMF / Zope2 stack....

The field and the instance provide different purposes to wicked as well as the close coupling between wicked and the zope / plone infrastructure.

instance: For most operations, the field uses the instance as a context object, a hook into what services are locally available to the field.  The FilterField in filter pass this context on to fields that require, such as the MacroSubstitionFilter, the base class for WickedFilter.  Wicked uses context information for resolving, access and set data in the cache, set and remove backlinks, and render links with the proper url.

field: 'filter' and wicked depend on the field object for configuration information.  In the case of wicked, the field stored what template, what macro and the 'scope' a search is to take place over when resolving a link. The FilterField use the field to derive what filters to apply to a block of text and to extract any configuration information.

Conceptually, field and instance provide similar but contextually different information.  An instance provides information about containment, and a hook to local utilities.  Fields provide more general configuration information (what filters for this type of instances, configuration information for particular filters).

Filter

Previously, filter's were registered in a module level dictionary at start up.  This ad hoc registery was replaced by refactoring each filter to be an adapter factory using an instance as the context, and registering them as adapters.   For simplicity, these factories were registered to a single interface, and a specific name: a named adapter.   

Within the field implementation, a call to FilterField.get would look at a configuration property set on the field and take the returned list of filter names and apply them in order to the text returned.   The filter is applied like so::

text = "some text to filter"
textfilter = zapi.getAdapter(instance, IFieldFilter, name='name')
text = textfilter(text)

Each adapter must have a __call__ method that applies a regular expression to the entire text area.  The resulting list of chunks gets process by the _filtercore method.

I started some preliminary work for registering filters via zcml, since filter include there own name and all used the same interface.  This work is yet to be finished.

Filter Patterns

filter's textfilters allow for simple extension of a textarea allowing for transactional interactivity (on page load). Changes in content relative to the area being rendered could be represented, allowing selective choice of behaviors (templating, wiki linking, etc.)

Since filters are run in an ordered and predictable sequence, it is possible for a creating a reactive chain with each filter passing information via the filtered text.  This is similar to a xslt pipeline and, writing adapters to use xslt on xml fields would be trivial with a package like lxml, and the effect of the xslt could be constrained to the contents of regex.

Caching is another pattern that such a pipeline lends itself. 

the Wicked filter

After the current refactoring of filter, the wicked filter got a similiar rewrite to act as a named adapter.  Unlike the textfilters in the filter package, wicked has grown a bit more rich behavior that extends beyond just filtering.  At the time of the initially refactoring, wicked the CMF product included a caching system, content types, a couple assisting skin scripts, and a backlinking behavior.  This create several hard dependencies on using a special wicked field.

The wicked filter behavior also included several distinct dependencies on CMF and Zope.