Refactoring Context

There are a few times I’ve needed some compile time information at run time inside a function. The current Fabulator engine doesn’t expose this information to a function definition, which can be problematic when you want a function to automagically use the right database, for example, without having to repeat yourself.

The actual use case that is driving this is building an Exhibit database by accumulating information over time and allowing editing of individual entries. I want to be able to specify the Exhibit database at a global level in the application using an @ex:database and have a function such as ex:item($id) retrieve the indicated item from the indicated database.

There’s no way to do that with the current code on Github, at least not without resorting to ugly globals that would break any kind of threaded approach in the future.

Instead, I’ve spent yesterday afternoon/evening and today ripping out how we handle execution context and slowly moving to a separation between context and data tree. Think of a ribosome walking an RNA strand. In our case, the RNA is a Fabulator::Expr::Context object and the RNA strand is the Fabulator::Expr::Node tree.

There are two types of information we want to access at run time: information available at compile time, and information that depends on what we’ve done at run time. Things like the namespace prefix mappings and ‘global’ attributes are compile time information and don’t change at run time. The current topical node and set of variables are run time scoped and depend on how we got to where we are.

The result is that a Context object that can have two parents: the compile time parent context that corresponds to the enclosing XML element, and the run time parent context that corresponds to the calling action (usually the XML element). The first has information that is constant across executions of the state machine. The second has information peculiar to this particular invocation. Before execution, all contexts only have a compile time parent. When executing, the compile time context is merged with the run time context to produce a new working context. I still need to tease a few things apart, but this should put us on the path to approximately proper support for multi-threaded execution.

Once we have all of our current behavior tests passing (currently at 64/73 scenarios passing), we’ll push our changes to Github and start fixing up the various extensions to use the new context system. This change will break compatibility with all previous versions of the extensions.

This is a major change in the backend, but it feels proper. I think once this is done, we’ll be well on our way to a proper gem cutting. I should be able to have that done by the end of August.

On a side note, I’m moving the regex support into an extension. I didn’t really have any good regex support yet anyway, and I like the way Perl 6 thinks about regexes as grammars with rules and tokens. My plan is to allow grammars to be like libraries that sit outside the code but can be referenced by it.