Mappings, Reductions, and Consolidations

The fabulator-0.0.2 gem will have a slight change in the expression language. We’re also removing a function from the namespace. Right now, I’d say that the tag lib schemas are subject to change until 0.1.0, at least. The 0.0.x releases are early alphas that are just trying to get all of the main points of functionality figured out.

For functions that operate on one type of argument, but possibly an array of that type, we have a way to define their scaling behavior: mappings (one-to-one domain to range), reductions (many-to-one domain to range), and consolidations (many-to-one range to range). Consolidations have the same idempotence requirements as reductions in map/reduce frameworks such as OpenCL.

There’s no difference in how mappings and reductions are invoked in expressions. Simply use their name and supply the data. Consolidations are a little different and are tied explicitly to the reduction that they consolidate.

For a reduction ‘foo’ that takes data from a domain of X to a range of Y, the corresponding consolidation ‘foo*’ will take the result of ‘foo’ (the range of Y) and take it to a range of Y. A consolidation is its own consolidation, though we don’t allow ‘foo**’.

This results in things like ‘f:count*’ == ‘f:sum’ and ‘f:sum*’ == ‘f:sum’ as well. But ‘f:avg*’ doesn’t exist because we lose required information when computing an average (but we can return to consolidations by working with the sums and counts that compose the average).

Anything that was working with histograms will need some changes. Instead of the ‘f:consolidate’ function, we now have the ‘f:histogram*’ function.

The ‘*’ (star) was chosen because it is reminiscent of how some algebras denote closure.