DesignNotes

L8 Design

The key idea is to break a javascript function into "steps" and then walk thru these steps much like the javascript interpreter runs thru the statements of a function. This is quite verbose however. But not so much when using CoffeeScript. This is why, after considering the idea years ago, I waited until now to implement it. That my cousin Jean Vincent would consider breaking a function into steps as something close enough to threading was another strong motivator.

To break functions into steps, I use a DSL (domain specific language) API. Once the AST (abstact syntax tree) is built, I interpret it.

Executable nodes in the AST are called "steps". They are the smallest non interruptible executable entities. Each Step belongs to a task. Task can involve sub tasks that cooperate across multiple paths.

This becomes really interesting when the AST gets dynamically modified! This happens when a step decides that it requires additional sub steps to complete.

On a path, execution goes from step to step. When a step involves sub-steps, the step is blocked until the sub-steps complete, unless sub-steps are created in a forked parallel path.

Example:

MainTask
  Task.1 - a task with a single path with a loop subpath
    MainPath
      Step
      Step
      RepeatPath
        Step
        Step
      Step
  Task.2 - a task with two paths (two forked subtasks)
    MainPath
      ForkedPath
        Step
        Step
      ForkedPath
        Step
      Step

Extensions

The l8 API defines a concept of Task/Path/Step entities that works nicely in the async/callback dominated world of Javascript and yet manage to provide some useful tools (hopefully) to address the infamous "callback hell" issue.

However these tools are very basic.

One way to improve on that situation is to use one of the multiple existing "promise" handling libraries, such as Q, when and rsvp. See also http://howtonode.org/promises and https://gist.github.com/3889970

What is also needed are more sophisticated yet simpler to use solutions. There are many styles of coding regarding the orchestration of complex interactions between fragments on code. We are no longer restricted to the signal/kill mechanism from the 70s in Unix!

In addition to the classics (semaphores, mutexes, message queues, conditional variables, signals, monitors, events...) newer mechanisms are experimented or rediscovered. Things like channels in Go, actors in Erlang, reactive system in Angular, these things are interesting to explore.

I believe l8 may help do that. For sure the granularity is not the same as in native implementations. Instead of dealing with statements or even machine code level instructions, l8 deals with much bigger "steps". However, regarding synchronisation, this difference of scale does not imply a difference of nature. As a consequence, solutions that work at the machine level may prove also productive at the "step" higher level.

Hence, proposals for extensions are welcome.

Enjoy.

Jean Hugues Robert, aka @jhr, october/november 2012.

PS: all this stuff is to relieve me from my "node anxiety". See http://news.ycombinator.com/item?id=2371152

PS: if you feel like threads are a low level tool, you may want to have look at a much higher level tool where data in sets flow thru much higher level tasks called datalet in a reactive functional style: Excess. See https://github.com/ConnectedSets/ConnectedSets