Coroutines As A Core Language Feature

[weswigham]

Co-routines are baby threads. Or perhaps super threads. In effect, they are threads whose scheduling is undecided. Python and Lua both implement co-routines at a language level, and for good reason. In python, they are used as an extension to generators, whereas in Lua they are used to model more complex iterator patterns and to handle concurrency issues in non-concurrent systems.

So, more to the point, the language should probably implement co-routines at the language level (for syntax prettiness purposes). A co-routine is a superclass of a function; any function is at least a one-step co-routine. What makes a co-routine different from a generator is that it can consume new inputs on every step (rather than just yielding outputs).

Since Brick is statically typed, co-routines may yield a problem - a co-routine may not want to yield the same type in all cases, and determining if the caller is expecting the correct ones can be difficult.

So, on to the actual syntax. In Python, coroutines were kind of hacked onto generators, so I'm going to ignore its ugly syntax for them:

def foo():
    for i in range(10):
        yield i # generator

def bar():
    state = 1
    for i in range(10):
        state += (yield state) # coroutine

def main():
    while True:
        print(bar.send(math.random(0,20)))

Lua's syntax is a bit less unnatural, but still cumbersome.

function bar()
    local state = 1
    for i=1,10 do
        state = state + coroutine.yield(state)
    end
end

local core = coroutine.wrap(bar)

function main()
    while true do
        print(core(math.random(0, 20)))
    end
end

(Lua has some more functions, such as resume, if you want to avoid the wrap shortcut)

So, what variety of coroutine syntax would fit in well with the language...?
I think something like

fn bar ->
    let x = 1
    10.times ->
        x += ^.receive(x)

fn main ->
   while true ->
        puts(bar.send(math.random(0, 20)))

Which has none of the yield-keyword-ambiguity that python has, while avoiding the high verbosity of lua's coroutines. Additionally, using ^ to store a function's state (if we view a function has a state machine) makes sense in this context, also making coroutine 'trampolining' (yielding all the way down to the initial thread/scheduler so it can schedule/start the next task) un-needed, since the child can simply go

^^.send(result)

(This is actually one of the major problems in a coroutine-based system, the need to trampoline back down to the scheduler to pass inputs around. Being able to avoid that is pretty cool.)

[toroidal-code]

I'll have to look into coroutines, as I know absolutely nothing about them. I do know that Ruby has coroutines as Fibers, and Scheme has proper coroutines, so I'll look at that.

[toroidal-code]

I was already considering supporting thunks. How does this relate?

[toroidal-code]

how would you write the examples using ruby fibers? I'm having trouble doing this and getting the same output as lua.

[toroidal-code]

Never mind, I got it.

fiber = Fiber.new do
  state = 1
  1.upto 10 do
    state = state + Fiber.yield(state)
  end
end

while true do 
  puts fiber.resume(20)
end

[weswigham]

Mmm, the one thing I don't like about the Ruby or Lua coroutine syntax is that it excepts coroutines as a special type of function (you have to use a special class/method), when it's the exact opposite. A function is a subroutine, a subclass of coroutine; every function is equivalent to a single-step coroutine. (Which is one bit the python syntax gets correct)

[weswigham]

As an aside about 'thunks'... thunks are lazy functions? Let me go make a thunk in Lua for my own reference...

local thunk = {
    __call = function(self, ...)
        return self.__func(unpack(self.__params))
    end
}

function lazy(f, ...)
    return setmetatable({__func = f, __params = {...}}, thunk)
end

local add = lazy(function(a,b) 
    return a+b
end, 5, 10)

print(add()) --not evaluated until called

--Or equivalently...

function lazy(f, ...)
    local a = {...}
    return coroutine.wrap(function()
        coroutine.yield(f(unpack(a)))
    end)
end

local add = lazy(function(a,b) 
    return a+b
end, 5, 10)

print(add())

--Or, you know...

function curry(f, ...)
    local a = {...}
    return function (...)
        for k,v in ipairs({...}) do
            table.insert(a, v)
        end
        return f(unpack(a))
    end
end

local add = curry(function(a,b)
    return a+b
end, 5, 10)

print(add())

Yeah, thunks don't seem all too different from currying. Really the main language feature that makes it different is when I have function func that takes no arguments, is if I simply go puts(func) does it evaluate and output the result of func (making it a thunk), or just print some representation of func? When I do x = 10 + 5, is x internally a function that will be evaluated when its value is needed, or is it 15? If I went

let x = -> 
    10 + 5
let y = 10 + 5

If x and y are compared, are they equivalent? Do you want them to be?

[toroidal-code]

A function with no call parameters ({ }, ( ), or [ ]) is the function itself.

When you do let! x = 10 + 5, that gets optimized to 15 by the compiler. That's done by the LLVM backend.

When x and y are compared, they would be equivalent, as accessing the x actually calls the function bound to it, evaluating to 15.

[weswigham]

When x and y are compared, they would be equivalent, as accessing the x actually calls the function bound to it, evaluating to 15.

You already have everything you need for 'thunks' then, from what I can tell. Anyway! Back to coroutines.

Coroutines would be nice to have at the language level, preferably with some nice native syntax support. Just having the functions that enable them (yield and resume at a minimum) on the base function class would really be all you need, I think.

[toroidal-code]

I agree. This should be pretty simple. Let's make note of this in the docs somewhere.