Introducing StepRewrite (aka I finally understand macros)

So it's finally done! I could probably do a bit more, but for the moment it's working and packaged! For a while now, I've been writing about a problem I want to solve, and discovering the tools to solve it. Of course it's also a problem that may not really exist, but fuck it! To really understand the motivations behind this thing I've built, follow those links above, but here is the short version.

I said that Evented IO makes you write ugly code, just to sequence a bunch of operations. I figured out that the only way to be able to sequence code normally but run it in an evented IO environment is to use macros and actually rewrite code. So I wrote step_rewrite (which has obtained that name since it was born out of an unholy union of step.js and rewrite). If you install, it you can write code like this

that behaves as if it was written like this.

This doesn't mean that you are forced to (or should in fact) write every block in this manner. It's meant to be used only when the blocks exist purely to sequence the rest of the method to occur inside a callback. i.e. when you really do intend what the first piece of code implies. That you have a series of operations that should occur one after another, and it just so happens, that they perform IO. (if you squint really hard, you can pretend the &_ bits are invisible).

So step_rewrite, Can be used either, as a function that takes a block to eval, or as a fuction that takes a block to define a method. It rewrites the code and converts every function call taking a special callback, followed by some other code into a form where the function now receives a block with the rest of the code in the block.

becomes

for situations where you intend the former but your environment requires code to do the latter.

It also converts return values into block arguments. So that

becomes

This is acceptable for the most part because using &_ has made hunter.kill the last statement in it's block. So anything it returns will be the return value of its block.

Of course this abstraction is leaky. There are a lot of complicated situations where you have to be aware of what the converted code will look like. I just hope that 80-90% of the time, you can be oblivious to it.

This works using the ParseTree gem. ParseTree converts code into S-Expressions, the language of macros. Here are some examples of S-expressions.

Given the S-expression, I can now manipulate it. Chopping out bits, wrapping it in other pieces. The resulting S-expression is converted back into Ruby using Ruby2Ruby and I'm done :).

So yeah, what I really do is convert an S-Expression of the first form to the second form.

I'm excited about this because I finally understood what the whole macro thing was about. I've always heard that it was about extending the language itself. But I never really got it. It seemed to me that anything I wanted to do, could either be implemented using good old fashioned meta programming, or was not possible even with macros. I could not see this middle ground of extending the language without writing a full fledged parser. But now I finally see it :)

Lastly, it looks like Narrative.js does something similar to what I was trying to do. It contains a full javascript parser, so its a bigger project. I need to look at it to see if it converts code into a similar form, and if so how it overcomes various problems that I have because of leaky abstractions.