Jupyter support?

[cunningjames]

I have a project in mind for which I'd like to use MacroPy, but the end result would not be an application but a library, and I'd like any eventual users to be able to use it interactively.*

I realize the projects are totally unconnected, but do you have any initial thoughts / cautions about how difficult it would be to bring MacroPy support to Jupyter as an extension (enabling a magic command, for example)? This is something I'd be happy to start looking into myself, but I'd more or less be starting from scratch (both in MacroPy and in Jupyter). If you don't know then no problem, I'll just trudge ahead.

*(I've very occasionally been knocking my head against a wall for the past few months, trying to figure out how to bring something like the R's dplyr and tidyr to Python in a generic and non-fiddly way. Once I started parsing strings and editing the resulting AST, I realized using macros is the only sensible way to get what I want ... but given the user base there needs to be some easy way of using it interactively, or it'd be an also-ran.)

[azazel75]

Hi @cunningjames and sorry for the late response.

Unfortunately I know little of the internals of Jupyter notebook and the library that "runs" them. MacroPy has some interactive capability (it can be run in a Python REPL) even if after the port to Python 3 it has some bugs that needs to be fixed.

If you have any more practical example of the AST mangling that you need to do, I'll try to help you out,

cheers

[chmp]

If my barge in: I did not use macropy so far, but encountered a similar issue myself. IPython support customizable AST rewriting, maybe this could help. The documentation can be found here. I am using the mechanism to apply pytest's assert rewriting. You can find my usage here.

For completeness: if you a have a function with the signature rewrite_ast(module_ast) -> module_ast, you should be able to hook it into IPython as follows:

class Rewriter(ast.NodeTransformer):
    def visit(self, node):
        return rewrite_ast(node)

get_python().ast_transformers.append(Rewriter())

This code will work both from inside the notebook itself and from a library.

[azazel75]

See also azazel75#20

[Technologicat]

The code in azazel75#20 has now been packaged into an add-on called imacropy. It's available on PyPI.

Beside macro support for the IPython REPL, imacropy provides also a generic bootstrap script that allows the main program to use macros. (The bootstrapper imports the intended main program, while pretending its name is main.)

This is possibly a temporary solution, while MacroPy itself concentrates on a stable Python 3 release. We'll decide later whether to merge the features into the MacroPy core, or keep them as an add-on.

[Technologicat]

Thanks @chmp , that's pretty much exactly the approach I ended up going with. :)

[chmp]

Great to hear. TBH, I am still amazed that this hook exists :).