|
| 1 | +## Lispython: the love child of Python and Scheme |
| 2 | + |
| 3 | + |
| 4 | + |
| 5 | +Powered by [`mcpyrate`](https://github.com/Technologicat/mcpyrate/) and `unpythonic`. |
| 6 | + |
| 7 | +```python |
| 8 | +from unpythonic.dialects import dialects, Lispython # noqa: F401 |
| 9 | + |
| 10 | +def factorial(n): |
| 11 | + def f(k, acc): |
| 12 | + if k == 1: |
| 13 | + return acc |
| 14 | + f(k - 1, k*acc) |
| 15 | + f(n, acc=1) |
| 16 | +assert factorial(4) == 24 |
| 17 | +factorial(5000) # no crash |
| 18 | + |
| 19 | +t = letrec[((evenp, lambda x: (x == 0) or oddp(x - 1)), |
| 20 | + (oddp, lambda x: (x != 0) and evenp(x - 1))) in |
| 21 | + evenp(10000)] |
| 22 | +assert t is True |
| 23 | + |
| 24 | +square = lambda x: x**2 |
| 25 | +assert square(3) == 9 |
| 26 | +assert square.__name__ == "square" |
| 27 | + |
| 28 | +g = lambda x: [local[y << 2*x], |
| 29 | + y + 1] |
| 30 | +assert g(10) == 21 |
| 31 | + |
| 32 | +c = cons(1, 2) |
| 33 | +assert tuple(c) == (1, 2) |
| 34 | +assert car(c) == 1 |
| 35 | +assert cdr(c) == 2 |
| 36 | +assert ll(1, 2, 3) == llist((1, 2, 3)) |
| 37 | +``` |
| 38 | + |
| 39 | +### Features |
| 40 | + |
| 41 | +In terms of ``unpythonic.syntax``, we implicitly enable ``tco``, ``autoreturn``, ``multilambda``, ``namedlambda``, and ``quicklambda`` for the whole module: |
| 42 | + |
| 43 | + - TCO in both ``def`` and ``lambda``, fully automatic |
| 44 | + - Omit ``return`` in any tail position, like in Lisps |
| 45 | + - Multiple-expression lambdas, ``lambda x: [expr0, ...]`` |
| 46 | + - Named lambdas (whenever the machinery can figure out a name) |
| 47 | + - The underscore: ``f[_*3] --> lambda x: x*3`` (name ``f`` is **reserved**) |
| 48 | + |
| 49 | +We also import some macros and functions to serve as dialect builtins: |
| 50 | + |
| 51 | + - All ``let[]`` and ``do[]`` constructs from ``unpythonic.syntax`` |
| 52 | + - ``cons``, ``car``, ``cdr``, ``ll``, ``llist``, ``nil``, ``prod`` |
| 53 | + - ``dyn``, for dynamic assignment |
| 54 | + |
| 55 | +For detailed documentation of the language features, see [``unpythonic.syntax``](https://github.com/Technologicat/unpythonic/tree/master/doc/macros.md), especially the macros ``tco``, ``autoreturn``, ``multilambda``, ``namedlambda``, ``quicklambda``, ``let`` and ``do``. |
| 56 | + |
| 57 | +The multi-expression lambda syntax uses ``do[]``, so it also allows lambdas to manage local variables using ``local[name << value]`` and ``delete[name]``. See the documentation of ``do[]`` for details. |
| 58 | + |
| 59 | +The builtin ``let[]`` constructs are ``let``, ``letseq``, ``letrec``, the decorator versions ``dlet``, ``dletseq``, ``dletrec``, the block versions (decorator, call immediately, replace def'd name with result) ``blet``, ``bletseq``, ``bletrec``, and the code-splicing variants ``let_syntax`` and ``abbrev``. Bindings may be made using any syntax variant supported by ``unpythonic.syntax``. |
| 60 | + |
| 61 | +The builtin ``do[]`` constructs are ``do`` and ``do0``. |
| 62 | + |
| 63 | +If you need more stuff, `unpythonic` is effectively the standard library of Lispython, on top of what Python itself already provides. |
| 64 | + |
| 65 | + |
| 66 | +### What Lispython is |
| 67 | + |
| 68 | +Lispython is a dialect of Python implemented via macros and a thin whole-module AST transformation. The dialect definition lives in [`unpythonic.dialects.lispython`](../../unpythonic/dialects/lispython.py). Usage examples can be found in [the unit tests](../../unpythonic/dialects/tests/test_lispython.py). |
| 69 | + |
| 70 | +The goal of the Lispython dialect is to fix some glaring issues that hamper Python when viewed from a Lisp/Scheme perspective, as well as make the popular almost-Lisp, Python, feel slightly more lispy. |
| 71 | + |
| 72 | +We take the approach of a relatively thin layer of macros (and underlying functions that implement the actual functionality), minimizing magic as far as reasonably possible. |
| 73 | + |
| 74 | +Performance is only a secondary concern; performance-critical parts fare better at the other end of [the wide spectrum](https://en.wikipedia.org/wiki/Wide-spectrum_language), with [Cython](http://cython.org/). Lispython is for [the remaining 80%](https://en.wikipedia.org/wiki/Pareto_principle), where the bottleneck is human developer time. |
| 75 | + |
| 76 | + |
| 77 | +### Comboability |
| 78 | + |
| 79 | +The aforementioned block macros are enabled implicitly for the whole module; this is the essence of the Lispython dialect. Other block macros can still be invoked manually in the user code. |
| 80 | + |
| 81 | +Of the other block macros in ``unpythonic.syntax``, code written in Lispython supports only ``continuations``. ``autoref`` should also be harmless enough (will expand too early, but shouldn't matter). |
| 82 | + |
| 83 | +``prefix``, ``curry``, ``lazify`` and ``envify`` are **not compatible** with the ordering of block macros implicit in the Lispython dialect. |
| 84 | + |
| 85 | +``prefix`` is an outside-in macro that should expand first, so it should be placed in a lexically outer position with respect to the ones Lispython invokes implicitly; but nothing can be more outer than the dialect template. |
| 86 | + |
| 87 | +The other three are inside-out macros that should expand later, so similarly, also they should be placed in a lexically outer position. |
| 88 | + |
| 89 | +Basically, any block macro that can be invoked *lexically inside* a ``with tco`` block will work, the rest will not. |
| 90 | + |
| 91 | +If you need e.g. a lazy Lispython, the way to do that is to make a copy of the dialect module, change the dialect template to import the ``lazify`` macro, and then include a ``with lazify`` in the appropriate position, outside the ``with namedlambda`` block. Other customizations can be made similarly. |
| 92 | + |
| 93 | + |
| 94 | +### Lispython and continuations (call/cc) |
| 95 | + |
| 96 | +Just use ``with continuations`` from ``unpythonic.syntax`` where needed. See its documentation for usage. |
| 97 | + |
| 98 | +Lispython works with ``with continuations``, because: |
| 99 | + |
| 100 | + - Nesting ``with continuations`` within a ``with tco`` block is allowed, for the specific reason of supporting continuations in Lispython. |
| 101 | + |
| 102 | + The dialect's implicit ``with tco`` will just skip the ``with continuations`` block (``continuations`` implies TCO). |
| 103 | + |
| 104 | + - ``autoreturn``, ``quicklambda`` and ``multilambda`` are outside-in macros, so although they will be in a lexically outer position with respect to the manually invoked ``with continuations`` in the user code, this is correct (because being on the outside, they run before ``continuations``, as they should). |
| 105 | + |
| 106 | + - The same applies to the outside-in pass of ``namedlambda``. Its inside-out pass, on the other hand, must come after ``continuations``, which it does, since the dialect's implicit ``with namedlambda`` is in a lexically outer position with respect to the ``with continuations``. |
| 107 | + |
| 108 | +Be aware, though, that the combination of the ``autoreturn`` implicit in the dialect and ``with continuations`` might have usability issues, because ``continuations`` handles tail calls specially (the target of a tail-call in a ``continuations`` block must be continuation-enabled; see the documentation of ``continuations``), and ``autoreturn`` makes it visually slightly less clear which positions are in factorial tail calls (since no explicit ``return``). Also, the top level of a ``with continuations`` block may not use ``return`` - while Lispython happily auto-injects a ``return`` to whatever is the last statement in any particular function. |
| 109 | + |
| 110 | + |
| 111 | +### Why extend Python? |
| 112 | + |
| 113 | +[Racket](https://racket-lang.org/) is an excellent Lisp, especially with [sweet](https://docs.racket-lang.org/sweet/), sweet expressions [[1]](https://sourceforge.net/projects/readable/) [[2]](https://srfi.schemers.org/srfi-110/srfi-110.html) [[3]](https://srfi.schemers.org/srfi-105/srfi-105.html), not to mention extremely pythonic. The word is *rackety*; the syntax of the language comes with an air of Zen minimalism (as perhaps expected of a descendant of Scheme), but the focus on *batteries included* and understandability are remarkably similar to the pythonic ideal. Racket even has an IDE (DrRacket) and an equivalent of PyPI, and the documentation is simply stellar. |
| 114 | + |
| 115 | +Python, on the other hand, has a slight edge in usability to the end-user programmer, and importantly, a huge ecosystem of libraries, second to ``None``. Python is where science happens (unless you're in CS). Python is an almost-Lisp that has delivered on [the productivity promise](http://paulgraham.com/icad.html) of Lisp. Python also gets many things right, such as well developed support for lazy sequences, and decorators. |
| 116 | + |
| 117 | +In certain other respects, Python the base language leaves something to be desired, if you have been exposed to Racket (or Haskell, but that's a different story). Writing macros is harder due to the irregular syntax, but thankfully MacroPy already exists, and any set of macros only needs to be created once. |
| 118 | + |
| 119 | +Practicality beats purity ([ZoP §9](https://www.python.org/dev/peps/pep-0020/)): hence, fix the minor annoyances that would otherwise quickly add up, and reap the benefits of both worlds. If Python is software glue, Lispython is an additive that makes it flow better. |
| 120 | + |
| 121 | + |
| 122 | +### PG's accumulator-generator puzzle |
| 123 | + |
| 124 | +The puzzle was posted by Paul Graham in 2002, in the essay [Revenge of the Nerds](http://paulgraham.com/icad.html). It asks to implement, in the shortest code possible, an accumulator-generator. The desired behavior is: |
| 125 | + |
| 126 | +```python |
| 127 | +f = foo(10) |
| 128 | +assert f(1) == 11 |
| 129 | +assert f(1) == 12 |
| 130 | +assert f(5) == 17 |
| 131 | +``` |
| 132 | + |
| 133 | +(The original name of the function is literally `foo`; we have chosen to keep the name, although [nowadays one should do better than that](https://docs.racket-lang.org/style/reference-style.html#%28part._examples-style%29).) |
| 134 | + |
| 135 | +Even Lispython can do no better than this let-over-lambda (here using the haskelly let-in syntax to establish let-bindings): |
| 136 | + |
| 137 | +```python |
| 138 | +foo = lambda n0: let[(n, n0) in |
| 139 | + (lambda i: n << n + i)] |
| 140 | +``` |
| 141 | + |
| 142 | +This still sets up a separate place for the accumulator (that is, separate from the argument of the outer function). The modern pure Python solution avoids that, but needs many lines: |
| 143 | + |
| 144 | +```python |
| 145 | +def foo(n): |
| 146 | + def accumulate(i): |
| 147 | + nonlocal n |
| 148 | + n += i |
| 149 | + return n |
| 150 | + return accumulate |
| 151 | +``` |
| 152 | + |
| 153 | +The problem is that assignment to a lexical variable (including formal parameters) is a statement in Python. Python 3.8's walrus operator does not solve this, because `n := n + i` by itself is a syntax error. |
| 154 | + |
| 155 | +If we abbreviate ``accumulate`` as a lambda, it needs a ``let`` environment to write in, to use `unpythonic`'s expression-assignment (`name << value`). |
| 156 | + |
| 157 | +But see ``envify`` in ``unpythonic.syntax``, which shallow-copies function arguments into an `env` implicitly: |
| 158 | + |
| 159 | +```python |
| 160 | +from unpythonic.syntax import macros, envify |
| 161 | + |
| 162 | +with envify: |
| 163 | + def foo(n): |
| 164 | + return lambda i: n << n + i |
| 165 | +``` |
| 166 | + |
| 167 | +or as a one-liner: |
| 168 | + |
| 169 | +```python |
| 170 | +with envify: |
| 171 | + foo = lambda n: lambda i: n << n + i |
| 172 | +``` |
| 173 | + |
| 174 | +``envify`` is not part of the Lispython dialect definition, because this particular, perhaps rarely used, feature is not really worth a global performance hit whenever a function is entered. |
| 175 | + |
| 176 | + |
| 177 | +### CAUTION |
| 178 | + |
| 179 | +No instrumentation exists (or is even planned) for the Lispython layer; you'll have to use regular Python tooling to profile, debug, and such. The Lispython layer should be thin enough for this not to be a major problem in practice. |
| 180 | + |
| 181 | + |
| 182 | +### Etymology? |
| 183 | + |
| 184 | +*Lispython* is obviously made of two parts: Python, and... |
0 commit comments