Skip to content

Commit bbf3a94

Browse files
committed
document dialects
Initial version of documentation from Pydialect; then updated.
1 parent f5e1a78 commit bbf3a94

File tree

10 files changed

+1293
-5
lines changed

10 files changed

+1293
-5
lines changed

doc/dialects.md

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,46 @@
1+
# Python dialect examples in ``unpythonic.dialects``
2+
3+
What if Python had automatic tail-call optimization, an implicit return statement, and automatically named, multi-expression lambdas? Look no further:
4+
5+
```python
6+
from unpythonic.dialects import dialects, Lispython # noqa: F401
7+
8+
def factorial(n):
9+
def f(k, acc):
10+
if k == 1:
11+
return acc
12+
f(k - 1, k * acc)
13+
f(n, acc=1)
14+
assert factorial(4) == 24
15+
factorial(5000) # no crash
16+
17+
# - brackets denote a multiple-expression lambda body
18+
# (if you want to have one expression that is a literal list,
19+
# double the brackets: `lambda x: [[5 * x]]`)
20+
# - local[name << value] makes an expression-local variable
21+
lam = lambda x: [local[y << 2 * x],
22+
y + 1]
23+
assert lam(10) == 21
24+
25+
t = letrec[((evenp, lambda x: (x == 0) or oddp(x - 1)),
26+
(oddp, lambda x:(x != 0) and evenp(x - 1))) in
27+
[local[x << evenp(100)],
28+
(x, evenp.__name__, oddp.__name__)]]
29+
assert t == (True, "evenp", "oddp")
30+
```
31+
32+
The [dialects subsystem of `mcpyrate`](https://github.com/Technologicat/mcpyrate/blob/master/doc/dialects.md) makes Python into a language platform, à la [Racket](https://racket-lang.org/).
33+
It provides the plumbing that allows to create, in Python, dialects that compile into Python
34+
at import time. It is geared toward creating languages that extend Python
35+
and look almost like Python, but extend or modify its syntax and/or semantics.
36+
Hence *dialects*.
37+
38+
As examples of what can be done with a dialects system together with a kitchen-sink language extension macro package such as `unpythonic`, we currently provide the following dialects:
39+
40+
- [**Lispython**: Python with tail-call optimization (TCO), implicit return, multi-expression lambdas](dialects/lispython.md)
41+
- [**Pytkell**: Python with automatic currying and lazy functions](dialects/pytkell.md)
42+
- [**Listhell**: Python with prefix syntax and automatic currying](dialects/listhell.md)
43+
44+
All three dialects support `unpythonic`'s ``continuations`` block macro, to add ``call/cc`` to the language; but it is not enabled automatically.
45+
46+
Mostly, these dialects are intended as a cross between teaching material and a (fully functional!) practical joke, but Lispython may occasionally come in handy.

doc/dialects/lis.png

35.3 KB
Loading

doc/dialects/lis.svg

Lines changed: 903 additions & 0 deletions
Loading

doc/dialects/lispython.md

Lines changed: 184 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,184 @@
1+
## Lispython: the love child of Python and Scheme
2+
3+
![mascot](lis.png)
4+
5+
Powered by [`mcpyrate`](https://github.com/Technologicat/mcpyrate/) and `unpythonic`.
6+
7+
```python
8+
from unpythonic.dialects import dialects, Lispython # noqa: F401
9+
10+
def factorial(n):
11+
def f(k, acc):
12+
if k == 1:
13+
return acc
14+
f(k - 1, k*acc)
15+
f(n, acc=1)
16+
assert factorial(4) == 24
17+
factorial(5000) # no crash
18+
19+
t = letrec[((evenp, lambda x: (x == 0) or oddp(x - 1)),
20+
(oddp, lambda x: (x != 0) and evenp(x - 1))) in
21+
evenp(10000)]
22+
assert t is True
23+
24+
square = lambda x: x**2
25+
assert square(3) == 9
26+
assert square.__name__ == "square"
27+
28+
g = lambda x: [local[y << 2*x],
29+
y + 1]
30+
assert g(10) == 21
31+
32+
c = cons(1, 2)
33+
assert tuple(c) == (1, 2)
34+
assert car(c) == 1
35+
assert cdr(c) == 2
36+
assert ll(1, 2, 3) == llist((1, 2, 3))
37+
```
38+
39+
### Features
40+
41+
In terms of ``unpythonic.syntax``, we implicitly enable ``tco``, ``autoreturn``, ``multilambda``, ``namedlambda``, and ``quicklambda`` for the whole module:
42+
43+
- TCO in both ``def`` and ``lambda``, fully automatic
44+
- Omit ``return`` in any tail position, like in Lisps
45+
- Multiple-expression lambdas, ``lambda x: [expr0, ...]``
46+
- Named lambdas (whenever the machinery can figure out a name)
47+
- The underscore: ``f[_*3] --> lambda x: x*3`` (name ``f`` is **reserved**)
48+
49+
We also import some macros and functions to serve as dialect builtins:
50+
51+
- All ``let[]`` and ``do[]`` constructs from ``unpythonic.syntax``
52+
- ``cons``, ``car``, ``cdr``, ``ll``, ``llist``, ``nil``, ``prod``
53+
- ``dyn``, for dynamic assignment
54+
55+
For detailed documentation of the language features, see [``unpythonic.syntax``](https://github.com/Technologicat/unpythonic/tree/master/doc/macros.md), especially the macros ``tco``, ``autoreturn``, ``multilambda``, ``namedlambda``, ``quicklambda``, ``let`` and ``do``.
56+
57+
The multi-expression lambda syntax uses ``do[]``, so it also allows lambdas to manage local variables using ``local[name << value]`` and ``delete[name]``. See the documentation of ``do[]`` for details.
58+
59+
The builtin ``let[]`` constructs are ``let``, ``letseq``, ``letrec``, the decorator versions ``dlet``, ``dletseq``, ``dletrec``, the block versions (decorator, call immediately, replace def'd name with result) ``blet``, ``bletseq``, ``bletrec``, and the code-splicing variants ``let_syntax`` and ``abbrev``. Bindings may be made using any syntax variant supported by ``unpythonic.syntax``.
60+
61+
The builtin ``do[]`` constructs are ``do`` and ``do0``.
62+
63+
If you need more stuff, `unpythonic` is effectively the standard library of Lispython, on top of what Python itself already provides.
64+
65+
66+
### What Lispython is
67+
68+
Lispython is a dialect of Python implemented via macros and a thin whole-module AST transformation. The dialect definition lives in [`unpythonic.dialects.lispython`](../../unpythonic/dialects/lispython.py). Usage examples can be found in [the unit tests](../../unpythonic/dialects/tests/test_lispython.py).
69+
70+
The goal of the Lispython dialect is to fix some glaring issues that hamper Python when viewed from a Lisp/Scheme perspective, as well as make the popular almost-Lisp, Python, feel slightly more lispy.
71+
72+
We take the approach of a relatively thin layer of macros (and underlying functions that implement the actual functionality), minimizing magic as far as reasonably possible.
73+
74+
Performance is only a secondary concern; performance-critical parts fare better at the other end of [the wide spectrum](https://en.wikipedia.org/wiki/Wide-spectrum_language), with [Cython](http://cython.org/). Lispython is for [the remaining 80%](https://en.wikipedia.org/wiki/Pareto_principle), where the bottleneck is human developer time.
75+
76+
77+
### Comboability
78+
79+
The aforementioned block macros are enabled implicitly for the whole module; this is the essence of the Lispython dialect. Other block macros can still be invoked manually in the user code.
80+
81+
Of the other block macros in ``unpythonic.syntax``, code written in Lispython supports only ``continuations``. ``autoref`` should also be harmless enough (will expand too early, but shouldn't matter).
82+
83+
``prefix``, ``curry``, ``lazify`` and ``envify`` are **not compatible** with the ordering of block macros implicit in the Lispython dialect.
84+
85+
``prefix`` is an outside-in macro that should expand first, so it should be placed in a lexically outer position with respect to the ones Lispython invokes implicitly; but nothing can be more outer than the dialect template.
86+
87+
The other three are inside-out macros that should expand later, so similarly, also they should be placed in a lexically outer position.
88+
89+
Basically, any block macro that can be invoked *lexically inside* a ``with tco`` block will work, the rest will not.
90+
91+
If you need e.g. a lazy Lispython, the way to do that is to make a copy of the dialect module, change the dialect template to import the ``lazify`` macro, and then include a ``with lazify`` in the appropriate position, outside the ``with namedlambda`` block. Other customizations can be made similarly.
92+
93+
94+
### Lispython and continuations (call/cc)
95+
96+
Just use ``with continuations`` from ``unpythonic.syntax`` where needed. See its documentation for usage.
97+
98+
Lispython works with ``with continuations``, because:
99+
100+
- Nesting ``with continuations`` within a ``with tco`` block is allowed, for the specific reason of supporting continuations in Lispython.
101+
102+
The dialect's implicit ``with tco`` will just skip the ``with continuations`` block (``continuations`` implies TCO).
103+
104+
- ``autoreturn``, ``quicklambda`` and ``multilambda`` are outside-in macros, so although they will be in a lexically outer position with respect to the manually invoked ``with continuations`` in the user code, this is correct (because being on the outside, they run before ``continuations``, as they should).
105+
106+
- The same applies to the outside-in pass of ``namedlambda``. Its inside-out pass, on the other hand, must come after ``continuations``, which it does, since the dialect's implicit ``with namedlambda`` is in a lexically outer position with respect to the ``with continuations``.
107+
108+
Be aware, though, that the combination of the ``autoreturn`` implicit in the dialect and ``with continuations`` might have usability issues, because ``continuations`` handles tail calls specially (the target of a tail-call in a ``continuations`` block must be continuation-enabled; see the documentation of ``continuations``), and ``autoreturn`` makes it visually slightly less clear which positions are in factorial tail calls (since no explicit ``return``). Also, the top level of a ``with continuations`` block may not use ``return`` - while Lispython happily auto-injects a ``return`` to whatever is the last statement in any particular function.
109+
110+
111+
### Why extend Python?
112+
113+
[Racket](https://racket-lang.org/) is an excellent Lisp, especially with [sweet](https://docs.racket-lang.org/sweet/), sweet expressions [[1]](https://sourceforge.net/projects/readable/) [[2]](https://srfi.schemers.org/srfi-110/srfi-110.html) [[3]](https://srfi.schemers.org/srfi-105/srfi-105.html), not to mention extremely pythonic. The word is *rackety*; the syntax of the language comes with an air of Zen minimalism (as perhaps expected of a descendant of Scheme), but the focus on *batteries included* and understandability are remarkably similar to the pythonic ideal. Racket even has an IDE (DrRacket) and an equivalent of PyPI, and the documentation is simply stellar.
114+
115+
Python, on the other hand, has a slight edge in usability to the end-user programmer, and importantly, a huge ecosystem of libraries, second to ``None``. Python is where science happens (unless you're in CS). Python is an almost-Lisp that has delivered on [the productivity promise](http://paulgraham.com/icad.html) of Lisp. Python also gets many things right, such as well developed support for lazy sequences, and decorators.
116+
117+
In certain other respects, Python the base language leaves something to be desired, if you have been exposed to Racket (or Haskell, but that's a different story). Writing macros is harder due to the irregular syntax, but thankfully MacroPy already exists, and any set of macros only needs to be created once.
118+
119+
Practicality beats purity ([ZoP §9](https://www.python.org/dev/peps/pep-0020/)): hence, fix the minor annoyances that would otherwise quickly add up, and reap the benefits of both worlds. If Python is software glue, Lispython is an additive that makes it flow better.
120+
121+
122+
### PG's accumulator-generator puzzle
123+
124+
The puzzle was posted by Paul Graham in 2002, in the essay [Revenge of the Nerds](http://paulgraham.com/icad.html). It asks to implement, in the shortest code possible, an accumulator-generator. The desired behavior is:
125+
126+
```python
127+
f = foo(10)
128+
assert f(1) == 11
129+
assert f(1) == 12
130+
assert f(5) == 17
131+
```
132+
133+
(The original name of the function is literally `foo`; we have chosen to keep the name, although [nowadays one should do better than that](https://docs.racket-lang.org/style/reference-style.html#%28part._examples-style%29).)
134+
135+
Even Lispython can do no better than this let-over-lambda (here using the haskelly let-in syntax to establish let-bindings):
136+
137+
```python
138+
foo = lambda n0: let[(n, n0) in
139+
(lambda i: n << n + i)]
140+
```
141+
142+
This still sets up a separate place for the accumulator (that is, separate from the argument of the outer function). The modern pure Python solution avoids that, but needs many lines:
143+
144+
```python
145+
def foo(n):
146+
def accumulate(i):
147+
nonlocal n
148+
n += i
149+
return n
150+
return accumulate
151+
```
152+
153+
The problem is that assignment to a lexical variable (including formal parameters) is a statement in Python. Python 3.8's walrus operator does not solve this, because `n := n + i` by itself is a syntax error.
154+
155+
If we abbreviate ``accumulate`` as a lambda, it needs a ``let`` environment to write in, to use `unpythonic`'s expression-assignment (`name << value`).
156+
157+
But see ``envify`` in ``unpythonic.syntax``, which shallow-copies function arguments into an `env` implicitly:
158+
159+
```python
160+
from unpythonic.syntax import macros, envify
161+
162+
with envify:
163+
def foo(n):
164+
return lambda i: n << n + i
165+
```
166+
167+
or as a one-liner:
168+
169+
```python
170+
with envify:
171+
foo = lambda n: lambda i: n << n + i
172+
```
173+
174+
``envify`` is not part of the Lispython dialect definition, because this particular, perhaps rarely used, feature is not really worth a global performance hit whenever a function is entered.
175+
176+
177+
### CAUTION
178+
179+
No instrumentation exists (or is even planned) for the Lispython layer; you'll have to use regular Python tooling to profile, debug, and such. The Lispython layer should be thin enough for this not to be a major problem in practice.
180+
181+
182+
### Etymology?
183+
184+
*Lispython* is obviously made of two parts: Python, and...

doc/dialects/listhell.md

Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,58 @@
1+
## Listhell: it's not Lisp, it's not Python, it's not Haskell
2+
3+
Powered by [`mcpyrate`](https://github.com/Technologicat/mcpyrate/) and `unpythonic`.
4+
5+
```python
6+
from unpythonic.dialects import dialects, Listhell # noqa: F401
7+
8+
from unpythonic import foldr, cons, nil, ll
9+
10+
(print, "hello from Listhell")
11+
12+
double = lambda x: 2 * x
13+
my_map = lambda f: (foldr, (compose, cons, f), nil)
14+
assert (my_map, double, (q, 1, 2, 3)) == (ll, 2, 4, 6)
15+
```
16+
17+
### Features
18+
19+
In terms of ``unpythonic.syntax``, we implicitly enable ``prefix`` and ``curry`` for the whole module.
20+
21+
The following are dialect builtins:
22+
23+
- ``apply``, aliased to ``unpythonic.fun.apply``
24+
- ``compose``, aliased to unpythonic's currying right-compose ``composerc``
25+
- ``q``, ``u``, ``kw`` for the prefix syntax (note these are not `mcpyrate`'s
26+
``q`` and ``u``, but those from `unpythonic.syntax`, specifically for ``prefix``)
27+
28+
For detailed documentation of the language features, see [``unpythonic.syntax``](https://github.com/Technologicat/unpythonic/tree/master/doc/macros.md).
29+
30+
If you need more stuff, `unpythonic` is effectively the standard library of Listhell, on top of what Python itself already provides.
31+
32+
33+
### What Listhell is
34+
35+
Listhell is a dialect of Python implemented via macros and a thin whole-module AST transformation. The dialect definition lives in [`unpythonic.dialects.listhell`](../../unpythonic/dialects/listhell.py). Usage examples can be found in [the unit tests](../../unpythonic/dialects/tests/test_listhell.py).
36+
37+
Listhell is essentially a demonstration of how Python could look, if it had Lisp's prefix syntax for function calls and Haskell's automatic currying.
38+
39+
It's also a minimal example of how to make an AST-transforming dialect.
40+
41+
42+
### Comboability
43+
44+
Only outside-in macros that should expand after ``curry`` (currently, `unpythonic` provides no such macros) and inside-out macros that should expand before ``curry`` (there are two, namely ``tco`` and ``continuations``) can be used in programs written in the Listhell dialect.
45+
46+
47+
### Notes
48+
49+
If you like the idea and want autocurry for a Lisp, try
50+
[spicy](https://github.com/Technologicat/spicy) for [Racket](https://racket-lang.org/).
51+
52+
### CAUTION
53+
54+
Not intended for serious use.
55+
56+
### Etymology?
57+
58+
Prefix syntax of **Lis**p, speed of Py**th**on, and readability of Hask**ell**, all in one.

doc/dialects/pytkell.md

Lines changed: 95 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,95 @@
1+
## Pytkell: Because it's good to have a kell
2+
3+
Powered by [`mcpyrate`](https://github.com/Technologicat/mcpyrate/) and `unpythonic`.
4+
5+
```python
6+
from unpythonic.dialects import dialects, Pytkell # noqa: F401
7+
8+
from operator import add, mul
9+
10+
def addfirst2(a, b, c):
11+
return a + b
12+
assert addfirst2(1)(2)(1/0) == 3
13+
14+
assert tuple(scanl(add, 0, (1, 2, 3))) == (0, 1, 3, 6)
15+
assert tuple(scanr(add, 0, (1, 2, 3))) == (0, 3, 5, 6)
16+
17+
my_sum = foldl(add, 0)
18+
my_prod = foldl(mul, 1)
19+
my_map = lambda f: foldr(compose(cons, f), nil)
20+
assert my_sum(range(1, 5)) == 10
21+
assert my_prod(range(1, 5)) == 24
22+
assert tuple(my_map((lambda x: 2*x), (1, 2, 3))) == (2, 4, 6)
23+
24+
pt = forall[z << range(1, 21), # hypotenuse
25+
x << range(1, z+1), # shorter leg
26+
y << range(x, z+1), # longer leg
27+
insist(x*x + y*y == z*z),
28+
(x, y, z)]
29+
assert tuple(sorted(pt)) == ((3, 4, 5), (5, 12, 13), (6, 8, 10),
30+
(8, 15, 17), (9, 12, 15), (12, 16, 20))
31+
32+
factorials = scanl(mul, 1, s(1, 2, ...)) # 0!, 1!, 2!, ...
33+
assert last(take(6, factorials)) == 120
34+
35+
x = let[(a, 21) in 2*a]
36+
assert x == 42
37+
x = let[2*a, where(a, 21)]
38+
assert x == 42
39+
```
40+
41+
### Features
42+
43+
In terms of ``unpythonic.syntax``, we implicitly enable ``curry`` and ``lazify`` for the whole module.
44+
45+
We also import some macros and functions to serve as dialect builtins:
46+
47+
- All ``let[]`` and ``do[]`` constructs from ``unpythonic.syntax``
48+
- ``lazy[]`` and ``lazyrec[]`` for manual lazification of atoms and data structure literals, respectively
49+
- If-elseif-else expression ``cond[]``
50+
- Nondeterministic evaluation ``forall[]`` (do-notation in the List monad)
51+
- Function composition, ``compose`` (like Haskell's ``.`` operator), aliased to `unpythonic`'s currying right-compose ``composerc``
52+
- Linked list utilities ``cons``, ``car``, ``cdr``, ``ll``, ``llist``, ``nil``
53+
- Folds and scans ``foldl``, ``foldr``, ``scanl``, ``scanr``
54+
- Memoization ``memoize``, ``gmemoize``, ``imemoize``, ``fimemoize``
55+
- Functional updates ``fup`` and ``fupdate``
56+
- Immutable dict ``frozendict``
57+
- Mathematical sequences ``s``, ``m``, ``mg``
58+
- Iterable utilities ``islice`` (`unpythonic`'s version), ``take``, ``drop``, ``split_at``, ``first``, ``second``, ``nth``, ``last``
59+
- Function arglist reordering utilities ``flip``, ``rotate``
60+
61+
For detailed documentation of the language features, see [``unpythonic.syntax``](https://github.com/Technologicat/unpythonic/tree/master/doc/macros.md).
62+
63+
The builtin ``let[]`` constructs are ``let``, ``letseq``, ``letrec``, the decorator versions ``dlet``, ``dletseq``, ``dletrec``, the block versions (decorator, call immediately, replace `def`'d name with result) ``blet``, ``bletseq``, ``bletrec``. Bindings may be made using any syntax variant supported by ``unpythonic.syntax``.
64+
65+
The builtin ``do[]`` constructs are ``do`` and ``do0``.
66+
67+
If you need more stuff, `unpythonic` is effectively the standard library of Pytkell, on top of what Python itself already provides.
68+
69+
70+
### What Pytkell is
71+
72+
Pytkell is a dialect of Python implemented via macros and a thin whole-module AST transformation. The dialect definition lives in [`unpythonic.dialects.pytkell`](../../unpythonic/dialects/lispython.py). Usage examples can be found in [the unit tests](../../unpythonic/dialects/tests/test_pytkell.py).
73+
74+
Pytkell essentially makes Python feel slightly more haskelly.
75+
76+
It's also a minimal example of how to make an AST-transforming dialect.
77+
78+
79+
### Comboability
80+
81+
**Not** comboable with most of the block macros in ``unpythonic.syntax``, because ``curry`` and ``lazify`` appear in the dialect template, hence at the lexically outermost position.
82+
83+
Only outside-in macros that should expand after ``lazify`` has recorded its userlambdas (currently, `unpythonic` provides no such macros) and inside-out macros that should expand before ``curry`` (there are two, namely ``tco`` and ``continuations``) can be used in programs written in the Pytkell dialect.
84+
85+
86+
### CAUTION
87+
88+
No instrumentation exists (or is even planned) for the Pytkell layer; you'll have to use regular Python tooling to profile, debug, and such.
89+
90+
This layer is not quite as thin as Lispython's, but the dialect is not intended for serious use, either.
91+
92+
93+
### Etymology?
94+
95+
The other obvious contraction, *Pyskell*, sounds like a serious programming language (or possibly the name of a fantasy airship), whereas *Pytkell* is obviously something quickly thrown together for system testing.

0 commit comments

Comments
 (0)