Skip to content

Commit 618dc5e

Browse files
loewisloewis
committed
Merged revisions 62004 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk ........ r62004 | georg.brandl | 2008-03-28 13:11:56 +0100 (Fr, 28 Mär 2008) | 4 lines Patch #1810 by Thomas Lee, reviewed by myself: allow compiling Python AST objects into code objects in compile(). ........
1 parent d337279 commit 618dc5e

8 files changed

Lines changed: 3474 additions & 51 deletions

File tree

‎Doc/library/_ast.rst‎

Lines changed: 13 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -10,16 +10,16 @@ Abstract Syntax Trees
1010

1111

1212
The ``_ast`` module helps Python applications to process trees of the Python
13-
abstract syntax grammar. The Python compiler currently provides read-only access
14-
to such trees, meaning that applications can only create a tree for a given
15-
piece of Python source code; generating :term:`bytecode` from a (potentially modified)
16-
tree is not supported. The abstract syntax itself might change with each Python
17-
release; this module helps to find out programmatically what the current grammar
18-
looks like.
13+
abstract syntax grammar. The abstract syntax itself might change with each
14+
Python release; this module helps to find out programmatically what the current
15+
grammar looks like.
1916

20-
An abstract syntax tree can be generated by passing ``_ast.PyCF_ONLY_AST`` as a
21-
flag to the :func:`compile` builtin function. The result will be a tree of
22-
objects whose classes all inherit from ``_ast.AST``.
17+
An abstract syntax tree can be generated by passing :data:`_ast.PyCF_ONLY_AST`
18+
as a flag to the :func:`compile` builtin function. The result will be a tree of
19+
objects whose classes all inherit from :class:`_ast.AST`.
20+
21+
A modified abstract syntax tree can be compiled into a Python code object using
22+
the built-in :func:`compile` function.
2323

2424
The actual classes are derived from the ``Parser/Python.asdl`` file, which is
2525
reproduced below. There is one class defined for each left-hand side symbol in
@@ -39,12 +39,15 @@ attribute ``left`` of type ``_ast.expr``. Instances of ``_ast.expr`` and
3939
``_ast.stmt`` subclasses also have lineno and col_offset attributes. The lineno
4040
is the line number of source text (1 indexed so the first line is line 1) and
4141
the col_offset is the utf8 byte offset of the first token that generated the
42-
node. The utf8 offset is recorded because the parser uses utf8 internally.
42+
node. The utf8 offset is recorded because the parser uses utf8 internally.
4343

4444
If these attributes are marked as optional in the grammar (using a question
4545
mark), the value might be ``None``. If the attributes can have zero-or-more
4646
values (marked with an asterisk), the values are represented as Python lists.
4747

48+
The constructors of all ``_ast`` classes don't take arguments; instead, if you
49+
create instances, you must assign the required attributes separately.
50+
4851

4952
Abstract Grammar
5053
----------------

‎Doc/library/functions.rst‎

Lines changed: 18 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -193,21 +193,21 @@ available. They are listed here in alphabetical order.
193193

194194
.. function:: compile(source, filename, mode[, flags[, dont_inherit]])
195195

196-
Compile the *source* into a code object. Code objects can be executed by a call
197-
to :func:`exec` or evaluated by a call to :func:`eval`. The *filename* argument
198-
should give the file from which the code was read; pass some recognizable value
199-
if it wasn't read from a file (``'<string>'`` is commonly used). The *mode*
200-
argument specifies what kind of code must be compiled; it can be ``'exec'`` if
201-
*source* consists of a sequence of statements, ``'eval'`` if it consists of a
202-
single expression, or ``'single'`` if it consists of a single interactive
203-
statement (in the latter case, expression statements that evaluate to something
204-
else than ``None`` will be printed).
205-
206-
When compiling multi-line statements, two caveats apply: line endings must be
207-
represented by a single newline character (``'\n'``), and the input must be
208-
terminated by at least one newline character. If line endings are represented
209-
by ``'\r\n'``, use the string :meth:`replace` method to change them into
210-
``'\n'``.
196+
Compile the *source* into a code object. Code objects can be
197+
executed by a call to :func:`exec` or evaluated by a call to
198+
:func:`eval`. *source* can either be a string or an AST object.
199+
Refer to the :mod:`_ast` module documentation for information on
200+
how to compile into and from AST objects.
201+
202+
The *filename* argument should give the file from
203+
which the code was read; pass some recognizable value if it wasn't
204+
read from a file (``'<string>'`` is commonly used). The *mode*
205+
argument specifies what kind of code must be compiled; it can be
206+
``'exec'`` if *source* consists of a sequence of statements,
207+
``'eval'`` if it consists of a single expression, or ``'single'``
208+
if it consists of a single interactive statement (in the latter
209+
case, expression statements that evaluate to something else than
210+
``None`` will be printed).
211211

212212
The optional arguments *flags* and *dont_inherit* (which are new in Python 2.2)
213213
control which future statements (see :pep:`236`) affect the compilation of
@@ -227,6 +227,9 @@ available. They are listed here in alphabetical order.
227227
This function raises :exc:`SyntaxError` if the compiled source is invalid,
228228
and :exc:`TypeError` if the source contains null bytes.
229229

230+
.. versionadded:: 2.6
231+
Support for compiling AST objects.
232+
230233

231234
.. function:: complex([real[, imag]])
232235

‎Include/Python-ast.h‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -542,3 +542,5 @@ keyword_ty _Py_keyword(identifier arg, expr_ty value, PyArena *arena);
542542
alias_ty _Py_alias(identifier name, identifier asname, PyArena *arena);
543543

544544
PyObject* PyAST_mod2obj(mod_ty t);
545+
mod_ty PyAST_obj2mod(PyObject* ast, PyArena* arena);
546+
int PyAST_Check(PyObject* obj);

‎Lib/test/test_compile.py‎

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
import unittest
22
import sys
3+
import _ast
34
from test import test_support
45

56
class TestSpecifics(unittest.TestCase):
@@ -406,6 +407,28 @@ def f():
406407
self.assert_("_A__mangled_mod" in A.f.__code__.co_varnames)
407408
self.assert_("__package__" in A.f.__code__.co_varnames)
408409

410+
def test_compile_ast(self):
411+
fname = __file__
412+
if fname.lower().endswith(('pyc', 'pyo')):
413+
fname = fname[:-1]
414+
with open(fname, 'r') as f:
415+
fcontents = f.read()
416+
sample_code = [
417+
['<assign>', 'x = 5'],
418+
['<ifblock>', """if True:\n pass\n"""],
419+
['<forblock>', """for n in [1, 2, 3]:\n print(n)\n"""],
420+
['<deffunc>', """def foo():\n pass\nfoo()\n"""],
421+
[fname, fcontents],
422+
]
423+
424+
for fname, code in sample_code:
425+
co1 = compile(code, '%s1' % fname, 'exec')
426+
ast = compile(code, '%s2' % fname, 'exec', _ast.PyCF_ONLY_AST)
427+
self.assert_(type(ast) == _ast.Module)
428+
co2 = compile(ast, '%s3' % fname, 'exec')
429+
self.assertEqual(co1, co2)
430+
431+
409432
def test_main():
410433
test_support.run_unittest(TestSpecifics)
411434

0 commit comments

Comments
 (0)