Skip to content

Commit 8457b69

Browse files
committed
Merged revisions 63829-63831,63858,63865,63879,63882,63948,63970-63972,63976,63989,64014-64015,64021-64022,64063-64065,64067 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk ........ r63829 | mark.summerfield | 2008-05-31 15:05:34 +0200 (Sat, 31 May 2008) | 4 lines Added a note to [] that special forms & special chars lose their meaning and backrefs can't be used inside [] ........ r63830 | georg.brandl | 2008-05-31 16:40:09 +0200 (Sat, 31 May 2008) | 2 lines #3010: clarification about stdin/use_rawinput. ........ r63831 | georg.brandl | 2008-05-31 16:45:55 +0200 (Sat, 31 May 2008) | 2 lines #3005: add explaining sentence to easydialogs docs. ........ r63858 | georg.brandl | 2008-06-01 18:41:31 +0200 (Sun, 01 Jun 2008) | 2 lines Add plain text make target. ........ r63865 | georg.brandl | 2008-06-01 21:24:36 +0200 (Sun, 01 Jun 2008) | 2 lines Spaces vs. tabs. ........ r63879 | gregory.p.smith | 2008-06-02 00:57:47 +0200 (Mon, 02 Jun 2008) | 3 lines Make the _H #define's match the header file names. Fix comments to mention the correct type names. ........ r63882 | gregory.p.smith | 2008-06-02 01:48:47 +0200 (Mon, 02 Jun 2008) | 3 lines Adds a Thread.getIdent() method to provide the _get_ident() value for any given threading.Thread object. feature request issue 2871. ........ r63948 | alexandre.vassalotti | 2008-06-04 22:41:44 +0200 (Wed, 04 Jun 2008) | 2 lines Fixed complex.__getnewargs__() to not emit another complex object. ........ r63970 | andrew.kuchling | 2008-06-06 01:33:54 +0200 (Fri, 06 Jun 2008) | 1 line Document 'utc' parameter ........ r63971 | andrew.kuchling | 2008-06-06 01:35:31 +0200 (Fri, 06 Jun 2008) | 1 line Add various items ........ r63972 | andrew.kuchling | 2008-06-06 01:35:48 +0200 (Fri, 06 Jun 2008) | 1 line Grammar fix ........ r63976 | georg.brandl | 2008-06-06 09:34:50 +0200 (Fri, 06 Jun 2008) | 2 lines Markup fix. ........ r63989 | thomas.heller | 2008-06-06 20:42:11 +0200 (Fri, 06 Jun 2008) | 2 lines Add a reminder for the maintainer of whatsnew. ........ r64014 | georg.brandl | 2008-06-07 17:59:10 +0200 (Sat, 07 Jun 2008) | 3 lines Factor out docstring dedenting from inspect.getdoc() into inspect.cleandoc() to ease standalone use of the algorithm. ........ r64015 | georg.brandl | 2008-06-07 18:04:01 +0200 (Sat, 07 Jun 2008) | 2 lines Revert unwanted changes. ........ r64021 | georg.brandl | 2008-06-07 20:16:12 +0200 (Sat, 07 Jun 2008) | 2 lines X-ref to numbers module. ........ r64022 | georg.brandl | 2008-06-07 20:17:37 +0200 (Sat, 07 Jun 2008) | 3 lines Document the "st" API, to avoid confusion with the "new" AST. Add a note about using the new AST module. ........ r64063 | martin.v.loewis | 2008-06-10 07:03:35 +0200 (Tue, 10 Jun 2008) | 2 lines Add Gregor Lingl. ........ r64064 | georg.brandl | 2008-06-10 09:45:28 +0200 (Tue, 10 Jun 2008) | 2 lines Add the "ast" module, containing helpers to ease use of the "_ast" classes. ........ r64065 | raymond.hettinger | 2008-06-10 09:57:15 +0200 (Tue, 10 Jun 2008) | 1 line Add Arnaud for his efforts on multi-arg set operations. ........ r64067 | georg.brandl | 2008-06-10 14:46:39 +0200 (Tue, 10 Jun 2008) | 2 lines #2536: fix itertools.permutations and itertools.combinations docstrings. ........
1 parent 1497d8e commit 8457b69

27 files changed

Lines changed: 931 additions & 212 deletions

Doc/ACKS.txt

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -157,6 +157,7 @@ docs@python.org), and we'll be glad to correct the problem.
157157
* Bernhard Reiter
158158
* Armin Rigo
159159
* Wes Rishel
160+
* Armin Ronacher
160161
* Jim Roskind
161162
* Guido van Rossum
162163
* Donald Wallace Rouse II

Doc/Makefile

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,7 @@ help:
2121
@echo " web to make file usable by Sphinx.web"
2222
@echo " htmlhelp to make HTML files and a HTML help project"
2323
@echo " latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter"
24+
@echo " text to make plain text files"
2425
@echo " changes to make an overview over all changed/added/deprecated items"
2526
@echo " linkcheck to check all external links for integrity"
2627
@echo " coverage to check documentation coverage for library and C API"
@@ -75,6 +76,10 @@ latex: build
7576
@echo "Run \`make all-pdf' or \`make all-ps' in that directory to" \
7677
"run these through (pdf)latex."
7778

79+
text: BUILDER = text
80+
text: build
81+
@echo "Build finished; the text files are in build/text."
82+
7883
changes: BUILDER = changes
7984
changes: build
8085
@echo "The overview file is in build/changes."

Doc/README.txt

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -56,6 +56,8 @@ Available make targets are:
5656
* "latex", which builds LaTeX source files that can be run with "pdflatex"
5757
to produce PDF documents.
5858

59+
* "text", which builds a plain text file for each source file.
60+
5961
* "linkcheck", which checks all external references to see whether they are
6062
broken, redirected or malformed, and outputs this information to stdout
6163
as well as a plain-text (.txt) file.

Doc/library/_ast.rst

Lines changed: 0 additions & 83 deletions
This file was deleted.

Doc/library/abc.rst

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -9,9 +9,9 @@
99
.. much of the content adapted from docstrings
1010
1111
This module provides the infrastructure for defining abstract base classes
12-
(ABCs) in Python, as outlined in :pep:`3119`; see the PEP for why this
13-
was added to Python. (See also, :pep:`3141` regarding a type hierarchy
14-
for numbers based on ABCs.)
12+
(ABCs) in Python, as outlined in :pep:`3119`; see the PEP for why this was added
13+
to Python. (See also :pep:`3141` and the :mod:`numbers` module regarding a type
14+
hierarchy for numbers based on ABCs.)
1515

1616
The :mod:`collections` module has some concrete classes that derive from
1717
ABCs; these can, of course, be further derived. In addition the

Doc/library/ast.rst

Lines changed: 257 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,257 @@
1+
.. _ast:
2+
3+
Abstract Syntax Trees
4+
=====================
5+
6+
.. module:: ast
7+
:synopsis: Abstract Syntax Tree classes and manipulation.
8+
9+
.. sectionauthor:: Martin v. Löwis <martin@v.loewis.de>
10+
.. sectionauthor:: Georg Brandl <georg@python.org>
11+
12+
.. versionadded:: 2.5
13+
The low-level ``_ast`` module containing only the node classes.
14+
15+
.. versionadded:: 2.6
16+
The high-level ``ast`` module containing all helpers.
17+
18+
19+
The :mod:`ast` module helps Python applications to process trees of the Python
20+
abstract syntax grammar. The abstract syntax itself might change with each
21+
Python release; this module helps to find out programmatically what the current
22+
grammar looks like.
23+
24+
An abstract syntax tree can be generated by passing :data:`_ast.PyCF_ONLY_AST`
25+
as a flag to the :func:`compile` builtin function, or using the :func:`parse`
26+
helper provided in this module. The result will be a tree of objects whose
27+
classes all inherit from :class:`ast.AST`.
28+
29+
A modified abstract syntax tree can be compiled into a Python code object using
30+
the built-in :func:`compile` function.
31+
32+
Node classes
33+
------------
34+
35+
.. class:: AST
36+
37+
This is the base of all AST node classes. The actual node classes are
38+
derived from the :file:`Parser/Python.asdl` file, which is reproduced
39+
:ref:`below <abstract-grammar>`. They are defined in the :mod:`_ast` C
40+
module and re-exported in :mod:`ast`.
41+
42+
There is one class defined for each left-hand side symbol in the abstract
43+
grammar (for example, :class:`ast.stmt` or :class:`ast.expr`). In addition,
44+
there is one class defined for each constructor on the right-hand side; these
45+
classes inherit from the classes for the left-hand side trees. For example,
46+
:class:`ast.BinOp` inherits from :class:`ast.expr`. For production rules
47+
with alternatives (aka "sums"), the left-hand side class is abstract: only
48+
instances of specific constructor nodes are ever created.
49+
50+
.. attribute:: _fields
51+
52+
Each concrete class has an attribute :attr:`_fields` which gives the names
53+
of all child nodes.
54+
55+
Each instance of a concrete class has one attribute for each child node,
56+
of the type as defined in the grammar. For example, :class:`ast.BinOp`
57+
instances have an attribute :attr:`left` of type :class:`ast.expr`.
58+
59+
If these attributes are marked as optional in the grammar (using a
60+
question mark), the value might be ``None``. If the attributes can have
61+
zero-or-more values (marked with an asterisk), the values are represented
62+
as Python lists. All possible attributes must be present and have valid
63+
values when compiling an AST with :func:`compile`.
64+
65+
.. attribute:: lineno
66+
col_offset
67+
68+
Instances of :class:`ast.expr` and :class:`ast.stmt` subclasses have
69+
:attr:`lineno` and :attr:`col_offset` attributes. The :attr:`lineno` is
70+
the line number of source text (1-indexed so the first line is line 1) and
71+
the :attr:`col_offset` is the UTF-8 byte offset of the first token that
72+
generated the node. The UTF-8 offset is recorded because the parser uses
73+
UTF-8 internally.
74+
75+
The constructor of a class :class:`ast.T` parses its arguments as follows:
76+
77+
* If there are positional arguments, there must be as many as there are items
78+
in :attr:`T._fields`; they will be assigned as attributes of these names.
79+
* If there are keyword arguments, they will set the attributes of the same
80+
names to the given values.
81+
82+
For example, to create and populate an :class:`ast.UnaryOp` node, you could
83+
use ::
84+
85+
node = ast.UnaryOp()
86+
node.op = ast.USub()
87+
node.operand = ast.Num()
88+
node.operand.n = 5
89+
node.operand.lineno = 0
90+
node.operand.col_offset = 0
91+
node.lineno = 0
92+
node.col_offset = 0
93+
94+
or the more compact ::
95+
96+
node = ast.UnaryOp(ast.USub(), ast.Num(5, lineno=0, col_offset=0),
97+
lineno=0, col_offset=0)
98+
99+
100+
.. _abstract-grammar:
101+
102+
Abstract Grammar
103+
----------------
104+
105+
The module defines a string constant ``__version__`` which is the decimal
106+
Subversion revision number of the file shown below.
107+
108+
The abstract grammar is currently defined as follows:
109+
110+
.. literalinclude:: ../../Parser/Python.asdl
111+
112+
113+
:mod:`ast` Helpers
114+
------------------
115+
116+
.. versionadded:: 2.6
117+
118+
Apart from the node classes, :mod:`ast` module defines these utility functions
119+
and classes for traversing abstract syntax trees:
120+
121+
.. function:: parse(expr, filename='<unknown>', mode='exec')
122+
123+
Parse an expression into an AST node. Equivalent to ``compile(expr,
124+
filename, mode, PyCF_ONLY_AST)``.
125+
126+
127+
.. function:: literal_eval(node_or_string)
128+
129+
Safely evaluate an expression node or a string containing a Python
130+
expression. The string or node provided may only consist of the following
131+
Python literal structures: strings, numbers, tuples, lists, dicts, booleans,
132+
and ``None``.
133+
134+
This can be used for safely evaluating strings containing Python expressions
135+
from untrusted sources without the need to parse the values oneself.
136+
137+
138+
.. function:: get_docstring(node, clean=True):
139+
140+
Return the docstring of the given *node* (which must be a
141+
:class:`FunctionDef`, :class:`ClassDef` or :class:`Module` node), or ``None``
142+
if it has no docstring. If *clean* is true, clean up the docstring's
143+
indentation with :func:`inspect.cleandoc`.
144+
145+
146+
.. function:: fix_missing_locations(node)
147+
148+
When you compile a node tree with :func:`compile`, the compiler expects
149+
:attr:`lineno` and :attr:`col_offset` attributes for every node that supports
150+
them. This is rather tedious to fill in for generated nodes, so this helper
151+
adds these attributes recursively where not already set, by setting them to
152+
the values of the parent node. It works recursively starting at *node*.
153+
154+
155+
.. function:: increment_lineno(node, n=1)
156+
157+
Increment the line number of each node in the tree starting at *node* by *n*.
158+
This is useful to "move code" to a different location in a file.
159+
160+
161+
.. function:: copy_location(new_node, old_node)
162+
163+
Copy source location (:attr:`lineno` and :attr:`col_offset`) from *old_node*
164+
to *new_node* if possible, and return *new_node*.
165+
166+
167+
.. function:: iter_fields(node)
168+
169+
Yield a tuple of ``(fieldname, value)`` for each field in ``node._fields``
170+
that is present on *node*.
171+
172+
173+
.. function:: iter_child_nodes(node)
174+
175+
Yield all direct child nodes of *node*, that is, all fields that are nodes
176+
and all items of fields that are lists of nodes.
177+
178+
179+
.. function:: walk(node)
180+
181+
Recursively yield all child nodes of *node*, in no specified order. This is
182+
useful if you only want to modify nodes in place and don't care about the
183+
context.
184+
185+
186+
.. class:: NodeVisitor()
187+
188+
A node visitor base class that walks the abstract syntax tree and calls a
189+
visitor function for every node found. This function may return a value
190+
which is forwarded by the `visit` method.
191+
192+
This class is meant to be subclassed, with the subclass adding visitor
193+
methods.
194+
195+
.. method:: visit(node)
196+
197+
Visit a node. The default implementation calls the method called
198+
:samp:`self.visit_{classname}` where *classname* is the name of the node
199+
class, or :meth:`generic_visit` if that method doesn't exist.
200+
201+
.. method:: generic_visit(node)
202+
203+
This visitor calls :meth:`visit` on all children of the node.
204+
205+
Note that child nodes of nodes that have a custom visitor method won't be
206+
visited unless the visitor calls :meth:`generic_visit` or visits them
207+
itself.
208+
209+
Don't use the :class:`NodeVisitor` if you want to apply changes to nodes
210+
during traversal. For this a special visitor exists
211+
(:class:`NodeTransformer`) that allows modifications.
212+
213+
214+
.. class:: NodeTransformer()
215+
216+
A :class:`NodeVisitor` subclass that walks the abstract syntax tree and
217+
allows modification of nodes.
218+
219+
The `NodeTransformer` will walk the AST and use the return value of the
220+
visitor methods to replace or remove the old node. If the return value of
221+
the visitor method is ``None``, the node will be removed from its location,
222+
otherwise it is replaced with the return value. The return value may be the
223+
original node in which case no replacement takes place.
224+
225+
Here is an example transformer that rewrites all occurrences of name lookups
226+
(``foo``) to ``data['foo']``::
227+
228+
class RewriteName(NodeTransformer):
229+
230+
def visit_Name(self, node):
231+
return copy_location(Subscript(
232+
value=Name(id='data', ctx=Load()),
233+
slice=Index(value=Str(s=node.id)),
234+
ctx=node.ctx
235+
), node)
236+
237+
Keep in mind that if the node you're operating on has child nodes you must
238+
either transform the child nodes yourself or call the :meth:`generic_visit`
239+
method for the node first.
240+
241+
For nodes that were part of a collection of statements (that applies to all
242+
statement nodes), the visitor may also return a list of nodes rather than
243+
just a single node.
244+
245+
Usually you use the transformer like this::
246+
247+
node = YourTransformer().visit(node)
248+
249+
250+
.. function:: dump(node, annotate_fields=True, include_attributes=False)
251+
252+
Return a formatted dump of the tree in *node*. This is mainly useful for
253+
debugging purposes. The returned string will show the names and the values
254+
for fields. This makes the code impossible to evaluate, so if evaluation is
255+
wanted *annotate_fields* must be set to False. Attributes such as line
256+
numbers and column offsets are dumped by default. If this is wanted,
257+
*include_attributes* can be set to ``True``.

0 commit comments

Comments
 (0)