% \iffalse meta-comment % % Copyright (C) 2012 by Geoffrey M. Poore % --------------------------------------------------------------------------- % This work may be distributed and/or modified under the % conditions of the LaTeX Project Public License, either version 1.3 % of this license or (at your option) any later version. % The latest version of this license is in % http://www.latex-project.org/lppl.txt % and version 1.3 or later is part of all distributions of LaTeX % version 2005/12/01 or later. % % This work has the LPPL maintenance status `maintained'. % % The Current Maintainer of this work is Geoffrey M. Poore. % % This work consists of the files pythontex.dtx and pythontex.ins % and the derived filebase pythontex.sty. % % \fi % % \iffalse %<*driver> \ProvidesFile{pythontex.dtx} % %\NeedsTeXFormat{LaTeX2e}[1999/12/01] %\ProvidesPackage{pythontex} %<*package> [2012/07/17 Version~0.9beta3 PythonTeX] % % %<*driver> \documentclass{ltxdoc} \usepackage[T1]{fontenc} \usepackage[utf8]{inputenc} \usepackage[svgnames]{xcolor} \usepackage[fixlr=false]{pythontex} %Need fixlr=false so can demo issue \usepackage{enumitem} \usepackage{hyperref} \hypersetup{colorlinks=true,urlcolor=Green,linkcolor=blue} \usepackage{textcomp} \newcommand{\pytxtodo}[1]{} %\newcommand{\pytxtodo}[1]{\textcolor{red}{TO~DO: \scantokens{#1}}} \EnableCrossrefs \CodelineIndex \RecordChanges \begin{document} \DocInput{pythontex.dtx} \PrintChanges \PrintIndex \end{document} % % \fi % % \CheckSum{1982} % % \CharacterTable % {Upper-case \A\B\C\D\E\F\G\H\I\J\K\L\M\N\O\P\Q\R\S\T\U\V\W\X\Y\Z % Lower-case \a\b\c\d\e\f\g\h\i\j\k\l\m\n\o\p\q\r\s\t\u\v\w\x\y\z % Digits \0\1\2\3\4\5\6\7\8\9 % Exclamation \! Double quote \" Hash (number) \# % Dollar \$ Percent \% Ampersand \& % Acute accent \' Left paren \( Right paren \) % Asterisk \* Plus \+ Comma \, % Minus \- Point \. Solidus \/ % Colon \: Semicolon \; Less than \< % Equals \= Greater than \> Question mark \? % Commercial at \@ Left bracket \[ Backslash \\ % Right bracket \] Circumflex \^ Underscore \_ % Grave accent \` Left brace \{ Vertical bar \| % Right brace \} Tilde \~} % % % \changes{v0.9beta3}{2012/07/17}{Added Unicode support, which required the Python code to be split into one set for Python 2 and another set for Python 3. This will require any old installation to be completely removed, and a new installation created from scratch. Refactoring of Python code. Documents should automatically re-execute all code after updating to the new version. Otherwise, you should delete the PythonTeX directory and run PythonTeX. Improved installation script. Added package options: pyfuture, stderr, upquote, pyglexer, pyginline. Renamed the pygextfile option to fvextfile. Added custom code and workingdir commands. Added the console environment and associated options. Rewrote pythontex\_utils*.py, creating a new, context-aware interface to SymPy's LatexPrinter class. Content brought in via macros no longer uses labels. Rather, long defs are used, which allows line breaks. Pygments highlighting is now default for PythonTeX commands and environments} % \changes{v0.9beta2}{2012/05/09}{Changed Python output extension to .stdout.} % \changes{v0.9beta}{2012/04/27}{Initial public beta release.} % % % \DoNotIndex{\newcommand,\newenvironment} % \DoNotIndex{\#,\$,\%,\&,\@,\\,\{,\},\^,\_,\~,\ } % \DoNotIndex{\@ne} % \DoNotIndex{\advance,\begingroup,\catcode,\closein} % \DoNotIndex{\closeout,\day,\def,\edef,\else,\empty,\endgroup} % \DoNotIndex{\begin,\end,\bgroup,\egroup} % % \providecommand*{\url}{\texttt} % \newcommand{\pytex}{Python\TeX} % \GetFileInfo{pythontex.dtx} % \title{The \textsf{pythontex} package} % \author{Geoffrey M.\ Poore \\ \url{gpoore@gmail.com}} % \date{\fileversion~from \filedate} % % \maketitle % % \begin{abstract} % \pytex\ allows Python code entered within a \LaTeX\ document to be executed, and the output to be included within the original document. This provides access to the full power of Python from within \LaTeX, simplifying Python-\LaTeX\ workflow and making possible a range of document customization and automation. It also allows macro definitions that mix Python and \LaTeX\ code. In addition, \pytex\ provides syntax highlighting for many programming languages via the Pygments Python package. % % \pytex\ is fast and user-friendly. Python code is only executed when it has been modified. When code is executed, it automatically attempts to run in parallel. If Python code produces errors, the error message line numbers are synchronized with the \LaTeX\ document line numbers, so that it is easy to find the misbehaving code. % \end{abstract} % % % \section*{Warning} % \pytex\ makes possible some pretty amazing things. But that power brings with it a certain risk and responsibility. Compiling a document that uses \pytex\ involves executing Python code on your computer. You should only compile \pytex\ documents from sources you trust. \pytex\ comes with NO WARRANTY.\footnote{All \LaTeX\ code is licensed under the \href{http://www.latex-project.org/lppl.txt}{\LaTeX\ Project Public License (LPPL)} and all Python code is licensed under the \href{http://www.opensource.org/licenses/BSD-3-Clause}{BSD 3-Clause License}.} The copyright holder and any additional authors will not be liable for any damages. % % % \section*{Package status} % \pytex\ is currently in ``beta.'' Almost all features intended for version 0.9 are present, and almost all are fully functional (at least, so far as is known!). Testing is the main task that remains. \pytex\ has been primarily developed and tested under Windows with \TeX\ Live and Python 2.7. It has also been tested with Python 3.2 under Windows, and has been used under OS X (10.7) with MacPort's \TeX\ Live and Python 2.7. % \vfill % % % \pagebreak % \tableofcontents % \pagebreak % % % \section{Introduction} % % \LaTeX\ can do a lot,\footnote{\TeX\ is a Turing-complete language.} but the programming required can sometimes be painful.\footnote{As I learned in creating this package.} Also, in spite of the many packages available for \LaTeX, the libraries and packages of a general-purpose programming language are lacking. For these reasons, there have been multiple attempts to allow other languages to be used within \LaTeX. % \begin{itemize} % \item \href{http://www.ctan.org/tex-archive/macros/latex/contrib/perltex/}{Perl\TeX} allows the bodies of \LaTeX\ macros to be written in Perl. % \item \href{http://www.ctan.org/tex-archive/macros/latex/contrib/sagetex/}{Sage\TeX} allows code for the Sage mathematics software to be executed from within a \LaTeX\ document. % \item Martin R.\ Ehmsen's \href{http://web.archive.org/web/20080728170129/www.imada.sdu.dk/~ehmsen/python.sty }{|python.sty|} provides a very basic method of executing Python code from within a \LaTeX\ document. % \item \href{http://elec.otago.ac.nz/w/index.php/SympyTeX}{Sympy\TeX} allows more sophisticated Python execution, and is largely based on a subset of Sage\TeX. % \item \href{http://www.luatex.org/}{Lua\TeX} extends the pdf\TeX\ engine to provide Lua as an embedded scripting language, and as a result yields tight, low-level Lua integration. % \end{itemize} % % \pytex\ attempts to fill a perceived gap in the current integrations of \LaTeX\ with an additional language. It has a number of objectives, only some of which have been met by previous packages. % \begin{description} % \item[Execution speed]\hfill\\ In the approaches mentioned above, all the non-\LaTeX\ code is executed at every compilation of the \LaTeX\ document (Perl\TeX, Lua\TeX, and |python.sty|), or all the non-\LaTeX\ code is executed every time it is modified (Sage\TeX\ and Sympy\TeX). However, many tasks such as plotting and data analysis take significant time to execute. We need a way to fine-tune code execution, so that independent blocks of slow code may be separated into their own sessions and are only executed when modified. If we are going to split code into multiple sessions, we might as well run these sessions in parallel, further increasing speed. A byproduct of this approach is that it now becomes much more feasible to include slower code, since we can still have fast compilations whenever the slow code isn't modified. % \item[Compiling without executing]\hfill\\ Even with all of these features to boost execution speed, there will be times when we have to run slow code. Thus, we need the execution of non-\LaTeX\ code to be separated from compiling the \LaTeX\ document. We need to be able to edit and compile a document containing unexecuted code. Unexecuted code should be invisible or be replaced by placeholders. Sage\TeX\ and Sympy\TeX\ have implemented such a separation of compiling and executing. In contrast, Lua\TeX\ and Perl\TeX\ execute all the code at each compilation---but that is appropriate given their goal of simplifying macro programming. % \item[Error messages]\hfill\\ Whenever code is saved from a \LaTeX\ document to an external file and then executed, the line numbers for any error messages will not correspond to the line numbering of the original \LaTeX\ document. At one extreme, |python.sty| doesn't attempt to deal with this issue, while at the other extreme, Sage\TeX\ uses an ingenous system of |Try|/|Except| statements on every line of code. We need a system that translates all error messages so that they correspond to the line numbering of the original \LaTeX\ document, with minimal overhead when there are no errors. % \item[Syntax highlighting]\hfill\\ Once we begin using non-\LaTeX\ code, sooner or later we will likely wish to typeset some of it, which means we need syntax highlighting. A number of syntax highlighting packages currently exist for \LaTeX; perhaps the most popular are |listings| and |minted|. |listings| uses pure \LaTeX. It has not been updated since 2007, which makes it a less ideal solution in some circumstances. |minted| uses the Python package Pygments to perform highlighting. Pygments can provide superior syntax highlighting, but |minted| can be slow because all code must be highlighted at each compilation. We need syntax highlighting via Pygments that saves all highlighted code, only re-highlighting when there are modifications. Ideally, we would also like a solution that overcomes some of |minted|'s longstanding issues.\footnote{http://code.google.com/p/minted/issues/list} % \item[Context awareness]\hfill\\ It would be nice for the non-\LaTeX\ code to have at least a minimal awareness of its context in the \LaTeX\ document. For example, it would be nice to know whether code is executing within math mode. % \item[Language-independent implementation]\hfill\\ It would be nice to have a system for executing non-\LaTeX\ code that depends very little on the language of the code. We should not expect to be able to escape all language dependence. But if the system is designed to be as general as possible, then it may be expanded in the future to support additional languages. % \item[Printing]\hfill\\ It would be nice for the |print| statement/function,\footnote{In Python, |print| was a statement until Python 3.0, when it became a function. The function form is available via import from |\textunderscore\textunderscore future\textunderscore\textunderscore| in Python 2.6 and later.} or its equivalent, to automatically return its output within the \LaTeX\ document. For example, using |python.sty| it is possible to generate some text while in Python, open a file, save the text to it, close the file, and then |\input| the file after returning to \LaTeX. But it is much simpler to generate the text and |print| it, since the printed content is automatically included in the \LaTeX\ document. This was one of the things that |python.sty| really got right. % \item[Pure code]\hfill\\ \LaTeX\ has a number of special characters (|# $ % & ~ _ ^ \ { }|), which complicates the entry of code in a non-\LaTeX\ language since these same characters are common in many languages. Sage\TeX\ and Sympy\TeX\ delimit all inline code with curly braces (|{}|), but this approach fails in the (somewhat unlikely) event that code needs to contain an unmatched brace. More seriously, they do not allow the percent symbol |%| (modular arithmetic and string formatting in Sage and Python) to be used within inline code. Rather, a |\percent| macro must be used instead. This means that code must (sometimes) be entered as a hybrid between \LaTeX\ and the non-\LaTeX\ language. Lua\TeX\ is somewhat similar: ``The main thing about Lua code in a TeX document is this: the code is expanded by TeX before Lua gets to it. This means that all the Lua code, even the comments, must be valid TeX!''\footnote{\url{http://wiki.contextgarden.net/Programming_in_LuaTeX}} % % This language hybridization is not terribly difficult to work around in the Sage\TeX\ and Sympy\TeX\ cases, and might even be considered a feature in Lua\TeX\ in some contexts. But if we are going to create a system for general-purpose access to a non-\LaTeX\ language, we need \textbf{all} valid code to work correctly in \textbf{all} contexts, with no hybridization of any sort required. We should be able to copy and paste valid code into a \LaTeX\ document, without having to worry about hybridizing it. Among other things, this means that inline code delimiters other than \LaTeX's default curly braces |{}| must be available. % \item[Hybrid code]\hfill\\ Although we need a system that allows input of pure non-\LaTeX\ code, it would also be convenient to allow hybrid code, or code in which \LaTeX\ macros may be present and are expanded before the code is executed. This allows \LaTeX\ data to be easily passed to the non-\LaTeX\ language, facilitating a tighter integration of the two languages and the use of the non-\LaTeX\ language in macro definitions. % \item[Math and science libraries]\hfill\\ The author decided to create \pytex\ after writing a physics dissertation using \LaTeX\ and realizing how frustrating it can be to switch back and forth between a \TeX\ editor and plotting software when fine-tuning figures. We need access to a non-\LaTeX\ language like Python, MATLAB, or Mathematica that provides strong support for data analysis and visualization. To maintain broad appeal, this language should primarily involve open-source tools, should have strong cross-platform support, and should also be suitable for general-purpose programming. % \end{description} % % Python was chosen as the language to fulfill these objectives for several reasons. % \begin{itemize} % \item It is open-source and has good cross-platform support.\footnote{Unfortunately, Sage can only run under Windows within a virtual machine at present; otherwise, an extension of Sage\TeX\ might have been tempting. Then again, for general computing, an approach that utilizes pure Python is probably superior.} % \item It has a strong set of scientific, numeric, and visualization packages, including \href{http://numpy.scipy.org/}{NumPy}, \href{http://www.scipy.org/}{SciPy}, \href{http://matplotlib.sourceforge.net/}{matplotlib}, and \href{http://sympy.org}{SymPy}. Much of the initial motivation for \pytex\ was the ability to create publication-quality plots and perform complex mathematical calculations without having to leave the \TeX\ editor. % \item We need a language that is suitable for scripting. Lua is already available via Lua\TeX, and in any case lacks the math and science tools.\footnote{One could use \href{http://labix.org/lunatic-python}{Lunatic Python}, and some numeric packages for Lua are \href{http://numlua.luaforge.net/}{in development}.} Perl is already available via Perl\TeX, although Perl\TeX's emphasis on Perl for macro creation makes it rather unsuitable for scientific work using the \href{http://pdl.perl.org/}{Perl Data Language (PDL)} or for more general programming. Python is one logical choice for scripting. % \end{itemize} % % Now at this point there will almost certainly be some reader, sooner or later, who wants to object, ``But what about language \textit{X}!'' Well, yes, in some respects the choice to use Python did come down to personal preference. But you should give Python a try, if you haven't already. You may also wish to consider the many interfaces that are available between Python and other languages. If you still aren't satisfied, keep in mind \pytex's ``language-independent'' implementation! Although \pytex\ is written to support Python within \LaTeX, the implementation has been specially crafted so that other languages may be supported in the future. See Section~\ref{sec:future} for more details. % % % % \section{Installing and running} % % \subsection{Installing \pytex} % % \pytex\ requires a \TeX\ installation. \href{http://www.tug.org/texlive/}{\TeX\ Live} or \href{http://miktex.org/}{MiK\TeX} are preferred. \pytex\ requires the |Kpathsea| library, which is available in both of these distributions. The following \LaTeX\ packages, with their dependencies, are also required: |fancyvrb|, |etex|, |etoolbox|, |xstring|, |pgfopts|, |newfloat|, and |color| or |xcolor|. If you are creating and importing graphics using Python, you will also need |graphicx|. The |mdframed| package is recommended for enclosing typeset code in boxes with fancy borders and/or background colors. % % \pytex\ also requires a \href{http://www.python.org/}{Python} installation. Python 2.7 is recommended for the greatest compatibility with scientific tools. Python 3.1 and later will work as well. Earlier versions of Python 2 and 3 are not compatible, at least not without some modifications to the \pytex\ scripts. The Python package \href{http://pygments.org/}{Pygments} must be installed for syntax highlighting to function. \pytex\ has been tested with Pygments 1.4 and later, but the latest version is recommended. For scientific work, or to compile or experiment with the \pytex\ gallery file, the following are also recommended: \href{http://numpy.scipy.org/}{NumPy}, \href{http://www.scipy.org/}{SciPy}, \href{http://matplotlib.sourceforge.net/}{matplotlib}, and \href{http://sympy.org}{SymPy}. % % \pytex\ consists of the following files: % \begin{itemize} % \item Installer file |pythontex.ins| % \item Documented \LaTeX\ source file |pythontex.dtx|, from which |pythontex.pdf| and |pythontex.sty| are generated % \item Main Python scripts |pythontex2.py| and |pythontex3.py| % \item Helper scripts |pythontex_utils2.py| and |pythontex_types2.py|, and |pythontex_utils3.py| and |pythontex_types3.py| % \item Installation script |pythontex_install_texlive| (for \TeX\ Live) % \item README % \item Optional batch file |pythontex.bat| for use in launching |pythontex*.py| under Windows % \end{itemize} % The style file |pythontex.sty| may be generated by running \LaTeX\ on |pythontex.ins|. The documentation you are reading may be generated by running \LaTeX\ on |pythontex.dtx|. Two versions of all of the Python scipts are supplied, one for Python 2 and one for Python 3.\footnote{Unfortunately, it is not possible to provide full Unicode support for both Python 2 and 3 using a single script. Currently, all code is written for Python 2, and then the Python 3 version is automatically generated via the |pythontex\_2to3.py| script. This script comments out code that is only for Python 2, and un-comments code that is only for Python 3.} % % Until \pytex\ is submitted to CTAN, it must be installed manually. The \pytex\ files should be installed within the \TeX\ directory structure as follows. % \begin{itemize} % \item \meta{\TeX\ tree root}|/doc/latex/pythontex/| % \begin{itemize} % \item |pythontex.pdf| % \item |README| % \end{itemize} % \item \meta{\TeX\ tree root}|/scripts/pythontex/| % \begin{itemize} % \item |pythontex2.py| and |pythontex3.py| % \item |pythontex_types2.py| and |pythontex_types3.py| % \item |pythontex_utils2.py| and |pythontex_utils3.py| % \end{itemize} % \item \meta{\TeX\ tree root}|/source/latex/pythontex/| % \begin{itemize} % \item |pythontex.dtx| % \end{itemize} % \item \meta{\TeX\ tree root}|/tex/latex/pythontex/| % \begin{itemize} % \item |pythontex.sty| % \end{itemize} % \end{itemize} % After the files are installed, the system must be made aware of their existence. Run |mktexlsr| or |texhash| to do this. In order for |pythontex*.py| to be executable, a symlink (\TeX\ Live under Linux), launching wrapper (\TeX\ Live under Windows), or batch file (general Windows) should be created in the |bin/| directory. For \TeX\ Live under Windows, simply copy |bin/win32/runscript.exe| to |bin/win32/pythontex*.exe| to create the wrapper (replace the |*| with the appropriate version).\footnote{See the output of |runscript -h| under Windows for additional details.} % % A Python installation script is provided for use with \TeX\ Live. It may need to be slightly modified based on your system. It performs all steps described above, except for creating a symlink under Linux. % % \subsection{Compiling documents using \pytex} % % To compile a document that uses \pytex, you should run \LaTeX, then run |pythontex*.py| (preferably via a symlink, wrapper, or batch file, as described above), and finally run \LaTeX\ again. |pythontex*.py| requires a single command-line argument, which must be passed to it directly or via symlink/wrapper/batch file: the name of the .tex file. The filename can be passed with or without the .tex extension, but no extension is preferred.\footnote{|pythontex*.py| will be happy to work with a file that does not have the .tex extension, so long as the file cooperates with |pythontex.sty|. In this case, the file extension should \textbf{not} be passed to |pythontex*.py|, because it won't be expecting it and won't be able to determine that it is indeed an extension. |pythontex*.py| just needs to know |\textbackslash jobname|.} The file name should be wrapped in double quotes |"| to allow for space characters.\footnote{Using spaces in the names of .tex files is apparently frowned upon. But if you configure things to handle spaces whenever it doesn't take much extra work, then that's one less thing that can go wrong.} For example, under Windows with \TeX\ Live and Python 2.7 we would create the wrapper |pythontex2.exe|. Then we could run \pytex\ on a file \meta{file~name}.tex using the command |pythontex2.exe "|\meta{file~name}|"|. In practice, you will probably want to configure your \TeX\ editor with a shortcut key for running \pytex. % % A second argument specifying the file encoding may also be passed to \pytex: |pythontex*.py| \meta{file} |--encoding| \meta{encoding}. Any encoding supported by Python's \href{http://docs.python.org/library/codecs.html}{codecs} module may be used. If an encoding is not specified, \pytex\ uses UTF-8. Note that the encoding \textbf{must} be used consistently; the .tex source, the \pytex\ output, and any external code files that \pytex\ highlights should all use the same encoding. If support for characters beyond ASCII is required, then the \LaTeX\ packages |fontenc| and |inputenc| should be used. % % \pytex\ currently does not provide means to choose between multiple Python installations; it will use the default Python installation. Support for multiple installations is unlikely to be added, since a cross-platform solution would be required. If you need to work with multiple installations, you may wish to modify |pythontex_types*.py| to create additional command and environment families that invoke different versions of Python, based on your system. % % \pytex\ attempts to check for a wide range of errors and return meaningful error messages. But due to the interaction of \LaTeX\ and Python code, some strange errors are possible. If you cannot make sense of errors when using \pytex, the simplest thing to try is deleting all files created by \pytex, then recompiling. By default, these files are stored in a directory called |pythontex-files-|\meta{jobname}, in the same directory as your .tex document. See Section \ref{sec:troubleshooting} for more details regarding Troubleshooting. % % % \section{Usage} % % \subsection{Package options} % Package options may be set in the standard manner when the package is loaded: % \begin{quote} % |\usepackage|\oarg{options}|{pythontex}| % \end{quote} % % All options are described as follows. The option is listed, followed by its possible values. When a value is not required, \meta{none} is listed as a possible value. In this case, what \meta{none} does is described. Each option lists its default setting, if the option is not invoked when the package is loaded. % % \DescribeMacro{autoprint=\meta{none}/true/false default:true \meta{none}=true} % % Whenever a |print| command/statement is used, the printed content will automatically be included in the document, unless the code doing the printing is being typeset. In that case, the printed content must be included using the |\printpythontex| or |\stdoutpythontex| commands, or one of their variants. % % Printed content is pulled in directly from the external file in which it is saved, and is interpreted by \LaTeX\ as \LaTeX\ code. If you wish to avoid this, you should print appropriate \LaTeX\ commands with your content to ensure that it is typeset as you desire. Alternatively, you may use |\printpythontex| or |\stdoutpythontex| to bring in printed content in verbatim form, using those commands' optional |verb| and |inlineverb| (|v|) options. % % \DescribeMacro{stderr=\meta{none}/true/false default:false \meta{none}=true} % % This option determines whether the stderr produced by scripts is available for input by \pytex, via the |\stderrpythontex| macro. This will not be needed in most situations. It is intended for typeseting incorrect code next to the errors that it produces. This option is not |true| by default, because additional processing is required to synchronize stderr with the document. % % \DescribeMacro{stderrfilename=full/session/genericfile/genericscript default:full} % % This option governs the file name that appears in |stderr|. Python errors begin with a line of the form % \begin{quote} % | File "", line | % \end{quote} % By default (option |full|), || is the actual name of the script that was executed. The name will be in the form \meta{family~name}|_|\meta{session}|_|\meta{group}|.|\meta{extension}. For example, an error produced by a |py| command or environment, in the session |mysession|, using the default group (that is, the default |\restartpythontexsession| treatment), would be reported in |py_mysession_default.py|. The |session| option replaces the full file name with the name of the session, |mysession.py| in this example. The |genericfile| and |genericscript| options replace the file name with || and |