diff --git a/NEWS.rst b/NEWS.rst new file mode 100644 index 0000000..69057dd --- /dev/null +++ b/NEWS.rst @@ -0,0 +1,700 @@ +================================================== + PythonTeX News +================================================== + + +Version History +=============== + + +v0.19 (2026/02/15) +------------------ + +* Fixed backslash escapes in docstrings and regexes for syntax compatibility + with Python 3.11+. + +* ``js`` now works as a command family for JavaScript (#197). + +* Python console commands and environments are now compatible with Python + 3.14 (#229). + +* Fixed a bug that could cause non-Windows operating systems to attempt to + invoke a nonexistent batch file when the executable associated with a + language is not found (#225). + +* Replaced doctype ``tex`` with ``texminted`` for ``Weave.jl`` (#219). + +* Fixed a bug that resulted in uncaught exceptions when ``\inputpygments`` + was used with files that do not exist (#64). + +* Fixed a bug in tracking the number of Pygments errors (#206). + +* Improved error messages (#207, #209). Improved documentation for + ``--error-exit-code`` (#208). + +* Several minor improvements to documentation. + + + +v0.18 (2021/06/06) +------------------ + +* ``\inputpygments`` now checks inputted files for modification, so that + typeset code will correctly update when the source is changed (#162). + +* Julia now uses project flag "``--project=@.``" (#157, #158). + +* Fixed bug in processing Pygments options (``pygopt``) when a key is used + without a value (#181). + +* Some error handling for Windows was incompatible with other operating + systems: replaced checks for ``WindowsError`` with checks for ``OSError`` + (#177). + +* Rust support is now compatible with document and working directory paths + that contain spaces (#167). + + + +v0.17 (2019/09/22) +------------------ + +* Pygments syntax highlighting for the Python console (``pycon`` lexer) now + uses the ``python3`` option, and the default Python lexer is now + ``python3`` (#156). + +* Added support for JavaScript (#147; thanks to Nathan Carter). + +* Updated Julia support for Julia versions 0.6 (#107), and 0.7 and 1.0 (#126, + #130). + +* There are now meaningful error messages for the Julia console when Weave.jl + is not installed or raises errors (#131). + +* ``pythontexcustomcode`` and ``\pythontexcustomc`` now set + ``pytex.context`` (#65). + +* Added support for R. The ``R`` family of commands and environments + (``\R``, ``\Rc``, ``Rcode``, ...) executes code as a script. + There is currently no utilities class or equivalent. The ``Rcon`` family + (``Rconsole``) executes code to emulate an interactive R session (#121). + +* ``fancyvrb`` settings from ``\setpythontexfv`` and console + environments now work with Julia and R consoles. + +* ``pythontexcustomcode`` now works with ``juliacon``. There are now proper + ``juliaconcode`` and ``Rconcode`` environments that execute code but + typeset nothing, to parallel ``pyconcode`` (#134). + +* Added support for Perl with the ``perl`` and ``pl`` families of commands + and environments. There is currently no utilities class or equivalent. + +* Added support for Perl 6 with the ``perlsix`` and ``psix`` families of + commands and environments (#104). There is currently no utilities class or + equivalent. + +* Updated Rust support by using ``dyn`` with traits in utilities object. + +* Under Windows, capitalization of script paths in ``stderr`` is now + preserved. + +* Fixed a bug that prevented the ``sub`` environment from working with + ``depythontex`` (#155). + +* Fixed a bug in checking mtime of dependencies to see if they have been + modified while ``pythontex`` is running. The check failed for dependencies + that do not exist or were deleted before ``pythontex`` can read them + (#136). + + + +v0.16 (2017/07/20) +------------------ + +* Added preliminary console support for Julia (#98). + +* Fixed Python console compatibility with Python 3.6 by setting the ``code`` + module's new ``exitmsg`` argument to suppress the exit message (#100). + +* Improved Rust support, including tracking of created files and + dependencies (#91). + + + +v0.15 (2016/07/21) +------------------ + +New features +~~~~~~~~~~~~ + +* The ``fvextra`` package is now required. This provides line breaking with + fine-grained control over break locations, the ability to highlight + specific lines or ranges of lines, improved handling of tabs, and several + additional features. + +* Added ``sub`` commands and environments (``\pys``, ``pysub``, ...). These + commands and environments perform string interpolation on text. Fields + delimited by ``!{...}`` are replaced by the result of evaluating and then + printing their content. This works for all families of commands and + environments, not just Python. See the documentation for details about + field delimiters and escaping. + +* Added ``rust`` and ``rs`` families of commands and environments. These + provide essentially complete support for Rust, except that + ``rstex.formatter()``, ``rstex.before()``, and ``rstex.after()`` will + likely need additional refinement (#90). + +* Added the ``sage`` family of commands and environments, which provide + support for Sage (#63). + +* Added ``bash`` family of commands and environments. This provides basic + support for bash (no utilities class or equivalent). Bash works with + Windows if it is installed. + +* Improved ``console`` compatibility under Linux with Python 3 (#70). + +* Counters for default sessions are now created automatically. This prevents + counter errors under some circumstances when working with ``\includeonly``. + +* Commands like ``\py`` can now output verbatim content under LuaTeX. + +Bugfixes +~~~~~~~~ + +* Fixed a bug that could cause an endless loop when a ``code`` command or + environment printed a ``code`` command or environment of the same family + with ``autoprint=true``. + + + +v0.14 (2014/07/17) +------------------ + +New features +~~~~~~~~~~~~ + +* All commands for working with code inline are now robust, via + ``etoolbox``'s ``\newrobustcmd``. Among other things, this allows + commands like ``\py`` to work in standard captions that have not been + redefined to avoid protection issues. +* Upgraded ``syncpdb`` to v0.2, which provides better list formatting. + +Backward-incompatible changes +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +* The default working directory is now the main document directory instead + of the output directory. Using the output directory was a common source + of confusion for new users and was incompatible with plans for future + development. Old documents in which the working directory was not + specified will continue to use the output directory, but PythonTeX will + print an upgrade message; new documents will use the new setting. The + output directory may be selected as the working directory manually, or + with the shorthand + "``\setpythontexworkingdir{}``". + +* Standardized version numbering by removing the "v" prefix from the stored + version numbers in Python variables and LaTeX macros. Standardized the + PythonTeX scripts by renaming ``version`` to ``__version__``. + + + +v0.13 (2014/07/14) +------------------ + +New features +~~~~~~~~~~~~ + +* Added ``--interactive`` command-line option. This runs a single + session in interactive mode, allowing user input. Among other things, + this is useful when working with debuggers. + +* Added ``--debug`` command-line option. This runs a single session + with the default debugger in interactive mode. Currently, only + standard (non-console) Python sessions are supported. The default + Python debugger is the new ``syncpdb``, which wraps ``pdb`` and + synchronizes code line numbers with document line numbers. All + ``pdb`` commands that take a line number or filename:lineno as an + argument will refer to document files and line numbers when the + argument has a percent symbol (``%``) as a prefix. For example, + ``list %50`` lists code that came from around line 50 in the + document. The ``--debug`` option will support other languages and + provide for customization in the future. + +* Added command-line option ``--jobs``, which allows the maximum number + of concurrent processes to be specified (#35). + +* Added support for GNU Octave, via the ``octave`` family of commands + and environments (#36). Parsing of Octave stderr is not ideal, though + synchronization works in most cases; this will be addressed by a + future rewrite of the stderr parser. + +* Installer now automatically works with MiKTeX, not just TeX Live. + +* The PythonTeX utilities class has a new ``open()`` method that opens + files and automatically tracks dependencies/created files. + +* When ``pythontex2.py`` and ``pythontex3.py`` are run directly, the + Python interpreter is automatically set to a reasonable default + (``py -2`` or ``py -3`` under Windows, using the Python 3.3+ wrapper; + ``python2`` or ``python3`` under other systems). + +* The installer now creates symlinks for the numbered scripts + ``pythontex*.py`` and ``depythontex*.py``. + +* Added Python version checking to all numbered scripts. + +* Under Python, the type of data passed via ``\setpythontexcontext`` may + now be set using YAML-style tags (``!!str``, ``!!int``, ``!!float``). For + example, ``{myint=!!int 123}``. + +* The ``fancyvrb`` options ``firstline`` and ``lastline`` now work with + the ``pygments`` environment and ``\inputpygments`` command. This required + some additional patching of ``fancyvrb``. + +* The ``pytx@Verbatim`` and ``pytx@SaveVerbatim`` environments are now + used for typesetting verbatim code. These are copies of the + ``fancyvrb`` environments. This prevents conflicts when literal + ``Verbatim`` and ``SaveVerbatim`` environments need to be typeset. + +* Improved ``latexmk`` compatibility (#40). Added discussion of + ``latexmk`` usage to documentation. + +* Tildes ``~`` may now be used in ``outputdir`` and ``workingdir`` to + refer to the user�s home directory, even under Windows. + +Bugfixes +~~~~~~~~ + +* Fixed a bug that prevented created files from being cleaned up when + the working directory was not the document root directory and the + full path to the files was not provided. + +* Fixed a bug that prevented the ``fvextfile`` option from working when + external files were highlighted. + + + +v0.13-beta (2014/02/06) +----------------------- + +New features +~~~~~~~~~~~~ + +* Switching to GitHub's Releases for downloads. + +* TeX information such as page dimensions may now be easily passed to the + programming-language side, using the new ``\setpythontexcontext`` command. + Contextual information is stored in the ``context`` attribute of the + utilities class, which is a dictionary (and also has attributes in Python). + +* The utilities class now has ``pt_to_in()``, ``pt_to_cm()``, and + ``pt_to_mm()`` methods for converting units of TeX points into inches, + centimeters, and millimeters. These work with integers and floats, as + well as strings that consist of numbers and optionally end in "pt". There + is also a ``pt_to_bp()`` for converting TeX points (1/72.27 inch) into big + (DTP or PostScript) points (1/72 inch). + +* Expanded Quickstart. Quickstart is now compatible with all LaTeX engines. + Quickstart now avoids ``microtype`` issues on some systems (#32). + +* Added information on citing PythonTeX (#28). + +* Utilities class has a new attribute ``id``, which is a string that joins + the command family name, session name, and session restart parameters with + underscores. This may be used in creating files that need a name that + contains a unique, session-based identifier (for example, names for + figures that are saved automatically). + +Backward-incompatible changes +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +* All utilities-class attributes with names of the form ``input_*`` have + been renamed with the "``input_``" removed. Among other things, this + makes it easier to access the ``context`` attribute (``pytex.context`` + vs. ``pytex.input_context``). + +* ``depythontex`` now has ``-o`` and ``--output`` command-line options for + specifying the name of the output file. If an output file is not + specified, then output is written to ``stdout``. This allows + ``depythontex`` output to be piped to another program. + +* All scripts ``*2.py`` now have shebangs with ``env python2``, and all + scripts ``*3.py`` now have shebangs with ``env python3``. This allows the + wrapper scripts (``env python`` shebang) to be used with the default + Python installation, and the numbered scripts to be used with specific + versions. Remember that except for console content, the ``--interpreter`` + option is what determines the Python version that actually executes code. + The version of Python used to launch ``pythontex.py`` merely determines + the version that manages code execution. (``--interpreter`` support for + console content is coming.) + +* Changed the template style used in the ``CodeEngine`` class. Replacement + fields are now surrounded by single curly braces (as in Python's format + string syntax), rather than double curly braces. Literal curly braces are + obtained by doubling braces. This allows the use of literal adjacent + double braces in templates, which was not possible previously. + +* The Julia template now uses the new ``in()`` function, replacing + ``contains()``. This requires Julia v0.2.0+. + +Bugfixes +~~~~~~~~ + +* Modified test for LuaTeX, so that ``\directlua`` is not ``\let`` to + ``\relax`` if it does not exist. This was causing incompatibility with + ``babel`` under pdfTeX and XeTeX (#33). + +* Added missing shebangs to ``depythontex*.py``. Handling of ``utilspath`` + is now more forgiving, so that ``pythontex_utils.py`` can be installed in + alternate locations (#23). + +* ``depythontex`` no longer leaves a blank line where + ``\usepackage{pythontex}`` was removed. + +* Console environments typeset with ``fancyvrb`` no longer end with an + unnecessary empty line. + +* Fixed bug in installer when ``kpsewhich`` was not found (#21). + + + +v0.12 (2013/08/26) +------------------ + +* Added support for the Julia language, with the ``julia`` and ``jl`` + families of commands and environments. (Note that Pygments only added + Julia support in version 1.6.) + +* Warnings and errors are now synchronized with the line numbers of files + brought in via ``\input``, ``\include``, etc. This is accomplished using + the ``currfile`` package. + +* Added package option ``gobble``. When ``gobble=auto``, all code is + dedented before being executed and/or typeset. The current + implementation is functional but basic; it will be improved and extended + in the future. + +* The document root directory is now always added to ``sys.path`` (or its + equivalent), even when it is not the working directory. (The working + directory has been added to ``sys.path`` since v0.12beta.) The document + directory is added after the working directory, so that the working + directory has precedence. + +* Fixed a bug in ``console`` commands and environments; ``sys.path`` now + contains the working and document directories, and the working directory + is now the output directory by default. This parallels the behavior of + non-``console`` commands and environments. + +* Added command-line option ``--interpreter`` that allows an interpreter to + be invoked via a specific command. This allows, for example, a specific + version of Python to be invoked. + +* Improved synchronization of stderr in cases when an error is triggered + far after its origin (for example, an error caused by a multiline string + that is lacking a closing quote/delimiter, and thus may span several + chunks of user code). + +* Modified usage of the ``shlex`` module to work around its lack of Unicode + support in Python versions prior to 2.7.3. + +* Fixed a bug from v0.12beta that prevented ``\inputpygments`` from working + when ``pygments=true``. + +* Fixed a bug with counters that caused errors when content spanning + multiple columns was created within a ``tabular`` environment. + +* Added checking for compatible Python versions in ``pythontex.py``. + +* Improved execution of ``*.bat`` and ``*.cmd`` files under Windows. The + solution from v0.12beta allowed ``*.bat`` and ``*.cmd`` to be found and + executed when the extension was not given, but did not give correct + return codes. + + +v0.12beta (2013/06/24) +---------------------- + +* Merged ``pythontex_types*.py`` into a single replacement + ``pythontex_engines.py`` compatible with both Python 2 and 3. It is + now much simpler to add support for additional languages. + +* Added support for the Ruby language as a demonstration of new + capabilities. The ``ruby`` and ``rb`` families of commands and + environments may be enabled via the new ``usefamily`` package option. + Support for additional languages is coming soon. See the new section + in the documentation on support for other languages for more + information. + +* Reimplemented treatment of Pygments content for better efficiency. + Now a Pygments process only runs if there is content to highlight. + Eliminated redundant highlighting of unmodified code. + +* Improved treatment of dependencies. If a dependency is modified + (``os.path.getmtime()``) after the current PythonTeX run starts, then + code that depends on it will be re-executed the next time PythonTeX + runs. A message is also issued to indicate that this is the case. + +* The utilities class now has ``before()`` and ``after()`` methods that + are called immediately before and after user code. These may be + redefined to customize output. For example, LaTeX commands could be + printed before and after user code; stdout could be redirected to + ``StringIO`` for further processing; or matplotlib figures could be + automatically detected, saved, and included in the document. + +* Added explanation of how to track dependencies and created files + automatically, and how to include matplotlib figures automatically, + to the documentation for the PythonTeX utilities class. + +* Created a new system for parsing and synchronizing stderr. + + - Exceptions that do not reference a line number in user code (such + as those from ``warnings.warn()`` in a module) are now traced back + to a single command or environment. Previously no synchronization + was attempted. This is accomplished by writing delimiters to + stderr before executing the code from each command/environment. + + - Exceptions that do reference a line in user code are more + efficiently synchronized with a document line number. This is + accomplished by careful record keeping as each script is + assembled. Line number synchronization no longer involves parsing + the script that was executed. + + - Improved and generalized parsing of stderr, in preparation for + supporting additional languages. Exceptions that cannot be + identified as errors or warnings are treated based on + ``Popen.returncode``. + +* Created a new system for ``console`` content. + + - There are now separate families of ``console`` commands and + environments. No Pygments or ``fancyvrb`` settings are shared with + the non-``console`` families, as was previously the case. There + is a new family of commands and environments based on ``pycon``, + including the ``\pycon`` command (inline reference to console variable), + ``pyconsole`` environment (same as the old one), ``\pyconc`` and + ``pyconcode`` (execute only), and ``\pyconv`` and ``pyconverbatim`` + (typeset only). There are equivalent families based on + ``pylabcon`` and ``sympycon``. + + - Each console session now runs in its own process and is cached + individually. Console output is now cached so that changing + Pygments settings no longer requires re-execution. + + - Unicode is now supported under Python 2. + + - The new package option ``pyconfuture`` allows automatic imports + from ``__future__`` for ``console`` families under Python 2, + paralleling the ``pyfuture`` option. + + - Any errors or warnings caused by code that is not typeset + (``code`` command and environment, startup code) are reported in + the run summary. This ensures that such code does not create + mischief. + + - ``customcode`` is now supported for ``console`` content. + +* Better support for ``latexmk`` and similar build tools. PythonTeX + creates a file of macros (``*.pytxmcr``) that is always included in a + document, and thus can be automatically detected and tracked by + ``latexmk``. This file now contains the time at which PythonTeX last + created files. When new files are created, the macro file will have a + new hash, triggering another document compile. + +* Improved the way in which the PythonTeX ``outputdir`` is added to the + graphics path. This had been done with ``\graphicspath``, but that + overwrites any graphics path previously specified by the user. Now the + ``outputdir`` is appended to any pre-existing path. + +* Added the ``depythontex`` option ``--graphicspath``. This adds the + ``outputdir`` to the graphics path of the ``depythontex`` document. + +* The installer now provides more options for installation locations. + It will now create missing directories if desired. + +* The working directory (``workingdir``) is now appended to + ``sys.path``, so that code there may be imported. + +* Under Windows, ``subprocess.Popen()`` is now invoked with + ``shell=True`` if ``shell=False`` results in a WindowsError. This + allows commands involving ``*.bat`` and ``*.cmd`` files to be + executed when the extension is not specified; otherwise, only ``*.exe`` + can be found and run. + +* The path to utils is now found in ``pythontex.py`` via + ``sys.path[0]`` rather than ``kpsewhich``. This allows the PythonTeX + scripts to be executed in an arbitrary location; they no longer must + be installed in a texmf tree where ``kpsewhich`` can find them. + +* Added ``rerun`` value ``never``. + +* At the end of each run, data and macros are only saved if modified, + improving efficiency. + +* The number of temporary files required by each process was reduced by + one. All macros for commands like ``\py`` are now returned within + stdout, rather than in their own file. + +* Fixed a bug with ``\stderrpythontex``; it was defaulting to ``verb`` + rather than ``verbatim`` mode. + + +v0.11 (2013/04/21) +------------------ + +* As the first non-beta release, this version adds several features and introduces several changes. You should read these release notes carefully, since some changes are not backwards-compatible. Changes are based on a thorough review of all current and planned features. PythonTeX's capabilities have already grown beyond what was originally intended, and a long list of features still remains to be implemented. As a result, some changes are needed to ensure consistent syntax and naming in the future. Insofar as possible, all command names and syntax will be frozen after this release. +* Added the ``pythontex.py`` and ``depythontex.py`` wrapper scripts. When run, these detect the current version of Python and import the correct PythonTeX code. It is still possible to run ``pythontex*.py`` and ``depythontex*.py`` directly, but the new wrapper scripts should be used instead for simplicity. There is now only a single ``pythontex_utils.py``, which works with both Python 2 and Python 3. +* Added the ``beta`` package option. This makes the current version behave like v0.11beta, for compatibility. This option is temporary and will probably only be retained for a few releases. +* Backward-incompatible changes (require the ``beta`` option to restore old behavior) + + - The ``pyverb`` environment has been renamed ``pyverbatim``. The old name was intended to be concise, but promoted confusion with LaTeX's ``\verb`` macro. + - For ``\printpythontex``, ``\stdoutpythontex``, and ``\stderrpythontex``, the modes ``inlineverb`` and ``v`` have been replaced by ``verb``, and the old mode ``verb`` has been replaced by ``verbatim``. This brings naming conventions in line with standard LaTeX ``\verb`` and ``verbatim``, avoiding a source of potential confusion. + - The ``\setpythontexpyglexer``, ``\setpythontexpygopt``, and ``\setpygmentspygopt`` commands now take an optional argument and a mandatory argument, rather than two mandatory arguments. This creates better uniformity among current and planned settings macros. + - The ``\setpythontexformatter`` and ``\setpygmentsformatter`` commands have been replaced by the ``\setpythontexprettyprinter`` and ``\setpygmentsprettyprinter`` commands. This anticipates possible upcoming features. It also avoids potential confusion with Pygments's formatters and the utilities class's ``formatter()`` method. + +* Deprecated (still work, but raise warnings; after a few releases, they will raise errors instead, and after that eventually be removed) + + - The ``rerun`` setting ``all`` was renamed ``always``, in preparation for upcoming features. + - The ``stderr`` option is replaced by ``makestderr``. The ``print``/``stdout`` option is replaced by ``debug``. These are intended to prevent confusion with future features. + - The ``fixlr`` option is deprecated. It was originally introduced to deal with some of SymPy's LaTeX formatting, which has since changed. + - The utilities class method ``init_sympy_latex()`` is deprecated. The ``sympy_latex()`` and ``set_sympy_latex()`` methods now automatically initialize themselves on first use. + +* Added ``autostdout`` package option and ``\setpythontexautostdout``, to complement ``autoprint``. Added ``prettyprinter`` and ``prettyprintinline`` package options to complement new settings commands. +* Added quickstart guide. +* Installer now installs gallery and quickstart files, if present. + + +v0.11beta (2013/02/17) +---------------------- + +* Commands like ``\py`` can now bring in any valid LaTeX code, including verbatim content, under the pdfTeX and XeTeX engines. Verbatim content was not allowed previously. LuaTeX cannot bring in verbatim, due to a known bug. +* Added package option ``depythontex`` and scripts ``depythontex*.py``. These allow a PythonTeX document to be converted into a pure LaTeX document, with no Python dependency. The package option creates an auxiliary file with extension ``.depytx``. The ``depythontex*.py`` scripts take this auxiliary file and the original LaTeX document, and combine the two to produce a new document that does not rely on the PythonTeX package. All PythonTeX commands and environments are replaced by their output. All Python-generated content is substituted directly into the document. By default, all typeset code is wrapped in ``\verb`` and ``verbatim``, but ``depythontex*.py`` has a ``--listing`` option that allows ``fancyvrb``, ``listings``, ``minted``, or ``pythontex`` to be used instead. +* The current PythonTeX version is now saved in the ``.pytxcode``. If this does not match the version of the PythonTeX scripts, a warning is issued. This makes it easier to determine errors due to version mismatches. +* Fixed an incompatibility with the latest release of ``xstring`` (version 1.7, 2013/01/13). +* Fixed a bug in the ``console`` environment that could cause problems when switching from Pygments highlighting to ``fancyvrb`` when using the ``fvextfile`` option. Fixed a bug introduced in the v0.10beta series that prevented the ``console`` environment from working with ``fancyvrb``. +* Fixed a bug with PythonTeX verbatim commands and environments that use Pygments. The verbatim commands and environments were incorrectly treated as if they had the attributes of executed code in the v0.10beta series. +* Fixed a bug from the v0.10beta series that sometimes prevented imports from ``__future__`` from working when there were multiple sessions. +* Fixed a bug related to hashing dependencies' mtime under Python 3. + + +v0.10beta2 (2013/01/23) +----------------------- + +* Improved ``pythontex*.py``'s handling of the name of the file being processed. A warning is no longer raised if the name is given with an extension; extensions are now processed (stripped) automatically. The filename may now contain a path to the file, so you need not run ``pythontex*.py`` from within the document's directory. +* Added command-line option ``--verbose`` for more verbose output. Currently, this prints a list of all processes that are launched. +* Fixed a bug that could crash ``pythontex*.py`` when the package option ``pygments=false``. +* Added documentation about ``autoprint`` behavior in the preamble. Summary: ``code`` commands and environments are allowed in the preamble as of v0.10beta. ``autoprint`` only applies to the body of the document, because nothing can be typeset in the preamble. Content printed in the preamble can be brought in by explicitly using ``\printpythontex``, but this should be used with great care. +* Revised ``\stdoutpythontex`` and ``\printpythontex`` so that they work in the preamble. Again, this should be used with great care if at all. +* Revised treatment of any content that custom code attempts to print. Custom code is not allowed to print to the document (see documentation). If custom code attempts to print, a warning is raised, and the printed content is included in the ``pythontex*.py`` run summary. +* One-line entries in stderr, such as those produced by Python's ``warnings.warn()``, were not previously parsed because they are of the form ``::`` rather than ``line ``. These are now parsed and synchronized with the document. They are also correctly parsed for inclusion in the document via ``\stderrpythontex``. +* If the package option ``stderrfilename`` is changed, all sessions that produced errors or warnings are now re-executed automatically, so that their stderr content is properly updated with the new filename. + + +v0.10beta (2013/01/09) +---------------------- + +* Backward-incompatible: Redid treatment of command-line options for + ``pythontex*.py``, using Python's ``argparse`` module. Run + ``pythontex*.py`` with option ``-h`` to see new command line options. +* Deprecated: ``\setpythontexcustomcode`` is deprecated in favor of the + ``\pythontexcustomc`` command and ``pythontexcustomcode`` + environment. These allow entry of pure code, unlike + ``\setpythontexcustomcode``. These also allow custom code to be + added to the beginning or end of a session, via an optional argument. + Improved treatment of errors and warnings associated with custom + code. +* The summary of errors and warnings now correctly differentiates + errors and warnings produced by user code, rather than treating all + of them as errors. By default, ``pythontex*.py`` now returns an + exit code of 1 if there were errors. +* The PythonTeX utilities class now allows external file dependencies + to be specified via ``pytex.add_dependencies()``, so that sessions + are automatically re-executed when external dependencies are + modified (modification is determined via either hash or mtime; this + is governed by the new ``hashdependencies`` option). +* The PythonTeX utilities class now allows created files to be + specified via ``pytex.add_created()``, so that created files may be + automatically cleaned up (deleted) when the code that created them + is modified (for example, name change for a saved plot). +* Added the following package options. + + - ``stdout`` (or ``print``): Allows input of stdout to be disabled. + Useful for debugging. + - ``runall``: Executes everything. Useful when code depends on + external data. + - ``rerun``: Determines when code is re-executed. Code may be set + to always run (same as ``runall`` option), or only run when it is + modified or when it produces errors or warnings. By default, + code is always re-executed if there are errors or modifications, + but not re-executed if there are warnings. + - ``hashdependencies``: Determines whether external dependencies + (data, external code files highlighted with Pygments, etc.) are + checked for modification via hashing or modification time. + Modification time is default for performance reasons. + +* Added the following new command line options. The options that are + equivalent to package options are overridden by the package options + when present. + + - ``--error-exit-code``: Determines whether an exit code of 1 is + returned if there were errors. On by default, but can be turned + off since it is undesirable when working with some editors. + - ``--runall``: Equivalent to new package option. + - ``--rerun``: Equivalent to new package option. + - ``--hashdependencies``: Equivalent to new package option. + +* Modified the ``fixlr`` option, so that it only patches commands if + they have not already been patched (avoids package conflicts). +* Added ``\setpythontexautoprint`` command for toggling autoprint + on/off within the body of the document. +* Installer now attempts to create symlinks under OS X and Linux with + TeX Live, and under OS X with MacPorts Tex Live. +* Performed compatibility testing under lualatex and xelatex + (previously, had only tested with pdflatex). Added documentation + for using these TeX engines; at most, slightly different preambles + are needed. Modified the PythonTeX gallery to support all three + engines. +* Code commands and environments may now be used in the preamble. + This, combined with the new treatment of custom code, allows + PythonTeX to be used in creating LaTeX packages. +* Added documentation for using PythonTeX in LaTeX programming. +* Fixed a bug that sometimes caused incorrect line numbers with + ``stderr`` content. Improved processing of stderr. +* Fixed a bug in automatic detection of pre-existing listings + environment. +* Improved the detection of imports from ``__future__``. Detection + should now be stricter, faster, and more accurate. + + +v0.9beta3 (2012/07/17) +---------------------- + +* Added Unicode support, which required the Python code to be split into + one set for Python 2 and another set for Python 3. This will require + any old installation to be completely removed, and a new installation + created from scratch. +* Refactoring of Python code. Documents should automatically re-execute + all code after updating to the new version. Otherwise, you should delete + the PythonTeX directory and run PythonTeX. +* Improved installation script. +* Added package options: ``pyfuture``, ``stderr``, ``upquote``, + ``pyglexer``, ``pyginline``. Renamed the ``pygextfile`` option to + ``fvextfile``. +* Added custom code and workingdir commands. +* Added the ``console`` environment and associated options. +* Rewrote ``pythontex_utils*.py``, creating a new, context-aware interface to + SymPy's LatexPrinter class. +* Content brought in via macros no longer uses labels. Rather, long defs + are used, which allows line breaks. +* Pygments highlighting is now default for PythonTeX commands and environments. + + +v0.9beta2 (2012/05/09) +---------------------- + +* Changed Python output extension to ``.stdout``. + + +v0.9beta (2012/04/27) +--------------------- + +* Initial public beta release. diff --git a/README.rst b/README.rst index 3a7cbc6..2854032 100644 --- a/README.rst +++ b/README.rst @@ -2,369 +2,121 @@ PythonTeX =============================================== ------------------------------------------------ - Fast Access to Python from within LaTeX ------------------------------------------------ +------------------------------------------------------------------------------------------ + Execute Python and other code in LaTeX documents, or typeset it with syntax highlighting +------------------------------------------------------------------------------------------ -:Author: Geoffrey Poore +PythonTeX executes code in LaTeX documents and allows the output to be +included in the original document. It supports Python as well as Bash, +JavaScript, Julia, Octave, Perl, R, Raku (Perl 6), Ruby, Rust, and SageMath. +PythonTeX also provides syntax highlighting for typeset code in LaTeX +documents via the `Pygments `_ syntax highlighter. -:Version: 0.12 +See ``pythontex_quickstart.pdf`` to get started, and ``pythontex_gallery.pdf`` +for examples of what is possible with PythonTeX. PythonTeX is included in TeX +Live and MiKTeX and may be installed via the package manager. See +``pythontex.pdf`` for detailed installation instructions if you want to +install the current development version, or just use the installation script +for TeX Live and MiKTeX. -:License: LPPL_ (LaTeX code) and `BSD 3-Clause`_ (Python code) +The ``depythontex`` utility creates a copy of a PythonTeX document in which +all code has been replaced by its output. This plain LaTeX document is +more suitable for journal submission, sharing, or conversion to other document +formats. See ``pythontex_gallery.html`` and the accompanying conversion +script for an example of a PythonTeX document that was converted to HTML via +``depythontex`` and `Pandoc `_. -.. _LPPL: http://www.latex-project.org/lppl.txt -.. _`BSD 3-Clause`: http://www.opensource.org/licenses/BSD-3-Clause +Example +======= + +* LaTeX document ``doc.tex``: + + .. code-block:: latex + + \documentclass{article} + + \usepackage{pythontex} + + \newcommand{\pymultiply}[2]{\py{#1*#2}} + + \begin{document} + + \begin{pycode} + print("Python says ``Hello!''") + \end{pycode} + + $8 \times 256 = \pymultiply{8}{256}$ + + \end{document} + +* Compiling under Windows: + + :: + + pdflatex -interaction=nonstopmode doc.tex + pythontex doc.tex + pdflatex -interaction=nonstopmode doc.tex + + +* Compiling under other operating systems: + + :: + + pdflatex -interaction=nonstopmode doc.tex + pythontex.py doc.tex + pdflatex -interaction=nonstopmode doc.tex -PythonTeX provides fast, user-friendly access to Python from within LaTeX. It allows Python code entered within a LaTeX document to be executed, and the results to be included within the original document. It also provides syntax highlighting for code within LaTeX documents via the Pygments syntax highlighter. - -PythonTeX also provides support for Ruby and Julia. Support for additional languages is coming soon. - -See ``pythontex.pdf`` for detailed installation instructions, or use the installation script for TeX Live. See ``pythontex_quickstart.pdf`` to get started, and ``pythontex_gallery.pdf`` for examples of what is possible with PythonTeX. - -The ``depythontex`` utility creates a copy of a PythonTeX document in which all Python code has been replaced by its output. This plain LaTeX document is more suitable for journal submission, sharing, or conversion to other document formats. See ``pythontex_gallery.html`` and the accompanying conversion script for an example of a PythonTeX document that was converted to HTML via ``depythontex`` and `Pandoc `_. - - -Current status --------------- - -Upcoming objectives: - -* Add better support for macro programming with PythonTeX. - - -Version History ---------------- - -v0.12 (2013/08/26) - -* Added support for the Julia language, with the ``julia`` and ``jl`` - families of commands and environments. (Note that Pygments only added - Julia support in version 1.6.) - -* Warnings and errors are now synchronized with the line numbers of files - brought in via ``\input``, ``\include``, etc. This is accomplished using - the ``currfile`` package. - -* Added package option ``gobble``. When ``gobble=auto``, all code is - dedented before being executed and/or typeset. The current - implementation is functional but basic; it will be improved and extended - in the future. - -* The document root directory is now always added to ``sys.path`` (or its - equivalent), even when it is not the working directory. (The working - directory has been added to ``sys.path`` since v0.12beta.) The document - directory is added after the working directory, so that the working - directory has precedence. - -* Fixed a bug in ``console`` commands and environments; ``sys.path`` now - contains the working and document directories, and the working directory - is now the output directory by default. This parallels the behavior of - non-``console`` commands and environments. - -* Added command-line option ``--interpreter`` that allows an interpreter to - be invoked via a specific command. This allows, for example, a specific - version of Python to be invoked. - -* Improved synchronization of stderr in cases when an error is triggered - far after its origin (for example, an error caused by a multiline string - that is lacking a closing quote/delimiter, and thus may span several - chunks of user code). - -* Modified usage of the ``shlex`` module to work around its lack of Unicode - support in Python versions prior to 2.7.3. - -* Fixed a bug from v0.12beta that prevented ``\inputpygments`` from working - when ``pygments=true``. - -* Fixed a bug with counters that caused errors when content spanning - multiple columns was created within a ``tabular`` environment. - -* Added checking for compatible Python versions in ``pythontex.py``. - -* Improved execution of ``*.bat`` and ``*.cmd`` files under Windows. The - solution from v0.12beta allowed ``*.bat`` and ``*.cmd`` to be found and - executed when the extension was not given, but did not give correct - return codes. - - -v0.12beta (2013/06/24) - -* Merged ``pythontex_types*.py`` into a single replacement - ``pythontex_engines.py`` compatible with both Python 2 and 3. It is - now much simpler to add support for additional languages. - -* Added support for the Ruby language as a demonstration of new - capabilities. The ``ruby`` and ``rb`` families of commands and - environments may be enabled via the new ``usefamily`` package option. - Support for additional languages is coming soon. See the new section - in the documentation on support for other languages for more - information. - -* Reimplemented treatment of Pygments content for better efficiency. - Now a Pygments process only runs if there is content to highlight. - Eliminated redundant highlighting of unmodified code. - -* Improved treatment of dependencies. If a dependency is modified - (``os.path.getmtime()``) after the current PythonTeX run starts, then - code that depends on it will be re-executed the next time PythonTeX - runs. A message is also issued to indicate that this is the case. - -* The utilities class now has ``before()`` and ``after()`` methods that - are called immediately before and after user code. These may be - redefined to customize output. For example, LaTeX commands could be - printed before and after user code; stdout could be redirected to - ``StringIO`` for further processing; or matplotlib figures could be - automatically detected, saved, and included in the document. - -* Added explanation of how to track dependencies and created files - automatically, and how to include matplotlib figures automatically, - to the documentation for the PythonTeX utilities class. - -* Created a new system for parsing and synchronizing stderr. - - - Exceptions that do not reference a line number in user code (such - as those from ``warnings.warn()`` in a module) are now traced back - to a single command or environment. Previously no synchronization - was attempted. This is accomplished by writing delimiters to - stderr before executing the code from each command/environment. - - - Exceptions that do reference a line in user code are more - efficiently synchronized with a document line number. This is - accomplished by careful record keeping as each script is - assembled. Line number synchronization no longer involves parsing - the script that was executed. - - - Improved and generalized parsing of stderr, in preparation for - supporting additional languages. Exceptions that cannot be - identified as errors or warnings are treated based on - ``Popen.returncode``. - -* Created a new system for ``console`` content. - - - There are now separate families of ``console`` commands and - environments. No Pygments or ``fancyvrb`` settings are shared with - the non-``console`` families, as was previously the case. There - is a new family of commands and environments based on ``pycon``, - including the ``\pycon`` command (inline reference to console variable), - ``pyconsole`` environment (same as the old one), ``\pyconc`` and - ``pyconcode`` (execute only), and ``\pyconv`` and ``pyconverbatim`` - (typeset only). There are equivalent families based on - ``pylabcon`` and ``sympycon``. - - - Each console session now runs in its own process and is cached - individually. Console output is now cached so that changing - Pygments settings no longer requires re-execution. - - - Unicode is now supported under Python 2. - - - The new package option ``pyconfuture`` allows automatic imports - from ``__future__`` for ``console`` families under Python 2, - paralleling the ``pyfuture`` option. - - - Any errors or warnings caused by code that is not typeset - (``code`` command and environment, startup code) are reported in - the run summary. This ensures that such code does not create - mischief. - - - ``customcode`` is now supported for ``console`` content. - -* Better support for ``latexmk`` and similar build tools. PythonTeX - creates a file of macros (``*.pytxmcr``) that is always included in a - document, and thus can be automatically detected and tracked by - ``latexmk``. This file now contains the time at which PythonTeX last - created files. When new files are created, the macro file will have a - new hash, triggering another document compile. - -* Improved the way in which the PythonTeX ``outputdir`` is added to the - graphics path. This had been done with ``\graphicspath``, but that - overwrites any graphics path previously specified by the user. Now the - ``outputdir`` is appended to any pre-existing path. - -* Added the ``depythontex`` option ``--graphicspath``. This adds the - ``outputdir`` to the graphics path of the ``depythontex`` document. - -* The installer now provides more options for installation locations. - It will now create missing directories if desired. - -* The working directory (``workingdir``) is now appended to - ``sys.path``, so that code there may be imported. - -* Under Windows, ``subprocess.Popen()`` is now invoked with - ``shell=True`` if ``shell=False`` results in a WindowsError. This - allows commands involving ``*.bat`` and ``*.cmd`` files to be - executed when the extension is not specified; otherwise, only ``*.exe`` - can be found and run. - -* The path to utils is now found in ``pythontex.py`` via - ``sys.path[0]`` rather than ``kpsewhich``. This allows the PythonTeX - scripts to be executed in an arbitrary location; they no longer must - be installed in a texmf tree where ``kpsewhich`` can find them. - -* Added ``rerun`` value ``never``. - -* At the end of each run, data and macros are only saved if modified, - improving efficiency. - -* The number of temporary files required by each process was reduced by - one. All macros for commands like ``\py`` are now returned within - stdout, rather than in their own file. - -* Fixed a bug with ``\stderrpythontex``; it was defaulting to ``verb`` - rather than ``verbatim`` mode. - - -v0.11 (2013/04/21) - -* As the first non-beta release, this version adds several features and introduces several changes. You should read these release notes carefully, since some changes are not backwards-compatible. Changes are based on a thorough review of all current and planned features. PythonTeX's capabilities have already grown beyond what was originally intended, and a long list of features still remains to be implemented. As a result, some changes are needed to ensure consistent syntax and naming in the future. Insofar as possible, all command names and syntax will be frozen after this release. -* Added the ``pythontex.py`` and ``depythontex.py`` wrapper scripts. When run, these detect the current version of Python and import the correct PythonTeX code. It is still possible to run ``pythontex*.py`` and ``depythontex*.py`` directly, but the new wrapper scripts should be used instead for simplicity. There is now only a single ``pythontex_utils.py``, which works with both Python 2 and Python 3. -* Added the ``beta`` package option. This makes the current version behave like v0.11beta, for compatibility. This option is temporary and will probably only be retained for a few releases. -* Backward-incompatible changes (require the ``beta`` option to restore old behavior) - - - The ``pyverb`` environment has been renamed ``pyverbatim``. The old name was intended to be concise, but promoted confusion with LaTeX's ``\verb`` macro. - - For ``\printpythontex``, ``\stdoutpythontex``, and ``\stderrpythontex``, the modes ``inlineverb`` and ``v`` have been replaced by ``verb``, and the old mode ``verb`` has been replaced by ``verbatim``. This brings naming conventions in line with standard LaTeX ``\verb`` and ``verbatim``, avoiding a source of potential confusion. - - The ``\setpythontexpyglexer``, ``\setpythontexpygopt``, and ``\setpygmentspygopt`` commands now take an optional argument and a mandatory argument, rather than two mandatory arguments. This creates better uniformity among current and planned settings macros. - - The ``\setpythontexformatter`` and ``\setpygmentsformatter`` commands have been replaced by the ``\setpythontexprettyprinter`` and ``\setpygmentsprettyprinter`` commands. This anticipates possible upcoming features. It also avoids potential confusion with Pygments's formatters and the utilities class's ``formatter()`` method. - -* Deprecated (still work, but raise warnings; after a few releases, they will raise errors instead, and after that eventually be removed) - - - The ``rerun`` setting ``all`` was renamed ``always``, in preparation for upcoming features. - - The ``stderr`` option is replaced by ``makestderr``. The ``print``/``stdout`` option is replaced by ``debug``. These are intended to prevent confusion with future features. - - The ``fixlr`` option is deprecated. It was originally introduced to deal with some of SymPy's LaTeX formatting, which has since changed. - - The utilities class method ``init_sympy_latex()`` is deprecated. The ``sympy_latex()`` and ``set_sympy_latex()`` methods now automatically initialize themselves on first use. - -* Added ``autostdout`` package option and ``\setpythontexautostdout``, to complement ``autoprint``. Added ``prettyprinter`` and ``prettyprintinline`` package options to complement new settings commands. -* Added quickstart guide. -* Installer now installs gallery and quickstart files, if present. - - -v0.11beta (2013/02/17) - -* Commands like ``\py`` can now bring in any valid LaTeX code, including verbatim content, under the pdfTeX and XeTeX engines. Verbatim content was not allowed previously. LuaTeX cannot bring in verbatim, due to a known bug. -* Added package option ``depythontex`` and scripts ``depythontex*.py``. These allow a PythonTeX document to be converted into a pure LaTeX document, with no Python dependency. The package option creates an auxiliary file with extension ``.depytx``. The ``depythontex*.py`` scripts take this auxiliary file and the original LaTeX document, and combine the two to produce a new document that does not rely on the PythonTeX package. All PythonTeX commands and environments are replaced by their output. All Python-generated content is substituted directly into the document. By default, all typeset code is wrapped in ``\verb`` and ``verbatim``, but ``depythontex*.py`` has a ``--listing`` option that allows ``fancyvrb``, ``listings``, ``minted``, or ``pythontex`` to be used instead. -* The current PythonTeX version is now saved in the ``.pytxcode``. If this does not match the version of the PythonTeX scripts, a warning is issued. This makes it easier to determine errors due to version mismatches. -* Fixed an incompatibility with the latest release of ``xstring`` (version 1.7, 2013/01/13). -* Fixed a bug in the ``console`` environment that could cause problems when switching from Pygments highlighting to ``fancyvrb`` when using the ``fvextfile`` option. Fixed a bug introduced in the v0.10beta series that prevented the ``console`` environment from working with ``fancyvrb``. -* Fixed a bug with PythonTeX verbatim commands and environments that use Pygments. The verbatim commands and environments were incorrectly treated as if they had the attributes of executed code in the v0.10beta series. -* Fixed a bug from the v0.10beta series that sometimes prevented imports from ``__future__`` from working when there were multiple sessions. -* Fixed a bug related to hashing dependencies' mtime under Python 3. - - -v0.10beta2 (2013/01/23) - -* Improved ``pythontex*.py``'s handling of the name of the file being processed. A warning is no longer raised if the name is given with an extension; extensions are now processed (stripped) automatically. The filename may now contain a path to the file, so you need not run ``pythontex*.py`` from within the document's directory. -* Added command-line option ``--verbose`` for more verbose output. Currently, this prints a list of all processes that are launched. -* Fixed a bug that could crash ``pythontex*.py`` when the package option ``pygments=false``. -* Added documentation about ``autoprint`` behavior in the preamble. Summary: ``code`` commands and environments are allowed in the preamble as of v0.10beta. ``autoprint`` only applies to the body of the document, because nothing can be typeset in the preamble. Content printed in the preamble can be brought in by explicitly using ``\printpythontex``, but this should be used with great care. -* Revised ``\stdoutpythontex`` and ``\printpythontex`` so that they work in the preamble. Again, this should be used with great care if at all. -* Revised treatment of any content that custom code attempts to print. Custom code is not allowed to print to the document (see documentation). If custom code attempts to print, a warning is raised, and the printed content is included in the ``pythontex*.py`` run summary. -* One-line entries in stderr, such as those produced by Python's ``warnings.warn()``, were not previously parsed because they are of the form ``::`` rather than ``line ``. These are now parsed and synchronized with the document. They are also correctly parsed for inclusion in the document via ``\stderrpythontex``. -* If the package option ``stderrfilename`` is changed, all sessions that produced errors or warnings are now re-executed automatically, so that their stderr content is properly updated with the new filename. - - -v0.10beta (2013/01/09) - -* Backward-incompatible: Redid treatment of command-line options for - ``pythontex*.py``, using Python's ``argparse`` module. Run - ``pythontex*.py`` with option ``-h`` to see new command line options. -* Deprecated: ``\setpythontexcustomcode`` is deprecated in favor of the - ``\pythontexcustomc`` command and ``pythontexcustomcode`` - environment. These allow entry of pure code, unlike - ``\setpythontexcustomcode``. These also allow custom code to be - added to the beginning or end of a session, via an optional argument. - Improved treatment of errors and warnings associated with custom - code. -* The summary of errors and warnings now correctly differentiates - errors and warnings produced by user code, rather than treating all - of them as errors. By default, ``pythontex*.py`` now returns an - exit code of 1 if there were errors. -* The PythonTeX utilities class now allows external file dependencies - to be specified via ``pytex.add_dependencies()``, so that sessions - are automatically re-executed when external dependencies are - modified (modification is determined via either hash or mtime; this - is governed by the new ``hashdependencies`` option). -* The PythonTeX utilities class now allows created files to be - specified via ``pytex.add_created()``, so that created files may be - automatically cleaned up (deleted) when the code that created them - is modified (for example, name change for a saved plot). -* Added the following package options. - - - ``stdout`` (or ``print``): Allows input of stdout to be disabled. - Useful for debugging. - - ``runall``: Executes everything. Useful when code depends on - external data. - - ``rerun``: Determines when code is re-executed. Code may be set - to always run (same as ``runall`` option), or only run when it is - modified or when it produces errors or warnings. By default, - code is always re-executed if there are errors or modifications, - but not re-executed if there are warnings. - - ``hashdependencies``: Determines whether external dependencies - (data, external code files highlighted with Pygments, etc.) are - checked for modification via hashing or modification time. - Modification time is default for performance reasons. - -* Added the following new command line options. The options that are - equivalent to package options are overridden by the package options - when present. - - - ``--error-exit-code``: Determines whether an exit code of 1 is - returned if there were errors. On by default, but can be turned - off since it is undesirable when working with some editors. - - ``--runall``: Equivalent to new package option. - - ``--rerun``: Equivalent to new package option. - - ``--hashdependencies``: Equivalent to new package option. - -* Modified the ``fixlr`` option, so that it only patches commands if - they have not already been patched (avoids package conflicts). -* Added ``\setpythontexautoprint`` command for toggling autoprint - on/off within the body of the document. -* Installer now attempts to create symlinks under OS X and Linux with - TeX Live, and under OS X with MacPorts Tex Live. -* Performed compatibility testing under lualatex and xelatex - (previously, had only tested with pdflatex). Added documentation - for using these TeX engines; at most, slightly different preambles - are needed. Modified the PythonTeX gallery to support all three - engines. -* Code commands and environments may now be used in the preamble. - This, combined with the new treatment of custom code, allows - PythonTeX to be used in creating LaTeX packages. -* Added documentation for using PythonTeX in LaTeX programming. -* Fixed a bug that sometimes caused incorrect line numbers with - ``stderr`` content. Improved processing of stderr. -* Fixed a bug in automatic detection of pre-existing listings - environment. -* Improved the detection of imports from ``__future__``. Detection - should now be stricter, faster, and more accurate. - - -v0.9beta3 (2012/07/17) - -* Added Unicode support, which required the Python code to be split into - one set for Python 2 and another set for Python 3. This will require - any old installation to be completely removed, and a new installation - created from scratch. -* Refactoring of Python code. Documents should automatically re-execute - all code after updating to the new version. Otherwise, you should delete - the PythonTeX directory and run PythonTeX. -* Improved installation script. -* Added package options: pyfuture, stderr, upquote, pyglexer, pyginline. - Renamed the pygextfile option to fvextfile. -* Added custom code and workingdir commands. -* Added the console environment and associated options. -* Rewrote pythontex_utils*.py, creating a new, context-aware interface to - SymPy's LatexPrinter class. -* Content brought in via macros no longer uses labels. Rather, long defs - are used, which allows line breaks. -* Pygments highlighting is now default for PythonTeX commands and environments. - - -v0.9beta2 (2012/05/09) - -* Changed Python output extension to .stdout. - -v0.9beta (2012/04/27) - -* Initial public beta release. +* Output: + + :: + + Python says “Hello!” + 8 × 256 = 2048 + +Notice that there is a three-step compile process. This is what makes +possible commands like ``\pymultiply`` that use Python or other languages +internally. You may want to configure your LaTeX editor with a shortcut for +running ``pythontex`` or ``pythontex.py``, or configure your LaTeX build +system to run ``pythontex`` or ``pythontex.py``. + + +Citing PythonTeX +================ + +If you use PythonTeX in your writing and research, please consider citing it +in any resulting publications. The best and most recent paper is in +`Computational Science & Discovery `_ +(doi:10.1088/1749-4699/8/1/014010). You may also cite the paper in the +`2013 SciPy proceedings `_. + + +Development status +================== + +Since 2020, I am increasingly creating new teaching materials with Markdown +and HTML instead of PythonTeX and LaTeX. I have limited time for developing +open-source software that I do not use frequently myself. There should be +occasional releases to keep PythonTeX running, but no major changes or +significant new features are anticipated. + +I have been developing `Codebraid `_ since 2019, +partially to have a PythonTeX equivalent for Markdown but also in the hope +that it could eventually be integrated with LaTeX as a PythonTeX replacement. +As part of `minted `_ version 3, I created +new software for passing data between LaTeX and Python. I cannot make any +guarantees, but I hope that this will eventually make it possible to create a +new LaTeX package based on Codebraid, with significant PythonTeX compatibility. + + +License +======= + +LPPL_ for LaTeX code and `BSD 3-Clause`_ for Python code. + +.. _LPPL: http://www.latex-project.org/lppl.txt + +.. _`BSD 3-Clause`: http://www.opensource.org/licenses/BSD-3-Clause diff --git a/package_downloads/old_versions/pythontex_0.10beta.zip b/package_downloads/old_versions/pythontex_0.10beta.zip deleted file mode 100644 index 11d47ff..0000000 Binary files a/package_downloads/old_versions/pythontex_0.10beta.zip and /dev/null differ diff --git a/package_downloads/old_versions/pythontex_0.10beta2.zip b/package_downloads/old_versions/pythontex_0.10beta2.zip deleted file mode 100644 index 8cb549d..0000000 Binary files a/package_downloads/old_versions/pythontex_0.10beta2.zip and /dev/null differ diff --git a/package_downloads/old_versions/pythontex_0.11.zip b/package_downloads/old_versions/pythontex_0.11.zip deleted file mode 100644 index b8210e9..0000000 Binary files a/package_downloads/old_versions/pythontex_0.11.zip and /dev/null differ diff --git a/package_downloads/old_versions/pythontex_0.11beta.zip b/package_downloads/old_versions/pythontex_0.11beta.zip deleted file mode 100644 index 084d8c9..0000000 Binary files a/package_downloads/old_versions/pythontex_0.11beta.zip and /dev/null differ diff --git a/package_downloads/old_versions/pythontex_0.12beta.zip b/package_downloads/old_versions/pythontex_0.12beta.zip deleted file mode 100644 index 5d4985d..0000000 Binary files a/package_downloads/old_versions/pythontex_0.12beta.zip and /dev/null differ diff --git a/package_downloads/old_versions/pythontex_0.9beta3.zip b/package_downloads/old_versions/pythontex_0.9beta3.zip deleted file mode 100644 index b8ef46b..0000000 Binary files a/package_downloads/old_versions/pythontex_0.9beta3.zip and /dev/null differ diff --git a/package_downloads/pythontex_0.12.zip b/package_downloads/pythontex_0.12.zip deleted file mode 100644 index b27d8b4..0000000 Binary files a/package_downloads/pythontex_0.12.zip and /dev/null differ diff --git a/pythontex/README b/pythontex/README index 3c7247e..14c2e82 100644 --- a/pythontex/README +++ b/pythontex/README @@ -3,28 +3,29 @@ =================================================================== :Author: Geoffrey Poore -:Version: 0.12 (2013/08/26) +:Version: 0.19 (2026/02/15) :License: LPPL v1.3 or later (LaTeX code) and BSD 3-Clause (Python code) :Development: https://github.com/gpoore/pythontex :Requirements: Python 2.7 or 3.2+; Pygments -The PythonTeX package allows you to enter Python code within a LaTeX -document, execute the code, and access its output in the original document. -Since v0.12, it also supports the Ruby and Julia languages. +The PythonTeX package allows you to enter Python code within a LaTeX document, +execute the code, and access its output in the original document. It also +supports the Ruby, Julia, Octave, Sage, Bash, Rust, R, Perl, Perl 6, and +JavaScript languages. PythonTeX provides fast, user-friendly access to Python from within LaTeX. -Python code is only executed when it has been modified, or when it meets +Python code is only executed when it has been modified, or when it meets user-specified criteria. Code may be divided into user-defined sessions, -which automatically run in parallel. Errors and warnings are synchronized -with the LaTeX document, so that they refer to the document's line numbers. -External dependencies can be tracked, so that code is re-executed when the -data it depends on is modified. PythonTeX also provides syntax highlighting +which automatically run in parallel. Errors and warnings are synchronized +with the LaTeX document, so that they refer to the document's line numbers. +External dependencies can be tracked, so that code is re-executed when the +data it depends on is modified. PythonTeX also provides syntax highlighting for code in LaTeX documents via the Pygments syntax highlighter. PythonTeX includes a "depythontex" utility that creates a copy of a document in which all Python code has been replaced by its output. This can be -convenient for journal submissions, sharing documents, and converting to +convenient for journal submissions, sharing documents, and converting to other formats. Consult pythontex_quickstart.pdf and pythontex_gallery.pdf to get @@ -36,10 +37,10 @@ Compiling a PythonTeX document involves three steps: 2. Run pythontex.py (all code is executed) 3. Run latex again (Python output is inputted) -See the main documentation in pythontex.pdf for detailed installation -instructions. An installation script is provided for TeX Live. Files -should be installed in the directories indicated below. In summary, -all files must be installed, and symlinks or launching wrappers should be +See the main documentation in pythontex.pdf for detailed installation +instructions. An installation script is provided for TeX Live and MiKTeX. +Files should be installed in the directories indicated below. In summary, +all files must be installed, and symlinks or launching wrappers should be created for pythontex.py and depythontex.py. * /doc/latex/pythontex/ @@ -57,6 +58,7 @@ created for pythontex.py and depythontex.py. - pythontex_engines.py - pythontex_utils.py - depythontex.py, depythontex2.py, and depythontex3.py + - syncpdb.py * /source/latex/pythontex/ diff --git a/pythontex/depythontex.bat b/pythontex/depythontex.bat new file mode 100644 index 0000000..d80ddad --- /dev/null +++ b/pythontex/depythontex.bat @@ -0,0 +1,2 @@ +@echo off +depythontex.py %* diff --git a/pythontex/depythontex.py b/pythontex/depythontex.py index 013e268..f015c15 100644 --- a/pythontex/depythontex.py +++ b/pythontex/depythontex.py @@ -1,10 +1,15 @@ +#!/usr/bin/env python # -*- coding: utf-8 -*- + ''' -This is the PythonTeX wrapper script. It automatically detects the version -of Python, and then imports the correct code from depythontex2.py or -depythontex3.py. +This is the depythontex wrapper script. It automatically detects the version +of Python, and then imports the correct code from depythontex2.py or +depythontex3.py. It is intended for use with the default Python installation +on your system. If you wish to use a different version of Python, you could +launch depythontex2.py or depythontex3.py directly. The version of Python +does not matter for depythontex, since no code is executed. -Copyright (c) 2013, Geoffrey M. Poore +Copyright (c) 2013-2026, Geoffrey M. Poore All rights reserved. Licensed under the BSD 3-Clause License: http://www.opensource.org/licenses/BSD-3-Clause diff --git a/pythontex/depythontex2.py b/pythontex/depythontex2.py index 028badc..6e5a7f8 100644 --- a/pythontex/depythontex2.py +++ b/pythontex/depythontex2.py @@ -1,51 +1,53 @@ +#!/usr/bin/env python2 # -*- coding: utf-8 -*- + ''' PythonTeX depythontex script. -This script takes a LaTeX document that uses the PythonTeX package and -creates a new document that does not depend on PythonTeX. It substitutes all -externally generated content into a copy of the original LaTeX document. -This is useful when you need a document that relies on few external packages -or custom macros (for example, for submission to a journal or conversion to +This script takes a LaTeX document that uses the PythonTeX package and +creates a new document that does not depend on PythonTeX. It substitutes all +externally generated content into a copy of the original LaTeX document. +This is useful when you need a document that relies on few external packages +or custom macros (for example, for submission to a journal or conversion to another document format). -If you just want to share a document that uses PythonTeX, keep in mind that -the document can be modified and compiled just like a regular LaTeX document, -without needing Python or any other external tools, so long as the following +If you just want to share a document that uses PythonTeX, keep in mind that +the document can be modified and compiled just like a regular LaTeX document, +without needing Python or any other external tools, so long as the following conditions are met: * A copy of pythontex.sty is included with the document. * The pythontex-files- directory is included with the document. * The PythonTeX-specific parts of the document are not modified. -To work, this script requires that the original LaTeX document be compiled -with the package option `depythontex`. That creates an auxiliary file with +To work, this script requires that the original LaTeX document be compiled +with the package option `depythontex`. That creates an auxiliary file with the extension .depytx that contains information about all content that needs to be substituted. -This script is purposely written in a simple, largely linear form to -facilitate customization. Most of the key substitutions are performed by a +This script is purposely written in a simple, largely linear form to +facilitate customization. Most of the key substitutions are performed by a few functions defined near the beginning of the script, so if you need custom -substitutions, you should begin there. By default, all typeset code is -wrapped in `\verb` commands and verbatim environments, since these have the -greatest generality. However, the command-line option --listing allows code -to be typeset with the fancyvrb, listings, minted, or PythonTeX packages +substitutions, you should begin there. By default, all typeset code is +wrapped in `\verb` commands and verbatim environments, since these have the +greatest generality. However, the command-line option --listing allows code +to be typeset with the fancyvrb, listings, minted, or PythonTeX packages instead. -The script automatically extracts all arguments of all commands and -environments that it replaces, so that these are available if desired for -customized substitution. Two additional pieces of information are also -available for any typeset code: the Pygments lexer (often the same as the +The script automatically extracts all arguments of all commands and +environments that it replaces, so that these are available if desired for +customized substitution. Two additional pieces of information are also +available for any typeset code: the Pygments lexer (often the same as the language) and the starting line number (if line numbering was used). Keep in mind that some manual adjustments may be required after a document is -depythontex'ed. While depythontex attempts to create an exact copy of the -original document, in many cases an identical copy is impossible. For -example, typeset code may have a different appearance or layout when it is +depythontex'ed. While depythontex attempts to create an exact copy of the +original document, in many cases an identical copy is impossible. For +example, typeset code may have a different appearance or layout when it is typeset with a different package. -Copyright (c) 2013, Geoffrey M. Poore +Copyright (c) 2013-2026, Geoffrey M. Poore All rights reserved. Licensed under the BSD 3-Clause License: http://www.opensource.org/licenses/BSD-3-Clause @@ -62,6 +64,15 @@ #\\ End Python 2 import sys import os +#// Python 2 +if sys.version_info.major != 2: + sys.exit('This version of the PythonTeX script requires Python 2.') +#\\ End Python 2 +#// Python 3 +#if sys.version_info.major != 3: +# sys.exit('This version of the PythonTeX script requires Python 3.') +#\\ End Python 3 + #// Python 2 from io import open input = raw_input @@ -70,13 +81,12 @@ from collections import defaultdict from re import match, sub, search import textwrap +import codecs # Script parameters # Version -version = 'v0.12' - - +__version__ = '0.19' # Functions and parameters for customizing the script output @@ -93,77 +103,77 @@ preamble_additions = list() # Lexer dict -# If you are using Pygments lexers that don't directly correspond to the -# languages used by the listings package, you can submit replacements via the -# command line option --lexer-dict, or edit this dict manually here. When -# listings is used, all lexers are checked against this dict to see if a -# substitution should be made. This approach could easily be modified to +# If you are using Pygments lexers that don't directly correspond to the +# languages used by the listings package, you can submit replacements via the +# command line option --lexer-dict, or edit this dict manually here. When +# listings is used, all lexers are checked against this dict to see if a +# substitution should be made. This approach could easily be modified to # work with another, non-Pygments highlighting package. lexer_dict = dict() -def replace_code_cmd(name, arglist, linenum, code_replacement, +def replace_code_cmd(name, arglist, linenum, code_replacement, code_replacement_mode, after, lexer, firstnumber): ''' Typeset code from a command with a command. - + It is only ever called if there is indeed code to typeset. - - Usually, code from a command is also typeset with a command. This - function primarily deals with that case. In cases where code from a + + Usually, code from a command is also typeset with a command. This + function primarily deals with that case. In cases where code from a command is typeset with an environment (for example, `\inputpygments`), - this function performs some preprocessing and then uses + this function performs some preprocessing and then uses replace_code_env() to do the real work. This approach prevents the two functions from unnecessarily duplicating each other, while still giving the desired output. - + Args: name (str): name of the command - arglist (list, of str/None): all arguments given to the original - command; the last argument is what is typeset, unless a + arglist (list, of str/None): all arguments given to the original + command; the last argument is what is typeset, unless a code_replacement is specified or other instructions are given linenum (int): line number in the original TeX document code_replacement (str/None): replacement for the code; usually None - for commands, because typically the code to be typeset is the - last argument passed to the command, rather than something + for commands, because typically the code to be typeset is the + last argument passed to the command, rather than something captured elsewhere (like the body of an environment) or something preprocessed (like a console environment's content) - code_replacement_mode (str/None): mode in which the replacement is - to be typeset; raw/None (as TeX; generally unused for code), + code_replacement_mode (str/None): mode in which the replacement is + to be typeset; raw/None (as TeX; generally unused for code), verb (inline), or verbatim (environment) - after (str): text immediately following the command; usually + after (str): text immediately following the command; usually shouldn't be needed lexer (str/None): Pygments lexer Returns: (replacement, after) (tuple, of str) - + ''' # Get the correct replacement if code_replacement is None: code_replacement = arglist[-1] - + # We only consider two possible modes of typesetting, verbatim and inline # verbatim if code_replacement_mode == 'verbatim': - # Sometimes we must replace a command with an environment, for + # Sometimes we must replace a command with an environment, for # example, for `\inputpygments` - - # Make sure the introduction of an environment where a command was - # previously won't produce errors with following content; make sure + + # Make sure the introduction of an environment where a command was + # previously won't produce errors with following content; make sure # that any following content is on a separate line if bool(match('[ \t]*\S', after)): after = '\n' + after # Rather than duplicating much of replace_code_env(), just use it - return replace_code_env(name, arglist, linenum, code_replacement, + return replace_code_env(name, arglist, linenum, code_replacement, code_replacement_mode, after, lexer, firstnumber) else: # Usually, we're replacing a command with a command - + # Wrap the replacement in appropriate delimiters - if (listing in ('verbatim', 'fancyvrb', 'minted') or - (listing in ('listings', 'pythontex') and + if (listing in ('verbatim', 'fancyvrb', 'minted') or + (listing in ('listings', 'pythontex') and ('{' in code_replacement or '}' in code_replacement))): - for delim in ('|', '/', '`', '!', '&', '#', '@', ':', '%', '~', '$', + for delim in ('|', '/', '`', '!', '&', '#', '@', ':', '%', '~', '$', '=', '+', '-', '^', '_', '?', ';'): if delim not in code_replacement: break @@ -188,40 +198,40 @@ def replace_code_cmd(name, arglist, linenum, code_replacement, else: code_replacement = r'\pygment{' + lexer + '}' + code_replacement return (code_replacement, after) - -def replace_code_env(name, arglist, linenum, code_replacement, + +def replace_code_env(name, arglist, linenum, code_replacement, code_replacement_mode, after, lexer, firstnumber): ''' Typeset code from an environment with an environment. - + It is only ever called if there is indeed code to typeset. - + Usually it is only used to typeset code from an environment. However, some commands bring in code that must be typeset as an environment. In - those cases, replace_code_cmd() is called initially, and after it + those cases, replace_code_cmd() is called initially, and after it performs some preprocessing, this function is called. This approach avoids unnecessary duplication between the two functions. - + Args: name (str): name of the environment - arglist (list, of str/None): all arguments given to the original + arglist (list, of str/None): all arguments given to the original environment - linenum (int): line number in the original TeX document where + linenum (int): line number in the original TeX document where the environment began code_replacement (str): replacement for the code; unlike the case of commands, this is always not None if the function is called - code_replacement_mode (str/None): mode in which the replacement is - to be typeset; raw/None (as TeX; generally unused for code), + code_replacement_mode (str/None): mode in which the replacement is + to be typeset; raw/None (as TeX; generally unused for code), verb (inline), or verbatim (environment) - after (str): text immediately following the environment; usually + after (str): text immediately following the environment; usually shouldn't be needed lexer (str/None): Pygments lexer firstnumber (str/None): the first number of the listing, if the listing had numbered lines Returns: (replacement, after) (tuple, of str) - + ''' # Currently, there is no need to test for code_replacement_mode, because # this function is only ever called if the mode is 'verbatim'. That may @@ -235,7 +245,7 @@ def replace_code_env(name, arglist, linenum, code_replacement, pre = '\\begin{Verbatim}' else: pre = '\\begin{{Verbatim}}[numbers=left,firstnumber={0}]'.format(firstnumber) - post = '\\end{Verbatim}' + post = '\\end{Verbatim}' elif listing == 'listings': if lexer is None: if firstnumber is None: @@ -290,26 +300,26 @@ def replace_print_cmd(name, arglist, linenum, after): ''' Typeset printed content from a command. - + It is only ever called if there is indeed printed content to typeset. - + Args: name (str): name of the command - arglist (list, of str/None): all arguments given to the original + arglist (list, of str/None): all arguments given to the original command linenum (int): line number in the original TeX document print_replacement (str): printed content, read directly from file into a single string - print_replacement_mode (str/None): mode in which the replacement is - to be typeset; raw/None (as TeX), inlineverb (or v) (as inline), + print_replacement_mode (str/None): mode in which the replacement is + to be typeset; raw/None (as TeX), inlineverb (or v) (as inline), or verb (as environment) source (str/None): source of the replacement content after (str): text immediately following the command; important in some situations, because spacing can depend on what's next Returns: (replacement, after) (tuple, of str) - - ''' + + ''' if print_replacement_mode == 'verb': if print_replacement.count('\n') > 1: print('* DePythonTeX error:') @@ -317,7 +327,7 @@ def replace_print_cmd(name, arglist, linenum, print(' This is not possible in inline verbatim mode') sys.exit(1) print_replacement = print_replacement.rstrip('\n') - for delim in ('|', '/', '`', '!', '&', '#', '@', ':', '%', '~', '$', + for delim in ('|', '/', '`', '!', '&', '#', '@', ':', '%', '~', '$', '=', '+', '-', '^', '_', '?', ';'): if delim not in print_replacement: break @@ -325,71 +335,71 @@ def replace_print_cmd(name, arglist, linenum, elif print_replacement_mode == 'verbatim': if bool(match('\s*?\n', after)): # Usually, we would end the verbatim environment with a newline. - # This is fine if there is content in `after` before the next + # This is fine if there is content in `after` before the next # newline---in fact, it's desirable, because the verbatim package - # doesn't allow for content on the same line as the end of the - # environment. But if `after` is an empty line, then adding a + # doesn't allow for content on the same line as the end of the + # environment. But if `after` is an empty line, then adding a # newline will throw off spacing and must be avoided print_replacement = '\\begin{verbatim}\n' + print_replacement + '\\end{verbatim}' else: print_replacement = '\\begin{verbatim}\n' + print_replacement + '\\end{verbatim}\n' else: - # When printed content from a file is included as LaTeX code, we have - # to be particularly careful to ensure that the content produces the - # same output when substituted as when brought in by `\input`. In - # particular, `\input` strips newlines from each line of content and + # When printed content from a file is included as LaTeX code, we have + # to be particularly careful to ensure that the content produces the + # same output when substituted as when brought in by `\input`. In + # particular, `\input` strips newlines from each line of content and # adds a space at the end of each line. This space is inside the - # `\input`, so it will not merge with following spaces. So when we - # substitute the content, sometimes we need to replace the final + # `\input`, so it will not merge with following spaces. So when we + # substitute the content, sometimes we need to replace the final # newline with a space that cannot be gobbled. # - # It gets more complicated. This final space is often not - # desirable. It can be prevented by either printing an `\endinput` - # command, to terminate the `\input`, or printing a percent + # It gets more complicated. This final space is often not + # desirable. It can be prevented by either printing an `\endinput` + # command, to terminate the `\input`, or printing a percent # character % in the last line of the content, which comments out the - # final newline. So we must check for `\endinput` anywhere in - # printed content, and % in the final line, and remove any content - # after them. It's also possible that the print is followed by + # final newline. So we must check for `\endinput` anywhere in + # printed content, and % in the final line, and remove any content + # after them. It's also possible that the print is followed by # an `\unskip` that eats the space, so we need to check for that too. # - # It turns out that the same approach is needed when a command like + # It turns out that the same approach is needed when a command like # `\py` brings in content ending in a newline - if (print_replacement.endswith('\\endinput\n') and + if (print_replacement.endswith('\\endinput\n') and not print_replacement.endswith('\\string\\endinput\n')): - # If `\endinput` is present, everything from it on should be + # If `\endinput` is present, everything from it on should be # discarded, unless the `\endinput` is not actually a command - # but rather a typeset name (for example, `\string\endinput` or - # `\verb|\endinput|`). It's impossible to check for all cases in - # which `\endinput` is not a command (at least, without actually - # using LaTeX), and even checking for most of them would require - # a good bit of parsing. We assume that `\endinput`, as a - # command, will only ever occur at the immediate end of the - # printed content. Later, we issue a warning in case it appears + # but rather a typeset name (for example, `\string\endinput` or + # `\verb|\endinput|`). It's impossible to check for all cases in + # which `\endinput` is not a command (at least, without actually + # using LaTeX), and even checking for most of them would require + # a good bit of parsing. We assume that `\endinput`, as a + # command, will only ever occur at the immediate end of the + # printed content. Later, we issue a warning in case it appears # anywhere else. print_replacement = print_replacement.rsplit(r'\endinput', 1)[0] - elif (print_replacement.endswith('%\n') and - not print_replacement.endswith('\\%\n') and + elif (print_replacement.endswith('%\n') and + not print_replacement.endswith('\\%\n') and not print_replacement.endswith('\\string%\n')): # Perform an analogous check for a terminating percent characer %. - # This case would be a bit easier to parse fully, since a percent - # that comments out the last newline would have to be in the - # final line of the replacement. But it would still be - # very difficult to perform a complete check. Later, we issue a - # warning if there is reason to think that a percent character + # This case would be a bit easier to parse fully, since a percent + # that comments out the last newline would have to be in the + # final line of the replacement. But it would still be + # very difficult to perform a complete check. Later, we issue a + # warning if there is reason to think that a percent character # was active in the last line. print_replacement = print_replacement.rsplit(r'%', 1)[0] elif print_replacement.endswith('\n'): # We can't just use `else` because that would catch content # from `\py` and similar - # By default, LaTeX strips newlines and adds a space at the end - # of each line of content that is brought in by `\input`. This - # may or may not be desirable, but we replicate the effect here - # for consistency with the original document. We use `\space{}` - # because plain `\space` would gobble a following space, which + # By default, LaTeX strips newlines and adds a space at the end + # of each line of content that is brought in by `\input`. This + # may or may not be desirable, but we replicate the effect here + # for consistency with the original document. We use `\space{}` + # because plain `\space` would gobble a following space, which # isn't consistent with the `\input` behavior being replicated. if bool(match(r'\\unskip\s+\S', after)): - # If there's an `\unskip`, fix the spacing and remove the - # `\unskip`. Since this is inline, the `\unskip` must + # If there's an `\unskip`, fix the spacing and remove the + # `\unskip`. Since this is inline, the `\unskip` must # immediately follow the command to do any good; otherwise, # it eliminates spaces that precede it, but doesn't get into # the `\input` content. @@ -400,9 +410,9 @@ def replace_print_cmd(name, arglist, linenum, # the `\n`, and it will yield a space. pass elif bool(match('\s*$', after)): - # If the rest of the current line, and the next line, are + # If the rest of the current line, and the next line, are # whitespace, we will get the correct spacing without needing - # `\space{}`. We could leave `\n`, but it would be + # `\space{}`. We could leave `\n`, but it would be # extraneous whitespace. print_replacement = print_replacement[:-1] else: @@ -421,7 +431,7 @@ def replace_print_cmd(name, arglist, linenum, after = sub('^\s+', '\n', after) # Issue warnings, if warranted # Warn about `\endinput` - if (r'\endinput' in print_replacement and + if (r'\endinput' in print_replacement and print_replacement.count(r'\endinput') != print_replacement.count(r'\string\endinput')): print('* DePythonTeX warning:') print(' "\\endinput" was present in printed content near line ' + str(linenum)) @@ -431,7 +441,7 @@ def replace_print_cmd(name, arglist, linenum, # Warn if it looks like there are active `%` that could comment # out part of the original document. We only need to check the # last line of printed content, because only there could - # percent characters escape from their original confines within + # percent characters escape from their original confines within # `\input`, and comment out part of the document. if print_replacement.endswith('\n'): if print_replacement.count('\n') > 1: @@ -461,42 +471,42 @@ def replace_print_cmd(name, arglist, linenum, print(' If it should have adjusted the spacing of printed content') print(' you should double-check the spacing') return (print_replacement, after) - + def replace_print_env(name, arglist, linenum, print_replacement, print_replacement_mode, source, after): ''' Typeset printed content from an environment. - + It is only ever called if there is indeed printed content to typeset. - + This should be similar to replace_print_cmd(). The main difference is - that the environment context typically ends with a newline, so + that the environment context typically ends with a newline, so substitution has to be a little different to ensure that spacing after the environment isn't modified. - + Args: name (str): name of the environment - arglist (list, of str/None): all arguments given to the original + arglist (list, of str/None): all arguments given to the original environment - linenum (int): line number in the original TeX document where the + linenum (int): line number in the original TeX document where the environment began print_replacement (str): printed content, read directly from file into a single string - print_replacement_mode (str/None): mode in which the replacement is - to be typeset; raw/None (as TeX), inlineverb (or v) (as inline), + print_replacement_mode (str/None): mode in which the replacement is + to be typeset; raw/None (as TeX), inlineverb (or v) (as inline), or verb (as environment) source (str/None): source of the replacement content after (str): text immediately following the command; important in some situations, because spacing can depend on what's next Returns: (replacement, after) (tuple, of str) - + #### The inlineverb and verb modes should work, but haven't been tested since there are currently no environments that use them; they are only used by `\printpythontex`, which is a command. - ''' + ''' if print_replacement_mode == 'verb': if print_replacement.count('\n') > 1: print('* DePythonTeX error:') @@ -504,119 +514,119 @@ def replace_print_env(name, arglist, linenum, print(' This is not possible in inline verbatim mode') sys.exit(1) print_replacement = print_replacement.rstrip('\n') - for delim in ('|', '/', '`', '!', '&', '#', '@', ':', '%', '~', '$', + for delim in ('|', '/', '`', '!', '&', '#', '@', ':', '%', '~', '$', '=', '+', '-', '^', '_', '?', ';'): if delim not in print_replacement: break print_replacement = r'\verb' + delim + print_replacement + delim if not bool(match('[ \t]+\S', after)): - # If there is text on the same line as the end of the - # environment, we're fine (this is unusual). Otherwise, + # If there is text on the same line as the end of the + # environment, we're fine (this is unusual). Otherwise, # we need to toss the newline at the end of the environment - # and gobble leading spaces. Leading spaces need to be - # gobbled because previously they were at the beginning of a + # and gobble leading spaces. Leading spaces need to be + # gobbled because previously they were at the beginning of a # line, where they would have been discarded. if not bool(match('\s*$', after)): after = sub('^\s*?\n\s*', '', after) elif print_replacement_mode == 'verbatim': if bool(match('\s*?\n', after)): # Usually, we would end the verbatim environment with a newline. - # This is fine if there is content in `after` before the next + # This is fine if there is content in `after` before the next # newline---in fact, it's desirable, because the verbatim package - # doesn't allow for content on the same line as the end of the - # environment. But if `after` is an empty line, then adding a + # doesn't allow for content on the same line as the end of the + # environment. But if `after` is an empty line, then adding a # newline will throw off spacing and must be avoided print_replacement = '\\begin{verbatim}\n' + print_replacement + '\\end{verbatim}' else: print_replacement = '\\begin{verbatim}\n' + print_replacement + '\\end{verbatim}\n' else: - # When printed content is included as LaTeX code, we have to be - # particularly careful to ensure that the content produces the same - # output when substituted as when brought in by `\input`. In - # particular, `\input` strips newlines from each line of content and + # When printed content is included as LaTeX code, we have to be + # particularly careful to ensure that the content produces the same + # output when substituted as when brought in by `\input`. In + # particular, `\input` strips newlines from each line of content and # adds a space at the end of each line. This space is inside the - # `\input`, so it will not merge with following spaces. So when we - # substitute the content, sometimes we need to replace the final + # `\input`, so it will not merge with following spaces. So when we + # substitute the content, sometimes we need to replace the final # newline with a space that cannot be gobbled. # - # It gets more complicated. This final space is often not - # desirable. It can be prevented by either printing an `\endinput` - # command, to terminate the `\input`, or printing a percent + # It gets more complicated. This final space is often not + # desirable. It can be prevented by either printing an `\endinput` + # command, to terminate the `\input`, or printing a percent # character % in the last line of the content, which comments out the - # final newline. So we must check for `\endinput` anywhere in - # printed content, and % in the final line, and remove any content - # after them. It's also possible that the print is followed by + # final newline. So we must check for `\endinput` anywhere in + # printed content, and % in the final line, and remove any content + # after them. It's also possible that the print is followed by # an `\unskip` that eats the space, so we need to check for that too. - if (print_replacement.endswith('\\endinput\n') and + if (print_replacement.endswith('\\endinput\n') and not print_replacement.endswith('\\string\\endinput\n')): - # If `\endinput` is present, everything from it on should be + # If `\endinput` is present, everything from it on should be # discarded, unless the `\endinput` is not actually a command - # but rather a typeset name (for example, `\string\endinput` or - # `\verb|\endinput|`). It's impossible to check for all cases in - # which `\endinput` is not a command (at least, without actually - # using LaTeX), and even checking for most of them would require - # a good bit of parsing. We assume that `\endinput`, as a - # command, will only ever occur at the immediate end of the - # printed content. Later, we issue a warning in case it appears + # but rather a typeset name (for example, `\string\endinput` or + # `\verb|\endinput|`). It's impossible to check for all cases in + # which `\endinput` is not a command (at least, without actually + # using LaTeX), and even checking for most of them would require + # a good bit of parsing. We assume that `\endinput`, as a + # command, will only ever occur at the immediate end of the + # printed content. Later, we issue a warning in case it appears # anywhere else. print_replacement = print_replacement.rsplit(r'\endinput', 1)[0] if not bool(match('[ \t]+\S', after)): - # If there is text on the same line as the end of the - # environment, we're fine (this is unusual). Otherwise, + # If there is text on the same line as the end of the + # environment, we're fine (this is unusual). Otherwise, # we need to toss the newline at the end of the environment - # and gobble leading spaces. Leading spaces need to be - # gobbled because previously they were at the beginning of a + # and gobble leading spaces. Leading spaces need to be + # gobbled because previously they were at the beginning of a # line, where they would have been discarded. if not bool(match('\s*$', after)): after = sub('^\s*?\n\s*', '', after) - elif (print_replacement.endswith('%\n') and - not print_replacement.endswith('\\%\n') and + elif (print_replacement.endswith('%\n') and + not print_replacement.endswith('\\%\n') and not print_replacement.endswith('\\string%\n')): # Perform an analogous check for a terminating percent characer %. - # This case would be a bit easier to parse fully, since a percent - # that comments out the last newline would have to be in the - # final line of the replacement. But it would still be - # very difficult to perform a complete check. Later, we issue a - # warning if there is reason to think that a percent character + # This case would be a bit easier to parse fully, since a percent + # that comments out the last newline would have to be in the + # final line of the replacement. But it would still be + # very difficult to perform a complete check. Later, we issue a + # warning if there is reason to think that a percent character # was active in the last line. print_replacement = print_replacement.rsplit(r'%', 1)[0] if not bool(match('[ \t]+\S', after)): - # If there is text on the same line as the end of the - # environment, we're fine (this is unusual). Otherwise, + # If there is text on the same line as the end of the + # environment, we're fine (this is unusual). Otherwise, # we need to toss the newline at the end of the environment - # and gobble leading spaces. Leading spaces need to be - # gobbled because previously they were at the beginning of a + # and gobble leading spaces. Leading spaces need to be + # gobbled because previously they were at the beginning of a # line, where they would have been discarded. if not bool(match('\s*$', after)): after = sub('^\s*?\n\s*', '', after) else: - # By default, LaTeX strips newlines and adds a space at the end - # of each line of content that is brought in by `\input`. This - # may or may not be desirable, but we replicate the effect here - # for consistency with the original document. We use `\space{}` - # because plain `\space` would gobble a following space, which + # By default, LaTeX strips newlines and adds a space at the end + # of each line of content that is brought in by `\input`. This + # may or may not be desirable, but we replicate the effect here + # for consistency with the original document. We use `\space{}` + # because plain `\space` would gobble a following space, which # isn't consistent with the `\input` behavior being replicated. if bool(match(r'\s*\\unskip\s+\S', after)): - # If there's an `\unskip`, fix the spacing and remove the + # If there's an `\unskip`, fix the spacing and remove the # `\unskip` print_replacement = print_replacement.rstrip(' \t\n') after = sub(r'^\s*\\unskip\s+', '', after) elif bool(match('[ \t]+\S', after)): - # If the next character after the end of the environment is + # If the next character after the end of the environment is # not whitespace (usually not allowed), we can just leave - # the `\n` in printed content, and it will yield a space. - # So we need do nothing. But if there is text on that line + # the `\n` in printed content, and it will yield a space. + # So we need do nothing. But if there is text on that line # we need `\space{}`. after = sub('^\s+', '\\space', after) forced_double_space_list.append((name, linenum)) else: # If the line at the end of the environment is blank, - # we can just discard it and keep the newline at the end of + # we can just discard it and keep the newline at the end of # the printed content; the newline gives us the needed space after = after.split('\n', 1)[1] # Issue warnings, if warranted # Warn about `\endinput` - if (r'\endinput' in print_replacement and + if (r'\endinput' in print_replacement and print_replacement.count(r'\endinput') != print_replacement.count(r'\string\endinput')): print('* DePythonTeX warning:') print(' "\\endinput" was present in printed content near line ' + str(linenum)) @@ -626,7 +636,7 @@ def replace_print_env(name, arglist, linenum, # Warn if it looks like there are active `%` that could comment # out part of the original document. We only need to check the # last line of printed content, because only there could - # percent characters escape from their original confines within + # percent characters escape from their original confines within # `\input`, and comment out part of the document. if print_replacement.endswith('\n'): if print_replacement.count('\n') > 1: @@ -663,9 +673,9 @@ def replace_print_env(name, arglist, linenum, # Deal with argv # Parse argv parser = argparse.ArgumentParser() -parser.add_argument('--version', action='version', - version='DePythonTeX {0}'.format(version)) -parser.add_argument('--encoding', default='utf-8', +parser.add_argument('--version', action='version', + version='DePythonTeX {0}'.format(__version__)) +parser.add_argument('--encoding', default='utf-8', help='encoding for all text files (see codecs module for encodings)') parser.add_argument('--overwrite', default=False, action='store_true', help='overwrite existing output, if it exists (off by default)') @@ -678,10 +688,10 @@ def replace_print_env(name, arglist, linenum, help='line of commands to add to output preamble') parser.add_argument('--graphicspath', default=False, action='store_true', help=r'Add the outputdir to the graphics path, by modifying an existing \graphicspath command or adding one.') +parser.add_argument('-o', '--output', default=None, + help='output file') parser.add_argument('TEXNAME', help='LaTeX file') -parser.add_argument('OUTFILE', nargs='?', default=None, - help='output file; by default, . is converted into depythontex_.') args = parser.parse_args() # Process argv @@ -715,8 +725,9 @@ def replace_print_env(name, arglist, linenum, # Let the user know things have started -print('This is DePythonTeX {0}'.format(version)) -sys.stdout.flush() +if args.output is not None: + print('This is DePythonTeX {0}'.format(__version__)) + sys.stdout.flush() @@ -736,23 +747,20 @@ def replace_print_env(name, arglist, linenum, print(' Could not locate file "' + texfile_name + '"') sys.exit(1) # Make sure we have a valid outfile -if args.OUTFILE is None: - p, f_name = os.path.split(texfile_name) - outfile_name = os.path.join(p, 'depythontex_' + f_name) -else: - outfile_name = os.path.expanduser(os.path.normcase(args.OUTFILE)) -if not args.overwrite and os.path.isfile(outfile_name): - print('* DePythonTeX warning:') - print(' Output file "' + outfile_name + '" already exists') - ans = input(' Do you want to overwrite this file? [y,n]\n ') - if ans != 'y': - sys.exit(1) -# Make sure the .depytx file exists +if args.output is not None: + outfile_name = os.path.expanduser(os.path.normcase(args.output)) + if not args.overwrite and os.path.isfile(outfile_name): + print('* DePythonTeX warning:') + print(' Output file "' + outfile_name + '" already exists') + ans = input(' Do you want to overwrite this file? [y,n]\n ') + if ans != 'y': + sys.exit(1) +# Make sure the .depytx file exists depytxfile_name = texfile_name.rsplit('.')[0] + '.depytx' if not os.path.isfile(depytxfile_name): print('* DePythonTeX error:') print(' Could not find DePythonTeX auxiliary file "' + depytxfile_name + '"') - print(' Use package option depythontex to creat it') + print(' Use package option depythontex to create it') sys.exit(1) @@ -760,7 +768,7 @@ def replace_print_env(name, arglist, linenum, # Start opening files and loading data # Read in the LaTeX file -# We read into a list with an empty first entry, so that we don't have to +# We read into a list with an empty first entry, so that we don't have to # worry about zero indexing when comparing list index to file line number f = open(texfile_name, 'r', encoding=encoding) tex = [''] @@ -784,33 +792,34 @@ def replace_print_env(name, arglist, linenum, depytx[n] = '' n -= 1 # Check .depytx version to make sure it is compatible -if settings['version'] != version: +if settings['version'] != __version__: print('* DePythonTeX warning:') print(' Version mismatch with DePythonTeX auxiliary file') print(' Do a complete compile cycle to update the auxiliary file') print(' Attempting to proceed') # Go ahead and open the outfile, even though we don't need it until the end -# This lets us change working directories for convenience without worrying +# This lets us change working directories for convenience without worrying # about having to modify the outfile path -outfile = open(outfile_name, 'w', encoding=encoding) +if args.output is not None: + outfile = open(outfile_name, 'w', encoding=encoding) # Change working directory to the document directory -# Technically, we could get by without this, but that would require a lot of -# path modification. This way, we can just use all paths straight out of the +# Technically, we could get by without this, but that would require a lot of +# path modification. This way, we can just use all paths straight out of the # .depytx without any modification, which is much simpler and less error-prone. if os.path.split(texfile_name)[0] != '': os.chdir(os.path.split(texfile_name)[0]) - + # Open and process the file of macros # Read in the macros -if os.path.isfile(settings['macrofile']): - f = open(settings['macrofile'], 'r', encoding=encoding) +if os.path.isfile(os.path.expanduser(os.path.normcase(settings['macrofile']))): + f = open(os.path.expanduser(os.path.normcase(settings['macrofile'])), 'r', encoding=encoding) macros = f.readlines() f.close() else: @@ -821,7 +830,7 @@ def replace_print_env(name, arglist, linenum, sys.exit(1) # Create a dict for storing macros macrodict = defaultdict(list) -# Create variables for keeping track of whether we're inside a macro or +# Create variables for keeping track of whether we're inside a macro or # environment # These must exist before we begin processing inside_macro = False @@ -833,13 +842,13 @@ def replace_print_env(name, arglist, linenum, if inside_macro: # If we're in a macro, look for the end-of-macro command if r'\endpytx@SVMCR' in line: - # If the current line contains the end-of-macro command, split - # off any content that comes before it. Also reset + # If the current line contains the end-of-macro command, split + # off any content that comes before it. Also reset # `inside_macro`. macrodict[current_macro].append(line.rsplit(r'\endpytx@SVMCR', 1)[0]) inside_macro = False else: - # If the current line doesn't end the macro, we add the whole + # If the current line doesn't end the macro, we add the whole # line to the macro dict macrodict[current_macro].append(line) elif inside_environment: @@ -847,7 +856,7 @@ def replace_print_env(name, arglist, linenum, # If the environment is ending, we reset inside_environment inside_environment = False else: - # If we're still in the environment, add the current line to the + # If we're still in the environment, add the current line to the # macro dict macrodict[current_macro].append(line) else: @@ -856,10 +865,10 @@ def replace_print_env(name, arglist, linenum, # file to increase readability). Once we've determined which one, # we need to get its name and extract any content. if line.startswith(r'\begin{'): - # Any \begin will indicate a use of fancyvrb to save verbatim - # content, since that is the only time an environment is used in + # Any \begin will indicate a use of fancyvrb to save verbatim + # content, since that is the only time an environment is used in # the macro file. All other content is saved in a standard macro. - # We extract the name of the macro in which the verbatim content + # We extract the name of the macro in which the verbatim content # is saved. current_macro = line.rsplit('{', 1)[1].rstrip('}\n') inside_environment = True @@ -885,8 +894,8 @@ def replace_print_env(name, arglist, linenum, # Start at 1, since the first entry in the tex list is `''` texlinenum = 1 # Create a variable for storing the current line(s) we are processing. -# This contains all lines from immediately after the last successfully -# processed line up to and including texlinenum. We may have to process +# This contains all lines from immediately after the last successfully +# processed line up to and including texlinenum. We may have to process # multiple lines at once if a macro is split over multiple lines, etc. texcontent = tex[texlinenum] # Create a list for storing processed content. @@ -899,21 +908,21 @@ def replace_print_env(name, arglist, linenum, depy_type, depy_name, depy_args, depy_typeset, depy_linenum, depy_lexer = depytxcontent.split(':') if depy_lexer == '': depy_lexer = None - + # Do a quick check on validity of info # #### Eventually add 'cp' and 'pc' if not (depy_type in ('cmd', 'env') and all([letter in ('o', 'm', 'v', 'n', '|') for letter in depy_args]) and ('|' not in depy_args or (depy_args.count('|') == 1 and depy_args.endswith('|'))) and - depy_typeset in ('c', 'p', 'n')): + depy_typeset in ('c', 'p', 'n')): print('* PythonTeX error:') print(' Invalid \\Depythontex string for operation on line ' + str(depy_linenum)) print(' The offending string was ' + depytxcontent) sys.exit(1) - # If depy_args contains a `|` to indicate `\obeylines`, strip it and - # store in a variable. Create a bool to keep track of obeylines - # status, which governs whether we can look on the next line for - # arguments. (If obeylines is active, a newline terminates the + # If depy_args contains a `|` to indicate `\obeylines`, strip it and + # store in a variable. Create a bool to keep track of obeylines + # status, which governs whether we can look on the next line for + # arguments. (If obeylines is active, a newline terminates the # argument search.) if depy_args.endswith('|'): obeylines = True @@ -923,8 +932,8 @@ def replace_print_env(name, arglist, linenum, # Get the line number as an integer # We don't have to adjust for zero indexing in tex depy_linenum = int(depy_linenum) - - + + # Check for information passed from LaTeX # This will be extra listings information, or replacements to plug in code_replacement = None @@ -974,51 +983,51 @@ def replace_print_env(name, arglist, linenum, elif nextdepytxline.startswith('FILE:'): source = 'file' try: - typeset, f_name = nextdepytxline.rstrip('\n').split(':', 2)[1:] + typeset, f_name = nextdepytxline.rstrip('\n').split(':', 2)[1:] except: print('* DePythonTeX error:') print(' Improperly formatted file information on line ' + str(depy_linenum)) print(' The file information was "' + nextdepytxline + '"') sys.exit(1) - # Files that are brought in have an optional mode that + # Files that are brought in have an optional mode that # determines if they need special handling (for example, verbatim) if ':mode=' in f_name: f_name, mode = f_name.split(':mode=') else: mode = None - f = open(f_name, 'r', encoding=encoding) + f = open(os.path.expanduser(os.path.normcase(f_name)), 'r', encoding=encoding) replacement = f.read() f.close() if typeset == 'c': code_replacement_mode = mode if depy_type == 'cmd' and code_replacement_mode != 'verbatim': # Usually, code from commands is typeset with commands - # and code from environments is typeset in - # environments. The except is code from commands + # and code from environments is typeset in + # environments. The except is code from commands # that bring in external files, like `\inputpygments` code_replacement = replacement else: # If we're replacing an environment of code with a # file, then we lose the newline at the beginning # of the environment, and need to get it back. - code_replacement = '\n' + replacement + code_replacement = '\n' + replacement elif typeset == 'p': print_replacement_mode = mode - print_replacement = replacement + print_replacement = replacement else: print('* DePythonTeX error:') print(' Improper typesetting information for file information on line ' + str(depy_linenum)) print(' The file information was "' + nextdepytxline + '"') sys.exit(1) - # Increment the line in depytx to check for more information + # Increment the line in depytx to check for more information # from LaTeX scan_ahead_line += 1 if scan_ahead_line == len(depytx): break else: nextdepytxline = depytx[scan_ahead_line] - - + + # If the line we're looking for is within the range currently held by # texcontent, do nothing. Otherwise, transfer content from tex # to texout until we get to the line of tex that we're looking for @@ -1030,12 +1039,12 @@ def replace_print_env(name, arglist, linenum, texlinenum += 1 texcontent = tex[texlinenum] - + # Deal with arguments - # All arguments are parsed and stored in a list variables, even if - # they are not used, for completeness; this makes it easy to add + # All arguments are parsed and stored in a list variables, even if + # they are not used, for completeness; this makes it easy to add # functionality - # Start by splitting the current line into what comes before the + # Start by splitting the current line into what comes before the # command or environment, and what is after it if depy_type == 'cmd': try: @@ -1051,10 +1060,10 @@ def replace_print_env(name, arglist, linenum, print('* DePythonTeX error:') print(' Could not find environment "' + depy_name + '" on line ' + str(depy_linenum)) sys.exit(1) - # We won't need the content from before the command or environment + # We won't need the content from before the command or environment # again, so we go ahead and store it texout.append(before) - + # Parse the arguments # Create a list for storing the recovered arguments arglist = list() @@ -1066,7 +1075,7 @@ def replace_print_env(name, arglist, linenum, # Account for possible line breaks before end of arg while ']' not in after: texlinenum += 1 - after += tex[texlinenum] + after += tex[texlinenum] optarg, after = after[1:].split(']', 1) else: if obeylines: @@ -1075,7 +1084,7 @@ def replace_print_env(name, arglist, linenum, after = after.split('[', 1)[1] while ']' not in after: texlinenum += 1 - after += tex[texlinenum] + after += tex[texlinenum] optarg, after = after.split(']', 1) else: optarg = None @@ -1093,7 +1102,7 @@ def replace_print_env(name, arglist, linenum, after = after.split('[', 1)[1] while ']' not in after: texlinenum += 1 - after += tex[texlinenum] + after += tex[texlinenum] optarg, after = after.split(']', 1) else: optarg = None @@ -1132,8 +1141,8 @@ def replace_print_env(name, arglist, linenum, # Go through the argument character by character to find the # closing brace. # If possible, use a very simple approach - if (r'\{' not in after and r'\}' not in after and - r'\string' not in after and + if (r'\{' not in after and r'\}' not in after and + r'\string' not in after and after.count('{') + 1 == after.count('}')): pos = 0 lbraces = 1 @@ -1149,7 +1158,7 @@ def replace_print_env(name, arglist, linenum, if pos == len(after): texlinenum += 1 after += tex[texlinenum] - # If a simple parsing approach won't work, parse in much + # If a simple parsing approach won't work, parse in much # greater depth else: pos = 0 @@ -1174,7 +1183,7 @@ def replace_print_env(name, arglist, linenum, # First, jump ahead to after `\string` pos += 7 #+= len(r'\string') # See if `\string` is followed by a regular macro - # If so, jump past it; otherwise, figure out if a + # If so, jump past it; otherwise, figure out if a # single-character macro, or just a single character, is next, # and jump past it standard_macro = match(r'\\[a-zA-Z]+', line[pos:]) @@ -1185,7 +1194,7 @@ def replace_print_env(name, arglist, linenum, else: pos += 1 elif line[pos] == '\\': - # If the current position is a backslash, figure out what + # If the current position is a backslash, figure out what # macro is used, and jump past it # The macro must either be a standard alphabetic macro, # or a single-character macro @@ -1230,8 +1239,8 @@ def replace_print_env(name, arglist, linenum, after += tex[texlinenum] mainarg, after = after[1:].split(delim, 1) arglist.append(mainarg) - - + + # Do substitution, depending on what is required # Need a variable for processed content to be added to texout processed = None @@ -1242,8 +1251,8 @@ def replace_print_env(name, arglist, linenum, if after.count('\n') < 2: texlinenum += 1 after += tex[texlinenum] - processed, texcontent = replace_code_cmd(depy_name, arglist, - depy_linenum, + processed, texcontent = replace_code_cmd(depy_name, arglist, + depy_linenum, code_replacement, code_replacement_mode, after, depy_lexer, @@ -1282,7 +1291,7 @@ def replace_print_env(name, arglist, linenum, depy_linenum, code_replacement, code_replacement_mode, - after, depy_lexer, + after, depy_lexer, firstnumber) elif depy_typeset == 'p' and print_replacement is not None: if depy_type == 'cmd': @@ -1291,7 +1300,7 @@ def replace_print_env(name, arglist, linenum, if after.count('\n') < 2: texlinenum += 1 after += tex[texlinenum] - processed, texcontent = replace_print_cmd(depy_name, arglist, + processed, texcontent = replace_print_cmd(depy_name, arglist, depy_linenum, print_replacement, print_replacement_mode, @@ -1311,9 +1320,9 @@ def replace_print_env(name, arglist, linenum, if after.count('\n') < 2: texlinenum += 1 after += tex[texlinenum] - processed, texcontent = replace_print_env(depy_name, arglist, + processed, texcontent = replace_print_env(depy_name, arglist, depy_linenum, - print_replacement, + print_replacement, print_replacement_mode, source, after) @@ -1337,8 +1346,8 @@ def replace_print_env(name, arglist, linenum, texcontent = after # #### Once it's supported on the TeX side, need to add support for # pc and cp - - + + # Store any processed content if processed is not None: texout.append(processed) @@ -1360,11 +1369,15 @@ def replace_print_env(name, arglist, linenum, if startline == n: if bool(search(r'\\usepackage(?:\[.*?\]){0,1}\{pythontex\}', line)): texout[n] = sub(r'\\usepackage(?:\[.*?\]){0,1}\{pythontex\}', '', line) + if texout[n].isspace(): + texout[n] = '' break else: content = ''.join(texout[startline:n+1]) if bool(search(r'(?s)\\usepackage(?:\[.*?\]\s*){0,1}\{pythontex\}', content)): replacement = sub(r'(?s)\\usepackage(?:\[.*?\]\s*){0,1}\{pythontex\}', '', content) + if replacement.isspace(): + replacement = '' texout[startline] = replacement for l in range(startline+1, n+1): texout[l] = '' @@ -1399,6 +1412,16 @@ def replace_print_env(name, arglist, linenum, # Write output -for line in texout: - outfile.write(line) -outfile.close() +if args.output is not None: + for line in texout: + outfile.write(line) + outfile.close() +else: + if sys.version_info[0] == 2: + sys.stdout = codecs.getwriter(encoding)(sys.stdout, 'strict') + sys.stderr = codecs.getwriter(encoding)(sys.stderr, 'strict') + else: + sys.stdout = codecs.getwriter(encoding)(sys.stdout.buffer, 'strict') + sys.stderr = codecs.getwriter(encoding)(sys.stderr.buffer, 'strict') + for line in texout: + sys.stdout.write(line) diff --git a/pythontex/depythontex3.py b/pythontex/depythontex3.py index 996b844..e19f102 100644 --- a/pythontex/depythontex3.py +++ b/pythontex/depythontex3.py @@ -1,51 +1,53 @@ +#!/usr/bin/env python3 # -*- coding: utf-8 -*- + ''' PythonTeX depythontex script. -This script takes a LaTeX document that uses the PythonTeX package and -creates a new document that does not depend on PythonTeX. It substitutes all -externally generated content into a copy of the original LaTeX document. -This is useful when you need a document that relies on few external packages -or custom macros (for example, for submission to a journal or conversion to +This script takes a LaTeX document that uses the PythonTeX package and +creates a new document that does not depend on PythonTeX. It substitutes all +externally generated content into a copy of the original LaTeX document. +This is useful when you need a document that relies on few external packages +or custom macros (for example, for submission to a journal or conversion to another document format). -If you just want to share a document that uses PythonTeX, keep in mind that -the document can be modified and compiled just like a regular LaTeX document, -without needing Python or any other external tools, so long as the following +If you just want to share a document that uses PythonTeX, keep in mind that +the document can be modified and compiled just like a regular LaTeX document, +without needing Python or any other external tools, so long as the following conditions are met: * A copy of pythontex.sty is included with the document. * The pythontex-files- directory is included with the document. * The PythonTeX-specific parts of the document are not modified. -To work, this script requires that the original LaTeX document be compiled -with the package option `depythontex`. That creates an auxiliary file with +To work, this script requires that the original LaTeX document be compiled +with the package option `depythontex`. That creates an auxiliary file with the extension .depytx that contains information about all content that needs to be substituted. -This script is purposely written in a simple, largely linear form to -facilitate customization. Most of the key substitutions are performed by a +This script is purposely written in a simple, largely linear form to +facilitate customization. Most of the key substitutions are performed by a few functions defined near the beginning of the script, so if you need custom -substitutions, you should begin there. By default, all typeset code is -wrapped in `\verb` commands and verbatim environments, since these have the -greatest generality. However, the command-line option --listing allows code -to be typeset with the fancyvrb, listings, minted, or PythonTeX packages +substitutions, you should begin there. By default, all typeset code is +wrapped in `\\verb` commands and verbatim environments, since these have the +greatest generality. However, the command-line option --listing allows code +to be typeset with the fancyvrb, listings, minted, or PythonTeX packages instead. -The script automatically extracts all arguments of all commands and -environments that it replaces, so that these are available if desired for -customized substitution. Two additional pieces of information are also -available for any typeset code: the Pygments lexer (often the same as the +The script automatically extracts all arguments of all commands and +environments that it replaces, so that these are available if desired for +customized substitution. Two additional pieces of information are also +available for any typeset code: the Pygments lexer (often the same as the language) and the starting line number (if line numbering was used). Keep in mind that some manual adjustments may be required after a document is -depythontex'ed. While depythontex attempts to create an exact copy of the -original document, in many cases an identical copy is impossible. For -example, typeset code may have a different appearance or layout when it is +depythontex'ed. While depythontex attempts to create an exact copy of the +original document, in many cases an identical copy is impossible. For +example, typeset code may have a different appearance or layout when it is typeset with a different package. -Copyright (c) 2013, Geoffrey M. Poore +Copyright (c) 2013-2026, Geoffrey M. Poore All rights reserved. Licensed under the BSD 3-Clause License: http://www.opensource.org/licenses/BSD-3-Clause @@ -62,6 +64,15 @@ #\\ End Python 2 import sys import os +#// Python 2 +#if sys.version_info.major != 2: +# sys.exit('This version of the PythonTeX script requires Python 2.') +#\\ End Python 2 +#// Python 3 +if sys.version_info.major != 3: + sys.exit('This version of the PythonTeX script requires Python 3.') +#\\ End Python 3 + #// Python 2 #from io import open #input = raw_input @@ -70,13 +81,12 @@ from collections import defaultdict from re import match, sub, search import textwrap +import codecs # Script parameters # Version -version = 'v0.12' - - +__version__ = '0.19' # Functions and parameters for customizing the script output @@ -93,77 +103,77 @@ preamble_additions = list() # Lexer dict -# If you are using Pygments lexers that don't directly correspond to the -# languages used by the listings package, you can submit replacements via the -# command line option --lexer-dict, or edit this dict manually here. When -# listings is used, all lexers are checked against this dict to see if a -# substitution should be made. This approach could easily be modified to +# If you are using Pygments lexers that don't directly correspond to the +# languages used by the listings package, you can submit replacements via the +# command line option --lexer-dict, or edit this dict manually here. When +# listings is used, all lexers are checked against this dict to see if a +# substitution should be made. This approach could easily be modified to # work with another, non-Pygments highlighting package. lexer_dict = dict() -def replace_code_cmd(name, arglist, linenum, code_replacement, +def replace_code_cmd(name, arglist, linenum, code_replacement, code_replacement_mode, after, lexer, firstnumber): ''' Typeset code from a command with a command. - + It is only ever called if there is indeed code to typeset. - - Usually, code from a command is also typeset with a command. This - function primarily deals with that case. In cases where code from a - command is typeset with an environment (for example, `\inputpygments`), - this function performs some preprocessing and then uses + + Usually, code from a command is also typeset with a command. This + function primarily deals with that case. In cases where code from a + command is typeset with an environment (for example, `\\inputpygments`), + this function performs some preprocessing and then uses replace_code_env() to do the real work. This approach prevents the two functions from unnecessarily duplicating each other, while still giving the desired output. - + Args: name (str): name of the command - arglist (list, of str/None): all arguments given to the original - command; the last argument is what is typeset, unless a + arglist (list, of str/None): all arguments given to the original + command; the last argument is what is typeset, unless a code_replacement is specified or other instructions are given linenum (int): line number in the original TeX document code_replacement (str/None): replacement for the code; usually None - for commands, because typically the code to be typeset is the - last argument passed to the command, rather than something + for commands, because typically the code to be typeset is the + last argument passed to the command, rather than something captured elsewhere (like the body of an environment) or something preprocessed (like a console environment's content) - code_replacement_mode (str/None): mode in which the replacement is - to be typeset; raw/None (as TeX; generally unused for code), + code_replacement_mode (str/None): mode in which the replacement is + to be typeset; raw/None (as TeX; generally unused for code), verb (inline), or verbatim (environment) - after (str): text immediately following the command; usually + after (str): text immediately following the command; usually shouldn't be needed lexer (str/None): Pygments lexer Returns: (replacement, after) (tuple, of str) - + ''' # Get the correct replacement if code_replacement is None: code_replacement = arglist[-1] - + # We only consider two possible modes of typesetting, verbatim and inline # verbatim if code_replacement_mode == 'verbatim': - # Sometimes we must replace a command with an environment, for + # Sometimes we must replace a command with an environment, for # example, for `\inputpygments` - - # Make sure the introduction of an environment where a command was - # previously won't produce errors with following content; make sure + + # Make sure the introduction of an environment where a command was + # previously won't produce errors with following content; make sure # that any following content is on a separate line - if bool(match('[ \t]*\S', after)): + if bool(match(r'[ \t]*\S', after)): after = '\n' + after # Rather than duplicating much of replace_code_env(), just use it - return replace_code_env(name, arglist, linenum, code_replacement, + return replace_code_env(name, arglist, linenum, code_replacement, code_replacement_mode, after, lexer, firstnumber) else: # Usually, we're replacing a command with a command - + # Wrap the replacement in appropriate delimiters - if (listing in ('verbatim', 'fancyvrb', 'minted') or - (listing in ('listings', 'pythontex') and + if (listing in ('verbatim', 'fancyvrb', 'minted') or + (listing in ('listings', 'pythontex') and ('{' in code_replacement or '}' in code_replacement))): - for delim in ('|', '/', '`', '!', '&', '#', '@', ':', '%', '~', '$', + for delim in ('|', '/', '`', '!', '&', '#', '@', ':', '%', '~', '$', '=', '+', '-', '^', '_', '?', ';'): if delim not in code_replacement: break @@ -188,40 +198,40 @@ def replace_code_cmd(name, arglist, linenum, code_replacement, else: code_replacement = r'\pygment{' + lexer + '}' + code_replacement return (code_replacement, after) - -def replace_code_env(name, arglist, linenum, code_replacement, + +def replace_code_env(name, arglist, linenum, code_replacement, code_replacement_mode, after, lexer, firstnumber): ''' Typeset code from an environment with an environment. - + It is only ever called if there is indeed code to typeset. - + Usually it is only used to typeset code from an environment. However, some commands bring in code that must be typeset as an environment. In - those cases, replace_code_cmd() is called initially, and after it + those cases, replace_code_cmd() is called initially, and after it performs some preprocessing, this function is called. This approach avoids unnecessary duplication between the two functions. - + Args: name (str): name of the environment - arglist (list, of str/None): all arguments given to the original + arglist (list, of str/None): all arguments given to the original environment - linenum (int): line number in the original TeX document where + linenum (int): line number in the original TeX document where the environment began code_replacement (str): replacement for the code; unlike the case of commands, this is always not None if the function is called - code_replacement_mode (str/None): mode in which the replacement is - to be typeset; raw/None (as TeX; generally unused for code), + code_replacement_mode (str/None): mode in which the replacement is + to be typeset; raw/None (as TeX; generally unused for code), verb (inline), or verbatim (environment) - after (str): text immediately following the environment; usually + after (str): text immediately following the environment; usually shouldn't be needed lexer (str/None): Pygments lexer firstnumber (str/None): the first number of the listing, if the listing had numbered lines Returns: (replacement, after) (tuple, of str) - + ''' # Currently, there is no need to test for code_replacement_mode, because # this function is only ever called if the mode is 'verbatim'. That may @@ -235,7 +245,7 @@ def replace_code_env(name, arglist, linenum, code_replacement, pre = '\\begin{Verbatim}' else: pre = '\\begin{{Verbatim}}[numbers=left,firstnumber={0}]'.format(firstnumber) - post = '\\end{Verbatim}' + post = '\\end{Verbatim}' elif listing == 'listings': if lexer is None: if firstnumber is None: @@ -290,26 +300,26 @@ def replace_print_cmd(name, arglist, linenum, after): ''' Typeset printed content from a command. - + It is only ever called if there is indeed printed content to typeset. - + Args: name (str): name of the command - arglist (list, of str/None): all arguments given to the original + arglist (list, of str/None): all arguments given to the original command linenum (int): line number in the original TeX document print_replacement (str): printed content, read directly from file into a single string - print_replacement_mode (str/None): mode in which the replacement is - to be typeset; raw/None (as TeX), inlineverb (or v) (as inline), + print_replacement_mode (str/None): mode in which the replacement is + to be typeset; raw/None (as TeX), inlineverb (or v) (as inline), or verb (as environment) source (str/None): source of the replacement content after (str): text immediately following the command; important in some situations, because spacing can depend on what's next Returns: (replacement, after) (tuple, of str) - - ''' + + ''' if print_replacement_mode == 'verb': if print_replacement.count('\n') > 1: print('* DePythonTeX error:') @@ -317,92 +327,92 @@ def replace_print_cmd(name, arglist, linenum, print(' This is not possible in inline verbatim mode') sys.exit(1) print_replacement = print_replacement.rstrip('\n') - for delim in ('|', '/', '`', '!', '&', '#', '@', ':', '%', '~', '$', + for delim in ('|', '/', '`', '!', '&', '#', '@', ':', '%', '~', '$', '=', '+', '-', '^', '_', '?', ';'): if delim not in print_replacement: break print_replacement = r'\verb' + delim + print_replacement + delim elif print_replacement_mode == 'verbatim': - if bool(match('\s*?\n', after)): + if bool(match(r'\s*?\n', after)): # Usually, we would end the verbatim environment with a newline. - # This is fine if there is content in `after` before the next + # This is fine if there is content in `after` before the next # newline---in fact, it's desirable, because the verbatim package - # doesn't allow for content on the same line as the end of the - # environment. But if `after` is an empty line, then adding a + # doesn't allow for content on the same line as the end of the + # environment. But if `after` is an empty line, then adding a # newline will throw off spacing and must be avoided print_replacement = '\\begin{verbatim}\n' + print_replacement + '\\end{verbatim}' else: print_replacement = '\\begin{verbatim}\n' + print_replacement + '\\end{verbatim}\n' else: - # When printed content from a file is included as LaTeX code, we have - # to be particularly careful to ensure that the content produces the - # same output when substituted as when brought in by `\input`. In - # particular, `\input` strips newlines from each line of content and + # When printed content from a file is included as LaTeX code, we have + # to be particularly careful to ensure that the content produces the + # same output when substituted as when brought in by `\input`. In + # particular, `\input` strips newlines from each line of content and # adds a space at the end of each line. This space is inside the - # `\input`, so it will not merge with following spaces. So when we - # substitute the content, sometimes we need to replace the final + # `\input`, so it will not merge with following spaces. So when we + # substitute the content, sometimes we need to replace the final # newline with a space that cannot be gobbled. # - # It gets more complicated. This final space is often not - # desirable. It can be prevented by either printing an `\endinput` - # command, to terminate the `\input`, or printing a percent + # It gets more complicated. This final space is often not + # desirable. It can be prevented by either printing an `\endinput` + # command, to terminate the `\input`, or printing a percent # character % in the last line of the content, which comments out the - # final newline. So we must check for `\endinput` anywhere in - # printed content, and % in the final line, and remove any content - # after them. It's also possible that the print is followed by + # final newline. So we must check for `\endinput` anywhere in + # printed content, and % in the final line, and remove any content + # after them. It's also possible that the print is followed by # an `\unskip` that eats the space, so we need to check for that too. # - # It turns out that the same approach is needed when a command like + # It turns out that the same approach is needed when a command like # `\py` brings in content ending in a newline - if (print_replacement.endswith('\\endinput\n') and + if (print_replacement.endswith('\\endinput\n') and not print_replacement.endswith('\\string\\endinput\n')): - # If `\endinput` is present, everything from it on should be + # If `\endinput` is present, everything from it on should be # discarded, unless the `\endinput` is not actually a command - # but rather a typeset name (for example, `\string\endinput` or - # `\verb|\endinput|`). It's impossible to check for all cases in - # which `\endinput` is not a command (at least, without actually - # using LaTeX), and even checking for most of them would require - # a good bit of parsing. We assume that `\endinput`, as a - # command, will only ever occur at the immediate end of the - # printed content. Later, we issue a warning in case it appears + # but rather a typeset name (for example, `\string\endinput` or + # `\verb|\endinput|`). It's impossible to check for all cases in + # which `\endinput` is not a command (at least, without actually + # using LaTeX), and even checking for most of them would require + # a good bit of parsing. We assume that `\endinput`, as a + # command, will only ever occur at the immediate end of the + # printed content. Later, we issue a warning in case it appears # anywhere else. print_replacement = print_replacement.rsplit(r'\endinput', 1)[0] - elif (print_replacement.endswith('%\n') and - not print_replacement.endswith('\\%\n') and + elif (print_replacement.endswith('%\n') and + not print_replacement.endswith('\\%\n') and not print_replacement.endswith('\\string%\n')): # Perform an analogous check for a terminating percent characer %. - # This case would be a bit easier to parse fully, since a percent - # that comments out the last newline would have to be in the - # final line of the replacement. But it would still be - # very difficult to perform a complete check. Later, we issue a - # warning if there is reason to think that a percent character + # This case would be a bit easier to parse fully, since a percent + # that comments out the last newline would have to be in the + # final line of the replacement. But it would still be + # very difficult to perform a complete check. Later, we issue a + # warning if there is reason to think that a percent character # was active in the last line. print_replacement = print_replacement.rsplit(r'%', 1)[0] elif print_replacement.endswith('\n'): # We can't just use `else` because that would catch content # from `\py` and similar - # By default, LaTeX strips newlines and adds a space at the end - # of each line of content that is brought in by `\input`. This - # may or may not be desirable, but we replicate the effect here - # for consistency with the original document. We use `\space{}` - # because plain `\space` would gobble a following space, which + # By default, LaTeX strips newlines and adds a space at the end + # of each line of content that is brought in by `\input`. This + # may or may not be desirable, but we replicate the effect here + # for consistency with the original document. We use `\space{}` + # because plain `\space` would gobble a following space, which # isn't consistent with the `\input` behavior being replicated. if bool(match(r'\\unskip\s+\S', after)): - # If there's an `\unskip`, fix the spacing and remove the - # `\unskip`. Since this is inline, the `\unskip` must + # If there's an `\unskip`, fix the spacing and remove the + # `\unskip`. Since this is inline, the `\unskip` must # immediately follow the command to do any good; otherwise, # it eliminates spaces that precede it, but doesn't get into # the `\input` content. print_replacement = print_replacement.rstrip(' \t\n') after = sub(r'^\\unskip\s+', '', after) - elif bool(match('\S', after)): + elif bool(match(r'\S', after)): # If the next character is not whitespace, we can just leave # the `\n`, and it will yield a space. pass - elif bool(match('\s*$', after)): - # If the rest of the current line, and the next line, are + elif bool(match(r'\s*$', after)): + # If the rest of the current line, and the next line, are # whitespace, we will get the correct spacing without needing - # `\space{}`. We could leave `\n`, but it would be + # `\space{}`. We could leave `\n`, but it would be # extraneous whitespace. print_replacement = print_replacement[:-1] else: @@ -411,17 +421,17 @@ def replace_print_cmd(name, arglist, linenum, # it's at the end of an environment, and thus is needed to # protect the following content print_replacement += '\\space{}' - after = sub('^\s+', '', after) + after = sub(r'^\s+', '', after) forced_double_space_list.append((name, linenum)) else: - if bool(match('\s+\S', after)): + if bool(match(r'\s+\S', after)): # If the following line starts with whitespace, replace it # with a newline, to protect in the event that the printed # content ended with an end-of-environment command - after = sub('^\s+', '\n', after) + after = sub(r'^\s+', '\n', after) # Issue warnings, if warranted # Warn about `\endinput` - if (r'\endinput' in print_replacement and + if (r'\endinput' in print_replacement and print_replacement.count(r'\endinput') != print_replacement.count(r'\string\endinput')): print('* DePythonTeX warning:') print(' "\\endinput" was present in printed content near line ' + str(linenum)) @@ -431,7 +441,7 @@ def replace_print_cmd(name, arglist, linenum, # Warn if it looks like there are active `%` that could comment # out part of the original document. We only need to check the # last line of printed content, because only there could - # percent characters escape from their original confines within + # percent characters escape from their original confines within # `\input`, and comment out part of the document. if print_replacement.endswith('\n'): if print_replacement.count('\n') > 1: @@ -461,42 +471,42 @@ def replace_print_cmd(name, arglist, linenum, print(' If it should have adjusted the spacing of printed content') print(' you should double-check the spacing') return (print_replacement, after) - + def replace_print_env(name, arglist, linenum, print_replacement, print_replacement_mode, source, after): ''' Typeset printed content from an environment. - + It is only ever called if there is indeed printed content to typeset. - + This should be similar to replace_print_cmd(). The main difference is - that the environment context typically ends with a newline, so + that the environment context typically ends with a newline, so substitution has to be a little different to ensure that spacing after the environment isn't modified. - + Args: name (str): name of the environment - arglist (list, of str/None): all arguments given to the original + arglist (list, of str/None): all arguments given to the original environment - linenum (int): line number in the original TeX document where the + linenum (int): line number in the original TeX document where the environment began print_replacement (str): printed content, read directly from file into a single string - print_replacement_mode (str/None): mode in which the replacement is - to be typeset; raw/None (as TeX), inlineverb (or v) (as inline), + print_replacement_mode (str/None): mode in which the replacement is + to be typeset; raw/None (as TeX), inlineverb (or v) (as inline), or verb (as environment) source (str/None): source of the replacement content after (str): text immediately following the command; important in some situations, because spacing can depend on what's next Returns: (replacement, after) (tuple, of str) - + #### The inlineverb and verb modes should work, but haven't been tested since there are currently no environments that use them; they are only - used by `\printpythontex`, which is a command. - ''' + used by `\\printpythontex`, which is a command. + ''' if print_replacement_mode == 'verb': if print_replacement.count('\n') > 1: print('* DePythonTeX error:') @@ -504,119 +514,119 @@ def replace_print_env(name, arglist, linenum, print(' This is not possible in inline verbatim mode') sys.exit(1) print_replacement = print_replacement.rstrip('\n') - for delim in ('|', '/', '`', '!', '&', '#', '@', ':', '%', '~', '$', + for delim in ('|', '/', '`', '!', '&', '#', '@', ':', '%', '~', '$', '=', '+', '-', '^', '_', '?', ';'): if delim not in print_replacement: break print_replacement = r'\verb' + delim + print_replacement + delim - if not bool(match('[ \t]+\S', after)): - # If there is text on the same line as the end of the - # environment, we're fine (this is unusual). Otherwise, + if not bool(match(r'[ \t]+\S', after)): + # If there is text on the same line as the end of the + # environment, we're fine (this is unusual). Otherwise, # we need to toss the newline at the end of the environment - # and gobble leading spaces. Leading spaces need to be - # gobbled because previously they were at the beginning of a + # and gobble leading spaces. Leading spaces need to be + # gobbled because previously they were at the beginning of a # line, where they would have been discarded. - if not bool(match('\s*$', after)): - after = sub('^\s*?\n\s*', '', after) + if not bool(match(r'\s*$', after)): + after = sub(r'^\s*?\n\s*', '', after) elif print_replacement_mode == 'verbatim': - if bool(match('\s*?\n', after)): + if bool(match(r'\s*?\n', after)): # Usually, we would end the verbatim environment with a newline. - # This is fine if there is content in `after` before the next + # This is fine if there is content in `after` before the next # newline---in fact, it's desirable, because the verbatim package - # doesn't allow for content on the same line as the end of the - # environment. But if `after` is an empty line, then adding a + # doesn't allow for content on the same line as the end of the + # environment. But if `after` is an empty line, then adding a # newline will throw off spacing and must be avoided print_replacement = '\\begin{verbatim}\n' + print_replacement + '\\end{verbatim}' else: print_replacement = '\\begin{verbatim}\n' + print_replacement + '\\end{verbatim}\n' else: - # When printed content is included as LaTeX code, we have to be - # particularly careful to ensure that the content produces the same - # output when substituted as when brought in by `\input`. In - # particular, `\input` strips newlines from each line of content and + # When printed content is included as LaTeX code, we have to be + # particularly careful to ensure that the content produces the same + # output when substituted as when brought in by `\input`. In + # particular, `\input` strips newlines from each line of content and # adds a space at the end of each line. This space is inside the - # `\input`, so it will not merge with following spaces. So when we - # substitute the content, sometimes we need to replace the final + # `\input`, so it will not merge with following spaces. So when we + # substitute the content, sometimes we need to replace the final # newline with a space that cannot be gobbled. # - # It gets more complicated. This final space is often not - # desirable. It can be prevented by either printing an `\endinput` - # command, to terminate the `\input`, or printing a percent + # It gets more complicated. This final space is often not + # desirable. It can be prevented by either printing an `\endinput` + # command, to terminate the `\input`, or printing a percent # character % in the last line of the content, which comments out the - # final newline. So we must check for `\endinput` anywhere in - # printed content, and % in the final line, and remove any content - # after them. It's also possible that the print is followed by + # final newline. So we must check for `\endinput` anywhere in + # printed content, and % in the final line, and remove any content + # after them. It's also possible that the print is followed by # an `\unskip` that eats the space, so we need to check for that too. - if (print_replacement.endswith('\\endinput\n') and + if (print_replacement.endswith('\\endinput\n') and not print_replacement.endswith('\\string\\endinput\n')): - # If `\endinput` is present, everything from it on should be + # If `\endinput` is present, everything from it on should be # discarded, unless the `\endinput` is not actually a command - # but rather a typeset name (for example, `\string\endinput` or - # `\verb|\endinput|`). It's impossible to check for all cases in - # which `\endinput` is not a command (at least, without actually - # using LaTeX), and even checking for most of them would require - # a good bit of parsing. We assume that `\endinput`, as a - # command, will only ever occur at the immediate end of the - # printed content. Later, we issue a warning in case it appears + # but rather a typeset name (for example, `\string\endinput` or + # `\verb|\endinput|`). It's impossible to check for all cases in + # which `\endinput` is not a command (at least, without actually + # using LaTeX), and even checking for most of them would require + # a good bit of parsing. We assume that `\endinput`, as a + # command, will only ever occur at the immediate end of the + # printed content. Later, we issue a warning in case it appears # anywhere else. print_replacement = print_replacement.rsplit(r'\endinput', 1)[0] - if not bool(match('[ \t]+\S', after)): - # If there is text on the same line as the end of the - # environment, we're fine (this is unusual). Otherwise, + if not bool(match(r'[ \t]+\S', after)): + # If there is text on the same line as the end of the + # environment, we're fine (this is unusual). Otherwise, # we need to toss the newline at the end of the environment - # and gobble leading spaces. Leading spaces need to be - # gobbled because previously they were at the beginning of a + # and gobble leading spaces. Leading spaces need to be + # gobbled because previously they were at the beginning of a # line, where they would have been discarded. - if not bool(match('\s*$', after)): - after = sub('^\s*?\n\s*', '', after) - elif (print_replacement.endswith('%\n') and - not print_replacement.endswith('\\%\n') and + if not bool(match(r'\s*$', after)): + after = sub(r'^\s*?\n\s*', '', after) + elif (print_replacement.endswith('%\n') and + not print_replacement.endswith('\\%\n') and not print_replacement.endswith('\\string%\n')): # Perform an analogous check for a terminating percent characer %. - # This case would be a bit easier to parse fully, since a percent - # that comments out the last newline would have to be in the - # final line of the replacement. But it would still be - # very difficult to perform a complete check. Later, we issue a - # warning if there is reason to think that a percent character + # This case would be a bit easier to parse fully, since a percent + # that comments out the last newline would have to be in the + # final line of the replacement. But it would still be + # very difficult to perform a complete check. Later, we issue a + # warning if there is reason to think that a percent character # was active in the last line. print_replacement = print_replacement.rsplit(r'%', 1)[0] - if not bool(match('[ \t]+\S', after)): - # If there is text on the same line as the end of the - # environment, we're fine (this is unusual). Otherwise, + if not bool(match(r'[ \t]+\S', after)): + # If there is text on the same line as the end of the + # environment, we're fine (this is unusual). Otherwise, # we need to toss the newline at the end of the environment - # and gobble leading spaces. Leading spaces need to be - # gobbled because previously they were at the beginning of a + # and gobble leading spaces. Leading spaces need to be + # gobbled because previously they were at the beginning of a # line, where they would have been discarded. - if not bool(match('\s*$', after)): - after = sub('^\s*?\n\s*', '', after) + if not bool(match(r'\s*$', after)): + after = sub(r'^\s*?\n\s*', '', after) else: - # By default, LaTeX strips newlines and adds a space at the end - # of each line of content that is brought in by `\input`. This - # may or may not be desirable, but we replicate the effect here - # for consistency with the original document. We use `\space{}` - # because plain `\space` would gobble a following space, which + # By default, LaTeX strips newlines and adds a space at the end + # of each line of content that is brought in by `\input`. This + # may or may not be desirable, but we replicate the effect here + # for consistency with the original document. We use `\space{}` + # because plain `\space` would gobble a following space, which # isn't consistent with the `\input` behavior being replicated. if bool(match(r'\s*\\unskip\s+\S', after)): - # If there's an `\unskip`, fix the spacing and remove the + # If there's an `\unskip`, fix the spacing and remove the # `\unskip` print_replacement = print_replacement.rstrip(' \t\n') after = sub(r'^\s*\\unskip\s+', '', after) - elif bool(match('[ \t]+\S', after)): - # If the next character after the end of the environment is + elif bool(match(r'[ \t]+\S', after)): + # If the next character after the end of the environment is # not whitespace (usually not allowed), we can just leave - # the `\n` in printed content, and it will yield a space. - # So we need do nothing. But if there is text on that line + # the `\n` in printed content, and it will yield a space. + # So we need do nothing. But if there is text on that line # we need `\space{}`. - after = sub('^\s+', '\\space', after) + after = sub(r'^\s+', '\\space', after) forced_double_space_list.append((name, linenum)) else: # If the line at the end of the environment is blank, - # we can just discard it and keep the newline at the end of + # we can just discard it and keep the newline at the end of # the printed content; the newline gives us the needed space after = after.split('\n', 1)[1] # Issue warnings, if warranted # Warn about `\endinput` - if (r'\endinput' in print_replacement and + if (r'\endinput' in print_replacement and print_replacement.count(r'\endinput') != print_replacement.count(r'\string\endinput')): print('* DePythonTeX warning:') print(' "\\endinput" was present in printed content near line ' + str(linenum)) @@ -626,7 +636,7 @@ def replace_print_env(name, arglist, linenum, # Warn if it looks like there are active `%` that could comment # out part of the original document. We only need to check the # last line of printed content, because only there could - # percent characters escape from their original confines within + # percent characters escape from their original confines within # `\input`, and comment out part of the document. if print_replacement.endswith('\n'): if print_replacement.count('\n') > 1: @@ -663,9 +673,9 @@ def replace_print_env(name, arglist, linenum, # Deal with argv # Parse argv parser = argparse.ArgumentParser() -parser.add_argument('--version', action='version', - version='DePythonTeX {0}'.format(version)) -parser.add_argument('--encoding', default='utf-8', +parser.add_argument('--version', action='version', + version='DePythonTeX {0}'.format(__version__)) +parser.add_argument('--encoding', default='utf-8', help='encoding for all text files (see codecs module for encodings)') parser.add_argument('--overwrite', default=False, action='store_true', help='overwrite existing output, if it exists (off by default)') @@ -678,10 +688,10 @@ def replace_print_env(name, arglist, linenum, help='line of commands to add to output preamble') parser.add_argument('--graphicspath', default=False, action='store_true', help=r'Add the outputdir to the graphics path, by modifying an existing \graphicspath command or adding one.') +parser.add_argument('-o', '--output', default=None, + help='output file') parser.add_argument('TEXNAME', help='LaTeX file') -parser.add_argument('OUTFILE', nargs='?', default=None, - help='output file; by default, . is converted into depythontex_.') args = parser.parse_args() # Process argv @@ -715,8 +725,9 @@ def replace_print_env(name, arglist, linenum, # Let the user know things have started -print('This is DePythonTeX {0}'.format(version)) -sys.stdout.flush() +if args.output is not None: + print('This is DePythonTeX {0}'.format(__version__)) + sys.stdout.flush() @@ -736,23 +747,20 @@ def replace_print_env(name, arglist, linenum, print(' Could not locate file "' + texfile_name + '"') sys.exit(1) # Make sure we have a valid outfile -if args.OUTFILE is None: - p, f_name = os.path.split(texfile_name) - outfile_name = os.path.join(p, 'depythontex_' + f_name) -else: - outfile_name = os.path.expanduser(os.path.normcase(args.OUTFILE)) -if not args.overwrite and os.path.isfile(outfile_name): - print('* DePythonTeX warning:') - print(' Output file "' + outfile_name + '" already exists') - ans = input(' Do you want to overwrite this file? [y,n]\n ') - if ans != 'y': - sys.exit(1) -# Make sure the .depytx file exists +if args.output is not None: + outfile_name = os.path.expanduser(os.path.normcase(args.output)) + if not args.overwrite and os.path.isfile(outfile_name): + print('* DePythonTeX warning:') + print(' Output file "' + outfile_name + '" already exists') + ans = input(' Do you want to overwrite this file? [y,n]\n ') + if ans != 'y': + sys.exit(1) +# Make sure the .depytx file exists depytxfile_name = texfile_name.rsplit('.')[0] + '.depytx' if not os.path.isfile(depytxfile_name): print('* DePythonTeX error:') print(' Could not find DePythonTeX auxiliary file "' + depytxfile_name + '"') - print(' Use package option depythontex to creat it') + print(' Use package option depythontex to create it') sys.exit(1) @@ -760,7 +768,7 @@ def replace_print_env(name, arglist, linenum, # Start opening files and loading data # Read in the LaTeX file -# We read into a list with an empty first entry, so that we don't have to +# We read into a list with an empty first entry, so that we don't have to # worry about zero indexing when comparing list index to file line number f = open(texfile_name, 'r', encoding=encoding) tex = [''] @@ -784,33 +792,34 @@ def replace_print_env(name, arglist, linenum, depytx[n] = '' n -= 1 # Check .depytx version to make sure it is compatible -if settings['version'] != version: +if settings['version'] != __version__: print('* DePythonTeX warning:') print(' Version mismatch with DePythonTeX auxiliary file') print(' Do a complete compile cycle to update the auxiliary file') print(' Attempting to proceed') # Go ahead and open the outfile, even though we don't need it until the end -# This lets us change working directories for convenience without worrying +# This lets us change working directories for convenience without worrying # about having to modify the outfile path -outfile = open(outfile_name, 'w', encoding=encoding) +if args.output is not None: + outfile = open(outfile_name, 'w', encoding=encoding) # Change working directory to the document directory -# Technically, we could get by without this, but that would require a lot of -# path modification. This way, we can just use all paths straight out of the +# Technically, we could get by without this, but that would require a lot of +# path modification. This way, we can just use all paths straight out of the # .depytx without any modification, which is much simpler and less error-prone. if os.path.split(texfile_name)[0] != '': os.chdir(os.path.split(texfile_name)[0]) - + # Open and process the file of macros # Read in the macros -if os.path.isfile(settings['macrofile']): - f = open(settings['macrofile'], 'r', encoding=encoding) +if os.path.isfile(os.path.expanduser(os.path.normcase(settings['macrofile']))): + f = open(os.path.expanduser(os.path.normcase(settings['macrofile'])), 'r', encoding=encoding) macros = f.readlines() f.close() else: @@ -821,7 +830,7 @@ def replace_print_env(name, arglist, linenum, sys.exit(1) # Create a dict for storing macros macrodict = defaultdict(list) -# Create variables for keeping track of whether we're inside a macro or +# Create variables for keeping track of whether we're inside a macro or # environment # These must exist before we begin processing inside_macro = False @@ -833,13 +842,13 @@ def replace_print_env(name, arglist, linenum, if inside_macro: # If we're in a macro, look for the end-of-macro command if r'\endpytx@SVMCR' in line: - # If the current line contains the end-of-macro command, split - # off any content that comes before it. Also reset + # If the current line contains the end-of-macro command, split + # off any content that comes before it. Also reset # `inside_macro`. macrodict[current_macro].append(line.rsplit(r'\endpytx@SVMCR', 1)[0]) inside_macro = False else: - # If the current line doesn't end the macro, we add the whole + # If the current line doesn't end the macro, we add the whole # line to the macro dict macrodict[current_macro].append(line) elif inside_environment: @@ -847,7 +856,7 @@ def replace_print_env(name, arglist, linenum, # If the environment is ending, we reset inside_environment inside_environment = False else: - # If we're still in the environment, add the current line to the + # If we're still in the environment, add the current line to the # macro dict macrodict[current_macro].append(line) else: @@ -856,10 +865,10 @@ def replace_print_env(name, arglist, linenum, # file to increase readability). Once we've determined which one, # we need to get its name and extract any content. if line.startswith(r'\begin{'): - # Any \begin will indicate a use of fancyvrb to save verbatim - # content, since that is the only time an environment is used in + # Any \begin will indicate a use of fancyvrb to save verbatim + # content, since that is the only time an environment is used in # the macro file. All other content is saved in a standard macro. - # We extract the name of the macro in which the verbatim content + # We extract the name of the macro in which the verbatim content # is saved. current_macro = line.rsplit('{', 1)[1].rstrip('}\n') inside_environment = True @@ -885,8 +894,8 @@ def replace_print_env(name, arglist, linenum, # Start at 1, since the first entry in the tex list is `''` texlinenum = 1 # Create a variable for storing the current line(s) we are processing. -# This contains all lines from immediately after the last successfully -# processed line up to and including texlinenum. We may have to process +# This contains all lines from immediately after the last successfully +# processed line up to and including texlinenum. We may have to process # multiple lines at once if a macro is split over multiple lines, etc. texcontent = tex[texlinenum] # Create a list for storing processed content. @@ -899,21 +908,21 @@ def replace_print_env(name, arglist, linenum, depy_type, depy_name, depy_args, depy_typeset, depy_linenum, depy_lexer = depytxcontent.split(':') if depy_lexer == '': depy_lexer = None - + # Do a quick check on validity of info # #### Eventually add 'cp' and 'pc' if not (depy_type in ('cmd', 'env') and all([letter in ('o', 'm', 'v', 'n', '|') for letter in depy_args]) and ('|' not in depy_args or (depy_args.count('|') == 1 and depy_args.endswith('|'))) and - depy_typeset in ('c', 'p', 'n')): + depy_typeset in ('c', 'p', 'n')): print('* PythonTeX error:') print(' Invalid \\Depythontex string for operation on line ' + str(depy_linenum)) print(' The offending string was ' + depytxcontent) sys.exit(1) - # If depy_args contains a `|` to indicate `\obeylines`, strip it and - # store in a variable. Create a bool to keep track of obeylines - # status, which governs whether we can look on the next line for - # arguments. (If obeylines is active, a newline terminates the + # If depy_args contains a `|` to indicate `\obeylines`, strip it and + # store in a variable. Create a bool to keep track of obeylines + # status, which governs whether we can look on the next line for + # arguments. (If obeylines is active, a newline terminates the # argument search.) if depy_args.endswith('|'): obeylines = True @@ -923,8 +932,8 @@ def replace_print_env(name, arglist, linenum, # Get the line number as an integer # We don't have to adjust for zero indexing in tex depy_linenum = int(depy_linenum) - - + + # Check for information passed from LaTeX # This will be extra listings information, or replacements to plug in code_replacement = None @@ -974,51 +983,51 @@ def replace_print_env(name, arglist, linenum, elif nextdepytxline.startswith('FILE:'): source = 'file' try: - typeset, f_name = nextdepytxline.rstrip('\n').split(':', 2)[1:] + typeset, f_name = nextdepytxline.rstrip('\n').split(':', 2)[1:] except: print('* DePythonTeX error:') print(' Improperly formatted file information on line ' + str(depy_linenum)) print(' The file information was "' + nextdepytxline + '"') sys.exit(1) - # Files that are brought in have an optional mode that + # Files that are brought in have an optional mode that # determines if they need special handling (for example, verbatim) if ':mode=' in f_name: f_name, mode = f_name.split(':mode=') else: mode = None - f = open(f_name, 'r', encoding=encoding) + f = open(os.path.expanduser(os.path.normcase(f_name)), 'r', encoding=encoding) replacement = f.read() f.close() if typeset == 'c': code_replacement_mode = mode if depy_type == 'cmd' and code_replacement_mode != 'verbatim': # Usually, code from commands is typeset with commands - # and code from environments is typeset in - # environments. The except is code from commands + # and code from environments is typeset in + # environments. The except is code from commands # that bring in external files, like `\inputpygments` code_replacement = replacement else: # If we're replacing an environment of code with a # file, then we lose the newline at the beginning # of the environment, and need to get it back. - code_replacement = '\n' + replacement + code_replacement = '\n' + replacement elif typeset == 'p': print_replacement_mode = mode - print_replacement = replacement + print_replacement = replacement else: print('* DePythonTeX error:') print(' Improper typesetting information for file information on line ' + str(depy_linenum)) print(' The file information was "' + nextdepytxline + '"') sys.exit(1) - # Increment the line in depytx to check for more information + # Increment the line in depytx to check for more information # from LaTeX scan_ahead_line += 1 if scan_ahead_line == len(depytx): break else: nextdepytxline = depytx[scan_ahead_line] - - + + # If the line we're looking for is within the range currently held by # texcontent, do nothing. Otherwise, transfer content from tex # to texout until we get to the line of tex that we're looking for @@ -1030,12 +1039,12 @@ def replace_print_env(name, arglist, linenum, texlinenum += 1 texcontent = tex[texlinenum] - + # Deal with arguments - # All arguments are parsed and stored in a list variables, even if - # they are not used, for completeness; this makes it easy to add + # All arguments are parsed and stored in a list variables, even if + # they are not used, for completeness; this makes it easy to add # functionality - # Start by splitting the current line into what comes before the + # Start by splitting the current line into what comes before the # command or environment, and what is after it if depy_type == 'cmd': try: @@ -1051,10 +1060,10 @@ def replace_print_env(name, arglist, linenum, print('* DePythonTeX error:') print(' Could not find environment "' + depy_name + '" on line ' + str(depy_linenum)) sys.exit(1) - # We won't need the content from before the command or environment + # We won't need the content from before the command or environment # again, so we go ahead and store it texout.append(before) - + # Parse the arguments # Create a list for storing the recovered arguments arglist = list() @@ -1066,16 +1075,16 @@ def replace_print_env(name, arglist, linenum, # Account for possible line breaks before end of arg while ']' not in after: texlinenum += 1 - after += tex[texlinenum] + after += tex[texlinenum] optarg, after = after[1:].split(']', 1) else: if obeylines: # Take into account possible whitespace before arg - if bool(match('[ \t]*\[', after)): + if bool(match(r'[ \t]*\[', after)): after = after.split('[', 1)[1] while ']' not in after: texlinenum += 1 - after += tex[texlinenum] + after += tex[texlinenum] optarg, after = after.split(']', 1) else: optarg = None @@ -1085,25 +1094,25 @@ def replace_print_env(name, arglist, linenum, after = sub('^[ \t]*', '', after) else: # Allow peeking ahead a line for the argument - if bool(match('\s*$', after)) and after.count('\n') < 2: + if bool(match(r'\s*$', after)) and after.count('\n') < 2: texlinenum += 1 after += tex[texlinenum] # Take into account possible whitespace before arg - if bool(match('\s*\[', after)): + if bool(match(r'\s*\[', after)): after = after.split('[', 1)[1] while ']' not in after: texlinenum += 1 - after += tex[texlinenum] + after += tex[texlinenum] optarg, after = after.split(']', 1) else: optarg = None # Account for eating whitespace afterward, if arg not found if argindex == len(depy_args) - 1: - if bool(match('\s*$', after)) and after.count('\n') < 2: + if bool(match(r'\s*$', after)) and after.count('\n') < 2: texlinenum += 1 after += tex[texlinenum] - if not bool(match('\s*$', after)): - after = sub('^\s*', '', after) + if not bool(match(r'\s*$', after)): + after = sub(r'^\s*', '', after) arglist.append(optarg) elif arg == 'm': # Account for possible line breaks or spaces before arg @@ -1112,7 +1121,7 @@ def replace_print_env(name, arglist, linenum, else: if obeylines: # Account for possible leading whitespace - if bool(match('[ \t\f\v]*\{', after)): + if bool(match(r'[ \t\f\v]*\{', after)): after = after.split('{', 1)[1] else: print('* DePythonTeX error:') @@ -1120,10 +1129,10 @@ def replace_print_env(name, arglist, linenum, sys.exit(1) else: # Peek ahead a line if needed - if bool(match('\s*$', after)) and after.count('\n') < 2: + if bool(match(r'\s*$', after)) and after.count('\n') < 2: texlinenum += 1 after += tex[texlinenum] - if bool(match('\s*\{', after)): + if bool(match(r'\s*\{', after)): after = after.split('{', 1)[1] else: print('* DePythonTeX error:') @@ -1132,8 +1141,8 @@ def replace_print_env(name, arglist, linenum, # Go through the argument character by character to find the # closing brace. # If possible, use a very simple approach - if (r'\{' not in after and r'\}' not in after and - r'\string' not in after and + if (r'\{' not in after and r'\}' not in after and + r'\string' not in after and after.count('{') + 1 == after.count('}')): pos = 0 lbraces = 1 @@ -1149,7 +1158,7 @@ def replace_print_env(name, arglist, linenum, if pos == len(after): texlinenum += 1 after += tex[texlinenum] - # If a simple parsing approach won't work, parse in much + # If a simple parsing approach won't work, parse in much # greater depth else: pos = 0 @@ -1174,7 +1183,7 @@ def replace_print_env(name, arglist, linenum, # First, jump ahead to after `\string` pos += 7 #+= len(r'\string') # See if `\string` is followed by a regular macro - # If so, jump past it; otherwise, figure out if a + # If so, jump past it; otherwise, figure out if a # single-character macro, or just a single character, is next, # and jump past it standard_macro = match(r'\\[a-zA-Z]+', line[pos:]) @@ -1185,7 +1194,7 @@ def replace_print_env(name, arglist, linenum, else: pos += 1 elif line[pos] == '\\': - # If the current position is a backslash, figure out what + # If the current position is a backslash, figure out what # macro is used, and jump past it # The macro must either be a standard alphabetic macro, # or a single-character macro @@ -1230,8 +1239,8 @@ def replace_print_env(name, arglist, linenum, after += tex[texlinenum] mainarg, after = after[1:].split(delim, 1) arglist.append(mainarg) - - + + # Do substitution, depending on what is required # Need a variable for processed content to be added to texout processed = None @@ -1242,8 +1251,8 @@ def replace_print_env(name, arglist, linenum, if after.count('\n') < 2: texlinenum += 1 after += tex[texlinenum] - processed, texcontent = replace_code_cmd(depy_name, arglist, - depy_linenum, + processed, texcontent = replace_code_cmd(depy_name, arglist, + depy_linenum, code_replacement, code_replacement_mode, after, depy_lexer, @@ -1282,7 +1291,7 @@ def replace_print_env(name, arglist, linenum, depy_linenum, code_replacement, code_replacement_mode, - after, depy_lexer, + after, depy_lexer, firstnumber) elif depy_typeset == 'p' and print_replacement is not None: if depy_type == 'cmd': @@ -1291,7 +1300,7 @@ def replace_print_env(name, arglist, linenum, if after.count('\n') < 2: texlinenum += 1 after += tex[texlinenum] - processed, texcontent = replace_print_cmd(depy_name, arglist, + processed, texcontent = replace_print_cmd(depy_name, arglist, depy_linenum, print_replacement, print_replacement_mode, @@ -1311,9 +1320,9 @@ def replace_print_env(name, arglist, linenum, if after.count('\n') < 2: texlinenum += 1 after += tex[texlinenum] - processed, texcontent = replace_print_env(depy_name, arglist, + processed, texcontent = replace_print_env(depy_name, arglist, depy_linenum, - print_replacement, + print_replacement, print_replacement_mode, source, after) @@ -1329,7 +1338,7 @@ def replace_print_env(name, arglist, linenum, after = tex[texlinenum] break after = after.split(end_environment, 1)[1] - if bool(match('\s*\n', after)): + if bool(match(r'\s*\n', after)): # If the line following `after` is whitespace, it should # be stripped, since most environments throw away # anything after the end of the environment @@ -1337,8 +1346,8 @@ def replace_print_env(name, arglist, linenum, texcontent = after # #### Once it's supported on the TeX side, need to add support for # pc and cp - - + + # Store any processed content if processed is not None: texout.append(processed) @@ -1360,11 +1369,15 @@ def replace_print_env(name, arglist, linenum, if startline == n: if bool(search(r'\\usepackage(?:\[.*?\]){0,1}\{pythontex\}', line)): texout[n] = sub(r'\\usepackage(?:\[.*?\]){0,1}\{pythontex\}', '', line) + if texout[n].isspace(): + texout[n] = '' break else: content = ''.join(texout[startline:n+1]) if bool(search(r'(?s)\\usepackage(?:\[.*?\]\s*){0,1}\{pythontex\}', content)): replacement = sub(r'(?s)\\usepackage(?:\[.*?\]\s*){0,1}\{pythontex\}', '', content) + if replacement.isspace(): + replacement = '' texout[startline] = replacement for l in range(startline+1, n+1): texout[l] = '' @@ -1376,7 +1389,7 @@ def replace_print_env(name, arglist, linenum, # Take care of graphicspath if args.graphicspath and settings['graphicx']: for n, line in enumerate(texout): - if '\\graphicspath' in line and not bool(match('\s*%', line)): + if '\\graphicspath' in line and not bool(match(r'\s*%', line)): texout[n] = line.replace('\\graphicspath{', '\\graphicspath{{' + settings['outputdir'] +'/}') break elif line.startswith(r'\begin{document}'): @@ -1399,6 +1412,16 @@ def replace_print_env(name, arglist, linenum, # Write output -for line in texout: - outfile.write(line) -outfile.close() +if args.output is not None: + for line in texout: + outfile.write(line) + outfile.close() +else: + if sys.version_info[0] == 2: + sys.stdout = codecs.getwriter(encoding)(sys.stdout, 'strict') + sys.stderr = codecs.getwriter(encoding)(sys.stderr, 'strict') + else: + sys.stdout = codecs.getwriter(encoding)(sys.stdout.buffer, 'strict') + sys.stderr = codecs.getwriter(encoding)(sys.stderr.buffer, 'strict') + for line in texout: + sys.stdout.write(line) diff --git a/pythontex/pythontex.dtx b/pythontex/pythontex.dtx index a7da101..be9c287 100644 --- a/pythontex/pythontex.dtx +++ b/pythontex/pythontex.dtx @@ -1,6 +1,6 @@ % \iffalse meta-comment % -% Copyright (C) 2012-2013 by Geoffrey M. Poore +% Copyright (C) 2012-2026 by Geoffrey M. Poore % --------------------------------------------------------------------------- % This work may be distributed and/or modified under the % conditions of the LaTeX Project Public License, either version 1.3 @@ -26,7 +26,7 @@ %\NeedsTeXFormat{LaTeX2e}[1999/12/01] %\ProvidesPackage{pythontex} %<*package> - [2013/08/26 Version~0.12] + [2026/02/15 v0.19 execute and typeset Python code and other languages] % % %<*driver> @@ -34,41 +34,172 @@ \makeatletter \usepackage[T1]{fontenc} \usepackage[utf8]{inputenc} +\usepackage{lmodern} +\usepackage{microtype} + \usepackage{textcomp} \usepackage[svgnames]{xcolor} \usepackage{environ} \usepackage{enumitem} +\usepackage{fvextra} +% The typesetting for macrocode doesn't use \@noligs, which upquote modifies. +% So apply the upquote fix to \verbatim@nolig@list as well, which is in macrocode. +\begingroup +\catcode`'=\active +\catcode``=\active +\g@addto@macro\verbatim@nolig@list{% + \let'\textquotesingle + \let`\textasciigrave + \ifx\encodingdefault\upquote@OTone + \ifx\ttdefault\upquote@cmtt + \def'{\char13 }% + \def`{\char18 }% + \fi\fi} +\endgroup \usepackage{pythontex} \usepackage{hyperref} -\hypersetup{colorlinks=true,urlcolor=Green,linkcolor=blue} +\hypersetup{ + pdftitle=The PythonTeX package, + pdfauthor=Geoffrey M. Poore, + pdfsubject={PythonTeX LaTeX package manual}, + colorlinks=true, + allcolors=ForestGreen, +} +\usepackage{cleveref} + \newcommand{\pytxtodo}[1]{} %\newcommand{\pytxtodo}[1]{\textcolor{red}{TO~DO: \scantokens{#1}}} \newcommand{\changestext}{} \NewEnviron{changelog}[2]{% - \g@addto@macro\changestext{\item[#1] (#2) \begin{itemize}}% + \g@addto@macro\changestext{\subsection*{\textbf{#1} (#2)}}% \expandafter\g@addto@macro\expandafter\changestext\expandafter{\BODY}% - \g@addto@macro\changestext{\end{itemize}}% + \g@addto@macro\changestext{}% } \newcommand{\PrintChangelog}{% \addcontentsline{toc}{section}{Version History} \section*{Version History}% - \begin{description}% \changestext - \end{description}% } -\EnableCrossrefs -\CodelineIndex + +% Create a short verbatim pipe that handles quotation marks properly +\begingroup +\catcode`\|=\active +\gdef\pipe@active@verbatim{% + \begingroup + \let\do\@makeother\dospecials + \catcode`\|=\active + \catcode`\`=\active + \catcode`\'=\active + \catcode`\<=\active + \catcode`\>=\active + \catcode`\-=\active + \catcode`\,=\active + \catcode`\ =\active + \pipe@active@verbatim@i} +\gdef\pipe@active@verbatim@i#1|{% + \endgroup + \begingroup + \def\FV@SV@pipe@active@verbatim{% + \FV@Gobble + \expandafter\FV@ProcessLine\expandafter{#1}}% + %\let\FV@BeginVBox\relax + %\let\FV@EndVBox\relax + %\def\FV@BProcessLine##1{\FancyVerbFormatLine{##1}}% + \BUseVerbatim{pipe@active@verbatim}% + \endgroup} +\AtBeginDocument{\let|\pipe@active@verbatim} +\endgroup + +\renewenvironment{verbatim}% + {\VerbatimEnvironment\begin{Verbatim}[gobble=1]}% + {\end{Verbatim}} + + +\def\MacroFont{% + \fontencoding\encodingdefault% + \fontfamily\ttdefault% + \fontseries\mddefault% + \fontshape\updefault% + \small} + +\def\PrintMacroName#1{{\strut\MacroFont\color{DarkGreen}\footnotesize\string #1\ }} + +\def\PrintDescribeMacro#1{\parbox[t]{1in}{\strut\MacroFont\textcolor{DarkGreen}{\string #1\ }}} +\let\PrintDescribeEnv\PrintDescribeMacro +%\let\PrintMacroName\PrintDescribeMacro +\let\PrintEnvName\PrintDescribeEnv + +\def\theCodelineNo{\textcolor{DarkGreen}{\sffamily\scriptsize{\arabic{CodelineNo}}}} + + +\renewenvironment{macro}{\macro@custom@arg}{} +\def\macro@custom@arg{% + \begingroup\makeatletter\macro@custom@arg@i} +\def\macro@custom@arg@i#1{% + \makeatother + \par\noindent + \ifstrempty{#1}% + {~\par}% + {\ttfamily\color{DarkGreen}\hspace*{-0.5in}% + \macro@custom@arg@split#1,\FV@Sentinel\par}% + \endgroup} +\def\macro@custom@arg@spacegobble#1{#1} +\def\macro@custom@arg@split#1,#2\FV@Sentinel{% + \expandafter\string\macro@custom@arg@spacegobble#1% + \if\relax\detokenize{#2}\relax + \expandafter\@firstoftwo + \else + \expandafter\@secondoftwo + \fi + {}{\\\hspace*{-0.5in}\macro@custom@arg@split#2\FV@Sentinel}} + +\renewenvironment{environment}{\env@custom@arg}{} +\def\env@custom@arg{% + \begingroup\makeatletter\env@custom@arg@i} +\def\env@custom@arg@i#1{% + \makeatother + \ifstrempty{#1}% + {~\par}% + {\ttfamily\color{DarkGreen}\hspace*{-0.5in}% + \env@custom@arg@split#1,\FV@Sentinel\par}% + \endgroup} +\def\env@custom@arg@spacegobble#1{#1} +\def\env@custom@arg@split#1,#2\FV@Sentinel{% + \env@custom@arg@spacegobble#1{\textrm{ \textit{(env.)}}}% + \if\relax\detokenize{#2}\relax + \expandafter\@firstoftwo + \else + \expandafter\@secondoftwo + \fi + {}{\\\hspace*{-0.5in}\env@custom@arg@split#2\FV@Sentinel}} + + +\let\orig@footnote\footnote +\renewcommand{\footnote}{% + \begingroup + \let\do\@makeother + \dospecials + \catcode`\{=1 + \catcode`\}=2 + \new@footnote} +\newcommand{\new@footnote}[1]{% + \endgroup + \orig@footnote{\scantokens{#1}}} + + +%\EnableCrossrefs +%\CodelineIndex %\RecordChanges \makeatother \begin{document} \DocInput{pythontex.dtx} %\PrintChanges - \PrintIndex + %\PrintIndex \end{document} % % \fi % -% \CheckSum{2928} +% \CheckSum{3289} % % \CharacterTable % {Upper-case \A\B\C\D\E\F\G\H\I\J\K\L\M\N\O\P\Q\R\S\T\U\V\W\X\Y\Z @@ -88,7 +219,148 @@ % % % +% \begin{changelog}{v0.19}{2026/02/15} +% \begin{itemize} +% \item Fixed backslash escapes in docstrings and regexes for syntax compatibility with Python 3.11+. +% \item \texttt{js} now works as a command family for JavaScript (\#197). +% \item Python console commands and environments are now compatible with Python 3.14 (\#229). +% \item Fixed a bug that could cause non-Windows operating systems to attempt to invoke a nonexistent batch file when the executable associated with a language is not found (\#225). +% \item Replaced doctype \texttt{tex} with \texttt{texminted} for \texttt{Weave.jl} (\#219). +% \item Fixed a bug that resulted in uncaught exceptions when \Verb{\inputpygments} was used with files that do not exist (\#64). +% \item Fixed a bug in tracking the number of Pygments errors (\#206). +% \item Improved error messages (\#207, \#209). Improved documentation for \Verb{--error-exit-code} (\#208). +% \item Several minor improvements to documentation. +% \end{itemize} +% \end{changelog} +% +% +% \begin{changelog}{v0.18}{2021/06/06} +% \begin{itemize} +% \item \texttt{\string\inputpygments} now checks inputted files for modification, so that typeset code will correctly update when the source is changed (\#162). +% \item Julia now uses project flag ``\texttt{-{}-project=@.}'' (\#157, \#158). +% \item Fixed bug in processing Pygments options (\texttt{pygopt}) when a key is used without a value (\#181). +% \item Some error handling for Windows was incompatible with other operating systems: replaced checks for \texttt{WindowsError} with checks for \texttt{OSError} (\#177). +% \item Rust support is now compatible with document and working directory paths that contain spaces (\#167). +% \end{itemize} +% \end{changelog} +% +% +% \begin{changelog}{v0.17}{2019/09/22} +% \begin{itemize} +% \item Pygments syntax highlighting for the Python console (\texttt{pycon} lexer) now uses the \texttt{python3} option, and the default Python lexer is now \texttt{python3} (\#156). +% \item Added support for JavaScript (\#147; thanks to Nathan Carter). +% \item Updated Julia support for Julia versions 0.6 (\#107), and 0.7 and 1.0 (\#126, \#130). +% \item There are now meaningful error messages for the Julia console when Weave.jl is not installed or raises errors (\#131). +% \item \texttt{pythontexcustomcode} and \texttt{\string\pythontexcustomc} now set \texttt{pytex.context} (\#65). +% \item Added support for R. The \texttt{R} family of commands and environments (\texttt{\string\R}, \texttt{\string\Rc}, \texttt{Rcode}, ...) executes code as a script. There is currently no utilities class or equivalent. The \texttt{Rcon} family (\texttt{Rconsole}) executes code to emulate an interactive R session (\#121). +% \item \texttt{fancyvrb} settings from \texttt{\string\setpythontexfv} and console environments now work with Julia and R consoles. +% \item \texttt{pythontexcustomcode} now works with \texttt{juliacon}. There are now proper \texttt{juliaconcode} and \texttt{Rconcode} environments that execute code but typeset nothing, to parallel \texttt{pyconcode} (\#134). +% \item Added support for Perl with the \texttt{perl} and \texttt{pl} families of commands and environments. There is currently no utilities class or equivalent. +% \item Added support for Perl 6 with the \texttt{perlsix} and \texttt{psix} families of commands and environments (\#104). There is currently no utilities class or equivalent. +% \item Updated Rust support by using \texttt{dyn} with traits in utilities object. +% \item Under Windows, capitalization of script paths in \texttt{stderr} is now preserved. +% \item Fixed a bug that prevented the \texttt{sub} environment from working with \texttt{depythontex} (\#155). +% \item Fixed a bug in checking mtime of dependencies to see if they have been modified while \texttt{pythontex} is running. The check failed for dependencies that do not exist or were deleted before \texttt{pythontex} can read them (\#136). +% \end{itemize} +% \end{changelog} +% +% +% \begin{changelog}{v0.16}{2017/07/20} +% \begin{itemize} +% \item Added preliminary console support for Julia (\#98). +% \item Fixed Python console compatibility with Python 3.6 by setting the \texttt{code} module's new \texttt{exitmsg} argument to suppress the exit message (\#100). +% \item Improved Rust support, including tracking of created files and dependencies (\#91). +% \end{itemize} +% \end{changelog} +% +% +% \begin{changelog}{v0.15}{2016/07/21} +% \textbf{New features} +% \begin{itemize} +% \item The \texttt{fvextra} package is now required. This provides line breaking with fine-grained control over break locations, the ability to highlight specific lines or ranges of lines, improved handling of tabs, and several additional features. +% \item Added \texttt{sub} commands and environments (\texttt{\string\pys}, \texttt{pysub}, ...). These commands and environments perform string interpolation on text. Fields delimited by \texttt{!\{...\}} are replaced by the result of evaluating and then printing their content. This works for all families of commands and environments, not just Python. See the documentation for details about field delimiters and escaping. +% \item Added \texttt{rust} and \texttt{rs} families of commands and environments. These provide essentially complete support for Rust, except that \texttt{rstex.formatter()}, \texttt{rstex.before()}, and \texttt{rstex.after()} will likely need additional refinement (\#90). +% \item Added the \texttt{sage} family of commands and environments, which provide support for Sage (\#63). +% \item Added \texttt{bash} family of commands and environments. This provides basic support for bash (no utilities class or equivalent). Bash works with Windows if it is installed. +% \item Improved \texttt{console} compatibility under Linux with Python 3 (\#70). +% \item Counters for default sessions are now created automatically. This prevents counter errors under some circumstances when working with \texttt{\string\includeonly}. +% \item Commands like \texttt{\string\py} can now output verbatim content under LuaTeX. +% \end{itemize} +% \textbf{Bugfixes} +% \begin{itemize} +% \item Fixed a bug that could cause an endless loop when a \texttt{code} command or environment printed a \texttt{code} command or environment of the same family with \texttt{autoprint=true}. +% \end{itemize} +% \end{changelog} +% +% +% \begin{changelog}{v0.14}{2014/07/17} +% \textbf{New features} +% \begin{itemize} +% \item All commands for working with code inline are now robust, via \texttt{etoolbox}'s \texttt{\string\newrobustcmd}. Among other things, this allows commands like \texttt{\string\py} to work in standard captions that have not been redefined to avoid protection issues. +% \item Upgraded \texttt{syncpdb} to v0.2, which provides better list formatting. +% \end{itemize} +% \textbf{Backward-incompatible changes} +% \begin{itemize} +% \item The default working directory is now the main document directory instead of the output directory. Using the output directory was a common source of confusion for new users and was incompatible with plans for future development. Old documents in which the working directory was not specified will continue to use the output directory, but PythonTeX will print an upgrade message; new documents will use the new setting. The output directory may be selected as the working directory manually, or with the shorthand ``\texttt{\string\setpythontexworkingdir\{\}}''. +% \item Standardized version numbering by removing the ``v'' prefix from the stored version numbers in Python variables and LaTeX macros. Standardized the PythonTeX scripts by renaming \texttt{version} to \texttt{\_\_version\_\_}. +% \end{itemize} +% \end{changelog} +% +% \begin{changelog}{v0.13}{2014/07/14} +% \textbf{New features} +% \begin{itemize} +% \item Added \texttt{-{}-interactive} command-line option. This runs a single session in interactive mode, allowing user input. Among other things, this is useful when working with debuggers. +% \item Added \texttt{-{}-debug} command-line option. This runs a single session with the default debugger in interactive mode. Currently, only standard (non-console) Python sessions are supported. The default Python debugger is the new \texttt{syncpdb}, which wraps \texttt{pdb} and synchronizes code line numbers with document line numbers. All \texttt{pdb} commands that take a line number or filename:lineno as an argument will refer to document files and line numbers when the argument has a percent symbol (\texttt{\%}) as a prefix. For example, \texttt{list \%50} lists code that came from around line 50 in the document. The \texttt{-{}-debug} option will support other languages and provide for customization in the future. +% \item Added command-line option \texttt{-{}-jobs}, which allows the maximum number of concurrent processes to be specified (\#35). +% \item Added support for GNU Octave, via the \texttt{octave} family of commands and environments (\#36). Parsing of Octave stderr is not ideal, though synchronization works in most cases; this will be addressed by a future rewrite of the stderr parser. +% \item Installer now automatically works with MiKTeX, not just TeX Live. +% \item The PythonTeX utilities class has a new \texttt{open()} method that opens files and automatically tracks dependencies/created files. +% \item When \texttt{pythontex2.py} and \texttt{pythontex3.py} are run directly, the Python interpreter is automatically set to a reasonable default (\texttt{py -2} or \texttt{py -3} under Windows, using the Python 3.3+ wrapper; \texttt{python2} or \texttt{python3} under other systems). +% \item The installer now creates symlinks for the numbered scripts \texttt{pythontex*.py} and \texttt{depythontex*.py}. +% \item Added Python version checking to all numbered scripts. +% \item Under Python, the type of data passed via \texttt{\string\setpythontexcontext} may now be set using YAML-style tags (\texttt{!!str}, \texttt{!!int}, \texttt{!!float}). For example, \texttt{\string\setpythontexcontext\{myint=!!int 123\}}. +% \item The \texttt{fancyvrb} options \texttt{firstline} and \texttt{lastline} now work with the \texttt{pygments} environment and \texttt{\string\inputpygments} command. This required some additional patching of \texttt{fancyvrb}. +% \item The \texttt{pytx@Verbatim} and \texttt{pytx@SaveVerbatim} environments are now used for typesetting verbatim code. These are copies of the \texttt{fancyvrb} environments. This prevents conflicts when literal \texttt{Verbatim} and \texttt{SaveVerbatim} environments need to be typeset. +% \item Improved \texttt{latexmk} compatibility (\#40). Added discussion of \texttt{latexmk} usage to documentation. +% \item Tildes \texttt{\string~} may now be used in \texttt{outputdir} and \texttt{workingdir} to refer to the user's home directory, even under Windows. +% \end{itemize} +% \textbf{Bugfixes} +% \begin{itemize} +% \item Fixed a bug that prevented created files from being cleaned up when the working directory was not the document root directory and the full path to the files was not provided. +% \item Fixed a bug that prevented the \texttt{fvextfile} option from working when external files were highlighted. +% \end{itemize} +% \end{changelog} +% +% \begin{changelog}{v0.13-beta}{2014/02/06} +% \textbf{New features} +% \begin{itemize} +% \item Switching to GitHub's Releases for downloads. +% \item TeX information such as page dimensions may now be easily passed to the programming-language side, using the new \texttt{\string\setpythontexcontext} command. Contextual information is stored in the \texttt{context} attribute of the utilities class, which is a dictionary (and also has attributes in Python). +% \item The utilities class now has \texttt{pt\_to\_in()}, \texttt{pt\_to\_cm()}, and \texttt{pt\_to\_mm()} methods for converting units of TeX points into inches, centimeters, and millimeters. These work with integers and floats, as well as strings that consist of numbers and optionally end in ``pt''. There is also a \texttt{pt\_to\_bp()} for converting TeX points (1/72.27 inch) into big (DTP or PostScript) points (1/72 inch). +% \item Expanded Quickstart. Quickstart is now compatible with all LaTeX engines. Quickstart now avoids \texttt{microtype} issues on some systems (\#32). +% \item Added information on citing PythonTeX (\#28). +% \item Utilities class has a new attribute \texttt{id}, which is a string that joins the command family name, session name, and session restart parameters with underscores. This may be used in creating files that need a name that contains a unique, session-based identifier (for example, names for figures that are saved automatically). +% \end{itemize} +% \textbf{Backward-incompatible changes} +% \begin{itemize} +% \item All utilities-class attributes with names of the form \texttt{input\_*} have been renamed with the ``\texttt{input\_}'' removed. Among other things, this makes it easier to access the \texttt{context} attribute (\texttt{pytex.context} vs.\ \texttt{pytex.input\_context}). +% \item \texttt{depythontex} now has \texttt{-o} and \texttt{-{}-output} command-line options for specifying the name of the output file. If an output file is not specified, then output is written to \texttt{stdout}. This allows \texttt{depythontex} output to be piped to another program. +% \item All scripts \texttt{*2.py} now have shebangs with \texttt{env python2}, and all scripts \texttt{*3.py} now have shebangs with \texttt{env python3}. This allows the wrapper scripts (\texttt{env python} shebang) to be used with the default Python installation, and the numbered scripts to be used with specific versions. Remember that except for console content, the \texttt{-{}-interpreter} option is what determines the Python version that actually executes code. The version of Python used to launch \texttt{pythontex.py} merely determines the version that manages code execution. (\texttt{-{}-interpreter} support for console content is coming.) +% \item Changed the template style used in the \texttt{CodeEngine} class. Replacement fields are now surrounded by single curly braces (as in Python's format string syntax), rather than double curly braces. Literal curly braces are obtained by doubling braces. This allows the use of literal adjacent double braces in templates, which was not possible previously. +% \item The Julia template now uses the new \texttt{in()} function, replacing \texttt{contains()}. This requires Julia v0.2.0+. +% \end{itemize} +% \textbf{Bugfixes} +% \begin{itemize} +% \item Modified test for LuaTeX, so that \texttt{\string\directlua} is not \texttt{\string\let} to \texttt{\string\relax} if it does not exist. This was causing incompatibility with \texttt{babel} under pdfTeX and XeTeX (\#33). +% \item Added missing shebangs to \texttt{depythontex*.py}. Handling of \texttt{utilspath} is now more forgiving, so that \texttt{pythontex\_utils.py} can be installed in alternate locations (\#23). +% \item \texttt{depythontex} no longer leaves a blank line where \texttt{\string\usepackage\{pythontex\}} was removed. +% \item Console environments typeset with \texttt{fancyvrb} no longer end with an unnecessary empty line. +% \item Fixed bug in installer when \texttt{kpsewhich} was not found (\#21). +% \end{itemize} +% \end{changelog} +% % \begin{changelog}{v0.12}{2013/08/26} +% \begin{itemize} % \item Added support for the Julia language, with the \texttt{julia} and \texttt{jl} families of commands and environments. (Note that Pygments only added Julia support in version 1.6.) % \item Warnings and errors are now synchronized with the line numbers of files brought in via \texttt{\string\input}, \texttt{\string\include}, etc. This is accomplished using the \texttt{currfile} package. % \item Added package option \texttt{gobble}. When \texttt{gobble=auto}, all code is dedented before being executed and/or typeset. The current implementation is functional but basic; it will be improved and extended in the future. @@ -101,9 +373,11 @@ % \item Fixed a bug with counters that caused errors when content spanning multiple columns was created within a \texttt{tabular} environment. % \item Added checking for compatible Python versions in \texttt{pythontex.py}. % \item Improved execution of \texttt{*.bat} and \texttt{*.cmd} files under Windows. The solution from v0.12beta allowed \texttt{*.bat} and \texttt{*.cmd} to be found and executed when the extension was not given, but did not give correct return codes. +% \end{itemize} % \end{changelog} % % \begin{changelog}{v0.12beta}{2013/06/24} +% \begin{itemize} % \item Merged \texttt{pythontex\_types*.py} into a single replacement \texttt{pythontex\_engines.py} compatible with both Python 2 and 3. It is now much simpler to add support for additional languages. % \item Added support for the Ruby language as a demonstration of new capabilities. The \texttt{ruby} and \texttt{rb} families of commands and environments may be enabled via the new \texttt{usefamily} package option. Support for additional languages is coming soon. See the new section in the documentation on support for other languages for more information. % \item Reimplemented treatment of Pygments content for better efficiency. Now a Pygments process only runs if there is content to highlight. Eliminated redundant highlighting of unmodified code. @@ -136,11 +410,13 @@ % \item At the end of each run, data and macros are only saved if modified, improving efficiency. % \item The number of temporary files required by each process was reduced by one. All macros for commands like \texttt{\string\py} are now returned within stdout, rather than in their own file. % \item Fixed a bug with \texttt{\string\stderrpythontex}; it was defaulting to \texttt{verb} rather than \texttt{verbatim} mode. +% \end{itemize} % \end{changelog} % % \begin{changelog}{v0.11}{2013/04/21} +% \begin{itemize} % \item As the first non-beta release, this version adds several features and introduces several changes. You should read these release notes carefully, since some changes are not backwards-compatible. Changes are based on a thorough review of all current and planned features. PythonTeX's capabilities have already grown beyond what was originally intended, and a long list of features still remains to be implemented. As a result, some changes are needed to ensure consistent syntax and naming in the future. Insofar as possible, all command names and syntax will be frozen after this release. -% \item Added the \texttt{pythontex.py} and \texttt{depythontex.py} wrapper scripts. When run, these detect the current version of Python and import the correct PythonTeX code. It is still possible to run \texttt{pythontex*.py} and \texttt{depythontex*.py} directly, but the new wrapper scripts should be used instead for simplicity. There is now only a single \texttt{pythontex\_utils.py}, which works with both Python 2 and Python 3. +% \item Added the \texttt{pythontex.py} and \texttt{depythontex.py} wrapper scripts. When run, these detect the current version of Python and import the correct PythonTeX code. It is still possible to run \texttt{pythontex*.py} and \texttt{depythontex*.py} directly, but the new wrapper scripts should be used instead for simplicity. There is now only a single \texttt{pythontex\_utils.py}, which works with both Python 2 and Python 3. % \item Added the \texttt{beta} package option. This makes the current version behave like v0.11beta, for compatibility. This option is temporary and will probably only be retained for a few releases. % \item Backward-incompatible changes (require the \texttt{beta} option to restore old behavior) % \begin{itemize} @@ -159,9 +435,11 @@ % \item Added \texttt{autostdout} package option and \texttt{\string\setpythontexautostdout}, to complement \texttt{autoprint}. Added |prettyprinter| and |prettyprintinline| package options to complement new settings commands. % \item Added quickstart guide. % \item Installer now installs gallery and quickstart files, if present. +% \end{itemize} % \end{changelog} % % \begin{changelog}{v0.11beta}{2013/02/17} +% \begin{itemize} % \item Commands like \texttt{\string\py} can now bring in any valid LaTeX code, including verbatim content, under the pdfTeX and XeTeX engines. Verbatim content was not allowed previously. LuaTeX cannot bring in verbatim, due to a known bug. % \item Added package option \texttt{depythontex} and scripts \texttt{depythontex*.py}. These allow a PythonTeX document to be converted into a pure LaTeX document, with no Python dependency. The package option creates an auxiliary file with extension \texttt{.depytx}. The \texttt{depythontex*.py} scripts take this auxiliary file and the original LaTeX document, and combine the two to produce a new document that does not rely on the PythonTeX package. All PythonTeX commands and environments are replaced by their output. All Python-generated content is substituted directly into the document. By default, all typeset code is wrapped in \texttt{\string\verb} and \texttt{verbatim}, but \texttt{depythontex*.py} has a \texttt{-{}-listing} option that allows \texttt{fancyvrb}, \texttt{listings}, \texttt{minted}, or \texttt{pythontex} to be used instead. % \item The current PythonTeX version is now saved in the \texttt{.pytxcode}. If this does not match the version of the PythonTeX scripts, a warning is issued. This makes it easier to determine errors due to version mismatches. @@ -170,9 +448,11 @@ % \item Fixed a bug with PythonTeX verbatim commands and environments that use Pygments. The verbatim commands and environments were incorrectly treated as if they had the attributes of executed code in the v0.10beta series. % \item Fixed a bug from the v0.10beta series that sometimes prevented imports from \texttt{\_\_future\_\_} from working when there were multiple sessions. % \item Fixed a bug related to hashing dependencies' mtime under Python 3. +% \end{itemize} % \end{changelog} % % \begin{changelog}{v0.10beta2}{2013/01/23} +% \begin{itemize} % \item Improved \texttt{pythontex*.py}'s handling of the name of the file being processed. A warning is no longer raised if the name is given with an extension; extensions are now processed (stripped) automatically. The filename may now contain a path to the file, so you need not run \texttt{pythontex*.py} from within the document's directory. % \item Added command-line option \texttt{-{}-verbose} for more verbose output. Currently, this prints a list of all processes that are launched. % \item Fixed a bug that could crash \texttt{pythontex*.py} when the package option \texttt{pygments=false}. @@ -181,9 +461,11 @@ % \item Revised treatment of any content that custom code attempts to print. Custom code is not allowed to print to the document (see documentation). If custom code attempts to print, a warning is raised, and the printed content is included in the \texttt{pythontex*.py} run summary. % \item One-line entries in stderr, such as those produced by Python's \texttt{warnings.warn()}, were not previously parsed because they are of the form \texttt{::} rather than \texttt{line~}. These are now parsed and synchronized with the document. They are also correctly parsed for inclusion in the document via \texttt{\string\stderrpythontex}. % \item If the package option \texttt{stderrfilename} is changed, all sessions that produced errors or warnings are now re-executed automatically, so that their stderr content is properly updated with the new filename. +% \end{itemize} % \end{changelog} % % \begin{changelog}{v0.10beta}{2013/01/09} +% \begin{itemize} % \item Backward-incompatible: Redid treatment of command-line options for \texttt{pythontex*.py}, using Python's \texttt{argparse} module. Run \texttt{pythontex*.py} with option \texttt{-h} to see new command line options. % \item Deprecated: \texttt{\string\setpythontexcustomcode} is deprecated in favor of the \texttt{\string\pythontexcustomc} command and \texttt{pythontexcustomcode} environment. These allow entry of pure code, unlike \texttt{\string\setpythontexcustomcode}. These also allow custom code to be added to the beginning or end of a session, via an optional argument. Improved treatment of errors and warnings associated with custom code. % \item The summary of errors and warnings now correctly differentiates errors and warnings produced by user code, rather than treating all of them as errors. By default, \texttt{pythontex*.py} now returns an exit code of 1 if there were errors. @@ -212,9 +494,11 @@ % \item Fixed a bug that sometimes caused incorrect line numbers with \texttt{stderr} content. Improved processing of stderr. % \item Fixed a bug in automatic detection of pre-existing listings environment. % \item Improved the detection of imports from \texttt{\_\_future\_\_}. Detection should now be stricter, faster, and more accurate. +% \end{itemize} % \end{changelog} % % \begin{changelog}{v0.9beta3}{2012/07/17} +% \begin{itemize} % \item Added Unicode support, which required the Python code to be split into one set for Python 2 and another set for Python 3. This will require any old installation to be completely removed, and a new installation created from scratch. % \item Refactoring of Python code. Documents should automatically re-execute all code after updating to the new version. Otherwise, you should delete the PythonTeX directory and run PythonTeX. % \item Improved installation script. @@ -224,14 +508,19 @@ % \item Rewrote pythontex\_utils*.py, creating a new, context-aware interface to SymPy's LatexPrinter class. % \item Content brought in via macros no longer uses labels. Rather, long defs are used, which allows line breaks. % \item Pygments highlighting is now default for PythonTeX commands and environments +% \end{itemize} % \end{changelog} % % \begin{changelog}{v0.9beta2}{2012/05/09} +% \begin{itemize} % \item Changed Python output extension to .stdout. +% \end{itemize} % \end{changelog} % % \begin{changelog}{v0.9beta}{2012/04/27} +% \begin{itemize} % \item Initial public beta release. +% \end{itemize} % \end{changelog} % % @@ -251,6 +540,7 @@ % % \maketitle % +% % \begin{abstract} % \pytex\ provides access to Python from within \LaTeX\ documents. It allows Python code entered within a \LaTeX\ document to be executed, and the results to be included within the original document. Python code may be adjacent to the figure or calculation it produces. The package also makes possible macro definitions that mix Python and \LaTeX\ code. In addition, \pytex\ provides syntax highlighting for many programming languages via the Pygments syntax highlighter. % @@ -258,14 +548,25 @@ % % Because documents that use \pytex\ mix \LaTeX\ and Python code, they are less suitable than plain \LaTeX\ documents for journal submission, sharing, and conversion to other formats. \pytex\ includes a |depythontex| utility that creates a copy of a document in which all \pytex\ content is replaced by its output. % -% While Python is the focus of \pytex, adding basic support for an additional language is usually as simple as creating a new class instance and a few templates, usually totaling less than 100 lines of code. The following languages are already have built-in support: Ruby. +% While Python is the focus of \pytex, adding basic support for an additional language is usually as simple as creating a new class instance and a few templates, usually totaling less than 100 lines of code. The following languages already have built-in support: Ruby, Julia, Octave, Bash, Rust, R, Perl, Perl 6, and JavaScript. % \end{abstract} % % -% \section*{\centering Warning} -% +% +% \begingroup +% \small +% +% \subsection*{\centering Development Status} +% Since 2020, I am increasingly creating new teaching materials with Markdown and HTML instead of \pytex\ and \LaTeX. I have limited time for developing open-source software that I do not use frequently myself. There should be occasional releases to keep \pytex\ running, but no major changes or significant new features are anticipated. +% +% +% +% \subsection*{\centering Warning} +% % \pytex\ makes possible some pretty amazing things. But that power brings with it a certain risk and responsibility. Compiling a document that uses \pytex\ involves executing Python code, and potentially other programs, on your computer. You should only compile \pytex\ documents from sources you trust. \pytex\ comes with NO WARRANTY.\footnote{All \LaTeX\ code is licensed under the \href{http://www.latex-project.org/lppl.txt}{\LaTeX\ Project Public License (LPPL)} and all Python code is licensed under the \href{http://www.opensource.org/licenses/BSD-3-Clause}{BSD 3-Clause License}.} The copyright holder and any additional authors will not be liable for any damages. % +% \endgroup +% % % \pagebreak % \tableofcontents @@ -278,8 +579,8 @@ % % \LaTeX\ can do a lot,\footnote{\TeX\ is a Turing-complete language.} but the programming required can sometimes be painful.\footnote{As I learned in creating this package.} In spite of the many packages available for \LaTeX, the libraries and packages of a general-purpose programming language are lacking. Furthermore, it can be convenient to include non-\LaTeX\ code in a document to make it more reproducible. For these reasons, there have been multiple systems that allow other languages to be used within \LaTeX\ documents.\footnote{I am not including the various web and weave dialects in my discussion, since they typically involve a web or weave document from which the .tex source is generated, and thus weaker integration with \LaTeX. Two sophisticated examples of this approach are \href{http://www.stat.uni-muenchen.de/~leisch/Sweave/}{Sweave} and \href{http://yihui.name/knitr/}{knitr}, both of which combine \LaTeX\ with the R language for tasks such as dynamic report generation.} % \begin{itemize} -% \item \href{http://www.ctan.org/tex-archive/macros/latex/contrib/perltex/}{Perl\TeX} allows the bodies of \LaTeX\ macros to be written in Perl. -% \item \href{http://www.ctan.org/tex-archive/macros/latex/contrib/sagetex/}{Sage\TeX} allows code for the Sage mathematics software to be executed from within a \LaTeX\ document. +% \item \href{http://www.ctan.org/pkg/perltex}{Perl\TeX} allows the bodies of \LaTeX\ macros to be written in Perl. +% \item \href{http://www.ctan.org/pkg/sagetex/}{Sage\TeX} allows code for the Sage mathematics software to be executed from within a \LaTeX\ document. % \item Martin R.\ Ehmsen's \href{http://www.ctan.org/pkg/python}{|python.sty|} provides a very basic method of executing Python code from within a \LaTeX\ document. % \item \href{http://elec.otago.ac.nz/w/index.php/SympyTeX}{Sympy\TeX} allows more sophisticated Python execution, and is largely based on a subset of Sage\TeX. % \item \href{http://www.luatex.org/}{Lua\TeX} extends the pdf\TeX\ engine to provide Lua as an embedded scripting language, and as a result yields tight, low-level Lua integration. @@ -291,7 +592,7 @@ % \item[Compiling without executing]\hfill\\ Even with all of these features to boost execution speed, there will be times when we have to run slow code. Thus, we need the execution of non-\LaTeX\ code to be separated from compiling the \LaTeX\ document. We need to be able to edit and compile a document containing unexecuted code. Unexecuted code should be invisible or be replaced by placeholders. Sage\TeX\ and Sympy\TeX\ have implemented such a separation of compiling and executing. In contrast, Lua\TeX\ and Perl\TeX\ execute all the code at each compilation---but that is appropriate given their goal of simplifying macro programming. % \item[Error messages]\hfill\\ Whenever code is saved from a \LaTeX\ document to an external file and then executed, the line numbers for any error messages will not correspond to the line numbering of the original \LaTeX\ document. At one extreme, |python.sty| doesn't attempt to deal with this issue, while at the other extreme, Sage\TeX\ uses an ingenous system of |Try|/|Except| statements on every chunk of code. We need a system that translates all error messages so that they correspond to the line numbering of the original \LaTeX\ document, with minimal overhead when there are no errors. % \item[Syntax highlighting]\hfill\\ Once we begin using non-\LaTeX\ code, sooner or later we will want to typeset some of it, which means we need syntax highlighting. A number of syntax highlighting packages currently exist for \LaTeX; perhaps the most popular are |listings| and |minted|. |listings| uses pure \LaTeX. It has not been updated since 2007, which makes it a less ideal solution in some circumstances. |minted| uses the Python-based syntax highlighter Pygments to perform highlighting. Pygments can provide superior syntax highlighting, but |minted| can be very slow because all code must be highlighted at each compilation and each instance of highlighting involves launching an external Python process. We need high-speed, user-friendly syntax highlighting via Pygments.\footnote{The author recently started maintaining the \texttt{minted} package. In the near future, \texttt{minted} will inherit \pytex's speed enhancements, and the two packages will become more compatible.} -% \item[Printing]\hfill\\ It would be nice for the |print| statement/function,\footnote{In Python, |print| was a statement until Python 3, when it became a function. The function form is available via import from |\textunderscore\textunderscore future\textunderscore\textunderscore| in Python 2.6 and later.} or its equivalent, to automatically return its output within the \LaTeX\ document. For example, using |python.sty| it is possible to generate some text while in Python, open a file, save the text to it, close the file, and then |\input| the file after returning to \LaTeX. But it is much simpler to generate the text and |print| it, since the printed content is automatically included in the \LaTeX\ document. This was one of the things that |python.sty| really got right. +% \item[Printing]\hfill\\ It would be nice for the |print| statement/function,\footnote{In Python, |print| was a statement until Python 3, when it became a function. The function form is available via import from |__future__| in Python 2.6 and later.} or its equivalent, to automatically return its output within the \LaTeX\ document. For example, using |python.sty| it is possible to generate some text while in Python, open a file, save the text to it, close the file, and then |\input| the file after returning to \LaTeX. But it is much simpler to generate the text and |print| it, since the printed content is automatically included in the \LaTeX\ document. This was one of the things that |python.sty| really got right. % \item[Pure code]\hfill\\ \LaTeX\ has a number of special characters (|#| \texttt{\$} |%| |&| |~| |_| |^| |\| |{| |}|), which complicates the entry of non-\LaTeX\ code since these same characters are common in many languages. Sage\TeX\ and Sympy\TeX\ delimit all inline code with curly braces (|{}|), but this approach fails in the (somewhat unlikely) event that code needs to contain an unmatched brace. More seriously, they do not allow the percent symbol |%| (modular arithmetic and string formatting in Sage and Python) to be used within inline code. Rather, a |\percent| macro must be used instead. This means that code must (sometimes) be entered as a hybrid between \LaTeX\ and the non-\LaTeX\ language. Lua\TeX\ is somewhat similar: ``The main thing about Lua code in a TeX document is this: the code is expanded by TeX before Lua gets to it. This means that all the Lua code, even the comments, must be valid TeX!''\footnote{\url{http://wiki.contextgarden.net/Programming_in_LuaTeX}} In the case of Lua\TeX, though, there is the \href{http://www.ctan.org/pkg/luacode}{\texttt{luacode}} package that allows for pure Lua. % % This language hybridization is not terribly difficult to work around in the Sage\TeX\ and Sympy\TeX\ cases, and is actually a Lua\TeX\ feature in many contexts. But if we are going to create a system for general-purpose access to a non-\LaTeX\ language, we need \textbf{all} valid code to work correctly in \textbf{all} contexts, with no hybridization of any sort required. We should be able to copy and paste valid code into a \LaTeX\ document, without having to worry about hybridizing it. Among other things, this means that inline code delimiters other than \LaTeX's default curly braces |{}| must be available. @@ -310,15 +611,24 @@ % Now at this point there will almost certainly be some reader, sooner or later, who wants to object, ``But what about language \textit{X}!'' Well, yes, in some respects the choice to use Python did come down to personal preference. But you should give Python a try, if you haven't already. You may also wish to consider the many interfaces that are available between Python and other languages. If you still aren't satisfied, keep in mind \pytex's ``language-independent'' implementation! In many cases, adding support for additional languages is relatively simple (see Section~\ref{sec:other-languages}). % % +% \section{Citing \pytex} +% \label{sec:citing} +% +% If you use \pytex\ in your writing and research, please consider citing it in any resulting publications. The best and most recent paper is in \textit{Computational Science \& Discovery}. +% \begin{itemize} +% \item ``PythonTeX: reproducible documents with LaTeX, Python, and more,'' Geoffrey M Poore. \textit{Computational Science \& Discovery} 8 (2015) 014010. Full text and Bib\TeX\ entry available at \url{http://stacks.iop.org/1749-4699/8/i=1/a=014010}. +% \item ``Reproducible Documents with PythonTeX,'' Geoffrey M.\ Poore. \textit{Proceedings of the 12th Python in Science Conference} (2013), pp.\ 73--79. Full text and Bib\TeX\ entry available at \url{http://conference.scipy.org/proceedings/scipy2013/poore.html}. +% \end{itemize} +% % % \section{Installing and running} % \label{sec:installing-and-running} % -% \subsection{Installing \pytex} +% \subsection{Installing \pytex} % -% \pytex\ requires a \TeX\ installation. It has been tested with \href{http://www.tug.org/texlive/}{\TeX\ Live} and \href{http://miktex.org/}{MiK\TeX}, but should work with other distributions. The following \LaTeX\ packages, with their dependencies, are required: |fancyvrb|, |etex|, |etoolbox|, |xstring|, |pgfopts|, |newfloat|, |currfile|, and |color| or |xcolor|. A current \TeX\ installation is recommended, since some features require recent versions of the packages. If you are creating and including graphics, you will also need |graphicx|. The \href{http://www.ctan.org/pkg/mdframed}{\texttt{mdframed}} package is recommended for enclosing typeset code in boxes with fancy borders and/or background colors; \href{http://www.ctan.org/pkg/tcolorbox}{\texttt{tcolorbox}} and \href{http://www.ctan.org/pkg/framed}{\texttt{framed}} are alternatives. +% \pytex\ requires a \TeX\ installation. It has been tested with \href{http://www.tug.org/texlive/}{\TeX\ Live} and \href{http://miktex.org/}{MiK\TeX}, but should work with other distributions. The following \LaTeX\ packages, with their dependencies, are required: |fancyvrb|, |fvextra|, |etoolbox|, |xstring|, |pgfopts|, |newfloat| (part of the |caption| bundle), |currfile|, and |color| or |xcolor|. A current \TeX\ installation is recommended, since some features require recent versions of the packages. If you are creating and including graphics, you will also need |graphicx|. The \href{http://www.ctan.org/pkg/mdframed}{\texttt{mdframed}} package is recommended for enclosing typeset code in boxes with fancy borders and/or background colors; \href{http://www.ctan.org/pkg/tcolorbox}{\texttt{tcolorbox}} and \href{http://www.ctan.org/pkg/framed}{\texttt{framed}} are alternatives. % -% \pytex\ also requires a \href{http://www.python.org/}{Python} installation. Python 2.7 is recommended for the greatest compatibility with scientific tools, although many scientific packages are now compatible with Python 3. \pytex\ is compatible with Python 2.7 and 3.2+. The Python package \href{http://pygments.org/}{Pygments} must be installed for syntax highlighting to function. \pytex\ has been tested with Pygments 1.4 and later, but the latest version is recommended. For scientific work, or to compile |pythontex_gallery.tex|, the following are also recommended: \href{http://numpy.scipy.org/}{NumPy}, \href{http://www.scipy.org/}{SciPy}, \href{http://matplotlib.sourceforge.net/}{matplotlib}, and \href{http://sympy.org}{SymPy}. +% \pytex\ also requires a \href{http://www.python.org/}{Python} installation. A recent Python 3 installation is recommended, but there is still support for Python 2.7. The Python package \href{http://pygments.org/}{Pygments} must be installed for syntax highlighting to function. \pytex\ has been tested with Pygments 1.4 and later, but the latest version is recommended. For scientific work, or to compile |pythontex_gallery.tex|, the following are also recommended: \href{http://numpy.scipy.org/}{NumPy}, \href{http://www.scipy.org/}{SciPy}, \href{http://matplotlib.sourceforge.net/}{matplotlib}, and \href{http://sympy.org}{SymPy}. When using \pytex\ with LyX, be aware that LyX may try to use its own version of Python; you may need to reconfigure LyX. % % \pytex\ also provides support for other languages such as Ruby, so you will need to install any additional languags you plan to use. Typically, the most recent major version of these languages is supported. % @@ -330,18 +640,23 @@ % \item Language definitions |pythontex_engines.py| % \item Utilities class |pythontex_utils.py| % \item |depythontex.py|, which imports from |depythontex2.py| or |depythontex3.py|, based on the Python version; used to remove \pytex\ dependence +% \item Synchronized Python Debugger |syncpdb.py| % \item |README| (in rst style) % \item |pythontex_gallery.tex| and |pythontex_gallery.pdf| % \item |pythontex_quickstart.tex| and |pythontex_quickstart.pdf| -% \item Optional installation script |pythontex_install_texlive.py| for \TeX\ Live +% \item Optional installation script |pythontex_install.py| for \TeX\ Live and MiKTeX % \item Optional batch file |pythontex.bat| for use in launching |pythontex.py| under Windows -% \item Optional conversion script |pythontex_2to3.py| for converting \pytex\ code written for Python 2 into a form compatible with Python 3 +% \item |pythontex_2to3.py| was originally included with \pytex, but it is no longer needed. Originally, the core of \pytex\ was developed in |pythontex2.py| for Python 2.7, and then |pythontex_2to3.py| was used convert |pythontex2.py| into |pythontex3.py| for Python 3.2+. Currently, all development is in |pythontex3.py|. |pythontex2.py| is retained for backward compatibility, but in most cases it is no longer updated to maintain parity with |pythontex3.py|. Similarly, |depythontex3.py| is now developed directly, separately from |depythontex2.py|, and |depythontex2.py| typically doesn't receive updates. % \end{itemize} -% The style file |pythontex.sty| may be generated by running \LaTeX\ on |pythontex.ins|. The documentation you are reading may be generated by running \LaTeX\ on |pythontex.dtx|. Some code is provided in two forms, one for Python 2 and one for Python 3 (names ending in |2| and |3|). Whenever this is the case, a version-independent wrapper is supplied that automatically runs the correct code based on the Python version. For example, there are two main scripts, |pythontex2.py| and |pythontex3.py|, but you should actually run |pythontex.py|, which imports the correct code based on the Python version.\footnote{Unfortunately, it is not possible to provide full Unicode support for both Python 2 and 3 using a single script. Currently, all code is written for Python 2, and then the Python 3 version is automatically generated via the |pythontex\_2to3.py| script. This script comments out code that is only for Python 2, and un-comments code that is only for Python 3.} +% The style file |pythontex.sty| may be generated by running \LaTeX\ on |pythontex.ins|. The documentation you are reading may be generated by running \LaTeX\ on |pythontex.dtx|. Some code is provided in two forms, one for Python 2 and one for Python 3 (names ending in |2| and |3|). Whenever this is the case, a version-independent wrapper is supplied that automatically runs the correct code based on the Python version. For example, there are two main scripts, |pythontex2.py| and |pythontex3.py|, but you can typically run |pythontex.py|, which imports the correct code based on the Python version. +% +% If you want the absolute latest version of \pytex, you should install it manually from \href{https://github.com/gpoore/pythontex}{github.com/gpoore/pythontex}. A Python installation script is provided for use with \TeX\ Live and MiKTeX. It has been tested with Windows, Linux, and OS~X, but may need manual input or slight modifications depending on your system. The installation script performs the steps described below. % -% If you want the absolute latest version of \pytex, you should install it manually from \href{https://github.com/gpoore/pythontex}{github.com/gpoore/pythontex}. A Python installation script is provided for use with \TeX\ Live. It has been tested with Windows, Linux, and OS~X, but may need manual input or slight modifications depending on your system. The installation script performs the steps described below. \textbf{Note that for a typical \TeX\ setup under Linux, you may need to run the script with elevated privileges, and may need to run it with the user's PATH.} This can be necessary when you are using a Linux distribution that includes an outdated version of \TeX\ Live, and have installed a new version manually. \textbf{If you are installing \pytex\ on a machine with multiple version of \TeX, make sure you install \pytex\ for the correct version.} For example, under Ubuntu Linux, you will probably need the following command if you have installed the latest version of \TeX\ Live manually: +% \textbf{For a MiKTeX installation, you may need administrator privileges; running \texttt{pythontex\_install.bat} as administrator may be simplest.} +% +% \textbf{Note that for a typical \TeX\ setup under Linux, you may need to run the script with elevated privileges, and may need to run it with the user's PATH.} This can be necessary when you are using a Linux distribution that includes an outdated version of \TeX\ Live, and have installed a new version manually. \textbf{If you are installing \pytex\ on a machine with multiple versions of \TeX, make sure you install \pytex\ for the correct version.} For example, under Ubuntu Linux, you will probably need the following command if you have installed the latest version of \TeX\ Live manually: % \begin{verbatim} -% sudo env PATH=$PATH python pythontex_install_texlive.py +%sudo env PATH=$PATH python pythontex_install.py % \end{verbatim} % % The installer creates the following files. It will offer to create the paths if they do not exist. If you are installing in TEXMFLOCAL, the paths will have an additional |local/| at the end. @@ -361,6 +676,7 @@ % \item |pythontex_engines.py| % \item |pythontex_utils.py| % \item |depythontex.py|, |depythontex2.py| and |depythontex3.py| +% \item |syncpdb.py| % \end{itemize} % \item \meta{\TeX\ tree root}|/source/latex/pythontex/| % \begin{itemize} @@ -376,6 +692,7 @@ % % % \subsection{Compiling documents using \pytex} +% \label{sec:installing-and-running:compiling} % % Compiling a document with \pytex\ involves three steps: running a \LaTeX-compatible \TeX\ engine (binary executable), running |pythontex.py| (preferably via a symlink, wrapper, or batch file, as described above), and finally running the \TeX\ engine again. The first \TeX\ run saves code into an external file where \pytex\ can access it. The second \TeX\ run pulls the \pytex\ output back into the document. % @@ -387,14 +704,14 @@ % \end{itemize} % For an example of a \pytex\ document that will correctly compile under all three engines, see the |pythontex_gallery.tex| source. % -% If you use XeLaTeX, and your non-\LaTeX\ code contains tabs, you \textbf{must} invoke XeLaTeX with the |-8bit| option so that tabs will be written to file as actual tab characters rather than as the character sequence |^^I|.\footnote{See \url{http://tex.stackexchange.com/questions/58732/how-to-output-a-tabulation-into-a-file} for more on tabs with XeTeX.} +% If you use XeLaTeX, and your non-\LaTeX\ code contains tabs, you \textbf{must} invoke XeLaTeX with the |-8bit| option so that tabs will be written to file as actual tab characters rather than as the character sequence |^^I|.\footnote{See \url{http://tex.stackexchange.com/questions/58732/how-to-output-a-tabulation-into-a-file} for more on tabs with XeTeX.} % % |pythontex.py| requires a single command-line argument: the name of the .tex file to process. The filename can be passed with or without an extension; the script really only needs the |\jobname|, so any extension is stripped off.\footnote{Thus, \pytex\ works happily with .tex, .ltx, .dtx, and any other extension.} The filename may include the path to the file; you do not have to be in the same directory as the file to run \pytex. If you are configuring your editor to run \pytex\ automatically via a shortcut, you may want to wrap the filename in double quotes |"| to allow for space characters.\footnote{Using spaces in the names of .tex files is apparently frowned upon. But if you configure things to handle spaces whenever it doesn't take much extra work, then that's one less thing that can go wrong.} For example, under Windows with \TeX\ Live and Python 2.7 we would create the wrapper |pythontex.exe|. Then we could run \pytex\ on a file \meta{file~name}.tex using the command |pythontex.exe "|\meta{file~name}|"|. % % |pythontex.py| accepts the following optional command-line arguments. Some of these options duplicate package-level options, so that settings may be configured either within the document or at the command line. In the event that the command-line and package options conflict, the package options always override the command-line options. For variations on these options that are acceptable, run |pythontex.py -h|. % \begin{itemize} % \item |--encoding=|\meta{encoding} This sets the file encoding. Any encoding supported by Python's \href{http://docs.python.org/library/codecs.html}{\texttt{codecs}} module may be used. The encoding should match that of the \LaTeX\ document. If an encoding is not specified, \pytex\ uses UTF-8. If support for characters beyond ASCII is required, then additional \LaTeX\ packages are required; see the discussion of \TeX\ engines above. -% \item |--error-exit-code={true,false}| By default, |pythontex.py| returns an exit code of 1 if there were any errors, and an exit code of 0 otherwise. This may be useful when \pytex\ is used in a scripting or command-line context, since the presence of errors may be easily detected. It is also useful with some \TeX\ editors. For example, \href{http://www.tug.org/texworks/}{TeXworks} automatically hides the output of external programs unless there are errors. +% \item |--error-exit-code={true,false}| By default, when |pythontex.py| finishes running, it returns an exit code of 1 if there were any errors (for example, in code execution), and an exit code of 0 otherwise. This may be useful when \pytex\ is used in a scripting or command-line context, since the presence of errors may be easily detected. It is also useful with some \TeX\ editors. For example, \href{http://www.tug.org/texworks/}{TeXworks} automatically hides the output of external programs unless there are errors. % % In some contexts, returning a nonzero exit code can be redundant. For example, with the \href{http://winshell.de/}{WinShell} editor under Windows with TeX Live, the complete output of \pytex\ is always available in the ``Output'' view, so it is clear if errors have occurred. Having a nonzero exit code causes |runscript.exe| to return an additional, redundant error message in the ``Output'' view. In such situations, it may be desirable to disable the nonzero exit code. % \item |--runall=[{true,false}]| This causes \textbf{all} code to be executed, regardless of modification or |rerun| settings. It is useful when code has not been modified, but a dependency such as a library or external data has changed. Note that the \pytex\ utilities class also provides a mechanism for automatically re-executing code that depends on external files when those external files are modified. @@ -405,6 +722,8 @@ % There is an equivalent |rerun| package option. % \item |--hashdependencies=[{true,false}]| This determines whether dependencies (external files highlighted by Pygments, code dependencies specified via |pytex.add_dependencies()|, etc.) are checked for changes via their hashes or modification times. By default, mtime is used, since it is faster. The package option |hashdependencies| is equivalent. % +% \item |--jobs| This sets the maximum number of concurrent processes. By default, this will be Python's |multiprocessing.cpu_count()|, which is the number of CPUs in the system. It may be useful to set a smaller value when some jobs are particularly resource intensive or themselves use subprocesses. +% % \item |--verbose| This gives more verbose output, including a list of all processes that are launched. % % \item |--interpreter| This allows the interpreter for a given language to be specified. The argument should be in the form @@ -414,6 +733,33 @@ % where || is |python|, |ruby|, etc., and || is the command for invoking the desired interpreter. The argument to |--interpreter| may also be in the form of a Python dictionary. The argument need not be enclosed in quotation marks if it contains no spaces. % % For example, by default Python code is executed with whatever interpreter the |python| command invokes. But Python 3 could be specified using |--interpreter python:python3| (many Linux distributions) or |--interpreter "python:py -3"| (Windows, with Python 3.3 installed so that the |py| wrapper is available). +% +% \item |--interactive [::]| This is used to run a single session in interactive mode. This allows user input. Code output is written to stdout. Interactive mode is particularly useful when working with debuggers (but also see the |--debug| option). +% +% |[::]| is optional; if it is not provided, the default session is executed. For non-default sessions (or if there are multiple default sessions, due to the use of multiple families of commands), simply supplying the session name is usually sufficient (for example, |--debug session|). The full combination of |[::]| (for example, |py:session:default|) is only necessary when the session name alone would be ambiguous. +% +% Note that when a session is run in interactive mode, it will \emph{not} save printed content in a form that may be brought back into the document. You will have to run the session again in normal mode to complete document compilation. +% +% Code that requires user input will cause \pytex\ to ``hang'' when \pytex\ is not running in interactive mode. This is because the code will request user input, but no input is possible given the way that the code is being executed, so the code will wait for input forever. It is inefficient constantly to add and then delete interactive code as you switch between normal and interactive modes. To avoid this, you can conditionally invoke code that requires input. In interactive mode, the temporary script that is executed is given the command-line argument |--interactive|. You can check for the presence of this argument, and only invoke interactive code if it is present. For example, under Python you could start the |pdb| debugger, only when the code is being executed in interactive mode, using commands such as the following. +%\begin{verbatim} +%import pdb +%import sys +%if '--interactive' in sys.argv[1:]: +% pdb.set_trace() +%\end{verbatim} +% +% This option is currently not compatible with Python console commands and environments. +% +% \item |--debug [::]| This is used to run a single session with the default debugger in interactive mode. Currently, only standard Python sessions are supported. (Python console commands and environments are not supported.) Support for other languages and support for customization will be added in the future. +% +% |[::]| is optional; if it is not provided, the default session is executed. For non-default sessions (or if there are multiple default sessions, due to the use of multiple families of commands), simply supplying the session name is usually sufficient (for example, |--debug session|). The full combination of |[::]| (for example, |py:session:default|) is only necessary when the session name alone would be ambiguous. +% +% Note that when a session is run in debug mode, it will \emph{not} save printed content in a form that may be brought back into the document. You will have to run the session again in normal mode to complete document compilation. +% +% The default Python debugger is |syncpdb|, the Synchronized Python Debugger. It provides a wrapper around |pdb| that is aware of the connection between the code and the \LaTeX\ document from which it was extracted. All |pdb| commands function normally. In addition, commands that take a line number or filename:lineno as an argument will also take these same values with a percent symbol |%| prefix. If the percent symbol is present, then |syncpdb| interprets the filename and line number as referring to the document, rather than to the code that is executed. It will translate the filename and line number to the corresponding code equivalents, and then pass these to the standard |pdb| internals. For example, the |pdb| command |list 50| would list the code that is being executed, centered around line 50. |syncpdb| allows the command |list %10|, which would list the code that is being executed, centered around the code that came from line 10 in the main \LaTeX\ document. (If no file name is given, then the main \LaTeX\ document is assumed.) If the code instead came from an inputed file |input.tex|, then |list %input.tex:10| could be used. Further details are provided at \href{https://github.com/gpoore/syncpdb}{github.com/gpoore/syncpdb}. +% +% The temporary script that is executed is given the command-line argument |--interactive| when run in debug mode. You can check for the presence of this argument if you wish to invoke code that requires user input conditionally. See the |--interactive| command-line option for more details. +% % \end{itemize} % % \pytex\ attempts to check for a wide range of errors and return meaningful error messages. But due to the interaction of \LaTeX\ and Python code, some strange errors are possible. If you cannot make sense of errors when using \pytex, the simplest thing to try is deleting all files created by \pytex, then recompiling. By default, these files are stored in a directory called |pythontex-files-|\meta{jobname}, in the same directory as your .tex document. See Section \ref{sec:troubleshooting} for more details regarding troubleshooting. @@ -437,12 +783,12 @@ % % \DescribeMacro{usefamily=\meta{basename}/\marg{basename1,~basename2,~...}} % -% By default, only the |py|, |sympy|, and |pylab| families of commands and environments are defined, to prevent possible package conflicts.\footnote{For example, a \texttt{\string\ruby} command for Ruby code, and the \texttt{\string\ruby} command defined by the Ruby package in the \href{http://www.ctan.org/pkg/cjk}{CJK package}.} This option defines preconfigured families for other available languages. It takes either a single language base name, or a list of comma-separated names enclosed in curly braces. Currently, the Ruby families |rb| and |ruby| may be created. +% By default, only the |py|, |sympy|, and |pylab| families of commands and environments are defined, to prevent possible package conflicts.\footnote{For example, a \texttt{\string\ruby} command for Ruby code, and the \texttt{\string\ruby} command defined by the Ruby package in the \href{http://www.ctan.org/pkg/cjk}{CJK package}.} This option defines preconfigured families for other available languages. It takes either a single language base name, or a list of comma-separated names enclosed in curly braces. For example, the Ruby families |rb| and |ruby|, the Julia families |jl| and |julia|, and the Octave family |octave| may be enabled. For a full list of supported languages, see \Cref{sec:other-languages}. % % % \DescribeMacro{gobble=none/auto default:none} % -% This option is still under development and may change somewhat in future releases. If that occurs, equivalent functionality will be provided. +% This option is still under development and may change somewhat in future releases. If that occurs, equivalent functionality will be provided. % % This option determines how code indentation is handled. By default, indentation is left as-is; leading whitespace is significant. |auto| will dedent all code by gobbling the largest common leading whitespace, using Python's |textwrap.dedent()|.\footnote{It would be possible to do the dedent on the \LaTeX\ side, as is done manually in the \texttt{fancyvrb} and \texttt{listings} packages with the \texttt{gobble} option and is done automatically in the \texttt{lstautogobble} package. This is not done for stability and security reasons. \texttt{lstautogobble} determines the dedent by extracting the leading whitespace from the first line of code, and then applying this dedent to each subsequent line. This is adequate for \textbf{typesetting} code, since the worst-case scenario is that a subsequent line with less indentation will be typeset with the first few characters missing. Such an approach is not acceptable when the code will be \textbf{executed}, since a few missing characters could in principle cause serious damage. Doing the dedent on the Python side ensures that no characters are discarded, even if that results in an indentation error.} Keep in mind that Python's dedent will not work correctly with mixed tabs and spaces. % @@ -455,7 +801,7 @@ % % This option provides compatibility with the beta releases from before the full v0.11 release, which introduced some changes in syntax and command names. This option should \textbf{only} be used with old \pytex\ documents that require it. % -% You are encouraged to update old documents, since this compatility option will only be provided for a few releases. +% You are encouraged to update old documents, since this compatibility option will only be provided for a few releases. % % % \DescribeMacro{runall=\meta{none}/true/false default:false \meta{none}=true} @@ -464,7 +810,7 @@ % % A command-line equivalent |--runall| exists for |pythontex.py|. The package option |rerun=always| is essentially equivelent. % -% +% % \DescribeMacro{rerun=never/modified/errors/warnings/always default:errors} % % This option sets the threshold for re-executing code. By default, \pytex\ will rerun code that has been modified or that produced errors on the last run. Sometimes, we may wish to have a more lenient setting (only rerun if modified) or a more stringent setting (rerun even for warnings, or always rerun). |never| never executes code; a warning is issued if there is modified code. |modified| only executes code that has been modified. |errors| executes all modified code as well as all code that produced errors on the last run; this is the default. |warnings| executes all modified code, as well as all code that produced errors or warnings. |always| executes all code regardless of its condition. @@ -481,9 +827,13 @@ % % \DescribeMacro{autoprint=\meta{none}/true/false default:true \meta{none}=true} % +% ~ +% +% ~ +% % \DescribeMacro{autostdout=\meta{none}/true/false default:true \meta{none}=true} % -% Whenever a |print| command/statement is used, the printed content will automatically be included in the document, unless the code doing the printing is being typeset.\footnote{Note that |autoprint| only works within the body of the document. The |code| command and environment can be used in the preamble, but |autoprint| is disabled there. It is usually a not a good idea to print in the preamble, because nothing can be typeset; the only thing that could be validly printed is \LaTeX\ commands that do not typeset content, such as macro definitions. Thus, it is appropriate that printed content is only brought in while in the preamble if it is explicitly requested via |\string\printpythontex|. This approach is also helpful for writing packages using \pytex, since the author does not have to worry about any \LaTeX\ commands printed by the package either not being included (if |autoprint| is relied upon, but the user turns it off) or being included twice (if |\string\printpythontex| is used and |autoprint| is enabled). Printing should only be used in the preamble with great care.} In that case, the printed content must be included using the |\printpythontex| or |\stdoutpythontex| commands. +% Whenever a |print| command/statement is used, the printed content will automatically be included in the document, unless the code doing the printing is being typeset.\footnote{Note that |autoprint| only works within the body of the document. The |code| command and environment can be used in the preamble, but |autoprint| is disabled there. It is usually a not a good idea to print in the preamble, because nothing can be typeset; the only thing that could be validly printed is \LaTeX\ commands that do not typeset content, such as macro definitions. Thus, it is appropriate that printed content is only brought in while in the preamble if it is explicitly requested via |\printpythontex|. This approach is also helpful for writing packages using \pytex, since the author does not have to worry about any \LaTeX\ commands printed by the package either not being included (if |autoprint| is relied upon, but the user turns it off) or being included twice (if |\printpythontex| is used and |autoprint| is enabled). Printing should only be used in the preamble with great care.} In that case, the printed content must be included using the |\printpythontex| or |\stdoutpythontex| commands. % % Printed content is pulled in directly from the external file in which it is saved, and is interpreted by \LaTeX\ as \LaTeX\ code. If you wish to avoid this, you should print appropriate \LaTeX\ commands with your content to ensure that it is typeset as you desire. Alternatively, you may use |\printpythontex| or |\stdoutpythontex| to bring in printed content in verbatim form, using those commands' optional |verb| and |verbatim| options. % @@ -526,17 +876,17 @@ % % This option determines whether the |upquote| package is loaded. In general, the |upquote| package should be loaded, because it ensures that quotes within verbatim contexts are ``upquotes,'' that is, \expandafter|\textquotesingle| rather than |'|. % -% Using |upquote| is important beyond mere presentation. It allows code to be copied directly from the compiled PDF and executed without any errors due to quotes |'| being copied as acute accents \texttt{\'}. +% Using |upquote| is important beyond mere presentation. It allows code to be copied directly from the compiled PDF and executed without any errors due to quotes |'| being copied as acute accents \texttt{\'}. % % \DescribeMacro{fixlr=\meta{none}/true/false default:false \meta{none}=true} % % This option removes ``extra'' spacing around |\left| and |\right| in math mode. This spacing is sometimes undesirable, especially when typesetting functions such as the trig functions. See the implementation for details. Similar functionality is provided by the \href{http://www.ctan.org/pkg/mleftright}{\texttt{mleftright}} package % -% \DescribeMacro{keeptemps=\meta{none}/all/code/none default:none \meta{none}=all} +% \DescribeMacro{keeptemps=\meta{none}/all/code/none default:none \meta{none}=all} % % When \pytex\ runs, it creates a number of temporary files. By default, none of these are kept. The |none| option keeps no temp files, the |code| option keeps only code temp files (these can be useful for debugging), and the |all| option keeps all temp files (code, stdout and stderr for each code file, etc.). Note that this option does not apply to any user-generated content, since \pytex\ knows very little about that; it only applies to files that \pytex\ automatically creates by itself. % -% \DescribeMacro{prettyprinter=pygments/fancyvrb default:pygments} +% \DescribeMacro{prettyprinter=pygments/fancyvrb default:pygments} % % This allows the user to determine at the document level whether code is typeset using Pygments or |fancyvrb|. % @@ -548,31 +898,31 @@ % This determines whether inline content is pretty printed. If it is turned off, inline content is typeset with |fancyvrb|. % % -% \DescribeMacro{pygments=\meta{none}/true/false default:true \meta{none}=true} +% \DescribeMacro{pygments=\meta{none}/true/false default:true \meta{none}=true} % % This allows the user to determine at the document level whether code is typeset using Pygments rather than |fancyvrb|. It is an alias for |prettyprinter=pygments|. % % % \DescribeMacro{pyginline=\meta{none}/true/false default:true \meta{none}=true} -% +% % This option governs whether inline code, not just code in environments, is highlighted when Pygments highlighting is in use. When Pygments is in use, it will highlight everything by default. % % It is an alias for |prettyprintinline|. % % -% \DescribeMacro{pyglexer=\meta{pygments~lexer} default:\meta{none}} +% \DescribeMacro{pyglexer=\meta{pygments~lexer} default:\meta{none}} % % This allows a Pygments lexer to be set at the document level. In general, this option should \textbf{not} be used. It overrides the default lexer for all commands and environments, for both \pytex\ and Pygments content, and this is usually not desirable. It should be useful primarily when all content uses the same lexer, and multiple lexers are compatible with the content. % % -% \DescribeMacro{pygopt=\marg{pygments~options} default:\meta{none}} +% \DescribeMacro{pygopt=\marg{pygments~options} default:\meta{none}} % -% This allows Pygments options to be set at the document level. The options must be enclosed in curly braces |{}|. Currently, three options may be passed in this manner: |style=|\meta{style~name}, which sets the formatting style; |texcomments|, which allows \LaTeX\ in code comments to be rendered; and |mathescape|, which allows \LaTeX\ math mode (|$...$|) in comments. The |texcomments| and |mathescape| options may be used with an argument (for example, |texcomments=True/False|); if an argument is not supplied, |True| is assumed. Example: |pygopt={style=colorful, texcomments=True, mathescape=False}|. +% This allows Pygments options to be set at the document level. The options must be enclosed in curly braces |{}|. Currently, three options may be passed in this manner: |style=|\meta{style~name}, which sets the formatting style; |texcomments|, which allows \LaTeX\ in code comments to be rendered; and |mathescape|, which allows \LaTeX\ math mode (|$...$|) in comments. The |texcomments| and |mathescape| options may be used with an argument (for example, |texcomments=true/false|); if an argument is not supplied, |true| is assumed. Example: |pygopt={style=colorful, texcomments=true, mathescape=false}|. % % Pygments options for individual command and environment families may be set with the |\setpythontexpygopt| macro; for Pygments content, there is |\setpygmentspygopt|. These individual settings are always overridden by the package option. % % -% \DescribeMacro{fvextfile=\meta{none}/\meta{integer} default:$\infty$ \meta{none}=25} +% \DescribeMacro{fvextfile=\meta{none}/\meta{integer} default:$\infty$ \meta{none}=25} % % This option speeds the typesetting of long blocks of code that are created on the Python side. This includes content highlighted using Pygments and the |console| environment. Typesetting speed is increased at the expense of creating additional external files (in the \pytex\ directory). The \meta{integer} determines the number of lines of code at which the system starts using multiple external files, rather than a single external file. See the implementation for the technical details; basically, an external file is used rather than |fancyvrb|'s |SaveVerbatim|, which becomes increasingly inefficient as the length of the saved verbatim content grows. In most situations, this option should not be needed, or should be fine with the default value or similar ``small'' integers. % @@ -586,11 +936,11 @@ % % This governs the form of the filename that appears in error messages in Python console environments. Python errors messages have a form such as the following: % \begin{verbatim} -% >>> z = 1 + 34 + -% File "", line 1 -% z = 1 + 34 + -% ^ -% SyntaxError: invalid syntax +%>>> z = 1 + 34 + +% File "", line 1 +% z = 1 + 34 + +% ^ +%SyntaxError: invalid syntax % \end{verbatim} % The |stdin| option replaces || with ||, as it appears in a standard Python interactive session. The |console| option uses || instead, which is the default setting for the Python |code| module used by \pytex\ to create Python console environments. % @@ -607,13 +957,13 @@ % % All commands and environments take a session name as an optional argument. The session name determines the session in which the code is executed. This allows code to be executed in multiple independent sessions, increasing speed (sessions run in parallel) and preventing naming conflicts. If a session is not specified, then the |default| session is used. Session names should use the characters a-z, A-Z, 0-9, the hyphen, and the underscore. All characters used \textbf{must} be valid in file names, since session names are used to create temporary files. The colon is also allowed, but it is replaced with a hyphen internally, so the sessions |code:1| and |code-1| are identical. % -% In addition, all environments take |fancyvrb| settings as a second, optional argument. See the \href{http://www.ctan.org/tex-archive/macros/latex/contrib/fancyvrb}{|fancyvrb| documentation} for an explanation of accepted settings. This second optional argument \textbf{must} be preceeded by the first optional argument (session name). If a named session is not desired, the optional argument can be left empty (|default| session), but the square brackets |[]| must be present so that the second optional argument may be correctly identified: +% In addition, all environments take |fancyvrb| settings as a second, optional argument. See the \href{http://www.ctan.org/pkg/fancyvrb}{|fancyvrb| documentation} for an explanation of accepted settings. This second optional argument \textbf{must} be preceeded by the first optional argument (session name). If a named session is not desired, the optional argument can be left empty (|default| session), but the square brackets |[]| must be present so that the second optional argument may be correctly identified: % \begin{quote} % |\begin{|\meta{environment}|}[]|\oarg{fancyvrb~settings} % \end{quote} % % -% \subsubsection{Inline commands} +% \subsubsection{Inline commands} % \pytxtodo{Fix spacing around |\DescribeMacro|!} % Inline commands are suitable for single lines of code that need to be executed within the body of a paragraph or within a larger body of text. The commands use arbitrary code delimiters (like |\verb| does), which allows the code to contain arbitrary characters. Note that this is only guaranteed to work properly when the inline commands are \textbf{not} inside other macros. If an inline command is used within another macro, the code will be read by the external macro before \pytex\ can read the special code characters (that is, \LaTeX\ will tokenize the code). The inline commands can work properly within other macros, but it is best to stick with curly braces for delimiters in this case and you may have trouble with the hash |#| and percent |%| characters. % @@ -621,16 +971,37 @@ % % This command is used for including variable values or other content that can be converted to a string. It is an alternative to including content via the |print| statement/function within other commands/environments. % -% The |\py| command sends \meta{code} to Python, and Python returns a string representation of \meta{code}.\pytxtodo{Link to details about Python built-ins} \meta{opening~delim} and \meta{closing~delim} must be either a pair of identical, non-space characters, or a pair of curly braces. If curly braces are used as delimiters, then curly braces may only be used within \meta{code} if they are paired. Thus, |\py{1+1}| sends the code |1+1| to Python, Python evaluates the string representation of this code, and the result is returned to \LaTeX\ and included as |2|. The commands |\py#1+1#| and |\py@1+1@| would have the same effect. The command can also be used to access variable values. For example, if the code |a=1| had been executed previously, then |\py{a}| simply brings the string represantation of |a| back into the document as |1|. +% The |\py| command sends \meta{code} to Python, and Python returns a string representation of \meta{code}.\pytxtodo{Link to details about Python built-ins} \meta{opening~delim} and \meta{closing~delim} must be either a pair of identical, non-space characters, or a pair of curly braces. If curly braces are used as delimiters, then curly braces may only be used within \meta{code} if they are paired. Thus, |\py{1+1}| sends the code |1+1| to Python, Python evaluates the string representation of this code, and the result is returned to \LaTeX\ and included as |2|. The commands |\py#1+1#| and |\py@1+1@| would have the same effect. The command can also be used to access variable values. For example, if the code |a=1| had been executed previously, then |\py{a}| simply brings the string represantation of |a| back into the document as |1|. % -% Assignment is \textbf{not} allowed using |\py|. For example, |\py{a=1}| is \textbf{not} valid. This is because assignment cannot be converted to a string.\footnote{It would be simple to allow any code within |\textbackslash py|, including assignment, by using a |try/except| statement. In this way, the functionality of |\textbackslash py| and |\textbackslash pyc| could be merged. While that would be simpler to use, it also has serious drawbacks. If |\textbackslash py| is not exclusively used to typeset string representations of \meta{code}, then it is no longer possible on the \LaTeX\ side to determine whether a command should return a string. Thus, it is harder to determine, from within a \TeX\ editor, whether |pythontex.py| needs to be run; warnings for missing Python content could not be issued, because the system wouldn't know (on the \LaTeX\ side) whether content was indeed missing.} +% Assignment is \textbf{not} allowed using |\py|. For example, |\py{a=1}| is \textbf{not} valid. This is because assignment cannot be converted to a string.\footnote{It would be simple to allow any code within |\py|, including assignment, by using a |try/except| statement. In this way, the functionality of |\py| and |\pyc| could be merged. While that would be simpler to use, it also has serious drawbacks. If |\py| is not exclusively used to typeset string representations of \meta{code}, then it is no longer possible on the \LaTeX\ side to determine whether a command should return a string. Thus, it is harder to determine, from within a \TeX\ editor, whether |pythontex.py| needs to be run; warnings for missing Python content could not be issued, because the system wouldn't know (on the \LaTeX\ side) whether content was indeed missing.} % -% The text returned by Python must be valid \LaTeX\ code. Verbatim and other special content is allowed under the pdfTeX and XeTeX engines (a known bug prevents it from working with LuaTeX). The primary reasons for using |\py| rather than |print| are (1) |\py| is more compact and (2) |print| requires an external file to be created for every command or environment in which it is used, while |\py| and equivalents for other families share a single external file. Thus, use of |\py| minimizes the creation of external files, which is a key design goal for \pytex.\footnote{For |\textbackslash py|, the text returned by Python is stored in macros and thus must be valid \LaTeX\ code, because \LaTeX\ interprets the returned content. The use of macros for storing returned content means that an external file need not be created for each use of |\textbackslash py|. Rather, all macros created by |\textbackslash py| and equivalent commands from other families are stored in a single file that is inputted. Note that even though the content is stored in macros, verbatim content is allowed, through the use of special macro definitions combined with \texttt{\string\scantokens}.} The main reason for using |print| rather than |\py| is if you need to include a very large amount of material; |print|'s use of external files won't use up \TeX's memory, and may give noticeably better performance once the material is sufficiently long. +% The text returned by Python must be valid \LaTeX\ code. Verbatim and other special content is allowed. The primary reasons for using |\py| rather than |print| are (1) |\py| is more compact and (2) |print| requires an external file to be created for every command or environment in which it is used, while |\py| and equivalents for other families share a single external file. Thus, use of |\py| minimizes the creation of external files, which is a key design goal for \pytex.\footnote{For |\py|, the text returned by Python is stored in macros and thus must be valid \LaTeX\ code, because \LaTeX\ interprets the returned content. The use of macros for storing returned content means that an external file need not be created for each use of |\py|. Rather, all macros created by |\py| and equivalent commands from other families are stored in a single file that is inputted. Note that even though the content is stored in macros, verbatim content is allowed, through the use of special macro definitions combined with \texttt{\string\scantokens}.} The main reason for using |print| rather than |\py| is if you need to include a very large amount of material; |print|'s use of external files won't use up \TeX's memory, and may give noticeably better performance once the material is sufficiently long. % % \DescribeMacro{\pyc\oarg{session}\meta{opening~delim}\meta{code}\meta{closing~delim}} % % This command is used for executing but not typesetting \meta{code}. The suffix |c| is an abbreviation of |code|. If the |print| statement/function is used within \meta{code}, printed content will be included automatically so long as the package |autoprint| option is set to true (the default setting). % +% \DescribeMacro{\pys\oarg{session}\meta{opening~delim}\meta{code}\meta{closing~delim}} +% +% This command performs variable and expression substitution, or string interpolation, on \meta{code}. Fields of the form |!{|\meta{expr}|}| in \meta{code} are replaced with the evaluated and printed output of \meta{expr}. Then the modified \meta{code} is inserted into the document and interpreted as \LaTeX. The suffix |s| is an abbreviation of |sub|, for ``substitute.'' +% +% This command is useful for inserting Python-generated content in contexts where the normal |\py| and |\pyc| would not function or would be inconvenient due to the restrictions imposed by \LaTeX. Since Python processes \meta{code} and performs substitutions \emph{before} the result is passed to \LaTeX, substitution fields may be anywhere, including within parts of \meta{code} that will become \LaTeX\ comments. +% +% Literal exclamation points |!| in \meta{code} only need to be escaped when they immediately precede an opening curly brace |{|, or when they precede exclamation points that precede a brace. Escaping is performed by doubling. Thus, |!!{| would indicate a literal exclamation point followed by a literal curly brace (|!{|), not the beginning of a substitution field. And |!!!{| would indicate a literal exclamation point (|!!|) followed by a substitution field (|!{...}|). Because curly braces |{}| only have the meaning of field delimiters when immediately following a non-escaped exclamation point, curly braces never need to be escaped. +% +% The rules for delimiting \meta{expr} depend on the number of curly braces used. +% \begin{description} +% \item[1 pair] If \meta{expr} is delimited by a single pair of braces, then \meta{expr} may contain curly braces so long as the braces only appear in matched pairs and are nested no more than 5 levels deep. This is essentially the same as standard \LaTeX\ tokenization rules except for the nesting limit. +% \item[2--6 pairs] If \meta{expr} is delimited by 2--6 immediately adjacent curly braces (|!{{...}}| to |!{{{{{{...}}}}}}|), then \meta{expr} may contain any combination of braces, paired or unpaired, so long as the longest sequence of identical brace characters is shorter than the delimiters. Thus, |!{{...}}| can only contain single braces |{| and |}| (paired or unpaired); |!{{{...}}}| can contain |{|, |}|, |{{|, or |}}|; and so forth. +% \end{description} +% In both cases, anything more than five identical, immediately adjacent braces will always trigger an error. If a greater level of nesting is needed, then a function should be created within a |pycode| environment and afterward used to assemble the desired result. +% +% Curly braces used for delimiting \meta{expr} must not be immediately adjacent to braces that are part of \meta{expr}, because it would be impossible to distinguish them in the general case. If \meta{expr} begins/ends with a literal curly brace, the brace should be preceded/followed by a space or other whitespace character; leading and trailing whitespace in \meta{expr} is stripped, so this will not affect the output. +% +% Besides braces, \meta{expr} may contain any character except for literal newlines. In some cases, it may be appropriate to represent newlines in escaped form (|\n|). In other cases, it will be more appropriate to perform most calculations within a preceding |pycode| environment, and then access them via a variable or function call. +% +% Because \meta{expr} is evaluated and printed, it must be suitable for insertion in a |print()| function (or the equivalent, for languages besides Python). If string conversion as performed by |print()| is not desirable, then commands for explicit string conversion should be used. +% % \DescribeMacro{\pyv\oarg{session}\meta{opening~delim}\meta{code}\meta{closing~delim}} % % This command is used for typesetting but not executing \meta{code}. The suffix |v| is an abbreviation for |verb|. @@ -647,6 +1018,13 @@ % This environment encloses code that is executed but not typeset. The second optional argument \meta{fancyvrb~settings} is irrelevant since nothing is typeset, but it is accepted to maintain parallelism with the |verbatim| and |block| environments. If the |print| statement/function is used within the environment, printed content will be included automatically so long as the package |autoprint| option is set to true (the default setting). % % +% \DescribeEnv{pysub~\oarg{session}\oarg{fancyvrb~settings}} +% +% This environment performs variable and expression substitution, or string interpolation, on the enclosed code. Fields of the form |!{|\meta{expr}|}| in \meta{code} are replaced with the evaluated and printed output of \meta{expr}. See the description of the |\pys| command for details about substitution and the substitution field syntax. +% +% +% +% % \DescribeEnv{pyverbatim~\oarg{session}\oarg{fancyvrb~settings}} % % This environment encloses code that is typeset but not executed. @@ -758,25 +1136,27 @@ % % All non-|console| families import |pythontex_utils.py|, and create an instance of the \pytex\ utilities class called |pytex|. This provides various utilities for interfacing with \LaTeX\ and \pytex. % +% The utilities class has an attribute |context|. This is a dictionary that can contain contextual information, such as page dimensions, from the \TeX\ side. Values may also be accessed as attributes rather than as dictionary keys. To determine what contextual information is available, and for additional details, see |\setpythontexcontext| under Section~\ref{sec:usage:advanced-usage}. For working with contextual data, the utilities class provides |pt_to_in()|, |pt_to_cm()|, |pt_to_mm()|, and |pt_to_bp()| methods for converting from \TeX\ points to other units. +% % The utilities class provides an interface for determining how Python objects are converted into strings in commands such as |\py|. The |pytex.set_formatter(|\meta{formatter}|)| method is used to set the conversion. Two formatters are provided: % \begin{itemize} % \item |'str'| converts Python objects to a string, using the |str()| function under Python 3 and the |unicode()| function under Python 2. (The use of |unicode()| under Python 2 should not cause problems, even if you have not imported |unicode_literals| and are not using unicode strings. All encoding issues should be taken care of automatically by the utilities class.) -% \item |'sympy_latex'| uses SymPy's |LatexPrinter| class to return context-sensitive \LaTeX\ representations of SymPy objects. Separate |LatexPrinter| settings may be created for the following contexts: |'display'| (displaystyle math), |'text'| (textstyle math), |'script'| (superscripts and subscripts), and |'scriptscript'| (superscripts and subscripts, of superscripts and subscripts). Settings are created via |pytex.set_sympy_latex(|\meta{context}|,|\meta{settings}|)|. For example, |pytex.set_sympy_latex('display', mul_symbol='times')| sets multiplication to use a multiplication symbol $\times$, but only when math is in displaystyle.\footnote{Internally, the |'sympy\_latex'| formatter uses the |\textbackslash mathchoice| macro to return multiple representations of a SymPy object, if needed by the current settings. Then |\textbackslash mathchoice| typesets the correct representation, based on context.} See the \href{http://docs.sympy.org/dev/modules/printing.html}{SymPy documentation} for a list of possible settings for the |LatexPrinter| class. +% \item |'sympy_latex'| uses SymPy's |LatexPrinter| class to return context-sensitive \LaTeX\ representations of SymPy objects. Separate |LatexPrinter| settings may be created for the following contexts: |'display'| (displaystyle math), |'text'| (textstyle math), |'script'| (superscripts and subscripts), and |'scriptscript'| (superscripts and subscripts, of superscripts and subscripts). Settings are created via |pytex.set_sympy_latex(|\meta{context}|,|\meta{settings}|)|. For example, |pytex.set_sympy_latex('display', mul_symbol='times')| sets multiplication to use a multiplication symbol $\times$, but only when math is in displaystyle.\footnote{Internally, the |'sympy_latex'| formatter uses the |\mathchoice| macro to return multiple representations of a SymPy object, if needed by the current settings. Then |\mathchoice| typesets the correct representation, based on context.} See the \href{http://docs.sympy.org/dev/modules/printing.html}{SymPy documentation} for a list of possible settings for the |LatexPrinter| class. % % By default, |'sympy_latex'| only treats matrices differently based on context. Matrices in displaystyle are typeset using |pmatrix|, while those in all other styles are typeset via |smallmatrix| with parentheses. % % The context-sensitive interface to SymPy's |LatexPrinter| is always available via |pytex.sympy_latex()|. % \end{itemize} % -% The \pytex\ utilities formatter may be set to a custom function that returns strings, simply by reassigning the |pytex.formatter()| method. For example, define a formatter function |my_func()|, and then |pytex.formatter=my_func|. +% The \pytex\ utilities formatter may be set to a custom function that returns strings, simply by reassigning the |pytex.formatter()| method. For example, define a formatter function |my_func()|, and then |pytex.formatter=my_func| within a |pycode| or |pythontexcustomcode| environment. Any subsequent uses of |\py| will then use |my_func()| to perform formatting. % % The utilities class also provides methods for tracking dependencies and created files. % \begin{itemize} % \item |pytex.add_dependencies(|\meta{dependencies}|)| This adds \meta{dependencies} to a list. If any dependencies in the list change, code is re-executed, even if the code itself has not changed (unless |rerun=never|). Modified dependencies are determined via either modification time (default) or hash; see the package option |hashdependencies| for details. This method is useful for tracking changes in external data and similar files. -% -% \meta{dependencies} should be one or more strings, separated by commas, that are the file names of dependencies. Dependencies should be given with relative paths from the current working directory, with absolute paths, or with paths based on the user's home directory (that is, starting with a tilde |~|). Paths can use a forward slash ``|/|'' even under Windows. Remember that by default, the working directory is the |pythontex-files-|\meta{jobname} directory where all \pytex\ temporary files are stored. This can be adjusted with |\setpythontexworkingdir|. % -% It is possible that a dependency of one session might be modified by another session while \pytex\ runs. The first session might not be executed during the \pytex\ run because its dependency was unmodified at the beginning. A more serious case occurs when the first session does run, but we don't know whether it accessed the dependency before or after the dependency was updated (remember, sessions run in parallel). \pytex\ keeps track of the time at which it started. Any sessions with dependencies that were modified after that time are set to re-execute on the next run. A warning is also issued to indicated that this is the case. +% \meta{dependencies} should be one or more strings, separated by commas, that are the file names of dependencies. Dependencies should be given with relative paths from the current working directory, with absolute paths, or with paths based on the user's home directory (that is, starting with a tilde |~|). Paths can use a forward slash ``|/|'' even under Windows. Remember that by default, the working directory is the main document directory. This can be adjusted with |\setpythontexworkingdir|. +% +% It is possible that a dependency of one session might be modified by another session while \pytex\ runs. The first session might not be executed during the \pytex\ run because its dependency was unmodified at the beginning. A more serious case occurs when the first session does run, but we don't know whether it accessed the dependency before or after the dependency was updated (remember, sessions run in parallel). \pytex\ keeps track of the time at which it started. Any sessions with dependencies that were modified after that time are set to re-execute on the next run. A warning is also issued to indicate that this is the case. % % \item |pytex.add_created(|\meta{created~files}|)| This adds \meta{created~files} to a list of files created by the current session. Any time the code for the current session is executed, \textbf{all of these files will be deleted}. Since this method deletes files, it should be used with care. It is intended for automating cleanup when code is modified. For example, if a figure's name is changed, the old figure would be deleted if its name had been added to the list. By default, \pytex\ can only clean up the temporary files it creates; it knows nothing about user-created files. This method allows user-created files to be specified, and thus added to \pytex's automatic cleanup. % @@ -785,20 +1165,28 @@ % Depending on how you use \pytex, this method may not be very beneficial. If all of the output is contained in the default output directory, or a similar directory of your choosing, then manual cleanup may be simple enough that this method is not needed. % \end{itemize} % -% These two methods for tracking files may be used manually. However, that is prone to errors, since you will have to modify both a \pytex\ utilities command and an open or save command every time you change a file name or add or remove a dependency or created file. It may be better to redefine your open and save commands, or define new ones, so that a single command opens (or saves) and adds a dependency (or adds a created file). For example, the following would create a version of the standard |open()| that automatically tracks dependencies and created files. +% These two methods for tracking files may be used manually. However, that is prone to errors, since you will have to modify both a \pytex\ utilities command and an open or save command every time you change a file name or add or remove a dependency or created file. It may be better to redefine your open and save commands, or define new ones, so that a single command opens (or saves) and adds a dependency (or adds a created file). For this reason, the \pytex\ utilities class provides an |open()| method that automatically tracks dependencies and created files. +% \begin{itemize} +% \item |pytex.open(|\meta{file}|, |\meta{mode}|, |\meta{args}|, | \meta{kwargs}|)| This method automatically tracks all files opened for reading (text or binary mode) as dependencies. It automatically tracks all files opened for writing (text or binary mode) as created files. Files opened for updating and appending will raise a warning, since it is not necessarily obvious how these files should be treated. The general form of the custom |open()| function is shown below. % \begin{verbatim} -%def track_open(file, mode='r', *args, **kwargs): -% if mode in ('r', 'rb'): -% pytex.add_dependencies(file) -% elif mode in ('w', 'wb'): -% pytex.add_created(file) -% return open(file, mode, *args, **kwargs) -%pytex.open = track_open +%def track_open(name, mode='r', *args, **kwargs): +% if mode in ('r', 'rt', 'rb'): +% pytex.add_dependencies(name) +% elif mode in ('w', 'wt', 'wb'): +% pytex.add_created(name) +% else: +% warnings.warn('Unsupported mode {0} for file tracking'.format(mode)) +% return open(name, mode, *args, **kwargs) % \end{verbatim} -% Notice that this approach does not deal with files opened for appending or updating; such cases require more complex logic. +% \textbf{Unicode note for Python 2:} By default, |pytex.open()| call the standard Python 2 |open()|. If more than 3 positional arguments are used, or if the |encoding| keyword is used, then \href{https://docs.python.org/2/library/io.html}{\texttt{io.open()}} will be called instead. So if you are working with Unicode, make sure to specify an encoding in |pytex.open()| so that |io.open()| will be used, or manually encode/decode everything. +% \end{itemize} % % The utilities class provides a pair of methods, |before()| and |after()|, that are called immediately before and after each chunk of user code. These may be redefined to customize the output of user code. For example, \LaTeX\ commands could be printed at the beginning and end of each command or environment, wrapping any content printed by the user. Or any matplotlib figures that were created in the chunk of code could be detected and saved, and \LaTeX\ commands to include them in the document could be printed. Or stdout could be redirected to a StringIO stream in |before()|, then processed in |after()| before being sent to the original stdout. % +% The |before()| and |after()| methods may be redefined in any code or block command or environment, using the techniques described below. Once they have been redefined, the new methods will be called for all subsequent commands and environments. When redefining these methods, it is important to realize that the order of redefinition may be important. For example, if the new |before()| and |after()| depend upon one another, then you should call the old |after()| (if it does anything), then redefine the methods, and finally call the new |before()|. This is necessary because |after()| will be called after the end of the command or environment in which redefinition takes place. If |after()| has been redefined so that it depends on the new |before()|, and the new |before()| has not yet been called, errors will likely result. Other methods of dealing with this scenario, involving disabling |before()| and |after()| for a given command or environment, are being considered as potential features for a future release. +% +% When redefining |before()| and |after()|, you may wish to have behavior that is command- or environment-specific. Information about the current command or environment is available in |pytex.command|. The string |i| corresponds to an inline command such as |\py|; |b|, to an inline block such as |\pyb|; |c|, to inline code such as |\pyc|. Similarly, |code| corresponds to a code environment and |block| to a block environment. +% % You may redefine |before()| and |after()| at the class level. For example, % \begin{verbatim} %def open(self): @@ -823,8 +1211,7 @@ % An example of using the |after()| method to automatically save and include all matplotlib figures created in a command or environment is shown below. This example is designed for the |pylab| family of commands, or when |from pylab import *| is used. If |pyplot| is imported as |plt| instead, then |plt.get_fignums()|, |plt.figure()|, |plt.savefig()|, |plt.close()|, etc., would be needed. % \begin{verbatim} %# Basename for figures that will be created -%pytex.basename = '_'.join([pytex.input_type, pytex.input_session, -% pytex.input_restart]) +%pytex.basename = '_'.join([pytex.family, pytex.session, pytex.restart]) % %# Need to keep track of total number of figures in each session %pytex.fignum = 0 @@ -885,19 +1272,23 @@ % % \DescribeMacro{\stdoutpythontex\oarg{mode}\oarg{options}} % +% ~\par +% % Unless the package option |autoprint| is true, printed content from |code| commands and environments will not be automatically included. Even when the |autoprint| option is turned on, |block| commands and environments do not automatically include printed content, since we will generally not want printed content immediately after typeset code. This macro brings in any printed content from the \textbf{last} command or environment. It is reset after each command/environment, so its scope for accessing particular printed content is very limited. It will return an error if no printed content exists. % % \meta{mode} determines how printed content is handled. It may be |raw| (interpreted as \LaTeX), |verb| (inline verbatim), or |verbatim|; |raw| is the default. Verbatim content is brought in via |fancyvrb|. \meta{options} consists of |fancyvrb| settings. % % % \DescribeMacro{\saveprintpythontex\marg{name}} -% +% % \DescribeMacro{\savestdoutpythontex\marg{name}} % % \DescribeMacro{\useprintpythontex\oarg{verbatim~options}\oarg{fancyvrb~options}\marg{name}} % % \DescribeMacro{\usestdoutpythontex\oarg{verbatim~options}\oarg{fancyvrb~options}\marg{name}} % +% ~\par~\par +% % We may wish to be able to access the printed content from a command or environment at any point after the code that prints it, not just before any additional commands or environments are used. In that case, we may save access to the content under \meta{name}, and access it later via |\useprintpythontex|\marg{name}. \meta{mode} must be |raw|, |verb|, or |verbatim|. If content is brought in verbatim, then \meta{fancyvrb~options} are applied. % % \DescribeMacro{\stderrpythontex\oarg{mode}\oarg{fancyvrb~options}} @@ -917,6 +1308,7 @@ % % \DescribeMacro{\usestderrpythontex\oarg{mode}\oarg{fancyvrb~options}\marg{name}} % +% ~\par % % Content written to |stderr| may be saved and accessed anywhere later in the document, just as |stdout| content may be. These commands should be used with care. Using Python-generated content at multiple locations within a document may often be appropriate. But an error message will usually be most meaningful in its context, next to the code that produced it. % @@ -924,15 +1316,17 @@ % % \DescribeMacro{\setpythontexautostdout\marg{boolean}} % +% ~\par +% % This allows autoprint behavior to be modified at various points within the document. The package-level |autoprint| option is also available for setting autoprint at the document level, but it is overridden by |\setpythontexautoprint|. \meta{boolean} should be |true| or |false|. -% +% % % \subsection{Pygments commands and environments} % % Although \pytex's goal is primarily the execution and typesetting of Python code from within \LaTeX, it also provides access to syntax highlighting for any language supported by Pygments. % % \DescribeMacro{\pygment\marg{lexer}\meta{opening~delim}\meta{code}\meta{closing~delim}} -% +% % This command typesets \meta{code} in a suitable form for inline use within a paragraph, using the specified Pygments \meta{lexer}. Internally, it uses the same macros as the \pytex\ inline commands. \meta{opening~delim} and \meta{closing~delim} may be a pair of any characters except for the space character, or a matched set of curly braces |{}|. % % As with the inline commands for code typesetting and execution, there is not an optional argument for |fancyvrb| settings, since almost all of them are not relevant for inline usage, and the few that might be should probably be used document-wide if at all. @@ -942,10 +1336,10 @@ % % This environment typesets its contents using the specified Pygments \meta{lexer} and applying the \meta{fancyvrb~settings}. % -% +% % \DescribeMacro{\inputpygments\oarg{fancyvrb~settings}\marg{lexer}\marg{external~file}} % -% This command brings in the contents of \meta{external~file}, highlights it using \meta{lexer}, and typesets it using \meta{fancyvrb~settings}. +% This command brings in the contents of \meta{external~file}, highlights it using \meta{lexer}, and typesets it using \meta{fancyvrb~settings}. % % % \DescribeMacro{\setpygmentsfv\oarg{lexer}\marg{fancyvrb~settings}} @@ -959,7 +1353,7 @@ % % If \meta{lexer} is not given, options are set for the entire document. % -% +% % \DescribeMacro{\setpygmentsprettyprinter\marg{printer}} % % This usually should not be needed. It allows the pretty printer for the document to be set; it is equivalent to using |\setpythontexprettyprinter| without an optional argument. Valid options for \meta{printer} are |fancyvrb| and |pygments|. @@ -984,7 +1378,7 @@ % % \pytex\ uses |fancyvrb| internally to typeset all code. Even code that is highlighted with Pygments is typeset afterwards with |fancyvrb|. Using |fancyvrb|, it is possible to set background colors for individual lines of code, but not for entire blocks of code, using |\FancyVerbFormatLine| (you may also wish to consider the |formatcom| option). For example, the following command puts a green background behind all the characters in each line of code: % \begin{verbatim} -% \renewcommand{\FancyVerbFormatLine}[1]{\colorbox{green}{#1}} +%\renewcommand{\FancyVerbFormatLine}[1]{\colorbox{green}{#1}} % \end{verbatim} % % If you need a completely solid colored background for an environment, or a highly customizable background, you should consider the |mdframed| package. Wrapping \pytex\ environments with |mdframed| frames works quite well. You can even automatically add a particular style of frame to all instances of an environment using the command @@ -1020,6 +1414,29 @@ % % % \subsection{Advanced \pytex\ usage} +% \label{sec:usage:advanced-usage} +% +% \DescribeMacro{\setpythontexcontext\marg{key-value~pairs}} +% +% This macro is used for passing contextual information such as page dimensions from the \TeX\ side to the Python/other language side. \meta{key-value~pairs} is a set of comma-delimited key-value pairs. An evaluated version of \meta{key-value~pairs} is passed to the programming language, wrapped in quotation marks to become a string. Thus, \meta{key-value~pairs} should \emph{not} contain quotation marks, backslashes, or other characters that would prevent the evaluated contents from being the body of a normal, quoted string. +% +% As an example, the following would pass the values of |\textwidth| and |\textheight| to the Python side. +%\begin{verbatim} +%\setpythontexcontext{textwidth=\the\textwidth, textheight=\the\textheight} +%\end{verbatim} +% Python would receive a string something like |"textwidth=390pt, textheight=592pt"|. This string would be parsed into key-value pairs, and the results stored in the |pytex.context| dictionary. For Python, the keys also become the names of attributes of |pytex.context|. Thus, the values may be accessed on the Python side via |pytex.context['textwidth']|, |pytex.context.textwidth|, etc. +% +% All contextual data is available as strings on the Python/other language side. For convenience, the utilities class provides unit conversion methods for converting from \TeX\ points to inches, centimeters, millimeters, and big (DTP or PostScript) points. These methods take integers, floats, or strings that consist of digits (optionally ending in ``pt''), and return floats. For example, |pytex.pt_to_in()|, |pytex.pt_to_cm()|, |pytex.pt_to_mm()|, |pytex.pt_to_bp()|. Keep in mind that the units of \TeX\ points are \href{http://tex.stackexchange.com/questions/41370/what-are-the-possible-dimensions-sizes-units-latex-understands}{$1/72.27$} of an inch, \emph{not} $1/72$ of an inch (which is a bp). +% +% There is also a type system for Python that allows the types of \meta{values} to be specified. Any \meta{value} beginning with |!!int| will become an integer; with |!!float|, a float; with |!!str|, a string. This notation is borrowed from \href{http://yaml.org/}{YAML}. For example, +%\begin{verbatim} +%\setpythontexcontext{a=!!int 42, b=!!float 42, c=!!str 42} +%\end{verbatim} +% \textbf{This type system is still under development and is subject to change in the future.} Once the system stabilizes, it will be extended to non-Python languages. Comments on the type system are welcome. +% +% The context may only be set in the preamble. +% +% Technical note: Contextual data is cached, so the dictionary (and its attributes, if applicable) is only updated when contextual data changes. This largely eliminates any potential overhead from contextual data. % % \DescribeMacro{\restartpythontexsession\marg{counter~value(s)}} % @@ -1035,29 +1452,58 @@ % % \DescribeMacro{\setpythontexoutputdir\marg{output~directory}} % -% By default, \pytex\ saves all temporary files and automatically generated content in a directory called |pythontex-files-|\meta{sanitized jobname}, where \meta{sanitized jobname} is just |\jobname| with any space characters or asterisks replaced with hyphens. This directory will be created by |pythontex.py|. If we wish to specify another directory (for example, if |\jobname| is long and complex, and there is no danger of two files trying to use the same directory), then we can use the |\setpythontexoutputdir| macro to redefine the output directory.\footnote{In the rare event that both |\string\setpythontexoutputdir| is used and |\string\printpythontex| is needed in the preamble, |\string\setpythontexoutputdir| must be used first, so that |\string\printpythontex| will know where to look for output.} +% By default, \pytex\ saves all temporary files and automatically generated content in a directory called |pythontex-files-|\meta{sanitized jobname}, where \meta{sanitized jobname} is just |\jobname| with any space characters or asterisks replaced with hyphens. This directory will be created by |pythontex.py|. If we wish to specify another directory (for example, if |\jobname| is long and complex, and there is no danger of two files trying to use the same directory), then we can use the |\setpythontexoutputdir| macro to redefine the output directory.\footnote{In the rare event that both |\setpythontexoutputdir| is used and |\printpythontex| is needed in the preamble, |\setpythontexoutputdir| must be used first, so that |\printpythontex| will know where to look for output.} % -% Any slashes in \meta{output~directory} should be forward slashes ``|/|'' (even under Windows). +% Any slashes in \meta{output~directory} should be forward slashes ``|/|'' (even under Windows). Tildes |~| may be used to refer to the user's home directory, including under Windows. % % \DescribeMacro{\setpythontexworkingdir\marg{working~directory}} % -% The \pytex\ working directory is the current working directory for \pytex\ scripts. This is the directory in which any open or save operations will take place, unless a path is explicitly specified. By default, the working directory is the same as the output directory. For example, if you are writing |my_file.tex| and save a matplotlib figure with |savefig('my_figure.pdf')|, then |my_figure.pdf| will be created in the output directory |pythontex-files-my_file|. But maybe you have a directory called |plots| in your document root directory. In that case, you could leave the working directory unchanged, and simply specify the relative path to |plots| when saving. Or you could set the working directory to |plots| using |\setpythontexworkingdir{plots}|, so that all content would automatically be saved there. If you want your working directory to be the document root directory, you should use a period (|.|) for \meta{working~directory}: |\setpythontexworkingdir{.}|. +% The \pytex\ working directory is the current working directory for \pytex\ scripts. This is the directory in which any open or save operations will take place, unless a path is explicitly specified. By default, the working directory is the same as the main document directory. For example, if you are writing |my_file.tex| and save a matplotlib figure with |savefig('my_figure.pdf')|, then |my_figure.pdf| will be created in the same directory as |my_file.tex|. But maybe you have a directory called |plots| in your document root directory. In that case, you could leave the working directory unchanged, and simply specify the relative path to |plots| when saving. Or you could set the working directory to |plots| using |\setpythontexworkingdir{plots}|, so that all content would automatically be saved there. % -% Any slashes in \meta{working~directory} should be forward slashes ``|/|'' (even under Windows). +% Any slashes in \meta{working~directory} should be forward slashes ``|/|'' (even under Windows). Tildes |~| may be used to refer to the user's home directory, including under Windows. % % The working directory is automatically added to Python's |sys.path|, so that code in the working directory there may be imported without a path being specified. % -% Note that in typical use scenarios, you should be able to use the output directory as the working directory. The |graphicx| package will automatically look for images and figures in the output directory, so long as you do not use the |\graphicspath| command outside the preamble.\footnote{\texttt{graphicx} looks for graphics in the document root directory and in the most recent graphics path defined by \texttt{\string\graphicspath}. \texttt{\string\graphicspath} stores the graphics path in \texttt{\string\Ginput@path}, overwriting any previous value. At the end of the preamble, \pytex\ appends the output directory to \texttt{\string\Ginput@path}. Thus, that directory will always be checked for graphics, so long as \texttt{\string\Ginput@path} is not overwritten by a subsequent use of \texttt{\string\graphicspath}. If you need to use \texttt{\string\graphicspath} within the document, you could consider creating a custom version that redefines \texttt{\string\Ginput@path} with the \pytex\ output directory automatically appended.} +% Note that in many use cases, you may be able to use the output directory as the working directory. The |graphicx| package will automatically look for images and figures in the output directory when it is used as the working directory, so long as you do not use the |\graphicspath| command outside the preamble.\footnote{\texttt{graphicx} looks for graphics in the document root directory and in the most recent graphics path defined by \texttt{\string\graphicspath}. \texttt{\string\graphicspath} stores the graphics path in \texttt{\string\Ginput@path}, overwriting any previous value. At the end of the preamble, \pytex\ appends the output directory to \texttt{\string\Ginput@path} if the output directory is being used as the working directory. Thus, that directory will always be checked for graphics, so long as \texttt{\string\Ginput@path} is not overwritten by a subsequent use of \texttt{\string\graphicspath}. If you need to use \texttt{\string\graphicspath} within the document, you could consider creating a custom version that redefines \texttt{\string\Ginput@path} with the \pytex\ output directory automatically appended.} To use the output directory as the working directory, you may enter the full name of the output directory manually, or use the text ``||'' as a shortcut: +%\begin{verbatim} +%\setpythontexworkingdir{} +%\end{verbatim} % % It is also possible to change the working directory from within Python code, via |os.chdir()|. % % +% \subsection{Working with other programs} +% +% \subsubsection{\texttt{latexmk}} +% +% \pytex\ is compatible with |latexmk|. How you configure |latexmk| largely depends on how you are using \pytex. +% +% If you are compiling in the same location as the document source (if you are \emph{not} using |-auxdir|, |-outdir|, or |$out_dir|, or alternatively |TEXINPUTS|), and are not using \pytex's dependency tracking, then adding a simple rule such as the following to your |.latexmkrc| should usually be sufficient. +%\begin{verbatim} +%add_cus_dep('pytxcode', 'tex', 0, 'pythontex'); +%sub pythontex { return system("pythontex \"$_[0]\""); } +%\end{verbatim} +% This tells |latexmk| that the document (|tex|) depends on the file of code extracted from the document (|pytxcode|).\footnote{This is a slightly atypical use, if not a ``misuse,'' of \texttt{add\_cus\_dep()}. In the standard usage, the first argument is the extension of a file that is used to create another file with the extension given in the second argument, via the rule named in the fourth argument. In this case, we just want to run the rule whenever files with the first extension are modified. The extension given in the second argument is irrelevant, so long as a file with the document name and that extension exists. Since the \texttt{tex} file itself will exist, its extension is a logical choice for the second argument.} Whenever the document is compiled, the file of code is updated. If |latexmk| detects that the code changed, then it will run \pytex. When \pytex\ runs, it will modify at least one file that is brought into the document. |latexmk| will detect this modification, and automatically recompile the document. +% +% If you are compiling to a different directory (using |-auxdir|, |-outdir|, or |$out_dir|, or alternatively |TEXINPUTS|), then the preceding rule may fail due to the different directory configuration. In that case, you should use |\setpythontexoutputdir{.}| so that \pytex\ will store its output in the current default location, rather than in a subdirectory, to ensure that |latexmk| will locate the output files. Since the |tex| source is no longer in the location of the compiled output, you also need a different dependency specification. It is probably simplest to use the |pytxmcr| file that \pytex\ always generates. +%\begin{verbatim} +%add_cus_dep('pytxcode','pytxmcr',0,'pythontex'); +%sub pythontex { return system("pythontex.py \"$_[0]\""); } +%\end{verbatim} +% Note that this configuration should \emph{always} work, but has the disadvantage of requiring that \pytex\ not use a subdirectory to isolate the files it automatically generates. +% +% If you are using \pytex's dependency tracking, then you should run \pytex\ once during \emph{every} compile cycle (unless you simply wish to run it manually, as needed). Checking the |pytxcode| for modification is not sufficient, because it does not reflect the state of dependencies. If you are testing for dependency modification using modification time (the default) rather than hashing, this should typically add very little overhead. If \pytex\ detects modified dependencies and actually does execute code, then the |pytxmcr| file will be updated, which will trigger another compile. It is possible to have \pytex\ run after each individual \LaTeX\ run by modifying |latexmk|'s |-latex|, |-pdflatex|, or |-xelatex| options. Ideally, however, \pytex\ would only run once per compile cycle. +% +% The situation is similar if you are using the |rerun=always| setting. The above rules will fail to run \pytex\ on each and every compile; in that situation, you should configure your |.latexmkrc| so that \pytex\ always runs at least once during \emph{every} compile cycle. +% +% +% % \section{\texttt{depythontex}} % \label{sec:depythontex} % % \pytex\ can greatly simplify the creation of documents. At the same time, by introducing dependence on non-\LaTeX\ external tools, it can constrain how these documents are used. For example, many publishers will not accept \LaTeX\ documents that require special packages or need special macros. To address this issue, the package includes a feature called |depythontex| that can convert a \pytex\ document into a plain \LaTeX\ document. -% -% +% +% % \subsection{Preparing a document that will be converted} % % The conversion process should work flawlessly in most cases, with no special formatting required. @@ -1069,7 +1515,7 @@ % \item If you need to insert content from Python in inline contexts, it is best to use |\py| or an equivalent command. If you use |print|, either directly (for example, from within |\pyc|) or via |\printpythontex|, make sure that the spacing following the printed content is correct. You may need to print an |\endinput| or |%| at the end of your content to prevent an extra trailing space. |depythontex| will attempt to reproduce the spacing of the original document, even if it is not ideal. See Section~\ref{sec:depythontex:technical-details} for additional details. % \item Some \LaTeX\ environments, such as the |verbatim| environment from the |verbatim| package and the |Verbatim| environment from |fancyvrb|, do not allow text to follow the |\end{|\meta{environment}|}|. If you bring Python-generated content that ends with one of these environments into your document, using |print| or |\py|, make sure that the end-of-environment command is followed by a newline. For example, if you are assembling a |Verbatim| environment to bring in, the last line should be the string % \begin{verbatim} -% '\\end{Verbatim}\n' +%'\\end{Verbatim}\n' % \end{verbatim} % Even if you neglect a final newline, |depythontex| will still function correctly in most cases. Whenever Python-generated content does not end with a newline, |depythontex| usually inserts one and gobbles spaces that follow the environment. This preserves the correct spacing while avoiding any issues produced by an end-of-environment command. But in some cases, |depythontex| cannot do this. For example, if |\py| is used to bring in a |Verbatim| environment, and there is text immediately after the |\py|, without any intervening space, |depythontex| cannot substitute a newline for spaces, because there are none. Because of the way that |print| and |\py| content is brought in, everything may still work correctly in the original \pytex\ document. But it would fail in the |depythontex| output. % \item Do not create \pytex\ commands or environments on the Python side and |print| or otherwise bring them in. That is too many levels of complexity! @@ -1080,7 +1526,7 @@ % \end{itemize} % % \subsection{Removing \pytex\ dependence} -% +% % Converting a document requires three steps. % \begin{enumerate} % \item Turn on the package option |depythontex|. Then compile the document, run |pythontex.py|, and compile the document again. Depending on the document, additional compiles may be necessary (for example, to resolve references). Any syntax highlighting will be turned off automatically during this process, to remove dependence on Pygments. @@ -1089,26 +1535,26 @@ % % \item Run the |depythontex.py| script. This takes the following arguments. % \begin{itemize} -% \item |--encoding ENCODING| This is the encoding of the \LaTeX\ file and all related files. If an encoding is not specified, UTF-8 is assumed. +% \item |--encoding| This is the encoding of the \LaTeX\ file and all related files. If an encoding is not specified, UTF-8 is assumed. % \item |--overwrite| This turns off the user prompt in the event that a file already exists with the output name, making overwriting automatic. % \item |--listing| This option specifies the commands and environments that are used for any typeset code. This can be |verbatim|, |fancyvrb|, |listings|, |minted|, or |pythontex|.\footnote{The \texttt{pythontex} option is included for completeness. In most cases, you would probably use \texttt{depythontex} to remove all dependence on \pytex. But sometimes it might be useful to remove all Python code while still using \pytex\ for syntax highlighting.} |verbatim| is used by default. An appropriate |\usepackage| command is automatically added to the output document's preamble. % % When code is typeset with any option other than |verbatim|, listing line numbering from the original document will be preserved. When code is typeset with any option other than |verbatim| and |fancyvrb|, syntax highlighting will also be preserved. The only exception is when |listings| is used, and |listings|'s language name does not correspond to Pygments' lexer name. In this case, you should use the |--lexer-dict| option to specify how the Pygments lexer is to be translated into a |listings| language. % \item |--lexer-dict| This option is used to specify how Pygments lexers are converted to |listings| languages, when the two do not have the same name. It takes a comma-separated list of the form % \begin{verbatim} -% ":, ..., ... " +%":, ..., ... " % \end{verbatim} % A Python-style dict will also be accepted. % \item |--preamble| This option allows additional commands to be added to the output document's preamble. This is useful when you want the output document to load a package that was automatically loaded by \pytex, such as |upquote|. % \item |--graphicspath| This option adds the |outputdir| to any existing graphics path defined by |\graphicspath|, or adds a |\graphicspath| command if one does not already exist. This causes the |depythontex| document to automatically look in the |outputdir| for graphics. Only use this option if you want to continue using the |outputdir| with the |depythontex| document. Graphics are further discussed below. -% \item |TEXNAME| The name of the \LaTeX\ file whose \pytex\ dependence is to be removed. -% \item |[OUTFILE]| By default, the script takes a file |.| and creates a new file called |depythontex_.|. If a name is given for the output file, that is used instead. +% \item |-o| |--output| The name of the output file. If no name is given, the converted file is written to |stdout|. +% \item |TEXNAME| The name of the \LaTeX\ file whose \pytex\ dependence is to be removed. % \end{itemize} % \item Compile the |depythontex| file, and compare it to the original. % % The original and |depythontex| files should be nearly identical. All Python-generated content is substituted directly, so it should be unchanged. Usually, any differences will be due to changes in the way that code is typeset. For example, by default all code in the |depythontex| file is typeset with |\verb| and |verbatim|. But |\verb| is more fragile than the inline \pytex\ commands (it isn't allowed inside other commands), and |verbatim| does not support line numbering or syntax highlighting. % -% Remember that the |depythontex| file will need to include any graphics created by \pytex. By default, these are saved in the \pytex\ |outputdir|, which is |pythontex-files-| unless you have customized it. They may be in other locations if you have set a non-default |workingdir| or have specified a path when saving graphics. Depending on your needs and configuration, you may wish to copy the graphics into a new location or specify their location via |\graphicspath|. Or you can run |depythontex| with the |--graphicspath| option, which will add the |outputdir| to any existing usage of |\graphicspath|, or add a |\graphicspath| command if one does not already exist.\footnote{Keep in mind that any time \texttt{\string\graphicspath} is used, it overwrites any previously specified path. If your document is using \texttt{\string\graphicspath} at multiple points in the preamble, or using it anywhere outside the preamble, then the \texttt{-{}-graphicspath} option will fail due to the path being overwritten.} +% Remember that the |depythontex| file will need to include any graphics created by \pytex. By default, these are saved in the document root directory. They may be in other locations if you have set a non-default |workingdir| or have specified a path when saving graphics. Depending on your needs and configuration, you may wish to copy the graphics into a new location or specify their location via |\graphicspath|. If you are using the |outputdir| as the |workingdir|, you can run |depythontex| with the |--graphicspath| option, which will add the |outputdir| to any existing usage of |\graphicspath|, or add a |\graphicspath| command if one does not already exist.\footnote{Keep in mind that any time \texttt{\string\graphicspath} is used, it overwrites any previously specified path. If your document is using \texttt{\string\graphicspath} at multiple points in the preamble, or using it anywhere outside the preamble, then the \texttt{-{}-graphicspath} option will fail due to the path being overwritten.} % % Depending on your needs, you may wish to customize |depythontex.py|. The actual substitutions are performed in a few functions that are defined at the beginning of the script. % \end{enumerate} @@ -1123,7 +1569,7 @@ % This spacing behavior is due to \LaTeX's |\input|. When the file of printed content is brought in via |\input|, \LaTeX\ removes any newline characters (|\n|, |\r|, or |\r\n|) at the end of each line, and adds a space at the end of each line (even if there wasn't a newline character). Thus, when the printed content is brought in, a space is added to its end. Since this space is within the |\input|'s curly braces |{}|, it is not combined with any following spaces in the \LaTeX\ document to make a single space. Rather, if the printed content is followed by one or more spaces, two spaces will result; and if it is followed immediately by text, there will be a single space before the text. % -% The space added by |\input| is often invisible, and even when it is not, it is sometimes desirable.\footnote{For example, |\string\printpythontex| behaves as a normal command, and gobbles following spaces, but the space from |\string\input| puts a space back. So you often get the space you want in inline contexts.} But this space can be an issue in some inline contexts. The simplest solution is to use a command like |\py| to bring in content inline. +% The space added by |\input| is often invisible, and even when it is not, it is sometimes desirable.\footnote{For example, |\printpythontex| behaves as a normal command, and gobbles following spaces, but the space from |\input| puts a space back. So you often get the space you want in inline contexts.} But this space can be an issue in some inline contexts. The simplest solution is to use a command like |\py| to bring in content inline. % % If a command like |\py| is not practical for some reason, there are at least three ways to deal with the space introduced by |\input|: by printing |\endinput| at the end of the printed content (ending the content before the final space), by printing |%| at the end of the printed content (commenting out the final space), or by using |\unskip| after the printed content (eating preceding spaces). |depythontex| will work with all three approaches, but only under a limited range of circumstances. In summary, |depythontex| works with |\endinput| and |%| only if they are the very last thing printed (before a final newline), and works with a following |\unskip|.\footnote{It would be possible to make \texttt{depythontex} work with \texttt{\string\endinput} and \texttt{\%} anywhere, not just at the very end of printed content. But doing so would require a lot of additional parsing, especially for \texttt{\string\endinput}, to be absolutely sure that we found an actual command rather than a string. Furthermore, there is no reason that there should be any content after an \texttt{\string\endinput} or \texttt{\%}, since such content would never be included in the document. Indeed, the current approach prevents any printed content from accidentally being eliminated in this manner.} % \begin{itemize} @@ -1157,16 +1603,16 @@ % This section will be expanded in the future. For now, it offers a brief summary. % % \subsection{Macro programming with \pytex} -% +% % In many situations, you can use \pytex\ commands inside macro definitions without any special consideration. For example, consider the following macro, for calculating powers. % \newcommand{\pow}[2]{\py{#1**#2}} % \begin{verbatim} -% \newcommand{\pow}[2]{\py{#1**#2}} +%\newcommand{\pow}[2]{\py{#1**#2}} % \end{verbatim} % Once this is defined, we can calculate |2**8| via |\pow{2}{8}|: \pow{2}{8}. Similarly, we can reverse a string. % \newcommand{\reverse}[1]{\py{"#1"[::-1]}} % \begin{verbatim} -% \newcommand{\reverse}[1]{\py{"#1"[::-1]}} +%\newcommand{\reverse}[1]{\py{"#1"[::-1]}} % \end{verbatim} % Now we can use |\reverse{``This is some text!''}|: \reverse{``This is some text!''}. % @@ -1174,16 +1620,16 @@ % % Of course, there are many cases where macros don't need arguments. Here is code for creating a macro that generates random polynomials. % \begin{verbatim} -% \begin{sympycode} -% from sympy.stats import DiscreteUniform, sample -% x = Symbol('x') -% a = DiscreteUniform('a', range(-10, 11)) -% b = DiscreteUniform('b', range(-10, 11)) -% c = DiscreteUniform('c', range(-10, 11)) -% def randquad(): -% return Eq(sample(a)*x**2 + sample(b)*x + sample(c)) -% \end{sympycode} -% \newcommand\randquad{\sympy{randquad()}} +%\begin{sympycode} +%from sympy.stats import DiscreteUniform, sample +%x = Symbol('x') +%a = DiscreteUniform('a', range(-10, 11)) +%b = DiscreteUniform('b', range(-10, 11)) +%c = DiscreteUniform('c', range(-10, 11)) +%def randquad(): +% return Eq(sample(a)*x**2 + sample(b)*x + sample(c), 0) +%\end{sympycode} +%\newcommand{\randquad}{\ensuremath{\sympy{randquad()}}} % \end{verbatim} % % If you are considering writing macros that involve \pytex, you should keep a few things in mind. @@ -1202,18 +1648,20 @@ % \section{Support for additional languages} % \label{sec:other-languages} % -% Beginning with v0.12, it is much simpler to add support for languages beyond Python. Support for several additional languages will be added in coming months. -% -% In the immediate future, support for additional languages will be part of \pytex. Later, it may make sense to provide an alternative interface for other languages. For example, a package could be created that provides access to \pytex\ internals in a language-agnostic manner, without having the word ``python'' as part of the command names. +% Details about adding support for additional languages are in \Cref{sec:other-languages:adding}. This section begins with a brief overview of supported languages and available features. % -% Languages beyond Python will typically not be enabled by default, to prevent potential macro naming conflicts with other packages. At least two possible base names for commands and environments will be provided for each language. Typically these will be the name of the language and the language's file extension. For example, Ruby has the |ruby| and |rb| base names. You can choose which base name to use for creating a family of commands and environments based on personal preference and potential naming conflicts. +% Languages beyond Python are typically not be enabled by default to prevent potential macro naming conflicts with other packages. Languages are enabled via the |usefamily| package option (\Cref{sec:usage}). For example, +%\begin{verbatim} +%\usepackage[usefamily=ruby]{pythontex} +%\end{verbatim} +% Usually at least two possible base names for commands and environments will be provided for each language. Typically these will be the name of the language and the language's file extension. For example, Ruby has the |ruby| and |rb| base names. You can choose which base name to use for creating a family of commands and environments based on personal preference and potential naming conflicts. % % % \subsection{Ruby} % % Support for Ruby was added in v0.12. Ruby support should be almost at the same level as that for Python. % -% The utilities class is called |RubyTeXUtils|, and the class instance is |rbtex|. The variables and methods are the same as those for Python (Section~\ref{sec:usage:commands:utilities-class}), except that there is not currently a |set_formatter()| method. (The Python utilities class has the special SymPy formatter, but there aren't yet any specialized formatters for Ruby.) +% The utilities class is called |RubyTeXUtils|, and the class instance is |rbtex|. The variables and methods are the same as those for Python (Section~\ref{sec:usage:commands:utilities-class}), except that there is not currently a |set_formatter()| or an |open()| method. (The Python utilities class has the special SymPy formatter, but there aren't yet any specialized formatters for Ruby.) % % A family of commands and environments for Ruby is not created by default. Two base names are provided for families: |ruby| and |rb|. Preconfigured families for these names may be created via the |usefamily| package option. Keep in mind that a |ruby| command is defined as part of the Ruby package in the \href{http://www.ctan.org/pkg/cjk}{CJK package}. I am unaware of a package that provides an |\rb| command. % @@ -1224,14 +1672,86 @@ % % Support for Julia was added in v0.12. Julia support should be at almost the same level as that for Python. The format of Julia stderr is somethat different from that of Python and Ruby. This required a modified parsing and synchronization algorithm. The current system is functional but will likely change somewhat in the future. % -% The utilities class is called |JuliaTeXUtils|, and the class instance is |jltex|. The variables and methods are the same as those for Python (Section~\ref{sec:usage:commands:utilities-class}), except that there is not currently a |set_formatter()| method. (The Python utilities class has the special SymPy formatter, but there aren't yet any specialized formatters for Julia.) +% The utilities ``class'' is called |JuliaTeXUtils| (it is actually a composite type, very similar to a struct), and the ``class'' instance is |jltex|. The variables and methods are the same as those for Python (Section~\ref{sec:usage:commands:utilities-class}), except that there is not currently a |set_formatter()| method or an |open()| method. (The Python utilities class has the special SymPy formatter, but there aren't yet any specialized formatters for Julia.) % -% A family of commands and environments for Ruby is not created by default. Two base names are provided for families: |julia| and |jl|. Preconfigured families for these names may be created via the |usefamily| package option. Keep in mind that Pygments only added Julia support in version 1.6, so you may need to update your Pygments installation, or just change the default lexer. +% A family of commands and environments for Julia is not created by default. Two base names are provided for families: |julia| and |jl|. Preconfigured families for these names may be created via the |usefamily| package option. Keep in mind that Pygments only added Julia support in version 1.6, so you may need to update your Pygments installation, or just change the default lexer. % % Julia exceptions are synchronized with the document, but the line numbering does not always correspond to the Python equivalent. This is because Julia allows expressions to be continued on subsequent lines in ways that Python does not. % +% \subsubsection*{Console} +% +% Julia console support was added in v0.16. It may be enabled by loading \pytex\ with |usefamily=juliacon|. The |juliaconsole| environment uses \href{https://github.com/mpastell/Weave.jl}{\textsf{Weave.jl}} internally to evaluate code. There is also a |juliaconcode| environment that executes code but typesets nothing. +% +% +% \subsection{Octave} +% +% Support for Octave was added in v0.13. Octave support should be at almost the same level as that for Python. Parsing of stderr for synchronization is successful in most cases but not ideal; this will be improved in a future release by a rewrite of the stderr parser. +% +% Octave does not have a genuine utilities class, since it only supports |@CLASS| classes and does not yet support newer MATLAB-style |classdef|. As a result of this limited support for classes, there is a struct |octavetex| rather than a utilities class instance |octavetex|. What would have been attributes of a utilities class instance are instead fields of the struct. What would have been methods of a class are instead anonymous functions. This allows |octavetex| to be used in most respects as if it were a class instance, especially insofar as syntax is concerned. +% +% There are no |set_formatter()| or |open()| methods. +% +% If any ``methods'' need to be overwritten, the simplest approach is probably to define a function and then set the appropriate struct field to an anonymous function that will call that function. For example, to replace the default |octavetex.before()|, we might define a function |before()|, and then use the command |octavetex.before = @() before();|. Of course, if the function is sufficiently short, it will be simpler just to put everything in the anonymous function: |octavetex.before = @() ;| +% +% A family of commands and environments for Octave is not created by default; the base name |octave| is provided. +% +% When |\setpythontexcontext| is used, it must be accessed as struct fields, of the form |octavetex.context.|. +% +% +% \subsection{bash} +% +% Support for bash was added in v0.15. Support for bash is very basic. Bash commands may be executed, and their output (stdout and stderr) may be typeset. As with other languages, all commands are executed in a single session unless the user specifies otherwise. There is not a utilities class or any related features. +% +% Bash will work with Windows if it is installed. +% +% +% \subsection{Rust} +% +% Support for Rust was added in v0.15, with the command/environment base names |rs| and |rust|. Complete support is provided, except that the utilities struct |rstex| does not have an |open()| method. Also |rstex.formatter()|, |rstex.before()|, and |rstex.after()| may need additional refinement in the future to make them more convenient to work with. All user code is inserted within a template-generated |main()| function; |main()| should not be defined explicitly. Future refinements of \pytex's template system may allow user code outside of |main()|. +% +% Because Rust typically gives a long sequence of errors, \pytex\ processing and synchronization of |stderr| is currently verbose and may need to be improved in the future as well. There is no support for encodings other than UTF-8. Currently, executables always use the |.exe| extension, even under non-Windows systems. +% +% Due to the way |rstex| is used in template-generated code, it needs to remain a mutable local variable. This means that, while there should be no problem using it through either shared or mutable references, taking it by value requires that the ``altered'' copy is reassigned to a new variable that shadows the old one. That is, code that needs to work with |rstex| by value should look like +% \begin{Verbatim}[commandchars=\\\{\}, gobble=2] +% let mut rstex = \meta{code}; +% \end{Verbatim} +% Additionally, when using |\rust| and |\rs|, keep in mind that these wrap code in a block, so you \emph{cannot} use |rstex| by value in these contexts (both shared and mutable references are still fine, though). +% +% +% \subsection{R} +% +% Support for R was added in v0.17. +% +% Loading \pytex\ with |usefamily=R| enables the |R| family of commands and environments (|\R|, |\Rc|, |Rcode|, ...). These execute code with |Rscript|. The |methods| library is loaded automatically as part of the template code. Expressions passed to the |\R| command are converted into strings via |toString()|. There is currently no utilities class or related features. A null graphics device, |pdf(file=NULL)|, is created by default to avoid the automatic, unintentional creation of plot files with default names. Plots that are to be saved require explicit graphics commands. +% +% \subsubsection*{Console} +% +% Loading \pytex\ with |usefamily=Rcon| enables the |Rconsole| environment, which executes code to emulate an interactive R session. There is also an |Rconcode| environment that executes code but typesets nothing. Code is executed with |Rscript|. The |methods| library is loaded automatically as part of the template code. The option |echo=TRUE| is used to intersperse code with output, while |error=function(){}| is used to avoid halting on errors. A null graphics device, |pdf(file=NULL)|, is created by default to avoid the automatic, unintentional creation of plot files with default names. Plots that are to be saved require explicit graphics commands. +% +% +% \subsection{Perl} +% +% Support for Perl was added in v0.17. +% +% Loading \pytex\ with |usefamily=perl| enables the |perl| family of commands and environments. Alternatively, |usefamily=pl| may be used to enable the |pl| family. There is currently no utilities class or related features. +% +% +% \subsection{Perl 6} +% +% Support for Perl 6 was added in v0.17. +% +% Loading \pytex\ with |usefamily=perlsix| enables the |perlsix| family of commands and environments. Alternatively, |usefamily=psix| may be used to enable the |psix| family. There is currently no utilities class or related features. +% +% +% \subsection{JavaScript} +% +% Support for JavaScript was added in v0.17. +% +% Loading \pytex\ with |usefamily=javascript| enables the |javascript| family of commands and environments. Alternatively, |usefamily=js| may be used to enable the |js| family. There is a utilities object |jstex|. +% % % \subsection{Adding support for a new language} +% \label{sec:other-languages:adding} % % Adding support for an additional language involves creating two templates, creating a new instance of a class, and using a \pytex\ macro. In some cases, additional changes may be necessary for full support. The information below does not deal with creating |console| families; additional support for user-defined |console| families will be added in the future. % @@ -1241,40 +1761,42 @@ % % \subsubsection{Template} % -% \pytex\ executes user code by inserting it in a script template. Replacement fields in the template are indicated by double curly braces: |{{|\meta{field}|}}|. Space between \meta{field} and the braces is allowed; |{{ |\meta{field}| }}| is valid. Replacement fields (\textbf{including} the braces) should be surrounded by quotation marks or equivalent when the replacement is to be a string rather than literal code. +% \pytex\ executes user code by inserting it in a script template. Replacement fields in the template are indicated by curly braces: |{|\meta{field}|}|.\footnote{This follows Python's \href{http://docs.python.org/2/library/string.html\#formatstrings}{format string syntax}. Literal curly braces are obtained by doubling.} Space between \meta{field} and the braces is not allowed. Replacement fields (\textbf{including} the braces) should be surrounded by quotation marks or equivalent when the replacement is to be a string rather than literal code. % % The template should perform the following tasks. % \begin{itemize} -% \item Set the script encoding. The |{{encoding}}| field will be replaced with a user-specified encoding or the default |UTF-8|. If you are not using anything beyond ASCII, this is not strictly necessary. -% \item Python templates should have a |{{future}}| field at the beginning, for compability with Python 2 and the package option |pyfuture|.\footnote{The beginning of user code is parsed for imports from \texttt{\_\_future\_\_}. Any imports are collected and inserted into the \texttt{\{\{future\}\}} field.} -% \item Set the |stdout| and |stderr| encoding, again using |{{encoding}}|. As before, this is not strictly necessary when only ASCII support is needed. -% \item Create a language-specific equivalent of the \pytex\ utilities class.\footnote{Python templates can import the \pytex\ utilities class. In that case, \texttt{sys.path.append('\{\{utilspath\}\}')} is needed before the import, so that the location of the utilities class is known.} Create an instance of this class. It is recommended that the class be called \meta{language~name}|TeXUtils| and the instance \meta{language~extension}|tex|, by analogy with the Python case.\footnote{The class could be called \meta{language~name}\texttt{TeX}. In that case, the class and the instance would have the very same name (except for capitalization) in cases where the language name and extension are the same (for example, Lua). That is probably not desirable, and besides, \texttt{Utils} adds additional clarity. The instance name \meta{language~extension}\texttt{tex} is recommended because it will be short and easily remembered. Plain \texttt{tex} could be used instead, but that would be less descriptive (it lacks the interface connotations) and would not remind the user of the language currently in use (which could be beneficial in a document combining multiple languages, each with its own slightly different utilities class).} When the \meta{language~extension} is only a single character or is shared by multiple languages, it may be better to use the full \meta{language~name} or an abbreviation in the name of the class instance. +% \item Set the script encoding. The |{encoding}| field will be replaced with a user-specified encoding or the default |UTF-8|. If you are not using anything beyond ASCII, this is not strictly necessary. +% \item Python templates should have a |{future}| field at the beginning, for compability with Python 2 and the package option |pyfuture|.\footnote{The beginning of user code is parsed for imports from \texttt{\_\_future\_\_}. Any imports are collected and inserted into the \texttt{\{\{future\}\}} field.} +% \item Set the |stdout| and |stderr| encoding, again using |{encoding}|. As before, this is not strictly necessary when only ASCII support is needed. +% \item Create a language-specific equivalent of the \pytex\ utilities class.\footnote{Python templates can import the \pytex\ utilities class. In that case, |sys.path.append('{{utilspath}}')| is needed before the import, so that the location of the utilities class is known.} Create an instance of this class. It is recommended that the class be called \meta{language~name}|TeXUtils| and the instance \meta{language~extension}|tex|, by analogy with the Python case.\footnote{The class could be called \meta{language~name}\texttt{TeX}. In that case, the class and the instance would have the very same name (except for capitalization) in cases where the language name and extension are the same (for example, Lua). That is probably not desirable, and besides, \texttt{Utils} adds additional clarity. The instance name \meta{language~extension}\texttt{tex} is recommended because it will be short and easily remembered. Plain \texttt{tex} could be used instead, but that would be less descriptive (it lacks the interface connotations) and would not remind the user of the language currently in use (which could be beneficial in a document combining multiple languages, each with its own slightly different utilities class).} When the \meta{language~extension} is only a single character or is shared by multiple languages, it may be better to use the full \meta{language~name} or an abbreviation in the name of the class instance. % % For full \pytex\ support, the utilities class should provide the following methods: % \begin{itemize} % \item |formatter()|: For formatting content for inline commands equivalent to |\py|. This should take a single argument of any type. By default, it should return a standard string representation of its argument. % \item |before()| and |after()|: Initially, these should do nothing; they are provided to be redefined by the user. They should take no arguments. % \item |add_dependencies()| and |add_created()|: These should accept an arbitrary number of comma-separated strings (if supported by the language). Each method should append its arguments to a list or equivalent data structure, for later use. -% \item |cleanup()|: This prints a dependencies delimiter string |{{dependencies_delim}}| to |stdout|, then prints all dependencies (one per line), then prints a created files delimiter string |{{created_delim}}|, then prints all created files (one per line). The delimiters should be printed even if there are no dependencies or created files. The delimiters contain no backslashes or quotes. +% \item |cleanup()|: This prints a dependencies delimiter string |{dependencies_delim}| to |stdout|, then prints all dependencies (one per line), then prints a created files delimiter string |{created_delim}|, then prints the names of all created files (one per line). The delimiters should be printed even if there are no dependencies or created files. The delimiters contain no backslashes or quotes. % \end{itemize} % The utilities class should also provide several variables, as described below. -% \item Attempt to change to the working directory |{{workingdir}}|. Raise an error and exit if this is not possible. For convenience, the script should check for a |--manual| command line argument. If this argument is present, the script should proceed even if the working directory cannot be found. This allows the user to manually invoke the script for debugging (the script can be saved via |keeptemps|). +% \item Attempt to change to the working directory |{workingdir}|. Raise an error and exit if this is not possible. For convenience, the script should check for a |--manual| command line argument. If this argument is present, the script should proceed even if the working directory cannot be found. This allows the user to manually invoke the script for debugging (the script can be saved via |keeptemps|). % % The working directory should be added to the module search path (Python |sys.path|, Ruby |$:| or |$LOAD_PATH|, etc.), unless it is the same as the document root directory or is otherwise already on the module search path. -% \item For full compatibility, the template should have an |{{extend}}| field where additional module imports or other code may be inserted. This allows a basic template to be a created for each language. The basic template may then be customized for specific purposes. The |{{extend}}| field should be after the utilities class instance has been created, so that the workings of the utilities class (|formatter()|, |before()|, |after()|, etc.) may be customized by it. -% \item \LaTeX-related variables of the utilities class instance that do not change should be set. These use the fields |{{input_family}}|, |{{input_session}}|, and |{{input_restart}}|; all should be strings. These variables should be named after the fields if possible (for example, |pytex.input_family|). These variables are not strictly necessary, but they allow user code to access information about its \LaTeX\ context. -% \item There should be a |{{body}}| field where the body of the script is inserted. +% \item For full compatibility, the template should have an |{extend}| field where additional module imports or other code may be inserted. This allows a basic template to be a created for each language. The basic template may then be customized for specific purposes. The |{extend}| field should be after the utilities class instance has been created, so that the workings of the utilities class (|formatter()|, |before()|, |after()|, etc.) may be customized by it. +% \item \LaTeX-related variables of the utilities class instance that do not change should be set. These use the fields |{family}|, |{session}|, and |{restart}|; all should be strings. These variables should be named after the fields if possible (for example, |pytex.family|). These variables are not strictly necessary, but they allow user code to access information about its origin on the \TeX\ side. +% \item There should be a |{body}| field where the body of the script is inserted. % \item The script should end by calling the |cleanup()| method. % \end{itemize} % % \subsubsection{Wrapper} % Each chunk of user code is inserted into a wrapper template. This performs the following tasks. % \begin{itemize} -% \item Set additional \LaTeX-related utilities variables: |{{input_command}}|, |{{input_context}}|, |{{input_args}}|, |{{input_instance}}|, |{{input_line}}|. All of these should be strings. They are not required, but make possible closer \LaTeX\ integration. |{{input_context}}| and |{{input_args}}| are not yet supported on the \LaTeX\ side, but will allow information about the \LaTeX\ context (for example, page width) and arguments from \LaTeX\ commands to be passed to user code. -% \item Write a delimiter |{{stdout_delim}}| to |stdout| and a delimiter |{{stderr_delim}}| to |stderr|. Both delimiters should be strings. Both should be written in such a way that the delimiter is followed by a newline; the delimiters that are inserted in the wrapper template \textbf{do not} contain a newline. For example, something like |"{{stderr_delim}}\n"| might be necessary. The delimiters contain no backslashes or quotation marks. +% \item Set additional \LaTeX-related utilities variables: |{command}|, |{context}|, |{args}|, |{instance}|, |{line}|. They are not required, but make possible closer \LaTeX\ integration. |{args}| is not yet supported on the \LaTeX\ side, but will allow arguments from \LaTeX\ commands to be passed to user code. +% +% All utilities variables should be stored as strings, except for |context| and |args|. If possible, these should be dictionaries or equivalent associative arrays of string keys that point to string values. The dictionaries should be created by processing |{context}| and |{args}| into comma-separated lists of key-value pairs. For example, if |{context}| is the string ``|k1=v1, k2=v2|'', then |pytex.context| should be a dictionary, and |pytex.context['k1']| should yield the string ``|v1|''. The key-value pairs may optionally be accessed as attributes, when this is possible with a given language. For example, |pytex.context.k1| could yield the string ``|v1|''. +% \item Write a delimiter |{stdoutdelim}| to |stdout| and a delimiter |{stderrdelim}| to |stderr|. Both delimiters should be strings. Both should be written in such a way that the delimiter is followed by a newline; the delimiters that are inserted in the wrapper template \textbf{do not} contain a newline.\footnote{This way, we don't have to assume that all languages will use \texttt{\string\n} for the newline character.} For example, something like |"{stderrdelim}\n"| might be necessary. The delimiters contain no backslashes or quotation marks. % \item Call |before()|. -% \item Have a |{{code}}| field into which the current chunk of user code is inserted. -% \item Call |after()|. For languages like Ruby that allow statements to continue onto subsequent lines, without enclosing parentheses or other delimiters, the call to |after()| should immediately follow user code, without any intervenening empty lines. That way, any syntax errors from the last line of user code will be caught as early as possible, and the corresponding line numbers will be more meaningful. +% \item Have a |{code}| field into which the current chunk of user code is inserted. +% \item Call |after()|. % \end{itemize} % % \subsubsection{The \texttt{CodeEngine} class} @@ -1285,28 +1807,28 @@ % \item The instance name. This will be the base name for commands and environments that use the instance. For example, |\py|, |\pyc|, |pycode|, etc., rely on the |py| instance of the |CodeEngine| class. % \item The name of the language. In some cases, this may be the same as the instance name. % \item The filename extension for scripts (with or without a period). -% \item The command for running scripts. The script that is executed should be referred to as ``|{{file}}.|\meta{extension}'' (without the quotes).\footnote{It might seem that the extension is redundant, since it is specified separately. The command is specified in this form to simplify cases where there may be intermediary files in the execution process.} The interpreter may be hardcoded (|python {{file}}.py|), but it is best to leave it as a substitution field (|{{python}} {{file}}.py|) so that the |--interpreter| command-line option can be used to provide a specific interpreter. +% \item The command for running scripts. The script that is executed should be referred to as ``|{file}.|\meta{extension}'' (without the quotes).\footnote{It might seem that the extension is redundant, since it is specified separately. The command is specified in this form to simplify cases where there may be intermediary files in the execution process.} The interpreter may be hardcoded (|python {file}.py|), but it is best to leave it as a substitution field (|{python} {file}.py|) so that the |--interpreter| command-line option can be used to provide a specific interpreter. % \item The script template. % \item The wrapper template. -% \item A template that specifies how code from commands like |\py| should be inserted into a call to the |formatter()| method. The user code is specified by |{{code}}|. The output of the |formatter()| method should be written to |stdout|, so something like |'print(pytex.formatter({{code}}))'| is needed. +% \item A template that specifies how code from commands like |\py| should be inserted into a call to the |formatter()| method. The user code is specified by |{code}|. The output of the |formatter()| method should be written to |stdout|, so something like |'print(pytex.formatter({code}))'| is needed. % \item An optional list of strings (or an individual string) that gives patterns for identifying error messages. % \item An optional list of strings (or an individual string) that gives patterns for identifying warning messages. -% \item An optional list of strings (or an individual string) that gives patterns for identifying code line numbers in |stderr|. These patterns use the field |{{number}}|. These patterns are searched for in any line of |stderr| that contains the name of the script that was executed. +% \item An optional list of strings (or an individual string) that gives patterns for identifying code line numbers in |stderr|. These patterns use the field |{number}|. These patterns are searched for in any line of |stderr| that contains the name of the script that was executed. % \item An optional boolean that specifies whether the engine emulates an interactive console. Currently, user-defined engines that emulate consoles are not supported. % \item An optional string of startup commands for engines that emulate consoles. -% \item An optional list of strings (or an individual string) that specifies any files created during execution, beyond the script |{{file}}.|\meta{extension}. The field |{{file}}| may be used in file names. +% \item An optional list of strings (or an individual string) that specifies any files created during execution, beyond the script |{file}.|\meta{extension}. The field |{file}| may be used in file names, if files are created with the same base name as the script; this could be useful with compiled languages, which might have a |{file}.|\meta{extension} that ultimately results in a |{file}.out|, |{file}.exe|, etc. % \end{itemize} % % An example of creating the |py| engine is shown below. The |python_template| and |python_wrapper| are long enough that they are defined separately. % \begin{verbatim} -%CodeEngine('py', 'python', '.py', 'python {{file}}.py', -% python_template, python_wrapper, -% 'print(pytex.formatter({{code}}))', -% 'Error:', 'Warning:', ['line {{number}}', ':{{number}}:']) +%CodeEngine('py', 'python', '.py', 'python {file}.py', +% python_template, python_wrapper, +% 'print(pytex.formatter({code}))', +% 'Error:', 'Warning:', ['line {number}', ':{number}:']) % \end{verbatim} % The script template and wrapper templates may be defined with Python's triple-quoted strings. All content within such a string may be indented for clarity, as can be seen in |pythontex_engines.py|. Strings are automatically dedented when |CodeEngine| instances are created. % -% In addition to the |CodeEngine| class, there is also a |SubCodeEngine| class. It allows a new engine to be created based on an existing engine. It requires the name of the engine from which to inherit and the name of the new engine. All of the other arguments listed above are optional; if any are provided, they overwrite the inherited arguments. The class also takes one additional optional argument, |extend|. This is a string that specifies additional code to be entered in the inherited template, in the |{{extend}}| field. Subengines of subengines may be created; in that case, any |extend|s are cumulative. +% In addition to the |CodeEngine| class, there is also a |SubCodeEngine| class. It allows a new engine to be created based on an existing engine. It requires the name of the engine from which to inherit and the name of the new engine. All of the other arguments listed above are optional; if any are provided, they overwrite the inherited arguments. The class also takes one additional optional argument, |extend|. This is a string that specifies additional code to be entered in the inherited template, in the |{extend}| field. Subengines of subengines may be created; in that case, any |extend|s are cumulative. % % \subsubsection{Creating the \LaTeX\ interface} % Once a new engine has been created, access from the \LaTeX\ side must be provided. \pytex\ provides a macro for this purpose. @@ -1324,9 +1846,63 @@ % \label{sec:troubleshooting} % % \begin{itemize} -% \item If a \pytex\ document will not compile, you may want to delete the directory in which \pytex\ content is stored and try compiling from scratch. It is possible for \pytex\ to become stuck in an unrecoverable loop. Suppose you tell Python to print some \LaTeX\ code back to your \LaTeX\ document, but make a fatal \LaTeX\ syntax error in the printed content. This syntax error prevents \LaTeX\ from compiling. Now suppose you realize what happened and correct the syntax error. The problem is that the corrected code cannot be executed until \LaTeX\ correctly compiles and saves the code externally, but \LaTeX\ cannot compile until the corrected code has already been executed. One solution in such cases is to correct the code, delete all files in the \pytex\ directory, compile the \LaTeX\ document, and then run \pytex\ from scratch. You can also disable the inclusion of printed content using the |debug| package options. +% \item If a \pytex\ document will not compile, you may want to delete the directory in which \pytex\ content is stored and try compiling from scratch. It is possible for \pytex\ to become stuck in an unrecoverable loop. Suppose you tell Python to print some \LaTeX\ code back to your \LaTeX\ document, but make a fatal \LaTeX\ syntax error in the printed content. This syntax error prevents \LaTeX\ from compiling. Now suppose you realize what happened and correct the syntax error. The problem is that the corrected code cannot be executed until \LaTeX\ correctly compiles and saves the code externally, but \LaTeX\ cannot compile until the corrected code has already been executed. One solution in such cases is to correct the code, delete all files in the \pytex\ directory, compile the \LaTeX\ document, and then run \pytex\ from scratch. You can also disable the inclusion of printed content using the |debug| package option. +% +% You may also run \pytex\ with the |--debug| option to launch the default debugger, or use the debugger of your choice by adding code that launches a debugger and then running \pytex\ with the |--interactive| option. See Section~\ref{sec:installing-and-running:compiling} for more details. % % \item Dollar signs \$ may appear as \textsterling\ in italic code comments typeset by Pygments. This is a font-related issue. One fix is to |\usepackage[T1]{fontenc}|. +% +% \item The |tabular| environment can conflict with \pytex\ under some circumstances, due to how |tabular| functions. Among other things, printing within a |tabular| environment can cause errors, because printing involves bringing in external content via |\InputIfFileExists|, but that macro is not expandable.\footnote{For more information, see \href{http://tex.stackexchange.com/questions/50820/expandable-version-of-inputiffileexists-or-iffileexists}{this}, \href{http://tex.stackexchange.com/questions/50828/execute-non-expandable-code-inside-a-tabular-environment}{this}, and \href{http://tex.stackexchange.com/questions/50694/cannot-use-toprule-when-doing-input-inside-tabular-why}{this}.} There are a few different ways to work around the limitations of |tabular|. +% \begin{itemize} +%\item Put the printed content in a macro definition, and use the macro in |tabular|. You will have to create a dummy version of the macro, to avoid errors before the macro is defined by \pytex. An example is given below. The |\global\def| is needed so that the macro is defined outside of the |pycode| environment. +%\begin{verbatim} +%\let\row\relax +%\begin{pycode} +%print("\\global\\def\\row{a & b & c & d & e \\\\}") +%\end{pycode} +% +%\begin{tabular}{|c|c|c|c|c|} +%\row +%\end{tabular} +%\end{verbatim} +% +% \item Use |\py|. The end-of-row |\\| must be outside of the command. Example: +%\begin{verbatim} +%\begin{tabular}{|c|c|c|c|c|} +%\py{"a & b & c & d & e"} \\ +%\end{tabular} +%\end{verbatim} +% \end{itemize} +% \item \pytex\ commands like |\py| won't work inside |siunitx| macros, because |\py| and company aren't fully expandable.\footnote{For more details, see \href{http://tex.stackexchange.com/questions/35039/why-isnt-everything-expandable}{this}, \href{http://tex.stackexchange.com/questions/66118/advantages-and-disadvantages-of-fully-expandable-macros}{this}, and \href{http://tex.stackexchange.com/questions/164918/how-do-i-use-ifstreqcase-within-a-siunitx-command}{this}.} There are different ways to work around this; some examples are shown below. +%\begin{verbatim} +%\documentclass{article} +%\usepackage{siunitx} +%\usepackage{pythontex} +% +%\begin{pycode} +%def SI(var, unit): +% return '\\SI{' + str(var) + '}{' + unit + '}' +%\end{pycode} +% +%\newcommand{\pySI}[2]{\py{'\\SI{' + str(#1) + '}{#2}'}} +% +%\begin{document} +%\pyc{y = 4} +% +%The value of y is \py{SI(y, r'\metre')}. +% +%The value of y is \pySI{y}{\metre}. +% +%\end{document} +%\end{verbatim} +% Another example, this time using SymPy: +%\begin{verbatim} +%\newcommand{\sympySI}[2]{\sympy{SI(#1,r"#2")}} +%\begin{sympycode} +%def SI(var, unit): +% return '\\SI{{{0}}}{{{1}}}'.format(N(var, 4), unit) +%\end{sympycode} +%\end{verbatim} % \end{itemize} % % @@ -1350,7 +1926,7 @@ % \item Improved testing framework. % \item It might nice to include some methods in the \pytex\ utilities for formatting numbers (especially with SymPy and Pylab). % \item Test the behavior of files brought in via |\input| and |\include| that contain \pytex\ content. -% \item Continue adding support for additional languages. In preparation: Julia. Under consideration: Perl, Lua, MATLAB, Mathematica, Sage, R, Octave. +% \item Continue adding support for additional languages. Under consideration: Perl, Lua, MATLAB, Mathematica, Sage, R, Octave. % \end{itemize} % % \subsubsection{Modifications to consider} @@ -1380,6 +1956,10 @@ % % Thanks to Øystein Bjørndal for many suggestions and for help with OS X compatibility. % +% Thanks to Alexander Altman for suggesting Rust support and providing template code. +% +% Thanks to Nathan Carter for suggesting JavaScript support and providing template code. +% % % \PrintChangelog % @@ -1399,20 +1979,19 @@ % We store the name of the package in a macro for later use in warnings and error messages. % \begin{macrocode} \newcommand{\pytx@packagename}{PythonTeX} -\newcommand{\pytx@packageversion}{v0.12} +\newcommand{\pytx@packageversion}{0.19} % \end{macrocode} % % \subsection{Required packages} -% A number of packages are required. |fancyvrb| is used to typeset all code that is not inline, and its internals are used to format inline code as well. |etex| provides extra registers, to avoid the (probably unlikely) possibility that the many counters required by \pytex\ will exhaust the supply. |etoolbox| is used for string comparison and boolean flags. |xstring| provides string manipulation. |pgfopts| is used to process package options, via the |pgfkeys| package. |newfloat| allows the creation of a floating environment for code listings. |currfile| is needed to allow errors and warnings to be synchronized with content brought in via |\input|, |\include|, etc. |xcolor| or |color| is needed for syntax highlighting with Pygments. +% A number of packages are required. |fvextra|, which loads and extends |fancyvrb|, is used to typeset all code that is not inline. |fancyvrb| internals are used to format inline code. |etoolbox| is used for string comparison and boolean flags. |xstring| provides string manipulation. |pgfopts| is used to process package options, via the |pgfkeys| package. |newfloat| allows the creation of a floating environment for code listings. |currfile| is needed to allow errors and warnings to be synchronized with content brought in via |\input|, |\include|, etc. |xcolor| or |color| is needed for syntax highlighting with Pygments. % \begin{macrocode} -\RequirePackage{fancyvrb} -\RequirePackage{etex} +\RequirePackage{fvextra} \RequirePackage{etoolbox} \RequirePackage{xstring} \RequirePackage{pgfopts} \RequirePackage{newfloat} -\RequirePackage{currfile} -\AtBeginDocument{\@ifpackageloaded{color}{}{\RequirePackage{xcolor}}} +\@ifpackageloaded{currfile}{}{\RequirePackage{currfile}} +\AtEndPreamble{\@ifpackageloaded{color}{}{\RequirePackage{xcolor}}} % \end{macrocode} % % \subsection{Package options} @@ -1621,7 +2200,7 @@ % % \subsubsection{Upquote} % \begin{macro}{pytx@opt@upquote} -% The |upquote| option determines whether the |upquote| package is loaded. It makes quotes within verbatim contexts \expandafter|\textquotesingle| rather than |'|. This is important, because it means that code may be copied directly from the compiled PDF and executed without any errors due to quotes |'| being copied as acute accents \texttt{\'}. +% The |upquote| option determines whether the |upquote| package is loaded. It makes quotes within verbatim contexts \expandafter|\textquotesingle| rather than |'|. This is important, because it means that code may be copied directly from the compiled PDF and executed without any errors due to quotes |'| being copied as acute accents \texttt{\'}. % \begin{macrocode} \newbool{pytx@opt@upquote} \booltrue{pytx@opt@upquote} @@ -1662,7 +2241,7 @@ % By default, \pytex\ uses |fancyvrb| to typeset code. This provides nice formatting and font options, but no syntax highlighting. The |prettyprinter| options, and |pygments| alias, determine whether Pygments or |fancyvrb| is used to typeset code. Pygments is a generic syntax highlighter written in Python. Since \pytex\ sends code to Python anyway, having Pygments process the code is only a small additional step and in many cases takes little if any extra time to execute.\footnote{Pygments code highlighting is executed as a separate process by |pythontex.py|, so it runs in parallel on a multicore system. Pygments usage is optimized by saving highlighted code and only reprocessing it when changed.} % % Command and environment families obey the |prettyprinter| option by default, but they may be set to override it and always use Pygments or always use |fancyvrb|, via |\setpythontexprettyprinter| and |\setpygmentsprettyprinter|. -% \begin{macrocode} +% \begin{macrocode} \newbool{pytx@opt@pygments} \booltrue{pytx@opt@pygments} \pgfkeys{/PYTX/pkgopt/prettyprinter/.is choice} @@ -1707,14 +2286,29 @@ % % Currently, three options may be passed in this manner: |style=|\meta{style}, which sets the formatting style; |texcomments|, which allows \LaTeX\ in code comments to be rendered; and |mathescape|, which allows \LaTeX\ math mode (|$...$|) in comments. The |texcomments| and |mathescape| options may be used with a boolean argument; if an argument is not supplied, true is assumed. As an example of |pygopt| usage, consider the following: % \begin{verbatim} -% pygopt={style=colorful, texcomments=True, mathescape=False} +%pygopt={style=colorful, texcomments=true, mathescape=false} % \end{verbatim} -% The usage of capitalized |True| and |False| is more pythonic, but is not strictly require. % -% While the package-level |pygments| option may be overridden by individual commands and environments (though it is not by default), the package-level Pygments options cannot be overridden by individual commands and environments. +% While the package-level |pygments| option may be overridden by individual commands and environments (though it is not by default), the package-level Pygments options cannot be overridden by individual commands and environments. While we're defining storage for |pygopt|, go ahead and define parsing to extract |style| for later use under all cicumstances. This should be reorganized during the next refactoring. % \begin{macrocode} \def\pytx@pygopt{} -\pgfkeys{/PYTX/pkgopt/pygopt/.code=\def\pytx@pygopt{#1}} +\pgfkeys{/PYTX/pkgopt/pygopt/.code=\def\pytx@pygopt{#1}\pgfkeys{/PYTX/gopt/pygopt/.cd, #1}} +\pgfkeys{/PYTX/gopt/pygopt/.is choice} +\pgfkeys{/PYTX/gopt/pygopt/texcomments/.code=\relax} +\pgfkeys{/PYTX/gopt/pygopt/mathescape/.code=\relax} +\pgfkeys{/PYTX/gopt/pygopt/style/.code=\ifstrempty{#1}{}{\def\pytx@style{#1}}} +\pgfkeys{/PYTX/lopt/pygopt/.is choice} +\pgfkeys{/PYTX/lopt/pygopt/name/.code=\def\pytx@tmp@name{#1}} +\pgfkeys{/PYTX/lopt/pygopt/texcomments/.code=\relax} +\pgfkeys{/PYTX/lopt/pygopt/mathescape/.code=\relax} +\pgfkeys{/PYTX/lopt/pygopt/style/.code=\ifstrempty{#1}{}{% + \expandafter\def\csname pytx@style@\pytx@tmp@name\endcsname{#1}}} +\pgfkeys{/PYTX/popt/pygopt/.is choice} +\pgfkeys{/PYTX/popt/pygopt/name/.code=\def\pytx@tmp@name{#1}} +\pgfkeys{/PYTX/popt/pygopt/texcomments/.code=\relax} +\pgfkeys{/PYTX/popt/pygopt/mathescape/.code=\relax} +\pgfkeys{/PYTX/popt/pygopt/style/.code=\ifstrempty{#1}{}{% + \expandafter\def\csname pytx@style@PYG\pytx@tmp@name\endcsname{#1}}} % \end{macrocode} % \end{macro} % @@ -1725,7 +2319,7 @@ % A default value of 25 is set. There is nothing special about 25; it is just a relatively reasonably cutoff. If the option is unused, it has a value of $-1$, which is converted to the maximum integer on the Python side. % \begin{macrocode} \def\pytx@fvextfile{-1} -\pgfkeys{/PYTX/pkgopt/fvextfile/.default=25} +\pgfkeys{/PYTX/pkgopt/fvextfile/.default=25} \pgfkeys{/PYTX/pkgopt/fvextfile/.code=\IfInteger{#1}{% \ifnum#1>0\relax \def\pytx@fvextfile{#1}% @@ -1736,7 +2330,7 @@ } % \end{macrocode} % \end{macro} -% +% % \subsubsection{Python console environment} % \begin{macro}{\pytx@opt@pyconbanner} % This option governs the appearance (or disappearance) of a banner at the beginning of Python console environments. The options |none| (no banner), |standard| (standard Python banner), |default| (default banner for Python's |code| module, standard banner plus interactive console class name), and |pyversion| (banner in the form |Python x.y.z|) are accepted. @@ -1753,8 +2347,8 @@ % \begin{macro}{\pytx@opt@pyconfilename} % This option governs the file name that appears in error messages in the console. The file name may be either |stdin|, as it is in a standard interactive interpreter, or |console|, as it would typically be for the Python |code| module. % \begin{verbatim} -% Traceback (most recent call last): -% File "", line , in +%Traceback (most recent call last): +% File "", line , in % \end{verbatim} % \begin{macrocode} \def\pytx@opt@pyconfilename{stdin} @@ -1815,7 +2409,7 @@ % Once options are processed, we proceed to define a number of utility macros and setup the file input/output that is required by \pytex. We also create macros and perform setup needed by depythontex, since these are closely related to input/output. % % \subsubsection{Automatic counter creation} -% +% % \begin{macro}{\pytx@CheckCounter} % We will be using counters to give each command/environment a unique identifier, as well as to manage line numbering of code when desired. We don't know the names of the counters ahead of time (this is actually determined by the user's naming of code sessions), so we need a macro that checks whether a counter exists, and if not, creates it. % \begin{macrocode} @@ -1833,8 +2427,6 @@ % % This prevents the creation of unnecessary files, but it also has a significant drawback: only some content can be saved in a standard macro. In particular, verbatim content using |\verb| and |verbatim| will not work. So we need a way to save anything in a macro. The solution is to create a special macro that captures its argument verbatim. The argument is then tokenized when it is used via |\scantokens|. All of this requires a certain amount of catcode trickery. % -% While this approach works with the XeTeX and pdfTeX engines, it does not work with the LuaTeX engine, which has a known bug in its implementation of |\scantokens|.\footnote{\url{http://tracker.luatex.org/view.php?id=733}} So we provide a separate version for Lua\TeX\ that does not support verbatim. The space after the |#1| is intentional, so that the newline at the beginning of the macro definition is gobbled, and the macro content will start with text rather than leading whitespace. -% % \begin{macrocode} \def\pytx@SVMCR#1{% \edef\pytx@tmp{\csname #1\endcsname}% @@ -1851,32 +2443,34 @@ !expandafter!scantokens!expandafter{#1!empty}}% }% !endgroup -\expandafter\ifx\csname directlua\endcsname\relax\else -\def\pytx@SVMCR#1 {% - \edef\pytx@tmp{\csname #1\endcsname}% - \pytx@SVMCR@i} -\long\def\pytx@SVMCR@i#1\endpytx@SVMCR{% - \expandafter\gdef\pytx@tmp{#1}% -} -\fi % \end{macrocode}\end{macro} % +% \begin{macro}{pytx@Verbatim} +% \begin{macro}{pytx@SaveVerbatim} +% We need custom versions of |fancyvrb|'s |Verbatim| and |SaveVerbatim| environments, because we don't want to have to worry about the possibility of these environments containing literal |Verbatim| and |SaveVerbatim| environments. +% \begin{macrocode} +\DefineVerbatimEnvironment{pytx@Verbatim}{Verbatim}{} +\DefineVerbatimEnvironment{pytx@SaveVerbatim}{SaveVerbatim}{} +% \end{macrocode} +% \end{macro}\end{macro} +% % % \subsubsection{Code context} -% \begin{macro}{\pytx@context}\begin{macro}{\pytx@SetContext}\begin{macro}{\definepythontexcontext} +% \begin{macro}{\pytx@context}\begin{macro}{\pytx@SetContext}\begin{macro}{\setpythontexcontext} % It would be nice if when our code is executed, we could know something about its context, such as the style of its surroundings or information about page size. % -% By default, no contextual information is passed to \LaTeX. There is a wide variety of information that could be passed, but most use cases would only need a very specific subset. Instead, the user can customize what information is passed to \LaTeX. The |\definepythontexcontext| macro defines what is passed. It creates the |\pytx@SetContext| macro, which creates |\pytx@context|, in which the expanded context information is stored. The context should only be defined in the preamble, so that it is consistent throughout the document. +% By default, no contextual information is passed to \LaTeX. There is a wide variety of information that could be passed, but most use cases would only need a very specific subset. Instead, the user can customize what information is passed to \LaTeX. The |\setpythontexcontext| macro defines what is passed. It creates the |\pytx@SetContext| macro, which creates |\pytx@context|, in which the expanded context information is stored. The context should only be defined in the preamble, so that it is consistent throughout the document. % % If you are interested in typesetting mathematics based on math styles, you should use the |\mathchoice| macro rather than attempting to pass contextual information. % \begin{macrocode} -\newcommand{\definepythontexcontext}[1]{% +\newcommand{\setpythontexcontext}[1]{% + \Depythontex{cmd:setpythontexcontext:m:n}% \def\pytx@SetContext{% \edef\pytx@context{#1}% }% } -\definepythontexcontext{} -\@onlypreamble\definepythontexcontext +\setpythontexcontext{} +\@onlypreamble\setpythontexcontext % \end{macrocode} % \end{macro}\end{macro}\end{macro} % @@ -1928,7 +2522,7 @@ \@onlypreamble\restartpythontexsession \restartpythontexsession{default} % \end{macrocode} -% +% % \subsubsection{File input and output} % % \begin{macro}{\pytx@jobname} @@ -1945,38 +2539,41 @@ % \begin{macro}{\pytx@outputdir}\begin{macro}{\setpythontexoutputdir} % To keep things tidy, all \pytex\ files are stored in a directory that is created in the document root directory. By default, this directory is called |pythontex-files-|\meta{sanitized~jobname}, but we want to provide the user with the option to customize this. For example, when \meta{sanitized~jobname} is very long, it might be convenient to use |pythontex-|\meta{abbreviated~name}. % -% The command |\setpythontexoutputdir| stores the name of \pytex's output directory in |\pytx@outputdir|. If the |graphicx| package is loaded, the output directory is also added to the graphics path at the beginning of the document, so that files in the output directory may be included within the main document without the necessity of specifying path information. The command |\setpythontexoutputdir| is only allowed in the preamble, because the location of \pytex\ content should be specified before the body of the document is typeset. +% The command |\setpythontexoutputdir| stores the name of \pytex's output directory in |\pytx@outputdir|. The command |\setpythontexoutputdir| is only allowed in the preamble, because the location of \pytex\ content should be specified before the body of the document is typeset. % \begin{macrocode} +\def\pytx@outputdir{pythontex-files-\pytx@jobname} \newcommand{\setpythontexoutputdir}[1]{% \Depythontex{cmd:setpythontexoutputdir:m:n}% \def\pytx@outputdir{#1}} -\setpythontexoutputdir{pythontex-files-\pytx@jobname} -\AtBeginDocument{% - \@ifpackageloaded{graphicx}{% - \ifx\Ginput@path\@undefined - \graphicspath{{\pytx@outputdir/}}% - \else - \g@addto@macro\Ginput@path{{\pytx@outputdir/}}% - \fi - }{}% -} \@onlypreamble\setpythontexoutputdir % \end{macrocode} % % % \begin{macro}{pytx@workingdir}\begin{macro}{\setpythontexworkingdir} -% We need to be able to set the current working directory for the scripts executed by \pytex. By default, the working directory should be the same as the output directory. That way, any files saved in the current working directory will be in the \pytex\ output directory, and will thus be kept separate. But in some cases the user may wish to specify a different working directory, such as the document root. +% We need to be able to set the current working directory for the scripts executed by \pytex. By default, the working directory should be the same as the document root directory. But in some cases the user may wish to specify a different working directory. We want to be able to use ``||'' as a shortcut for setting the working directory to the output directory. % +% If the |graphicx| package is loaded, and the output directory is being used as the working directory, then the output directory is added to the graphics path at the beginning of the document, so that files in the output directory may be included within the main document without the necessity of specifying path information. % \begin{macrocode} +\def\pytx@workingdir{.} +\def\pytx@workingdirset{false} \newcommand{\setpythontexworkingdir}[1]{% \Depythontex{cmd:setpythontexworkingdir:m:n}% \def\pytx@workingdir{#1}% + \def\pytx@workingdirset{true}% } \@onlypreamble\setpythontexworkingdir \AtBeginDocument{% - \ifcsname pytx@workingdir\endcsname\else - %\setpythontexworkingdir{\pytx@outputdir}\fi Depythontex - \let\pytx@workingdir\pytx@outputdir\fi + \ifdefstring{\pytx@workingdir}{}% + {\let\pytx@workingdir\pytx@outputdir}{}% + \ifdefstrequal{\pytx@workingdir}{\pytx@outputdir}{% + \@ifpackageloaded{graphicx}{% + \ifx\Ginput@path\@undefined + \graphicspath{{\pytx@outputdir/}}% + \else + \g@addto@macro\Ginput@path{{\pytx@outputdir/}}% + \fi + }{}% + }{}% } % \end{macrocode} % \end{macro}\end{macro} @@ -1984,31 +2581,46 @@ % % \begin{macro}{pytx@usedpygments} % Once we have specified the output directory, we are free to pull in content from it. Most content from the output directory will be pulled in manually by the user (for example, via |\includegraphics|) or automatically by \pytex\ as it goes along. But content ``printed'' by code commands and environments (via macros) as well as code typeset by Pygments needs to be included conditionally, based on whether it exists and on user preferences. -% +% % This gets a little tricky. We only want to pull in the Pygments content if it is actually used, since Pygments content will typically use |fancyvrb|'s |SaveVerb| environment, and this can slow down compilation when very large chunks of code are saved. It doesn't matter if the code is actually used; saving it in a macro is what potentially slows things down. So we create a bool to keep track of whether Pygments is ever actually used, and only bring in Pygments content if it is.\footnote{The same effect could be achieved by having |pythontex.py| delete the Pygments content whenever it is run and Pygments is not used. But that approach is faulty in two regards. First, it requires that |pythontex.py| be run, which is not necessarily the case if the user simply sets the package option |pygments| to |false| and the recompiles. Second, even if it could be guaranteed that the content would be deleted, such an approach would not be optimal. It is quite possible that the user wishes to temporarily turn off Pygments usage to speed compilation while working on other parts of the document. In this case, deleting the Pygments content is simply deleting data that must be recreated when Pygments is turned back on.} This bool must be set to |true| whenever a command or environment is created that makes use of Pygments (in practice, we will simply set it to true when a family is created). Note that we cannot use the |pytx@opt@pygments| bool for this purpose, because it only tells us if the package option for Pygments usage is |true| or |false|. Typically, this will determine if any Pygments content is used. But it is possible for the user to create a command and environment family that overrides the package option (indeed, this may sometimes be desirable, for example, if the user wishes code in a particular language never to be highlighted). Thus, a new bool is needed to allow detection in such nonstandard cases. % \begin{macrocode} \newbool{pytx@usedpygments} % \end{macrocode} % \end{macro} % -% Now we can conditionally bring in the Pygments content. Note that we must use the |etoolbox| macro |\AfterEndPreamble|. This is because commands and environments are created using |\AtBeginDocument|, so that the user can change their properties in the preamble before they are created. And since the commands and environments must be created before we know the final state of |pytx@usedpygments|, we must bring in Pygments content after that. +% Now we can conditionally bring in the Pygments content. Note that we must use the |etoolbox| macro |\AfterEndPreamble|. This is because commands and environments are created using |\AtBeginDocument|, so that the user can change their properties in the preamble before they are created. And since the commands and environments must be created before we know the final state of |pytx@usedpygments|, we must bring in Pygments content after that. We typically need to patch the Pygments single quote macro so that it cooperates with |upquote|. % \begin{macrocode} \AfterEndPreamble{% - \ifbool{pytx@usedpygments}% - {\InputIfFileExists{\pytx@outputdir/\pytx@jobname.pytxpyg}{}{}}{}% + \ifbool{pytx@usedpygments}% + {\InputIfFileExists{\pytx@outputdir/\pytx@jobname.pytxpyg}{}{}% + \ifcsname PYGZsq\endcsname + \ifdefstring{\PYGZsq}{\char`\'}{\pytx@patch@PYGZsq}{}% + \fi}% + {}% } +\begingroup +\catcode`\'=\active +\gdef\pytx@patch@PYGZsq{\gdef\PYGZsq{'}} +\endgroup % \end{macrocode} % % % While we are pulling in content, we also pull in the file of macros that stores some inline ``printed'' content, if the file exists. Since we need this file in general, and since it will not typically invole a noticeable speed penalty, we bring it in at the beginning of the document without any special conditions. % \begin{macrocode} \AtBeginDocument{% \makeatletter - \InputIfFileExists{\pytx@outputdir/\pytx@jobname.pytxmcr}{}{}% + \InputIfFileExists{\pytx@outputdir/\pytx@jobname.pytxmcr}{}% + {\ifstrempty{\pytx@outputdir}% + {\typeout{No file \pytx@jobname.pytxmcr.}}% + {}% + \IfStrEq{\pytx@outputdir}{.}% + {\typeout{No file \pytx@jobname.pytxmcr.}}% + {\typeout{No file \pytx@outputdir/\pytx@jobname.pytxmcr.}}% + \typeout{Run \pytx@packagename\space to create it.}}% \makeatother } % \end{macrocode} % \end{macro}\end{macro} -% +% % % \begin{macro}{\pytx@codefile} % We create a new write, named |\pytx@codefile|, to which we will save code. All the code from the document will be written to this single file, interspersed with information specifying where in the document it came from. \pytex\ parses this file to separate the code into individual sessions and groups. These are then executed, and the identifying information is used to tie code output back to the original code in the document.\footnote{The choice to write all code to a single file is the result of two factors. First, \TeX\ has a limited number of output registers available (16), so having a separate output stream for each group or session is not possible. The |morewrites| package from Bruno Le Floch potentially removes this obstacle, but since this package is very recent (README from 2011/7/10), we will not consider using additional writes in the immediate future. Second, one of the design goals of \pytex\ is to minimize the number of persistent files created by a run. This keeps directories cleaner and makes file synchronization/transfer somewhat simpler. Using one write per session or group could result in numerous code files, and these could only be cleaned up by |pythontex.py| since \LaTeX\ cannot delete files itself (well, without unrestricted |write18|). Using a single output file for code does introduce a speed penalty since the code does not come pre-sorted by session or group, but in typical usage this should be minimal. Adding an option for single or multiple code files may be something to reconsider at a later date.} @@ -2017,7 +2629,7 @@ \immediate\openout\pytx@codefile=\jobname.pytxcode % \end{macrocode} % \end{macro} -% +% % In the code file, information from \pytex\ must be interspersed with the code. Some type of delimiting is needed for \pytex\ information. All \pytex\ content is written to the file in the form |=>PYTHONTEX#|\meta{content}|#|. When this content involves package options, the delimiter is modified to the form |=>PYTHONTEX:SETTINGS#|\meta{content}|#|. The |#| symbol is also used as a subdelimiter within \meta{content}. The |#| symbol is convenient as a delimiter since it has a special meaning in \TeX\ and is very unlikely to be accidentally entered by the user in unexpected locations without producing errors. Note that the usage of ``|=>PYTHONTEX#|'' as a beginning delimiter for \pytex\ data means that this string should \textbf{never} be written by the user at the beginning of a line, because |pythontex.py| will try to intepret it as data and will fail. % % \begin{macro}{\pytx@delimchar} @@ -2046,6 +2658,7 @@ \immediate\write\pytx@codefile{version=\pytx@packageversion}% \immediate\write\pytx@codefile{outputdir=\pytx@outputdir}% \immediate\write\pytx@codefile{workingdir=\pytx@workingdir}% + \immediate\write\pytx@codefile{workingdirset=\pytx@workingdirset}% \immediate\write\pytx@codefile{gobble=\pytx@opt@gobble}% \immediate\write\pytx@codefile{rerun=\pytx@opt@rerun}% \immediate\write\pytx@codefile{hashdependencies=\pytx@opt@hashdependencies}% @@ -2064,7 +2677,7 @@ % \end{macrocode} % % \begin{macro}{\pytx@WriteCodefileInfo}\begin{macro}{\pytx@WriteCodefileInfoExt} -% Later, we will frequently need to write \pytex\ information to the code file in standardized form. We create a macro to simplify that process. We also create an alternate form, for use with external files that must be inputted or read in by \pytex\ and processed. While the standard form employs a counter that is incremented elsewhere, the version for external files substitutes a zero (0) for the counter, because each external file must be unique in name and thus numbering via a counter is redundant.\footnote{The external-file form also takes an optional argument. This corresponds to a command-line argument that is passed to an external file during the file's execution. Currently, executing external files, with or without arguments, is not implemented. But this feature is under consideration, and the macro retains the optional argument for the potential future compatibility.} +% Later, we will frequently need to write \pytex\ information to the code file in standardized form. We create a macro to simplify that process. We also create an alternate form, for use with external files that must be inputted or read in by \pytex\ and processed.\footnote{The external-file form also takes an optional argument. This corresponds to a command-line argument that is passed to an external file during the file's execution. Currently, executing external files, with or without arguments, is not implemented. But this feature is under consideration, and the macro retains the optional argument for the potential future compatibility. Originally, the external version used a fixed instance, but that conflicted with the \texttt{fancyvrb} options \texttt{firstline} and \texttt{lastline}, so instances had to be added.} % \begin{macrocode} \def\pytx@argsrun{} \def\pytx@argspprint{} @@ -2084,7 +2697,7 @@ {\let\pytx@currfile\@empty}{\let\pytx@currfile\currfilename}% \immediate\write\pytx@codefile{\pytx@delim\pytx@type\pytx@delimchar% \pytx@session\pytx@delimchar\pytx@group\pytx@delimchar% - 0\pytx@delimchar\pytx@cmd\pytx@delimchar% + \arabic{\pytx@counter}\pytx@delimchar\pytx@cmd\pytx@delimchar% \pytx@context\pytx@delimchar\pytx@argsrun\pytx@delimchar% \pytx@argspprint\pytx@delimchar% \pytx@currfile\pytx@delimchar% @@ -2099,7 +2712,7 @@ % \begin{macro}{\pytx@fvsettings}\begin{macro}{\setpythontexfv} % The macro |\setpythontexfv|\oarg{family}\marg{settings} takes \meta{settings} and stores them in a macro that is run through |fancyvrb|'s |\fvset| at the beginning of \pytex\ code. If a \meta{family} is specified, the settings are stored in |\pytx@fvsettings@|\meta{family}, and the settings only apply to typeset code belonging to that family. If no optional argument is given, then the settings are stored in |\pytx@fvsettings|, and the settings apply to all typeset code. % -% In the current implementation, |\setpythontexfv| and |\fvset| differ because the former is not persistent in the same sense as the latter. If we use |\fvset| to set one property, and then use it later to set another property, the setting for the original property is persistent. It remains until another |\fvset| command is issued to change it. In contrast, every time |\setpythontexfv| is used, it clears all prior settings and only the current settings actually apply. This is because |\fvset| stores the state of each setting in its own macro, while |\setpythontexfv| simply stores a string of settings that is passed to |\fvset| at the appropriate times. For typical use scenarios, this distinction shouldn't be important---usually, we will want to set the behavior of |fancyvrb| for all \pytex\ content, or for a family of \pytex\ content, and leave those settings constant throughout the document. Furthermore, environments that typeset code take |fancyvrb| commands as their second optional argument, so there is already a mechanism in place for changing the settings for a single environment. However, if we ever want to change the typesetting of code for only a small portion of a document (larger than a single environment), this persistence distinction does become important.\footnote{An argument could be made for having |\textbackslash setpythontexfv| behave exactly like |\textbackslash fvset|. Properly implementing this behavior would be tricky, because of inheritance issues between \pytex-wide and family-specific settings (this is probably a job for |pgfkeys|). Full persistence would likely require a large number of macros and conditionals. At least from the perspective of keeping the code clean and concise, the current approach is superior, and probably introduces minor annoyances at worst.} +% In the current implementation, |\setpythontexfv| and |\fvset| differ because the former is not persistent in the same sense as the latter. If we use |\fvset| to set one property, and then use it later to set another property, the setting for the original property is persistent. It remains until another |\fvset| command is issued to change it. In contrast, every time |\setpythontexfv| is used, it clears all prior settings and only the current settings actually apply. This is because |\fvset| stores the state of each setting in its own macro, while |\setpythontexfv| simply stores a string of settings that is passed to |\fvset| at the appropriate times. For typical use scenarios, this distinction shouldn't be important---usually, we will want to set the behavior of |fancyvrb| for all \pytex\ content, or for a family of \pytex\ content, and leave those settings constant throughout the document. Furthermore, environments that typeset code take |fancyvrb| commands as their second optional argument, so there is already a mechanism in place for changing the settings for a single environment. However, if we ever want to change the typesetting of code for only a small portion of a document (larger than a single environment), this persistence distinction does become important.\footnote{An argument could be made for having |\setpythontexfv| behave exactly like |\fvset|. Properly implementing this behavior would be tricky, because of inheritance issues between \pytex-wide and family-specific settings (this is probably a job for |pgfkeys|). Full persistence would likely require a large number of macros and conditionals. At least from the perspective of keeping the code clean and concise, the current approach is superior, and probably introduces minor annoyances at worst.} % \begin{macrocode} \newcommand{\setpythontexfv}[2][]{% \Depythontex{cmd:setpythontexfv:om:n}% @@ -2108,7 +2721,7 @@ {\expandafter\gdef\csname pytx@fvsettings@#1\endcsname{#2}}% }% % \end{macrocode} -% +% % Now that we have a mechanism for applying global settings to typeset \pytex\ code, we go ahead and set a default tab size for all environments. If |\setpythontexfv| is ever invoked, this setting will be overwritten, so that must be kept in mind. % \begin{macrocode} \setpythontexfv{tabsize=4} @@ -2117,6 +2730,7 @@ % % \begin{macro}{\pytx@FVSet} % Once the |fancyvrb| settings for \pytex\ are stored in macros, we need a way to actually invoke them. |\pytx@FVSet| applies family-specific settings first, then \pytex-wide settings second, so that \pytex-wide settings have precedence and will override family-specific settings. Note that by using |\fvset|, we are overwriting |fancyvrb|'s settings. Thus, to keep the settings local to the \pytex\ code, |\pytx@FVSet| must always be used within a |\begingroup ... \endgroup| block. +% % \begin{macrocode} \def\pytx@FVSet{% \expandafter\let\expandafter\pytx@fvsettings@@% @@ -2138,6 +2752,8 @@ % \begin{macro}{pytx@FancyVerbLineTemp} % |fancyvrb|'s |SaveVerbatim| environment will be used extensively to include code highlighted by Pygments and other processed content. Unfortunately, when the saved content is included in a document with the corresponding |UseVerbatim|, line numbering does not work correctly. Based on a web search, this appears to be a known bug in |fancyvrb|. We begin by fixing this, which requires patching |fancyvrb|'s |\FVB@SaveVerbatim| and |\FVE@SaveVerbatim|. We create a patched |\pytx@FVB@SaveVerbatim| by inserting |\FV@StepLineNo| and |\FV@CodeLineNo=1| at appropriate locations. We also delete an unnecessary |\gdef\SaveVerbatim@Name{#1}|. Then we create a |\pytx@FVE@SaveVerbatim|, and add code so that the two macros work together to prevent |FancyVerbLine| from incorrectly being incremented within the |SaveVerbatim| environment. This involves using the counter |pytx@FancyVerbLineTemp| to temporarily store the value of |FancyVerbLine|, so that it may be restored to its original value after verbatim content has been saved. % +% There is an additional line-numbering issue when the |firstline| option is used with |SaveVerbatim|. This is fixed by globally reseting |\FV@CodeLineNo| to zero. That was originally done in |fancyvrb|, via |\FV@FormattingPrep|, but this macro is commented out in the current version of |fancyvrb|, which throws off line numbering. +% % Typically, we |\let| our own custom macros to the corresponding macros within |fancyvrb|, but only within a command or environment. In this case, however, we are fixing behavior that should be considered a bug even for normal |fancyvrb| usage. So we let the buggy macros to the patched macros immediately after defining the patched versions. % \begin{macrocode} \newcounter{pytx@FancyVerbLineTemp} @@ -2145,6 +2761,7 @@ % \begin{macrocode} \def\pytx@FVB@SaveVerbatim#1{% \setcounter{pytx@FancyVerbLineTemp}{\value{FancyVerbLine}}% + \global\FV@CodeLineNo\z@ \@bsphack \begingroup \FV@UseKeyValues @@ -2165,6 +2782,27 @@ % \end{macro}\end{macro} % % +% +% \subsubsection{Enabling \texttt{fvextra} support for Pygments macros} +% +% \begin{macro}{\pytx@ConfigPygments} +% The |fvextra| package provides Pygments support. We need a macro that can be used to turn this on at the appropriate points. +% \begin{macrocode} +\def\pytx@ConfigPygments{% + \def\pytx@currentstyle{default}% + \ifcsname pytx@style\endcsname + \let\pytx@currentstyle\pytx@style + \else + \ifcsname pytx@style@\pytx@type\endcsname + \expandafter\let\expandafter\pytx@currentstyle\csname pytx@style@\pytx@type\endcsname + \fi + \fi + \expandafter\let\expandafter\PYG@style\csname PYG\pytx@currentstyle\endcsname + \VerbatimPygments{\PYG}{\PYG@style}} +% \end{macrocode} +% \end{macro} +% +% % \subsubsection{Access to printed content (stdout)} % The |autoprint| package option automatically pulls in printed content from |code| commands and environments. But this does not cover all possible use cases, because we could have print statements/functions in |block| commands and environments as well. Furthermore, sometimes we may print content, but then desire to bring it back into the document multiple times, without duplicating the code that creates the content. Here, we create a number of macros that allow access to printed content. All macros are created in two identical forms, one based on the name |print| and one based on the name |stdout|. Which macros are used depends on user preference. The macros based on |stdout| provide symmetry with |stderr| access. % @@ -2307,7 +2945,7 @@ % % A mechanism is provided for saving and later using stderr. This should be used with care, since stderr content may lose some of its meaning if isolated from the larger code context that produced it. % -% \begin{macro}{\savestderrpythontex} +% \begin{macro}{\savestderrpythontex} % \begin{macrocode} \def\savestderrpythontex#1{% \Depythontex{cmd:savestderrpythontex:m:n}% @@ -2364,7 +3002,7 @@ % \begin{macro}{\Depythontex} % Each command or environment that is to work with depythontex will write the following information to the auxiliary file: % \begin{verbatim} -% =>DEPYTHONTEX#:::::[]# +%=>DEPYTHONTEX#:::::[]# % \end{verbatim} % where || is |cmd| or |env|; || is the complete name of the command or environment; || is a string representing the arguments taken (|o|=optional, |m|=mandatory, |v|=mandatory~verbatim, |n|=none); || is a string representing what is typeset (|c|=code, |p|=printed, |n|=null), and || is |\the\inputlineno|. The last one can be determined automatically without user input, but the first four must be entered when a macro is created. Optionally, the Pygments lexer is written to file if it is available (if |\pytx@lexer| is not |\relax|). These pieces of information are needed for the following reasons. % \begin{itemize} @@ -2555,7 +3193,7 @@ % We save a retokenized version of the argument in |\pytx@argretok|. This is needed for typesetting with |fancyvrb|. The code must be retokenized so that space characters are active, since |fancyvrb| allows space characters to be visible or invisible by making them active. % % The \textbf{name} of the counter corresponding to this code is assembled. It is needed for keeping track of the instance, and is used for bringing in content created by the code and for bringing in highlighting created by Pygments. -% +% % Next we call a series of macros that determine whether the code is shown (typeset), whether it is saved to the code file, and whether content created by the code (``printed'') should be brought in. These macros are |\let| to appropriate values when an inline command is called; they are not defined independently. % % Finally, the counter for the code is incremented. @@ -2584,7 +3222,7 @@ % % % \begin{macro}{\pytx@InlineShowFV} -% Code may be typeset with |fancyvrb|. |fancyvrb| settings are invoked via |pytx@FVSet|, but this must be done within a group so that the settings remain local. Most of the remainder of the commands are from |fancyvrb|'s |\FV@FormattingPrep|, and take care of various formatting matters, including spacing, font, whether space characters are shown, and any user-defined formatting. Finally, we create an |\hbox| and invoke |\FancyVerbFormatLine| to maintain parallelism with |BVerbatim|, which is used for inline content highlighted with Pygments. |\FancyVerbFormatLine| may be redefined to alter the typeset code, for example, by putting it in a colorbox via the following command:\footnote{Currently, |\textbackslash FancyVerbFormatLine| is global, as in |fancyvrb|. Allowing a family-specific variant may be considered in the future. In most cases, the |fancyvrb| option |formatcom|, combined with external formatting from packages like |mdframed|, should provide all formatting desired. But something family-specific might occasionally prove useful.} +% Code may be typeset with |fancyvrb|. |fancyvrb| settings are invoked via |pytx@FVSet|, but this must be done within a group so that the settings remain local. Most of the remainder of the commands are from |fancyvrb|'s |\FV@FormattingPrep|, and take care of various formatting matters, including spacing, font, whether space characters are shown, and any user-defined formatting. Finally, we create an |\hbox| and invoke |\FancyVerbFormatLine| to maintain parallelism with |BVerbatim|, which is used for inline content highlighted with Pygments. |\FancyVerbFormatLine| may be redefined to alter the typeset code, for example, by putting it in a colorbox via the following command:\footnote{Currently, |\FancyVerbFormatLine| is global, as in |fancyvrb|. Allowing a family-specific variant may be considered in the future. In most cases, the |fancyvrb| option |formatcom|, combined with external formatting from packages like |mdframed|, should provide all formatting desired. But something family-specific might occasionally prove useful.} % \begin{quote} % |\renewcommand{\FancyVerbFormatLine}[1]{\colorbox{green}{#1}}| % \end{quote} @@ -2622,6 +3260,7 @@ \def\pytx@InlineShowPyg{% \begingroup \pytx@FVSet + \pytx@ConfigPygments \ifcsname FV@SV@\pytx@counter @\arabic{\pytx@counter}\endcsname \setcounter{pytx@FancyVerbLineTemp}{\value{FancyVerbLine}}% \BUseVerbatim{\pytx@counter @\arabic{\pytx@counter}}% @@ -2664,6 +3303,18 @@ % \end{macro} % % +% \begin{macro}{\pytx@InlineAlwaysprint} +% This is like |\pytx@InlineAutoprint|, except that it always prints rather than depending on |autoprint|. It is used for the |s| commands, which are always expected to have output. +% \begin{macrocode} +\def\pytx@InlineAlwaysprint{% + \InputIfFileExists{\pytx@outputdir/\pytx@stdfile.stdout}% + {\DepyFile{p:\pytx@outputdir/\pytx@stdfile.stdout}}% + {\textbf{??}% + \PackageWarning{\pytx@packagename}{Missing sub content}}} +% \end{macrocode} +% \end{macro} +% +% % \begin{macro}{\pytx@InlineMacroprint} % This macro brings in ``printed'' content that is brought in via macros in the .pytxmcr file. We must disable the macro in the event that the |debug| option is false. % \begin{macrocode} @@ -2700,13 +3351,13 @@ % % % \subsubsection{Inline command constructors} -% With the core inline macros complete, we are ready to create constructors for different kinds of inline commands. All of these consctructors take a string and define an inline command named using that string as a base name. Two forms of each constructor are created, one that uses Pygments and one that does not. The Pygments variants have names ending in ``Pyg''. +% With the core inline macros complete, we are ready to create constructors for different kinds of inline commands. All of these consctructors take a string and define an inline command named using that string as a base name. Two forms of each constructor are created, one that uses Pygments and one that does not. The Pygments variants have names ending in ``Pyg''. All commands are created using |etoolbox|'s |\newrobustcmd|. Among other things, this is needed so that commands will work within the default caption command. % % \begin{macro}{\pytx@MakeInlinebFV}\begin{macro}{\pytx@MakeInlinebPyg} % These macros creates inline block commands, which both typeset code and save it so that it may be executed. The base name of the command is stored in |\pytx@type|. A string representing the kind of command is stored in |\pytx@cmd|. Then |\pytx@SetContext| is used to set |\pytx@context| and |\pytx@SetGroup| is used to set |\pytx@group|. Macros for showing, saving, and printing are set to appropriate values. Then the core inline macros are invoked through |\pytx@Inline|. % \begin{macrocode} \newcommand{\pytx@MakeInlinebFV}[1]{% - \expandafter\newcommand\expandafter{\csname #1b\endcsname}{% + \expandafter\newrobustcmd\expandafter{\csname #1b\endcsname}{% \expandafter\let\expandafter\pytx@lexer\csname pytx@pyglexer@#1\endcsname \Depythontex{cmd:#1b:ov:c}% \xdef\pytx@type{#1}% @@ -2720,7 +3371,7 @@ }% }% \newcommand{\pytx@MakeInlinebPyg}[1]{% - \expandafter\newcommand\expandafter{\csname #1b\endcsname}{% + \expandafter\newrobustcmd\expandafter{\csname #1b\endcsname}{% \xdef\pytx@type{#1}% \edef\pytx@cmd{b}% \pytx@SetContext @@ -2739,7 +3390,7 @@ % This macro creates inline verbatim commands, which only typeset code. |\pytx@type|, |\pytx@cmd|, |\pytx@context|, and |\pytx@group| are still set, for symmetry with other commands. They are not needed for |fancyvrb| typesetting, though. We use |\pytx@SetGroupVerb| to split verbatim content (|v| and |verb|) off into its own group. That way, verbatim content doesn't affect the instance numbers of executed code, and thus executed code is not affected by the addition or removal of verbatim content. % \begin{macrocode} \newcommand{\pytx@MakeInlinevFV}[1]{% - \expandafter\newcommand\expandafter{\csname #1v\endcsname}{% + \expandafter\newrobustcmd\expandafter{\csname #1v\endcsname}{% \expandafter\let\expandafter\pytx@lexer\csname pytx@pyglexer@#1\endcsname \Depythontex{cmd:#1v:ov:c}% \xdef\pytx@type{#1}% @@ -2753,7 +3404,7 @@ }% }% \newcommand{\pytx@MakeInlinevPyg}[1]{% - \expandafter\newcommand\expandafter{\csname #1v\endcsname}{% + \expandafter\newrobustcmd\expandafter{\csname #1v\endcsname}{% \xdef\pytx@type{#1}% \edef\pytx@cmd{v}% \pytx@SetContext @@ -2772,7 +3423,7 @@ % This macro creates inline code commands, which save code for execution but do not typeset it. If the code prints content, this content is inputted automatically if the package option |autoprint| is on. Since no code is typeset, there is no difference between the |fancyvrb| and Pygments forms. % \begin{macrocode} \newcommand{\pytx@MakeInlinecFV}[1]{% - \expandafter\newcommand\expandafter{\csname #1c\endcsname}{% + \expandafter\newrobustcmd\expandafter{\csname #1c\endcsname}{% \Depythontex{cmd:#1c:ov:p}% \xdef\pytx@type{#1}% \edef\pytx@cmd{c}% @@ -2789,11 +3440,32 @@ % \end{macro}\end{macro} % % +% \begin{macro}{\pytx@MakeInlinesFV}\begin{macro}{\pytx@MakeInlinesPyg} +% This macro behaves almost exactly like code commands on the \LaTeX\ side, but on the Python side, the argument is treated as a template in which fields are evaluated and replaced with the result. Since no code is typeset, there is no difference between the |fancyvrb| and Pygments forms. +% \begin{macrocode} +\newcommand{\pytx@MakeInlinesFV}[1]{% + \expandafter\newrobustcmd\expandafter{\csname #1s\endcsname}{% + \Depythontex{cmd:#1s:ov:p}% + \xdef\pytx@type{#1}% + \edef\pytx@cmd{s}% + \pytx@SetContext + \pytx@SetGroup + \let\pytx@InlineShow\@empty + \let\pytx@InlineSave\pytx@InlineSaveCode + \let\pytx@InlinePrint\pytx@InlineAlwaysprint + \pytx@Inline + }% +}% +\let\pytx@MakeInlinesPyg\pytx@MakeInlinesFV +% \end{macrocode} +% \end{macro}\end{macro} +% +% % \begin{macro}{\pytx@MakeInlineFV}\begin{macro}{\pytx@MakeInlinePyg} % This macro creates plain inline commands, which save code and then bring in the output of |pytex.formatter(|\meta{code}|)| (|pytex.formatter()| is the formatter function in Python sessions that is provided by |pythontex_utils*.py|). The Python output is saved in a \TeX\ macro, and the macro is written to a file shared by all \pytex\ sessions. This greatly reduces the number of external files needed. Since no code is typeset, there is no difference between the |fancyvrb| and Pygments forms. % \begin{macrocode} \newcommand{\pytx@MakeInlineFV}[1]{% - \expandafter\newcommand\expandafter{\csname #1\endcsname}{% + \expandafter\newrobustcmd\expandafter{\csname #1\endcsname}{% \Depythontex{cmd:#1:ov:p}% \xdef\pytx@type{#1}% \edef\pytx@cmd{i}% @@ -2814,7 +3486,7 @@ % This is the inline form for |console| types. It brings in |SaveVerbatim|. % \begin{macrocode} \newcommand{\pytx@MakeInlineConsFV}[1]{% - \expandafter\newcommand\expandafter{\csname #1\endcsname}{% + \expandafter\newrobustcmd\expandafter{\csname #1\endcsname}{% \Depythontex{cmd:#1:ov:c}% \xdef\pytx@type{#1}% \edef\pytx@cmd{i}% @@ -2834,7 +3506,7 @@ %\begin{macro}{\pythontexcustomc} % This macro takes a single line of code and adds it to all sessions within a family. It is the inline version of the |pythontexcustomcode| environment. % \begin{macrocode} -\newcommand{\pythontexcustomc}[2][begin]{% +\newrobustcmd{\pythontexcustomc}[2][begin]{% \Depythontex{cmd:pythontexcustomc:omv:p}% \ifstrequal{#1}{begin}{}{% \ifstrequal{#1}{end}{}{\PackageError{\pytx@packagename}% @@ -2843,7 +3515,7 @@ }% \xdef\pytx@type{CC:#2:#1}% \edef\pytx@cmd{c}% - \def\pytx@context{}% + \pytx@SetContext \def\pytx@group{none}% \let\pytx@InlineShow\@empty \let\pytx@InlineSave\pytx@InlineSaveCode @@ -2881,7 +3553,7 @@ % % % \subsection{Environments} -% +% % The inline commands were all created using a common core set of macros, combined with short, command-specific constructors. In the case of environments, we do not have a common core set of macros. Each environment is coded separately, though there are similarities among environments. In the future, it may be worthwhile to attempt to consolidate the environment code base. % % One of the differences between inline commands and environments is that environments may need to typeset code with line numbers. Each family of code needs to have its own line numbering (actually, its own numbering for code, verbatim, and console groups), and this line numbering should not overwrite any line numbering that may separately be in use by |fancyvrb|. To make this possible, we use a temporary counter extensively. When line numbers are used, |fancyvrb|'s line counter is copied into |pytx@FancyVerbLineTemp|, lines are numbered, and then |fancyvrb|'s line counter is restored from |pytx@FancyVerbLineTemp|. This keeps |fancyvrb| and \pytex's line numbering separate, even though \pytex\ is using |fancyvrb| and its macros internally. @@ -2897,7 +3569,7 @@ \gdef\pytx@FancyVerbGetLine#1^^M{% \@nil% \FV@CheckEnd{#1}% - \ifx\@tempa\FV@EnvironName% + \ifx\@tempa\FV@EnvironName% \ifx\@tempb\FV@@@CheckEnd\else\FV@BadEndError\fi% \let\next\FV@EndScanning% \else% @@ -3045,6 +3717,7 @@ \setcounter{FancyVerbLine}{\value{\pytx@linecount}}% \pytx@FVSet \ifdefstring{\pytx@fvopttmp}{}{}{\expandafter\fvset\expandafter{\pytx@fvopttmp}}% + \pytx@ConfigPygments \ifcsname FV@SV@\pytx@counter @\arabic{\pytx@counter}\endcsname \UseVerbatim{\pytx@counter @\arabic{\pytx@counter}}% \else @@ -3084,6 +3757,7 @@ % \begin{macrocode} \def\pytx@BeginEnvPyg@i[#1]{% \def\pytx@fvopttmp{#1}% + \def\pytx@argspprint{#1}% \setcounter{pytx@FancyVerbLineTemp}{\value{FancyVerbLine}}% \let\FVB@VerbatimOut\pytx@FVB@VerbatimOut \let\FVE@VerbatimOut\pytx@FVE@VerbatimOut @@ -3110,7 +3784,7 @@ % % % \subsubsection{Code environment constructor} -% The |code| environment merely saves code to the code file; nothing is typeset. To accomplish this, we use a slightly modified version of |fancyvrb|'s |VerbatimOut|. +% The |code| environment merely saves code to the code file; nothing is typeset. To accomplish this, we use a slightly modified version of |fancyvrb|'s |VerbatimOut|. % \begin{macro}{\pytx@WriteDetok} % We can use |fancyvrb| to capture the code, but we will need a way to write the code in detokenized form. This is necessary so that \TeX\ doesn't try to process the code as it is written, which would generally be disastrous. % \begin{macrocode} @@ -3175,9 +3849,25 @@ }% {\end{VerbatimOut}% \xdef\pytx@stdfile{\pytx@type_\pytx@session_\pytx@group_\arabic{\pytx@counter}}% - \pytx@EnvAutoprint + \ifcsname pytx@nonpyconsole@\pytx@type\endcsname + \ifcsname pytx@code@as@console\endcsname + \setcounter{FancyVerbLine}{\value{\pytx@linecount}}% + \pytx@FVSet + \ifdefstring{\pytx@fvopttmp}{}{}{\expandafter\fvset\expandafter{\pytx@fvopttmp}}% + \pytx@ConfigPygments + \InputIfFileExists{\pytx@outputdir/\pytx@stdfile.stdout}% + {\DepyFile{p:\pytx@outputdir/\pytx@stdfile.stdout}}% + {\par\textbf{??~\pytx@packagename~??}\par + \PackageWarning{\pytx@packagename}{Non-existent console content}}% + \setcounter{\pytx@linecount}{\value{FancyVerbLine}}% + \else + \fi + \let\pytx@EnvAutoprint\relax + \else + \fi \setcounter{FancyVerbLine}{\value{pytx@FancyVerbLineTemp}}% \stepcounter{\pytx@counter}% + \pytx@EnvAutoprint }% }% % \end{macrocode} @@ -3191,6 +3881,8 @@ \ifstrempty{#1}{\edef\pytx@session{default}}{\StrSubstitute{#1}{:}{-}[\pytx@session]}% \xdef\pytx@counter{pytx@\pytx@type @\pytx@session @\pytx@group}% \pytx@CheckCounter{\pytx@counter}% + \edef\pytx@linecount{\pytx@counter @line}% + \pytx@CheckCounter{\pytx@linecount}% \pytx@WriteCodefileInfo \begingroup \obeylines @@ -3204,6 +3896,8 @@ % As described above, this macro captures a second optional argument, if present, and then starts the |VerbatimOut| environment. Note that |VerbatimOut| does not have a mandatory argument, because we are invoking our custom |\pytx@FVB@VerbatimOut| macro. The default |fancyvrb| macro needs an argument to tell it the name of the file to which to save the verbatim content. But in our case, we are always writing to the same file, and the custom macro accounts for this by not having a mandatory file name argument. We must perform the typical |FancyVerbLine| trickery, to prevent the |fancyvrb| line counter from being affected by \textbf{writing} content! % \begin{macrocode} \def\pytx@BeginCodeEnv@i[#1]{% + \def\pytx@fvopttmp{#1}% + \def\pytx@argspprint{#1}% \setcounter{pytx@FancyVerbLineTemp}{\value{FancyVerbLine}}% \let\FVB@VerbatimOut\pytx@FVB@VerbatimOut \let\FVE@VerbatimOut\pytx@FVE@VerbatimOut @@ -3234,7 +3928,7 @@ }% \xdef\pytx@type{CC:#2:#1}% \edef\pytx@cmd{code}% - \def\pytx@context{}% + \pytx@SetContext \def\pytx@group{none}% \pytx@BeginCodeEnv[none]}% {\end{VerbatimOut}% @@ -3245,8 +3939,50 @@ % \end{macro} % % +% \subsubsection{Sub environment constructor} +% +% The |sub| environment behaves exactly like a |code| environment on the \LaTeX\ side: environment content is saved to the code file, and then the output is brought back in. The difference is on the Python side, where the environment content is treated as a template in which fields are evaluated and replaced with the result. +% +% \begin{macro}{\pytx@MakeSubFV} +% Create a |sub| environment compatible with |fancyvrb|, reusing the |code| approach almost entirely. +% \begin{macrocode} +\newcommand{\pytx@MakeSubFV}[1]{% + \expandafter\newenvironment{#1sub}{% + \VerbatimEnvironment + \Depythontex{env:#1sub:oo|:p}% + \xdef\pytx@type{#1}% + \edef\pytx@cmd{sub}% + \pytx@SetContext + \pytx@SetGroup + \begingroup + \obeylines + \@ifnextchar[{\endgroup\pytx@BeginCodeEnv}{\endgroup\pytx@BeginCodeEnv[]}% + }% + {\end{VerbatimOut}% + \xdef\pytx@stdfile{\pytx@type_\pytx@session_\pytx@group_\arabic{\pytx@counter}}% + \setcounter{FancyVerbLine}{\value{pytx@FancyVerbLineTemp}}% + \stepcounter{\pytx@counter}% + \InputIfFileExists{\pytx@outputdir/\pytx@stdfile.stdout}% + {\DepyFile{p:\pytx@outputdir/\pytx@stdfile.stdout}}% + {\textbf{??~\pytx@packagename~??}% + \PackageWarning{\pytx@packagename}{Non-existent substituted content}}% + }% +}% +% \end{macrocode} +% \end{macro} +% +% \begin{macro}{\pytx@MakeSubPyg} +% The Pygments-compatible version is the same. +% \begin{macrocode} +\let\pytx@MakeSubPyg\pytx@MakeSubFV +% \end{macrocode} +% \end{macro} +% +% +% +% % \subsubsection{Console environment constructor} -% +% % The |console| environment needs to write all code contained in the environment to the code file, and then bring in the console output. % % An environment suffix is not enforced for flexibility. For Python, the convention is that |console| type names will end with |con|, and then the environment will use the suffix |sole|. For example, the |pycon| type has the |pyconsole| environment. @@ -3308,6 +4044,7 @@ % \begin{macrocode} \def\pytx@BeginConsEnvFV@i[#1]{% \def\pytx@fvopttmp{#1}% + \def\pytx@argspprint{#1}% \setcounter{pytx@FancyVerbLineTemp}{\value{FancyVerbLine}}% \let\FVB@VerbatimOut\pytx@FVB@VerbatimOut \let\FVE@VerbatimOut\pytx@FVE@VerbatimOut @@ -3338,6 +4075,7 @@ \setcounter{FancyVerbLine}{\value{\pytx@linecount}}% \pytx@FVSet \ifdefstring{\pytx@fvopttmp}{}{}{\expandafter\fvset\expandafter{\pytx@fvopttmp}}% + \pytx@ConfigPygments \ifcsname FV@SV@\pytx@counter @\arabic{\pytx@counter}\endcsname \UseVerbatim{\pytx@counter @\arabic{\pytx@counter}}% \DepyMacro{c:\pytx@counter @\arabic{\pytx@counter}}% @@ -3361,13 +4099,16 @@ % % \begin{macro}{\makepythontexfamily} % This macro creates a family of commands. It needs a some |pgfkeys| to handle the optional arguments. The actual creation of all non-code commands and environments is delayed using |\AtBeginDocument|, so that the user has the option to choose whether |fancyvrb| or Pygments is used for the family. +% +% We need to create a counter for the default session for each family to avoid (some of the) issues with |\includeonly| and counters. See \url{http://tug.org/pipermail/macostex-archives/2010-December/046007.html} for more on the problematic counter behavior with |\includeonly|. % \begin{macrocode} \pgfkeys{ /PYTX/family/.is family, /PYTX/family, + name/.estore in = \pytx@tmp@name, prettyprinter/.estore in = \pytx@tmp@pprinter, pyglexer/.estore in = \pytx@tmp@pyglexer, - pygopt/.estore in = \pytx@tmp@pygopt, + pygopt/.code = \def\pytx@tmp@pygopt{#1}\pgfkeys{/PYTX/lopt/pygopt/.cd, #1}, console/.estore in = \pytx@tmp@console, default/.style = {prettyprinter=auto, pyglexer=text, pygopt={}, console=false} } @@ -3414,14 +4155,16 @@ \IfBeginWith{#2}{PYG}% {\PackageError{\pytx@packagename}% {Attempt to create macros with reserved prefix PYG}{}}{}% - \pgfkeys{/PYTX/family, default, #1} + \pgfkeys{/PYTX/family, name=#2, default, #1} \expandafter\xdef\csname pytx@macroformatter@#2\endcsname{\pytx@tmp@pprinter} \expandafter\gdef\csname pytx@fvsettings@#2\endcsname{} \expandafter\xdef\csname pytx@pyglexer@#2\endcsname{\pytx@tmp@pyglexer} \expandafter\xdef\csname pytx@pygopt@#2\endcsname{\pytx@tmp@pygopt} \expandafter\xdef\csname pytx@console@#2\endcsname{\pytx@tmp@console} \pytx@MakeInlinecFV{#2} + \pytx@MakeInlinesFV{#2} \pytx@MakeCodeFV{#2} + \pytx@MakeSubFV{#2} \AtBeginDocument{% \ifcsstring{pytx@macroformatter@#2}{auto}{% \ifbool{pytx@opt@pygments}% @@ -3437,10 +4180,39 @@ {\ifcsstring{pytx@console@#2}{true}% {\pytx@MakeFamilyPygCons{#2}}{\pytx@MakeFamilyPyg{#2}}}{}% }% + \newcounter{pytx@#2@default@default}% } \@onlypreamble\makepythontexfamily % \end{macrocode} % \end{macro} +% +% +% \begin{macro}{\makepythontexfamily@con} +% This macro creates |console| and |code| environments for non-Python consoles. \pytex\ was not designed with commands and environments for non-Python consoles. Non-Python consoles are currently created via specially customized |code| environments. Note that simply creating these |console| and |code| environments is typically not enough to create non-Python consoles; |pythontex2.py| and |pythontex3.py| usually also require customization. This macro's definition should not be treated as stable; it will change in the future. The ultimate long-term goal is to eliminate it entirely, by redesigning the code execution core of \pytex\ to accomodate non-Python consoles more easily. +% \begin{macrocode} +\newcommand{\makepythontexfamily@con}[2][text]{% + \pgfkeys{/PYTX/family, name=#2con, default, pyglexer=#1, console=true}% + \expandafter\xdef\csname pytx@macroformatter@#2con\endcsname{\pytx@tmp@pprinter}% + \expandafter\gdef\csname pytx@fvsettings@#2con\endcsname{}% + \expandafter\xdef\csname pytx@pyglexer@#2con\endcsname{\pytx@tmp@pyglexer}% + \expandafter\xdef\csname pytx@pygopt@#2con\endcsname{\pytx@tmp@pygopt}% + \expandafter\xdef\csname pytx@console@#2con\endcsname{\pytx@tmp@console}% + \AtEndDocument{\immediate\write\pytx@codefile{pygfamily=#2con|% + \csname pytx@pyglexer@#2con\endcsname|% + \csname pytx@pygopt@#2con\endcsname}% + }% + \pytx@MakeCodeFV{#2con}% + \expandafter\global\expandafter\let\csname pytx@nonpyconsole@#2con\endcsname\relax + \newenvironment{#2console}% + {\VerbatimEnvironment + \def\pytx@type{#2con}% + \let\pytx@code@as@console\relax + \begin{#2concode}}% + {\end{#2concode}}% +} +% \end{macrocode} +% \end{macro} + % % \begin{macro}{\setpythontexpyglexer} % We need to be able to reset the lexer associated with a family after the family has already been created. @@ -3467,14 +4239,15 @@ % \begin{macrocode} \newcommand{\setpythontexpygopt}[2][]{% \Depythontex{cmd:setpythontexpygopt:om:n}% - \ifstrempty{#1}{\def\pytx@pygopt{#2}}{% - \ifcsname pytx@pygopt@#1\endcsname + \ifstrempty{#1}% + {\def\pytx@pygopt{#2}\pgfkeys{/PYTX/gopt/pygopt/.cd, #2}}% + {\ifcsname pytx@pygopt@#1\endcsname \expandafter\xdef\csname pytx@pygopt@#1\endcsname{#2}% - \else + \pgfkeys{/PYTX/lopt/pygopt/.cd, name=#1, #2} + \else \PackageError{\pytx@packagename}% - {Cannot modify Pygments options for a non-existent family}{}% - \fi - }% + {Cannot modify Pygments options for a non-existent family}{}% + \fi}% } \@onlypreamble\setpythontexpygopt % \end{macrocode} @@ -3509,11 +4282,11 @@ % % All of these command and environment families are created conditionally, depending on whether the package option |pygments| is used, via |\makepythontexfamily|. We recommend that any custom families created by the user be constructed in the same manner. % \begin{macrocode} -\makepythontexfamily[pyglexer=python]{py} +\makepythontexfamily[pyglexer=python3]{py} \makepythontexfamily[pyglexer=pycon, console=true]{pycon} -\makepythontexfamily[pyglexer=python]{sympy} +\makepythontexfamily[pyglexer=python3]{sympy} \makepythontexfamily[pyglexer=pycon, console=true]{sympycon} -\makepythontexfamily[pyglexer=python]{pylab} +\makepythontexfamily[pyglexer=python3]{pylab} \makepythontexfamily[pyglexer=pycon, console=true]{pylabcon} % \end{macrocode} % We also need to create any additional families specified via the |usefamily| package option.\footnote{The loop here is accomplished via \texttt{etoolbox}. \texttt{pgffor} might be an alternative, but making definitions global requires trickery.} @@ -3522,7 +4295,22 @@ \ifstrequal{#1}{ruby}{\makepythontexfamily[pyglexer=ruby]{ruby}}{}% \ifstrequal{#1}{rb}{\makepythontexfamily[pyglexer=ruby]{rb}}{}% \ifstrequal{#1}{julia}{\makepythontexfamily[pyglexer=julia]{julia}}{}% + \ifstrequal{#1}{juliacon}{\makepythontexfamily@con[jlcon]{julia}}{}% \ifstrequal{#1}{jl}{\makepythontexfamily[pyglexer=julia]{jl}}{}% + \ifstrequal{#1}{matlab}{\makepythontexfamily[pyglexer=matlab]{matlab}}{}% + \ifstrequal{#1}{octave}{\makepythontexfamily[pyglexer=octave]{octave}}{}% + \ifstrequal{#1}{bash}{\makepythontexfamily[pyglexer=bash]{bash}}{}% + \ifstrequal{#1}{sage}{\makepythontexfamily[pyglexer=sage]{sage}}{}% + \ifstrequal{#1}{rust}{\makepythontexfamily[pyglexer=rust]{rust}}{}% + \ifstrequal{#1}{rs}{\makepythontexfamily[pyglexer=rust]{rs}}{}% + \ifstrequal{#1}{R}{\makepythontexfamily[pyglexer=r]{R}}{}% + \ifstrequal{#1}{Rcon}{\makepythontexfamily@con[rconsole]{R}}{}% + \ifstrequal{#1}{perl}{\makepythontexfamily[pyglexer=perl]{perl}}{}% + \ifstrequal{#1}{pl}{\makepythontexfamily[pyglexer=perl]{pl}}{}% + \ifstrequal{#1}{perlsix}{\makepythontexfamily[pyglexer=perl6]{perlsix}}{}% + \ifstrequal{#1}{psix}{\makepythontexfamily[pyglexer=perl6]{psix}}{}% + \ifstrequal{#1}{javascript}{\makepythontexfamily[pyglexer=js]{javascript}}{}% + \ifstrequal{#1}{js}{\makepythontexfamily[pyglexer=js]{js}}{}% } \expandafter\docsvlist\expandafter{\pytx@families} % \end{macrocode} @@ -3557,7 +4345,7 @@ {\PackageWarning{\pytx@packagename}% {A "listing" environment already exists \MessageBreak \pytx@packagename\space will not create one \MessageBreak - Use \string\setpythontexlistingenv to create a custom listing environment}}% + Use \string\setpythontexlistingenv\space to create a custom listing environment}}% \else \ifbool{pytx@listingenv}{}{\DeclareFloatingEnvironment[fileext=lopytx]{listing}} \fi @@ -3663,6 +4451,7 @@ % \begin{macrocode} \def\pytx@BEPygmentsFV[#1]{% \def\pytx@fvopttmp{#1}% + \def\pytx@argspprint{#1}% \begingroup \obeylines \pytx@BEPygmentsFV@i @@ -3711,6 +4500,7 @@ \setcounter{FancyVerbLine}{\value{\pytx@linecount}}% \pytx@FVSet \ifdefstring{\pytx@fvopttmp}{}{}{\expandafter\fvset\expandafter{\pytx@fvopttmp}}% + \pytx@ConfigPygments \ifcsname FV@SV@\pytx@counter @\arabic{\pytx@counter}\endcsname \UseVerbatim{\pytx@counter @\arabic{\pytx@counter}}% \else @@ -3733,6 +4523,7 @@ % \begin{macrocode} \def\pytx@BEPygmentsPyg[#1]{% \def\pytx@fvopttmp{#1}% + \def\pytx@argspprint{#1}% \begingroup \obeylines \pytx@BEPygmentsPyg@i @@ -3769,12 +4560,12 @@ % % \begin{macro}{\pytx@MakePygmentsInputFV} % \begin{macro}{\pytx@MakePygmentsInputPyg} -% For completeness, we need to be able to read in a file and highlight it. This is done through some trickery with the current system. We define the type as |PYG|\meta{lexer}, and the command as |verb|. We set the context for consistency. We set the session as |EXT:|\meta{file~name}.\footnote{There is no possibility of this session being confused with a user-defined session, because colons are substituted for hyphens in all user-defined sessions, before they are written to the code file.} Next we define a |fancyvrb| settings macro for the type if it does not already exist. We write info to the code file using |\pytx@WriteCodefileInfoExt|, which writes the standard info to the code file but uses zero for the instance, since external files that are not executed can only have one instance. +% For completeness, we need to be able to read in a file and highlight it. This is done through some trickery with the current system. We define the type as |PYG|\meta{lexer}, and the command as |verb|. We set the context for consistency. We set the session as |EXT:|\meta{file~name}.\footnote{There is no possibility of this session being confused with a user-defined session, because colons are substituted for hyphens in all user-defined sessions, before they are written to the code file.} Next we define a |fancyvrb| settings macro for the type if it does not already exist. We write info to the code file using |\pytx@WriteCodefileInfoExt|. % % Then we check to see if the file actually exists, and issue a warning if not. This saves the user from running |pythontex.py| to get the same error. We perform our typical |FancyVerbLine| trickery. Next we make use of the saved content in the same way as the |pygments| environment. Note that we do not create a counter for the line numbers. This is because under typical usage an external file should have its lines numbered beginning with 1. We also encourage this by setting |firstnumber=auto| before bringing in the content. % % The current naming of the macro in which the Pygments content is saved is probably excessive. In almost every situation, a unique name could be formed with less information. The current approach has been taken to maintain parallelism, thus simplifying |pythontex.py|, and to avoid any rare potential conflicts. -% +% % \begin{macrocode} \def\pytx@MakePygmentsInputFV{ \newcommand{\inputpygments}[3][]{% @@ -3786,6 +4577,8 @@ \pytx@SetGroupVerb \edef\pytx@session{EXT:##3}% \pytx@CheckPygmentsInit{##2}% + \xdef\pytx@counter{pytx@\pytx@type @\pytx@session @\pytx@group}% + \pytx@CheckCounter{\pytx@counter}% \setcounter{pytx@FancyVerbLineTemp}{\value{FancyVerbLine}}% \begingroup \DepyListing %Always must be in a group @@ -3796,32 +4589,43 @@ {\PackageWarning{\pytx@packagename}{Input file <##3> doesn't exist}}% \endgroup \setcounter{FancyVerbLine}{\value{pytx@FancyVerbLineTemp}}% + \stepcounter{\pytx@counter}% }% } \def\pytx@MakePygmentsInputPyg{ \newcommand{\inputpygments}[3][]{% + \begingroup \edef\pytx@type{PYG##2}% \edef\pytx@cmd{verbatim}% \pytx@SetContext \pytx@SetGroupVerb + \def\pytx@argspprint{##1}% \edef\pytx@session{EXT:##3}% \pytx@CheckPygmentsInit{##2}% + \xdef\pytx@counter{pytx@\pytx@type @\pytx@session @\pytx@group}% + \pytx@CheckCounter{\pytx@counter}% \pytx@WriteCodefileInfoExt \IfFileExists{##3}{}{\PackageWarning{\pytx@packagename}% - {Input file <##3> does not exist}} + {Input file <##3> does not exist}}% \setcounter{pytx@FancyVerbLineTemp}{\value{FancyVerbLine}}% \begingroup \pytx@FVSet \fvset{firstnumber=auto}% - \ifcsname FV@SV@pytx@\pytx@type @\pytx@session @\pytx@group @0\endcsname - \UseVerbatim[##1]{pytx@\pytx@type @\pytx@session @\pytx@group @0}% + \pytx@ConfigPygments + \ifcsname FV@SV@pytx@\pytx@type @\pytx@session @\pytx@group + @\arabic{\pytx@counter}\endcsname + \UseVerbatim[##1]{pytx@\pytx@type @\pytx@session @\pytx@group + @\arabic{\pytx@counter}}% \else - \InputIfFileExists{\pytx@outputdir/##3_##2.pygtex}{}% + \InputIfFileExists{\pytx@outputdir/\pytx@type_##3_\pytx@group + _\arabic{\pytx@counter}.pygtex}{}% {\textbf{??~\pytx@packagename~??}% \PackageWarning{\pytx@packagename}{Non-existent Pygments content}}% \fi \endgroup \setcounter{FancyVerbLine}{\value{pytx@FancyVerbLineTemp}}% + \stepcounter{\pytx@counter}% + \endgroup }% } % \end{macrocode} @@ -3851,8 +4655,10 @@ % \begin{macrocode} \newcommand{\setpygmentspygopt}[2][]{% \Depythontex{cmd:setpygmentspygopt:om:n}% - \ifstrempty{#1}{\def\pytx@pygopt{#2}}{% - \expandafter\gdef\csname pytx@pygopt@PYG#1\endcsname{#2}}% + \ifstrempty{#1}% + {\def\pytx@pygopt{#2}\pgfkeys{/PYTX/gopt/pygopt/.cd, #2}}% + {\expandafter\gdef\csname pytx@pygopt@PYG#1\endcsname{#2}% + \pgfkeys{/PYTX/popt/pygopt/.cd, name=#1, #2}}% } \@onlypreamble\setpygmentspygopt % \end{macrocode} @@ -4075,7 +4881,7 @@ }{} %End beta % \end{macrocode} -% +% % % % \iffalse diff --git a/pythontex/pythontex.ins b/pythontex/pythontex.ins index 7eb2530..2d7f3c1 100644 --- a/pythontex/pythontex.ins +++ b/pythontex/pythontex.ins @@ -1,4 +1,4 @@ -%% Copyright (C) 2012-2013 by Geoffrey M. Poore +%% Copyright (C) 2012-2026 by Geoffrey M. Poore %% -------------------------------------------------------------------------- %% This work may be distributed and/or modified under the %% conditions of the LaTeX Project Public License, either version 1.3 @@ -25,7 +25,7 @@ This is a generated file. -Copyright (C) 2012-2013 by Geoffrey M. Poore +Copyright (C) 2012-2026 by Geoffrey M. Poore -------------------------------------------------------------------------- This work may be distributed and/or modified under the conditions of the LaTeX Project Public License, either version 1.3 @@ -45,9 +45,9 @@ version 2005/12/01 or later. \Msg{* To finish the installation you have to move the following *} \Msg{* file into a directory searched by TeX: *} \Msg{* *} -\Msg{* pythontex.sty *} +\Msg{* pythontex.sty *} \Msg{* *} -\Msg{* To produce the documentation run the file pythontex.dtx *} +\Msg{* To produce the documentation run the file pythontex.dtx *} \Msg{* through LaTeX. *} \Msg{* *} \Msg{* Happy TeXing! *} diff --git a/pythontex/pythontex.pdf b/pythontex/pythontex.pdf index 4d3c866..635a412 100644 Binary files a/pythontex/pythontex.pdf and b/pythontex/pythontex.pdf differ diff --git a/pythontex/pythontex.py b/pythontex/pythontex.py index 1be1f6d..21c2966 100644 --- a/pythontex/pythontex.py +++ b/pythontex/pythontex.py @@ -3,12 +3,20 @@ ''' This is the PythonTeX wrapper script. It automatically detects the version -of Python, and then imports the correct code from pythontex2.py or -pythontex3.py. +of Python, and then imports the correct code from pythontex2.py or +pythontex3.py. It is intended for use with the default Python installation +on your system. If you wish to use a different version of Python, you could +launch pythontex2.py or pythontex3.py directly. You should also consider the +command-line option `--interpreter`. This allows you to specify the command +that is actually used to execute the code from your LaTeX documents. Except +for Python console content, it doesn't matter which version of Python is used +to launch pythontex.py; pythontex.py just manages the execution of code from +your LaTeX document. The interpreter setting is what determines the version +under which your code is actually executed. Licensed under the BSD 3-Clause License: -Copyright (c) 2012-2013, Geoffrey M. Poore +Copyright (c) 2012-2026, Geoffrey M. Poore All rights reserved. @@ -47,8 +55,8 @@ import pythontex3 as pythontex else: sys.exit('PythonTeX require Python 3.2+; you are using 3.{0}'.format(sys.version_info.minor)) - -# The "if" statement is needed for multiprocessing under Windows; see the + +# The "if" statement is needed for multiprocessing under Windows; see the # multiprocessing documentation. if __name__ == '__main__': pythontex.main() diff --git a/pythontex/pythontex.sty b/pythontex/pythontex.sty index a71b4f4..d2640c7 100644 --- a/pythontex/pythontex.sty +++ b/pythontex/pythontex.sty @@ -8,7 +8,7 @@ %% %% This is a generated file. %% -%% Copyright (C) 2012-2013 by Geoffrey M. Poore +%% Copyright (C) 2012-2026 by Geoffrey M. Poore %% -------------------------------------------------------------------------- %% This work may be distributed and/or modified under the %% conditions of the LaTeX Project Public License, either version 1.3 @@ -20,18 +20,17 @@ %% \NeedsTeXFormat{LaTeX2e}[1999/12/01] \ProvidesPackage{pythontex} - [2013/08/26 Version~0.12] + [2026/02/15 v0.19 execute and typeset Python code and other languages] \newcommand{\pytx@packagename}{PythonTeX} -\newcommand{\pytx@packageversion}{v0.12} -\RequirePackage{fancyvrb} -\RequirePackage{etex} +\newcommand{\pytx@packageversion}{0.19} +\RequirePackage{fvextra} \RequirePackage{etoolbox} \RequirePackage{xstring} \RequirePackage{pgfopts} \RequirePackage{newfloat} -\RequirePackage{currfile} -\AtBeginDocument{\@ifpackageloaded{color}{}{\RequirePackage{xcolor}}} +\@ifpackageloaded{currfile}{}{\RequirePackage{currfile}} +\AtEndPreamble{\@ifpackageloaded{color}{}{\RequirePackage{xcolor}}} \def\pytx@families{} \pgfkeys{/PYTX/pkgopt/usefamily/.estore in=\pytx@families} \def\pytx@opt@gobble{none} @@ -171,7 +170,23 @@ \def\pytx@pyglexer{} \pgfkeys{/PYTX/pkgopt/pyglexer/.code=\def\pytx@pyglexer{#1}} \def\pytx@pygopt{} -\pgfkeys{/PYTX/pkgopt/pygopt/.code=\def\pytx@pygopt{#1}} +\pgfkeys{/PYTX/pkgopt/pygopt/.code=\def\pytx@pygopt{#1}\pgfkeys{/PYTX/gopt/pygopt/.cd, #1}} +\pgfkeys{/PYTX/gopt/pygopt/.is choice} +\pgfkeys{/PYTX/gopt/pygopt/texcomments/.code=\relax} +\pgfkeys{/PYTX/gopt/pygopt/mathescape/.code=\relax} +\pgfkeys{/PYTX/gopt/pygopt/style/.code=\ifstrempty{#1}{}{\def\pytx@style{#1}}} +\pgfkeys{/PYTX/lopt/pygopt/.is choice} +\pgfkeys{/PYTX/lopt/pygopt/name/.code=\def\pytx@tmp@name{#1}} +\pgfkeys{/PYTX/lopt/pygopt/texcomments/.code=\relax} +\pgfkeys{/PYTX/lopt/pygopt/mathescape/.code=\relax} +\pgfkeys{/PYTX/lopt/pygopt/style/.code=\ifstrempty{#1}{}{% + \expandafter\def\csname pytx@style@\pytx@tmp@name\endcsname{#1}}} +\pgfkeys{/PYTX/popt/pygopt/.is choice} +\pgfkeys{/PYTX/popt/pygopt/name/.code=\def\pytx@tmp@name{#1}} +\pgfkeys{/PYTX/popt/pygopt/texcomments/.code=\relax} +\pgfkeys{/PYTX/popt/pygopt/mathescape/.code=\relax} +\pgfkeys{/PYTX/popt/pygopt/style/.code=\ifstrempty{#1}{}{% + \expandafter\def\csname pytx@style@PYG\pytx@tmp@name\endcsname{#1}}} \def\pytx@fvextfile{-1} \pgfkeys{/PYTX/pkgopt/fvextfile/.default=25} \pgfkeys{/PYTX/pkgopt/fvextfile/.code=\IfInteger{#1}{% @@ -230,21 +245,16 @@ !expandafter!scantokens!expandafter{#1!empty}}% }% !endgroup -\expandafter\ifx\csname directlua\endcsname\relax\else -\def\pytx@SVMCR#1 {% - \edef\pytx@tmp{\csname #1\endcsname}% - \pytx@SVMCR@i} -\long\def\pytx@SVMCR@i#1\endpytx@SVMCR{% - \expandafter\gdef\pytx@tmp{#1}% -} -\fi -\newcommand{\definepythontexcontext}[1]{% +\DefineVerbatimEnvironment{pytx@Verbatim}{Verbatim}{} +\DefineVerbatimEnvironment{pytx@SaveVerbatim}{SaveVerbatim}{} +\newcommand{\setpythontexcontext}[1]{% + \Depythontex{cmd:setpythontexcontext:m:n}% \def\pytx@SetContext{% \edef\pytx@context{#1}% }% } -\definepythontexcontext{} -\@onlypreamble\definepythontexcontext +\setpythontexcontext{} +\@onlypreamble\setpythontexcontext \newcommand{\restartpythontexsession}[1]{% \Depythontex{cmd:restartpythontexsession:m:n}% \def\pytx@SetGroup{% @@ -273,38 +283,55 @@ \StrSubstitute{\jobname}{ }{-}[\pytx@jobname] \StrSubstitute{\pytx@jobname}{"}{}[\pytx@jobname] \StrSubstitute{\pytx@jobname}{*}{-}[\pytx@jobname] +\def\pytx@outputdir{pythontex-files-\pytx@jobname} \newcommand{\setpythontexoutputdir}[1]{% \Depythontex{cmd:setpythontexoutputdir:m:n}% \def\pytx@outputdir{#1}} -\setpythontexoutputdir{pythontex-files-\pytx@jobname} -\AtBeginDocument{% - \@ifpackageloaded{graphicx}{% - \ifx\Ginput@path\@undefined - \graphicspath{{\pytx@outputdir/}}% - \else - \g@addto@macro\Ginput@path{{\pytx@outputdir/}}% - \fi - }{}% -} \@onlypreamble\setpythontexoutputdir +\def\pytx@workingdir{.} +\def\pytx@workingdirset{false} \newcommand{\setpythontexworkingdir}[1]{% \Depythontex{cmd:setpythontexworkingdir:m:n}% \def\pytx@workingdir{#1}% + \def\pytx@workingdirset{true}% } \@onlypreamble\setpythontexworkingdir \AtBeginDocument{% - \ifcsname pytx@workingdir\endcsname\else - %\setpythontexworkingdir{\pytx@outputdir}\fi Depythontex - \let\pytx@workingdir\pytx@outputdir\fi + \ifdefstring{\pytx@workingdir}{}% + {\let\pytx@workingdir\pytx@outputdir}{}% + \ifdefstrequal{\pytx@workingdir}{\pytx@outputdir}{% + \@ifpackageloaded{graphicx}{% + \ifx\Ginput@path\@undefined + \graphicspath{{\pytx@outputdir/}}% + \else + \g@addto@macro\Ginput@path{{\pytx@outputdir/}}% + \fi + }{}% + }{}% } \newbool{pytx@usedpygments} \AfterEndPreamble{% - \ifbool{pytx@usedpygments}% - {\InputIfFileExists{\pytx@outputdir/\pytx@jobname.pytxpyg}{}{}}{}% + \ifbool{pytx@usedpygments}% + {\InputIfFileExists{\pytx@outputdir/\pytx@jobname.pytxpyg}{}{}% + \ifcsname PYGZsq\endcsname + \ifdefstring{\PYGZsq}{\char`\'}{\pytx@patch@PYGZsq}{}% + \fi}% + {}% } +\begingroup +\catcode`\'=\active +\gdef\pytx@patch@PYGZsq{\gdef\PYGZsq{'}} +\endgroup \AtBeginDocument{% \makeatletter - \InputIfFileExists{\pytx@outputdir/\pytx@jobname.pytxmcr}{}{}% + \InputIfFileExists{\pytx@outputdir/\pytx@jobname.pytxmcr}{}% + {\ifstrempty{\pytx@outputdir}% + {\typeout{No file \pytx@jobname.pytxmcr.}}% + {}% + \IfStrEq{\pytx@outputdir}{.}% + {\typeout{No file \pytx@jobname.pytxmcr.}}% + {\typeout{No file \pytx@outputdir/\pytx@jobname.pytxmcr.}}% + \typeout{Run \pytx@packagename\space to create it.}}% \makeatother } \newwrite\pytx@codefile @@ -317,6 +344,7 @@ \immediate\write\pytx@codefile{version=\pytx@packageversion}% \immediate\write\pytx@codefile{outputdir=\pytx@outputdir}% \immediate\write\pytx@codefile{workingdir=\pytx@workingdir}% + \immediate\write\pytx@codefile{workingdirset=\pytx@workingdirset}% \immediate\write\pytx@codefile{gobble=\pytx@opt@gobble}% \immediate\write\pytx@codefile{rerun=\pytx@opt@rerun}% \immediate\write\pytx@codefile{hashdependencies=\pytx@opt@hashdependencies}% @@ -350,7 +378,7 @@ {\let\pytx@currfile\@empty}{\let\pytx@currfile\currfilename}% \immediate\write\pytx@codefile{\pytx@delim\pytx@type\pytx@delimchar% \pytx@session\pytx@delimchar\pytx@group\pytx@delimchar% - 0\pytx@delimchar\pytx@cmd\pytx@delimchar% + \arabic{\pytx@counter}\pytx@delimchar\pytx@cmd\pytx@delimchar% \pytx@context\pytx@delimchar\pytx@argsrun\pytx@delimchar% \pytx@argspprint\pytx@delimchar% \pytx@currfile\pytx@delimchar% @@ -377,6 +405,7 @@ \newcounter{pytx@FancyVerbLineTemp} \def\pytx@FVB@SaveVerbatim#1{% \setcounter{pytx@FancyVerbLineTemp}{\value{FancyVerbLine}}% + \global\FV@CodeLineNo\z@ \@bsphack \begingroup \FV@UseKeyValues @@ -393,6 +422,17 @@ \setcounter{FancyVerbLine}{\value{pytx@FancyVerbLineTemp}}} \let\FVB@SaveVerbatim\pytx@FVB@SaveVerbatim \let\FVE@SaveVerbatim\pytx@FVE@SaveVerbatim +\def\pytx@ConfigPygments{% + \def\pytx@currentstyle{default}% + \ifcsname pytx@style\endcsname + \let\pytx@currentstyle\pytx@style + \else + \ifcsname pytx@style@\pytx@type\endcsname + \expandafter\let\expandafter\pytx@currentstyle\csname pytx@style@\pytx@type\endcsname + \fi + \fi + \expandafter\let\expandafter\PYG@style\csname PYG\pytx@currentstyle\endcsname + \VerbatimPygments{\PYG}{\PYG@style}} \def\pytx@stdfile{} \def\pytx@stdout@warntext{} \def\pytx@FetchStdoutfile[#1][#2]#3{% @@ -638,6 +678,7 @@ \def\pytx@InlineShowPyg{% \begingroup \pytx@FVSet + \pytx@ConfigPygments \ifcsname FV@SV@\pytx@counter @\arabic{\pytx@counter}\endcsname \setcounter{pytx@FancyVerbLineTemp}{\value{FancyVerbLine}}% \BUseVerbatim{\pytx@counter @\arabic{\pytx@counter}}% @@ -661,6 +702,11 @@ } \ifbool{pytx@opt@stdout}{}{\let\pytx@InlineAutoprint\@empty} } +\def\pytx@InlineAlwaysprint{% + \InputIfFileExists{\pytx@outputdir/\pytx@stdfile.stdout}% + {\DepyFile{p:\pytx@outputdir/\pytx@stdfile.stdout}}% + {\textbf{??}% + \PackageWarning{\pytx@packagename}{Missing sub content}}} \def\pytx@InlineMacroprint{% \edef\pytx@mcr{pytx@MCR@\pytx@type @\pytx@session @\pytx@group @\arabic{\pytx@counter}}% \ifcsname\pytx@mcr\endcsname @@ -684,7 +730,7 @@ } \ifbool{pytx@opt@stdout}{}{\let\pytx@InlineMacroprint\@empty} \newcommand{\pytx@MakeInlinebFV}[1]{% - \expandafter\newcommand\expandafter{\csname #1b\endcsname}{% + \expandafter\newrobustcmd\expandafter{\csname #1b\endcsname}{% \expandafter\let\expandafter\pytx@lexer\csname pytx@pyglexer@#1\endcsname \Depythontex{cmd:#1b:ov:c}% \xdef\pytx@type{#1}% @@ -698,7 +744,7 @@ }% }% \newcommand{\pytx@MakeInlinebPyg}[1]{% - \expandafter\newcommand\expandafter{\csname #1b\endcsname}{% + \expandafter\newrobustcmd\expandafter{\csname #1b\endcsname}{% \xdef\pytx@type{#1}% \edef\pytx@cmd{b}% \pytx@SetContext @@ -710,7 +756,7 @@ }% }% \newcommand{\pytx@MakeInlinevFV}[1]{% - \expandafter\newcommand\expandafter{\csname #1v\endcsname}{% + \expandafter\newrobustcmd\expandafter{\csname #1v\endcsname}{% \expandafter\let\expandafter\pytx@lexer\csname pytx@pyglexer@#1\endcsname \Depythontex{cmd:#1v:ov:c}% \xdef\pytx@type{#1}% @@ -724,7 +770,7 @@ }% }% \newcommand{\pytx@MakeInlinevPyg}[1]{% - \expandafter\newcommand\expandafter{\csname #1v\endcsname}{% + \expandafter\newrobustcmd\expandafter{\csname #1v\endcsname}{% \xdef\pytx@type{#1}% \edef\pytx@cmd{v}% \pytx@SetContext @@ -736,7 +782,7 @@ }% }% \newcommand{\pytx@MakeInlinecFV}[1]{% - \expandafter\newcommand\expandafter{\csname #1c\endcsname}{% + \expandafter\newrobustcmd\expandafter{\csname #1c\endcsname}{% \Depythontex{cmd:#1c:ov:p}% \xdef\pytx@type{#1}% \edef\pytx@cmd{c}% @@ -749,8 +795,22 @@ }% }% \let\pytx@MakeInlinecPyg\pytx@MakeInlinecFV +\newcommand{\pytx@MakeInlinesFV}[1]{% + \expandafter\newrobustcmd\expandafter{\csname #1s\endcsname}{% + \Depythontex{cmd:#1s:ov:p}% + \xdef\pytx@type{#1}% + \edef\pytx@cmd{s}% + \pytx@SetContext + \pytx@SetGroup + \let\pytx@InlineShow\@empty + \let\pytx@InlineSave\pytx@InlineSaveCode + \let\pytx@InlinePrint\pytx@InlineAlwaysprint + \pytx@Inline + }% +}% +\let\pytx@MakeInlinesPyg\pytx@MakeInlinesFV \newcommand{\pytx@MakeInlineFV}[1]{% - \expandafter\newcommand\expandafter{\csname #1\endcsname}{% + \expandafter\newrobustcmd\expandafter{\csname #1\endcsname}{% \Depythontex{cmd:#1:ov:p}% \xdef\pytx@type{#1}% \edef\pytx@cmd{i}% @@ -764,7 +824,7 @@ }% \let\pytx@MakeInlinePyg\pytx@MakeInlineFV \newcommand{\pytx@MakeInlineConsFV}[1]{% - \expandafter\newcommand\expandafter{\csname #1\endcsname}{% + \expandafter\newrobustcmd\expandafter{\csname #1\endcsname}{% \Depythontex{cmd:#1:ov:c}% \xdef\pytx@type{#1}% \edef\pytx@cmd{i}% @@ -777,7 +837,7 @@ }% }% \let\pytx@MakeInlineConsPyg\pytx@MakeInlineConsFV -\newcommand{\pythontexcustomc}[2][begin]{% +\newrobustcmd{\pythontexcustomc}[2][begin]{% \Depythontex{cmd:pythontexcustomc:omv:p}% \ifstrequal{#1}{begin}{}{% \ifstrequal{#1}{end}{}{\PackageError{\pytx@packagename}% @@ -786,7 +846,7 @@ }% \xdef\pytx@type{CC:#2:#1}% \edef\pytx@cmd{c}% - \def\pytx@context{}% + \pytx@SetContext \def\pytx@group{none}% \let\pytx@InlineShow\@empty \let\pytx@InlineSave\pytx@InlineSaveCode @@ -910,6 +970,7 @@ \setcounter{FancyVerbLine}{\value{\pytx@linecount}}% \pytx@FVSet \ifdefstring{\pytx@fvopttmp}{}{}{\expandafter\fvset\expandafter{\pytx@fvopttmp}}% + \pytx@ConfigPygments \ifcsname FV@SV@\pytx@counter @\arabic{\pytx@counter}\endcsname \UseVerbatim{\pytx@counter @\arabic{\pytx@counter}}% \else @@ -935,6 +996,7 @@ }% \def\pytx@BeginEnvPyg@i[#1]{% \def\pytx@fvopttmp{#1}% + \def\pytx@argspprint{#1}% \setcounter{pytx@FancyVerbLineTemp}{\value{FancyVerbLine}}% \let\FVB@VerbatimOut\pytx@FVB@VerbatimOut \let\FVE@VerbatimOut\pytx@FVE@VerbatimOut @@ -975,21 +1037,41 @@ }% {\end{VerbatimOut}% \xdef\pytx@stdfile{\pytx@type_\pytx@session_\pytx@group_\arabic{\pytx@counter}}% - \pytx@EnvAutoprint + \ifcsname pytx@nonpyconsole@\pytx@type\endcsname + \ifcsname pytx@code@as@console\endcsname + \setcounter{FancyVerbLine}{\value{\pytx@linecount}}% + \pytx@FVSet + \ifdefstring{\pytx@fvopttmp}{}{}{\expandafter\fvset\expandafter{\pytx@fvopttmp}}% + \pytx@ConfigPygments + \InputIfFileExists{\pytx@outputdir/\pytx@stdfile.stdout}% + {\DepyFile{p:\pytx@outputdir/\pytx@stdfile.stdout}}% + {\par\textbf{??~\pytx@packagename~??}\par + \PackageWarning{\pytx@packagename}{Non-existent console content}}% + \setcounter{\pytx@linecount}{\value{FancyVerbLine}}% + \else + \fi + \let\pytx@EnvAutoprint\relax + \else + \fi \setcounter{FancyVerbLine}{\value{pytx@FancyVerbLineTemp}}% \stepcounter{\pytx@counter}% + \pytx@EnvAutoprint }% }% \def\pytx@BeginCodeEnv[#1]{% \ifstrempty{#1}{\edef\pytx@session{default}}{\StrSubstitute{#1}{:}{-}[\pytx@session]}% \xdef\pytx@counter{pytx@\pytx@type @\pytx@session @\pytx@group}% \pytx@CheckCounter{\pytx@counter}% + \edef\pytx@linecount{\pytx@counter @line}% + \pytx@CheckCounter{\pytx@linecount}% \pytx@WriteCodefileInfo \begingroup \obeylines \@ifnextchar[{\endgroup\pytx@BeginCodeEnv@i}{\endgroup\pytx@BeginCodeEnv@i[]}% }% \def\pytx@BeginCodeEnv@i[#1]{% + \def\pytx@fvopttmp{#1}% + \def\pytx@argspprint{#1}% \setcounter{pytx@FancyVerbLineTemp}{\value{FancyVerbLine}}% \let\FVB@VerbatimOut\pytx@FVB@VerbatimOut \let\FVE@VerbatimOut\pytx@FVE@VerbatimOut @@ -1006,13 +1088,36 @@ }% \xdef\pytx@type{CC:#2:#1}% \edef\pytx@cmd{code}% - \def\pytx@context{}% + \pytx@SetContext \def\pytx@group{none}% \pytx@BeginCodeEnv[none]}% {\end{VerbatimOut}% \setcounter{FancyVerbLine}{\value{pytx@FancyVerbLineTemp}}% \stepcounter{\pytx@counter}% }% +\newcommand{\pytx@MakeSubFV}[1]{% + \expandafter\newenvironment{#1sub}{% + \VerbatimEnvironment + \Depythontex{env:#1sub:oo|:p}% + \xdef\pytx@type{#1}% + \edef\pytx@cmd{sub}% + \pytx@SetContext + \pytx@SetGroup + \begingroup + \obeylines + \@ifnextchar[{\endgroup\pytx@BeginCodeEnv}{\endgroup\pytx@BeginCodeEnv[]}% + }% + {\end{VerbatimOut}% + \xdef\pytx@stdfile{\pytx@type_\pytx@session_\pytx@group_\arabic{\pytx@counter}}% + \setcounter{FancyVerbLine}{\value{pytx@FancyVerbLineTemp}}% + \stepcounter{\pytx@counter}% + \InputIfFileExists{\pytx@outputdir/\pytx@stdfile.stdout}% + {\DepyFile{p:\pytx@outputdir/\pytx@stdfile.stdout}}% + {\textbf{??~\pytx@packagename~??}% + \PackageWarning{\pytx@packagename}{Non-existent substituted content}}% + }% +}% +\let\pytx@MakeSubPyg\pytx@MakeSubFV \newcommand{\pytx@MakeConsFV}[2]{% \expandafter\newenvironment{#1#2}{% \VerbatimEnvironment @@ -1059,6 +1164,7 @@ }% \def\pytx@BeginConsEnvFV@i[#1]{% \def\pytx@fvopttmp{#1}% + \def\pytx@argspprint{#1}% \setcounter{pytx@FancyVerbLineTemp}{\value{FancyVerbLine}}% \let\FVB@VerbatimOut\pytx@FVB@VerbatimOut \let\FVE@VerbatimOut\pytx@FVE@VerbatimOut @@ -1083,6 +1189,7 @@ \setcounter{FancyVerbLine}{\value{\pytx@linecount}}% \pytx@FVSet \ifdefstring{\pytx@fvopttmp}{}{}{\expandafter\fvset\expandafter{\pytx@fvopttmp}}% + \pytx@ConfigPygments \ifcsname FV@SV@\pytx@counter @\arabic{\pytx@counter}\endcsname \UseVerbatim{\pytx@counter @\arabic{\pytx@counter}}% \DepyMacro{c:\pytx@counter @\arabic{\pytx@counter}}% @@ -1100,9 +1207,10 @@ \pgfkeys{ /PYTX/family/.is family, /PYTX/family, + name/.estore in = \pytx@tmp@name, prettyprinter/.estore in = \pytx@tmp@pprinter, pyglexer/.estore in = \pytx@tmp@pyglexer, - pygopt/.estore in = \pytx@tmp@pygopt, + pygopt/.code = \def\pytx@tmp@pygopt{#1}\pgfkeys{/PYTX/lopt/pygopt/.cd, #1}, console/.estore in = \pytx@tmp@console, default/.style = {prettyprinter=auto, pyglexer=text, pygopt={}, console=false} } @@ -1149,14 +1257,16 @@ \IfBeginWith{#2}{PYG}% {\PackageError{\pytx@packagename}% {Attempt to create macros with reserved prefix PYG}{}}{}% - \pgfkeys{/PYTX/family, default, #1} + \pgfkeys{/PYTX/family, name=#2, default, #1} \expandafter\xdef\csname pytx@macroformatter@#2\endcsname{\pytx@tmp@pprinter} \expandafter\gdef\csname pytx@fvsettings@#2\endcsname{} \expandafter\xdef\csname pytx@pyglexer@#2\endcsname{\pytx@tmp@pyglexer} \expandafter\xdef\csname pytx@pygopt@#2\endcsname{\pytx@tmp@pygopt} \expandafter\xdef\csname pytx@console@#2\endcsname{\pytx@tmp@console} \pytx@MakeInlinecFV{#2} + \pytx@MakeInlinesFV{#2} \pytx@MakeCodeFV{#2} + \pytx@MakeSubFV{#2} \AtBeginDocument{% \ifcsstring{pytx@macroformatter@#2}{auto}{% \ifbool{pytx@opt@pygments}% @@ -1172,8 +1282,30 @@ {\ifcsstring{pytx@console@#2}{true}% {\pytx@MakeFamilyPygCons{#2}}{\pytx@MakeFamilyPyg{#2}}}{}% }% + \newcounter{pytx@#2@default@default}% } \@onlypreamble\makepythontexfamily +\newcommand{\makepythontexfamily@con}[2][text]{% + \pgfkeys{/PYTX/family, name=#2con, default, pyglexer=#1, console=true}% + \expandafter\xdef\csname pytx@macroformatter@#2con\endcsname{\pytx@tmp@pprinter}% + \expandafter\gdef\csname pytx@fvsettings@#2con\endcsname{}% + \expandafter\xdef\csname pytx@pyglexer@#2con\endcsname{\pytx@tmp@pyglexer}% + \expandafter\xdef\csname pytx@pygopt@#2con\endcsname{\pytx@tmp@pygopt}% + \expandafter\xdef\csname pytx@console@#2con\endcsname{\pytx@tmp@console}% + \AtEndDocument{\immediate\write\pytx@codefile{pygfamily=#2con|% + \csname pytx@pyglexer@#2con\endcsname|% + \csname pytx@pygopt@#2con\endcsname}% + }% + \pytx@MakeCodeFV{#2con}% + \expandafter\global\expandafter\let\csname pytx@nonpyconsole@#2con\endcsname\relax + \newenvironment{#2console}% + {\VerbatimEnvironment + \def\pytx@type{#2con}% + \let\pytx@code@as@console\relax + \begin{#2concode}}% + {\end{#2concode}}% +} + \newcommand{\setpythontexpyglexer}[2][]{% \Depythontex{cmd:setpythontexpyglexer:om:n}% \ifstrempty{#1}{\def\pytx@pyglexer{#2}}{% @@ -1188,14 +1320,15 @@ \@onlypreamble\setpythontexpyglexer \newcommand{\setpythontexpygopt}[2][]{% \Depythontex{cmd:setpythontexpygopt:om:n}% - \ifstrempty{#1}{\def\pytx@pygopt{#2}}{% - \ifcsname pytx@pygopt@#1\endcsname + \ifstrempty{#1}% + {\def\pytx@pygopt{#2}\pgfkeys{/PYTX/gopt/pygopt/.cd, #2}}% + {\ifcsname pytx@pygopt@#1\endcsname \expandafter\xdef\csname pytx@pygopt@#1\endcsname{#2}% - \else + \pgfkeys{/PYTX/lopt/pygopt/.cd, name=#1, #2} + \else \PackageError{\pytx@packagename}% - {Cannot modify Pygments options for a non-existent family}{}% - \fi - }% + {Cannot modify Pygments options for a non-existent family}{}% + \fi}% } \@onlypreamble\setpythontexpygopt \newcommand{\setpythontexprettyprinter}[2][]{% @@ -1215,17 +1348,32 @@ }% } \@onlypreamble\setpythontexprettyprinter -\makepythontexfamily[pyglexer=python]{py} +\makepythontexfamily[pyglexer=python3]{py} \makepythontexfamily[pyglexer=pycon, console=true]{pycon} -\makepythontexfamily[pyglexer=python]{sympy} +\makepythontexfamily[pyglexer=python3]{sympy} \makepythontexfamily[pyglexer=pycon, console=true]{sympycon} -\makepythontexfamily[pyglexer=python]{pylab} +\makepythontexfamily[pyglexer=python3]{pylab} \makepythontexfamily[pyglexer=pycon, console=true]{pylabcon} \renewcommand{\do}[1]{% \ifstrequal{#1}{ruby}{\makepythontexfamily[pyglexer=ruby]{ruby}}{}% \ifstrequal{#1}{rb}{\makepythontexfamily[pyglexer=ruby]{rb}}{}% \ifstrequal{#1}{julia}{\makepythontexfamily[pyglexer=julia]{julia}}{}% + \ifstrequal{#1}{juliacon}{\makepythontexfamily@con[jlcon]{julia}}{}% \ifstrequal{#1}{jl}{\makepythontexfamily[pyglexer=julia]{jl}}{}% + \ifstrequal{#1}{matlab}{\makepythontexfamily[pyglexer=matlab]{matlab}}{}% + \ifstrequal{#1}{octave}{\makepythontexfamily[pyglexer=octave]{octave}}{}% + \ifstrequal{#1}{bash}{\makepythontexfamily[pyglexer=bash]{bash}}{}% + \ifstrequal{#1}{sage}{\makepythontexfamily[pyglexer=sage]{sage}}{}% + \ifstrequal{#1}{rust}{\makepythontexfamily[pyglexer=rust]{rust}}{}% + \ifstrequal{#1}{rs}{\makepythontexfamily[pyglexer=rust]{rs}}{}% + \ifstrequal{#1}{R}{\makepythontexfamily[pyglexer=r]{R}}{}% + \ifstrequal{#1}{Rcon}{\makepythontexfamily@con[rconsole]{R}}{}% + \ifstrequal{#1}{perl}{\makepythontexfamily[pyglexer=perl]{perl}}{}% + \ifstrequal{#1}{pl}{\makepythontexfamily[pyglexer=perl]{pl}}{}% + \ifstrequal{#1}{perlsix}{\makepythontexfamily[pyglexer=perl6]{perlsix}}{}% + \ifstrequal{#1}{psix}{\makepythontexfamily[pyglexer=perl6]{psix}}{}% + \ifstrequal{#1}{javascript}{\makepythontexfamily[pyglexer=js]{javascript}}{}% + \ifstrequal{#1}{js}{\makepythontexfamily[pyglexer=js]{js}}{}% } \expandafter\docsvlist\expandafter{\pytx@families} \newbool{pytx@listingenv} @@ -1241,7 +1389,7 @@ {\PackageWarning{\pytx@packagename}% {A "listing" environment already exists \MessageBreak \pytx@packagename\space will not create one \MessageBreak - Use \string\setpythontexlistingenv to create a custom listing environment}}% + Use \string\setpythontexlistingenv\space to create a custom listing environment}}% \else \ifbool{pytx@listingenv}{}{\DeclareFloatingEnvironment[fileext=lopytx]{listing}} \fi @@ -1298,6 +1446,7 @@ } \def\pytx@BEPygmentsFV[#1]{% \def\pytx@fvopttmp{#1}% + \def\pytx@argspprint{#1}% \begingroup \obeylines \pytx@BEPygmentsFV@i @@ -1332,6 +1481,7 @@ \setcounter{FancyVerbLine}{\value{\pytx@linecount}}% \pytx@FVSet \ifdefstring{\pytx@fvopttmp}{}{}{\expandafter\fvset\expandafter{\pytx@fvopttmp}}% + \pytx@ConfigPygments \ifcsname FV@SV@\pytx@counter @\arabic{\pytx@counter}\endcsname \UseVerbatim{\pytx@counter @\arabic{\pytx@counter}}% \else @@ -1347,6 +1497,7 @@ } \def\pytx@BEPygmentsPyg[#1]{% \def\pytx@fvopttmp{#1}% + \def\pytx@argspprint{#1}% \begingroup \obeylines \pytx@BEPygmentsPyg@i @@ -1377,6 +1528,8 @@ \pytx@SetGroupVerb \edef\pytx@session{EXT:##3}% \pytx@CheckPygmentsInit{##2}% + \xdef\pytx@counter{pytx@\pytx@type @\pytx@session @\pytx@group}% + \pytx@CheckCounter{\pytx@counter}% \setcounter{pytx@FancyVerbLineTemp}{\value{FancyVerbLine}}% \begingroup \DepyListing %Always must be in a group @@ -1387,32 +1540,43 @@ {\PackageWarning{\pytx@packagename}{Input file <##3> doesn't exist}}% \endgroup \setcounter{FancyVerbLine}{\value{pytx@FancyVerbLineTemp}}% + \stepcounter{\pytx@counter}% }% } \def\pytx@MakePygmentsInputPyg{ \newcommand{\inputpygments}[3][]{% + \begingroup \edef\pytx@type{PYG##2}% \edef\pytx@cmd{verbatim}% \pytx@SetContext \pytx@SetGroupVerb + \def\pytx@argspprint{##1}% \edef\pytx@session{EXT:##3}% \pytx@CheckPygmentsInit{##2}% + \xdef\pytx@counter{pytx@\pytx@type @\pytx@session @\pytx@group}% + \pytx@CheckCounter{\pytx@counter}% \pytx@WriteCodefileInfoExt \IfFileExists{##3}{}{\PackageWarning{\pytx@packagename}% - {Input file <##3> does not exist}} + {Input file <##3> does not exist}}% \setcounter{pytx@FancyVerbLineTemp}{\value{FancyVerbLine}}% \begingroup \pytx@FVSet \fvset{firstnumber=auto}% - \ifcsname FV@SV@pytx@\pytx@type @\pytx@session @\pytx@group @0\endcsname - \UseVerbatim[##1]{pytx@\pytx@type @\pytx@session @\pytx@group @0}% + \pytx@ConfigPygments + \ifcsname FV@SV@pytx@\pytx@type @\pytx@session @\pytx@group + @\arabic{\pytx@counter}\endcsname + \UseVerbatim[##1]{pytx@\pytx@type @\pytx@session @\pytx@group + @\arabic{\pytx@counter}}% \else - \InputIfFileExists{\pytx@outputdir/##3_##2.pygtex}{}% + \InputIfFileExists{\pytx@outputdir/\pytx@type_##3_\pytx@group + _\arabic{\pytx@counter}.pygtex}{}% {\textbf{??~\pytx@packagename~??}% \PackageWarning{\pytx@packagename}{Non-existent Pygments content}}% \fi \endgroup \setcounter{FancyVerbLine}{\value{pytx@FancyVerbLineTemp}}% + \stepcounter{\pytx@counter}% + \endgroup }% } \newcommand{\setpygmentsfv}[2][]{% @@ -1423,8 +1587,10 @@ }% \newcommand{\setpygmentspygopt}[2][]{% \Depythontex{cmd:setpygmentspygopt:om:n}% - \ifstrempty{#1}{\def\pytx@pygopt{#2}}{% - \expandafter\gdef\csname pytx@pygopt@PYG#1\endcsname{#2}}% + \ifstrempty{#1}% + {\def\pytx@pygopt{#2}\pgfkeys{/PYTX/gopt/pygopt/.cd, #2}}% + {\expandafter\gdef\csname pytx@pygopt@PYG#1\endcsname{#2}% + \pgfkeys{/PYTX/popt/pygopt/.cd, name=#1, #2}}% } \@onlypreamble\setpygmentspygopt \newcommand{\setpygmentsprettyprinter}[1]{% diff --git a/pythontex/pythontex2.py b/pythontex/pythontex2.py index c123ade..c83a81f 100644 --- a/pythontex/pythontex2.py +++ b/pythontex/pythontex2.py @@ -1,19 +1,19 @@ -#!/usr/bin/env python +#!/usr/bin/env python2 # -*- coding: utf-8 -*- ''' This is the main PythonTeX script. It should be launched via pythontex.py. -Two versions of this script are provided. One, with name ending in "2", runs +Two versions of this script are provided. One, with name ending in "2", runs under Python 2.7. The other, with name ending in "3", runs under Python 3.2+. -This script needs to be able to import pythontex_engines.py; in general it +This script needs to be able to import pythontex_engines.py; in general it should be in the same directory. Licensed under the BSD 3-Clause License: -Copyright (c) 2012-2013, Geoffrey M. Poore +Copyright (c) 2012-2026, Geoffrey M. Poore All rights reserved. @@ -61,6 +61,8 @@ from pygments.styles import get_all_styles from pythontex_engines import * import textwrap +import platform +import itertools if sys.version_info[0] == 2: try: @@ -76,7 +78,7 @@ # Script parameters # Version -version = 'v0.12' +__version__ = '0.19' @@ -84,80 +86,94 @@ class Pytxcode(object): def __init__(self, data, gobble): self.delims, self.code = data.split('#\n', 1) - self.input_family, self.input_session, self.input_restart, self.input_instance, self.input_command, self.input_context, self.input_args_run, self.input_args_prettyprint, self.input_file, self.input_line = self.delims.split('#') - self.input_instance_int = int(self.input_instance) - self.input_line_int = int(self.input_line) - self.key_run = self.input_family + '#' + self.input_session + '#' + self.input_restart - self.key_typeset = self.key_run + '#' + self.input_instance - self.hashable_delims_run = self.key_typeset + '#' + self.input_command + '#' + self.input_context + '#' + self.input_args_run - self.hashable_delims_typeset = self.key_typeset + '#' + self.input_command + '#' + self.input_context + '#' + self.input_args_run - if len(self.input_command) > 1: + self.family, self.session, self.restart, self.instance, self.command, self.context, self.args_run, self.args_prettyprint, self.input_file, self.line = self.delims.split('#') + self.instance_int = int(self.instance) + self.line_int = int(self.line) + self.key_run = self.family + '#' + self.session + '#' + self.restart + self.key_typeset = self.key_run + '#' + self.instance + self.hashable_delims_run = self.key_typeset + '#' + self.command + '#' + self.context + '#' + self.args_run + self.hashable_delims_typeset = self.key_typeset + '#' + self.command + '#' + self.context + '#' + self.args_run + if len(self.command) > 1: self.is_inline = False # Environments start on the next line - self.input_line_int += 1 - self.input_line = str(self.input_line_int) + self.line_int += 1 + self.line = str(self.line_int) else: self.is_inline = True - self.is_extfile = True if self.input_session.startswith('EXT:') else False + self.is_extfile = True if self.session.startswith('EXT:') else False if self.is_extfile: - self.extfile = os.path.expanduser(os.path.normcase(self.input_session.replace('EXT:', '', 1))) - self.is_cc = True if self.input_family.startswith('CC:') else False - self.is_pyg = True if self.input_family.startswith('PYG') else False - self.is_verb = True if self.input_restart.endswith('verb') else False + self.extfile = os.path.expanduser(os.path.normcase(self.session.replace('EXT:', '', 1))) + self.key_typeset = self.key_typeset.replace('EXT:', '') + self.is_cc = True if self.family.startswith('CC:') else False + self.is_pyg = True if self.family.startswith('PYG') else False + self.is_verb = True if self.restart.endswith('verb') else False if self.is_cc: - self.input_instance += 'CC' - self.cc_type, self.cc_pos = self.input_family.split(':')[1:] + self.instance += 'CC' + self.cc_type, self.cc_pos = self.family.split(':')[1:] if self.is_verb or self.is_pyg or self.is_cc: self.is_cons = False else: - self.is_cons = engine_dict[self.input_family].console + self.is_cons = engine_dict[self.family].console self.is_code = False if self.is_verb or self.is_pyg or self.is_cc or self.is_cons else True - if self.input_command in ('c', 'code') or (self.input_command == 'i' and not self.is_cons): + if self.command in ('c', 'code') or (self.command == 'i' and not self.is_cons): self.is_typeset = False else: self.is_typeset = True - + if gobble == 'auto': self.code = textwrap.dedent(self.code) - + + self.sub_template = None + def process_argv(data, temp_data): ''' Process command line options using the argparse module. - + Most options are passed via the file of code, rather than via the command line. ''' - + # Create a command line argument parser parser = argparse.ArgumentParser() parser.add_argument('TEXNAME', help='LaTeX file, with or without .tex extension') - parser.add_argument('--version', action='version', - version='PythonTeX {0}'.format(data['version'])) - parser.add_argument('--encoding', default='UTF-8', + parser.add_argument('--version', action='version', + version='PythonTeX {0}'.format(data['version'])) + parser.add_argument('--encoding', default='UTF-8', help='encoding for all text files (see codecs module for encodings)') - parser.add_argument('--error-exit-code', default='true', - choices=('true', 'false'), + parser.add_argument('--error-exit-code', default='true', + choices=('true', 'false'), help='return exit code of 1 if there are errors (not desirable with some TeX editors and workflows)') - group = parser.add_mutually_exclusive_group() - group.add_argument('--runall', nargs='?', default='false', - const='true', choices=('true', 'false'), - help='run ALL code; equivalent to package option') - group.add_argument('--rerun', default='errors', - choices=('never', 'modified', 'errors', 'warnings', 'always'), - help='set conditions for rerunning code; equivalent to package option') - parser.add_argument('--hashdependencies', nargs='?', default='false', - const='true', choices=('true', 'false'), + group_run = parser.add_mutually_exclusive_group() + group_run.add_argument('--runall', nargs='?', default='false', + const='true', choices=('true', 'false'), + help='run ALL code; equivalent to package option') + group_run.add_argument('--rerun', default='errors', + choices=('never', 'modified', 'errors', 'warnings', 'always'), + help='set conditions for rerunning code; equivalent to package option') + parser.add_argument('--hashdependencies', nargs='?', default='false', + const='true', choices=('true', 'false'), help='hash dependencies (such as external data) to check for modification, rather than using mtime; equivalent to package option') + parser.add_argument('-j', '--jobs', metavar='N', default=None, type=int, + help='Allow N jobs at once; defaults to cpu_count().') parser.add_argument('-v', '--verbose', default=False, action='store_true', help='verbose output') parser.add_argument('--interpreter', default=None, help='set a custom interpreter; argument should be in the form ":, :, ..." where is "python", "ruby", etc., and is the command for invoking the interpreter; argument may also be in the form of a Python dictionary') + group_debug = parser.add_mutually_exclusive_group() + group_debug.add_argument('--debug', nargs='?', default=None, + const='default', + metavar='::', + help='Run the specified session (or default session) with the default debugger, if available. If there is only one session, it need not be specified. If the session name is unambiguous, it is sufficient. The full :: (for example, py:default:default) is only needed when the session name alone would be ambiguous.') + group_debug.add_argument('--interactive', nargs='?', default=None, + const='default', + metavar='::', + help='Run the specified session (or default session) in interactive mode. If there is only one session, it need not be specified. If the session name is unambiguous, it is sufficient. The full :: (for example, py:default:default) is only needed when the session name alone would be ambiguous.') args = parser.parse_args() - - # Store the parsed argv in data and temp_data + + # Store the parsed argv in data and temp_data data['encoding'] = args.encoding if args.error_exit_code == 'true': temp_data['error_exit_code'] = True @@ -174,25 +190,91 @@ def process_argv(data, temp_data): temp_data['hashdependencies'] = True else: temp_data['hashdependencies'] = False + if args.jobs is None: + try: + jobs = multiprocessing.cpu_count() + except NotImplementedError: + jobs = 1 + temp_data['jobs'] = jobs + else: + temp_data['jobs'] = args.jobs temp_data['verbose'] = args.verbose + temp_data['debug'] = args.debug + temp_data['interactive'] = args.interactive # Update interpreter_dict based on interpreter + set_python_interpreter = False if args.interpreter is not None: interp_list = args.interpreter.lstrip('{').rstrip('}').split(',') for interp in interp_list: if interp: try: - k, v = interp.split(':') + k, v = interp.split(':', 1) k = k.strip(' \'"') v = v.strip(' \'"') interpreter_dict[k] = v + if k == 'python': + set_python_interpreter = True except: print('Invalid --interpreter argument') return sys.exit(2) - - + # If the Python interpreter wasn't set, then try to set an appropriate + # default value, based on how PythonTeX was launched (pythontex.py, + # pythontex2.py, or pythontex3.py). + if not set_python_interpreter: + if temp_data['python'] == 2: + if platform.system() == 'Windows': + try: + subprocess.check_output(['py', '--version']) + interpreter_dict['python'] = 'py -2' + except: + msg = ''' + * PythonTeX error: + You have launched PythonTeX using pythontex{0}.py + directly. This should only be done when you want + to use Python version {0}, but have a different + version installed as the default. (Otherwise, you + should start PythonTeX with pythontex.py.) For + this to work correctly, you should install Python + version 3.3+, which has a Windows wrapper (py) that + PythonTeX can use to run the correct version of + Python. If you do not want to install Python 3.3+, + you can also use the --interpreter command-line + option to tell PythonTeX how to access the version + of Python you wish to use. + '''.format(temp_data['python']) + print(textwrap.dedent(msg[1:])) + return sys.exit(2) + else: + interpreter_dict['python'] = 'python2' + elif temp_data['python'] == 3: + if platform.system() == 'Windows': + try: + subprocess.check_output(['py', '--version']) + interpreter_dict['python'] = 'py -3' + except: + msg = ''' + * PythonTeX error: + You have launched PythonTeX using pythontex{0}.py + directly. This should only be done when you want + to use Python version {0}, but have a different + version installed as the default. (Otherwise, you + should start PythonTeX with pythontex.py.) For + this to work correctly, you should install Python + version 3.3+, which has a Windows wrapper (py) that + PythonTeX can use to run the correct version of + Python. If you do not want to install Python 3.3+, + you can also use the --interpreter command-line + option to tell PythonTeX how to access the version + of Python you wish to use. + '''.format(temp_data['python']) + print(textwrap.dedent(msg[1:])) + return sys.exit(2) + else: + interpreter_dict['python'] = 'python3' + if args.TEXNAME is not None: - # Determine if we a dealing with just a filename, or a name plus - # path. If there's a path, we need to make the document directory + # Determine if we a dealing with just a filename, or a name plus + # path. If there's a path, we need to make the document directory # the current working directory. dir, raw_jobname = os.path.split(args.TEXNAME) dir = os.path.expanduser(os.path.normcase(dir)) @@ -208,30 +290,30 @@ def process_argv(data, temp_data): print(' Code file ' + raw_jobname + '.pytxcode does not exist.') print(' Run LaTeX to create it.') return sys.exit(1) - - # We need a "sanitized" version of the jobname, with spaces and - # asterisks replaced with hyphens. This is done to avoid TeX issues - # with spaces in file names, paralleling the approach taken in - # pythontex.sty. From now on, we will use the sanitized version every - # time we create a file that contains the jobname string. The raw - # version will only be used in reference to pre-existing files created + + # We need a "sanitized" version of the jobname, with spaces and + # asterisks replaced with hyphens. This is done to avoid TeX issues + # with spaces in file names, paralleling the approach taken in + # pythontex.sty. From now on, we will use the sanitized version every + # time we create a file that contains the jobname string. The raw + # version will only be used in reference to pre-existing files created # on the TeX side, such as the .pytxcode file. jobname = raw_jobname.replace(' ', '-').replace('"', '').replace('*', '-') # Store the results in data data['raw_jobname'] = raw_jobname data['jobname'] = jobname - - # We need to check to make sure that the "sanitized" jobname doesn't - # lead to a collision with a file that already has that name, so that + + # We need to check to make sure that the "sanitized" jobname doesn't + # lead to a collision with a file that already has that name, so that # two files attempt to use the same PythonTeX folder. - # + # # If . and . both exist, where - # is a common LaTeX extension, we exit. We operate under the - # assumption that there should be only a single file in the - # document root directory that has a common LaTeX extension. That - # could be false, but if so, the user probably has worse things to + # is a common LaTeX extension, we exit. We operate under the + # assumption that there should be only a single file in the + # document root directory that has a common LaTeX extension. That + # could be false, but if so, the user probably has worse things to # worry about than a potential PythonTeX output collision. - # If * and * both exist, we issue a warning but + # If * and * both exist, we issue a warning but # attempt to proceed. if jobname != raw_jobname: resolved = False @@ -256,9 +338,9 @@ def process_argv(data, temp_data): print(' ' + jobname + '*') print(' Attempting to proceed.') temp_data['warnings'] += 1 - break - - + break + + def load_code_get_settings(data, temp_data): @@ -278,7 +360,7 @@ def load_code_get_settings(data, temp_data): print(' Code file ' + raw_jobname + '.pytxcode does not exist.') print(' Run LaTeX to create it.') return sys.exit(1) - + # Split code and settings try: pytxcode, pytxsettings = pytxcode.rsplit('=>PYTHONTEX:SETTINGS#', 1) @@ -286,8 +368,8 @@ def load_code_get_settings(data, temp_data): print('The .pytxcode file appears to have an outdated format or be invalid') print('Run LaTeX to make sure the file is current') return sys.exit(1) - - + + # Prepare to process settings # # Create a dict for storing settings. @@ -295,7 +377,7 @@ def load_code_get_settings(data, temp_data): # Create a dict for storing Pygments settings. # Each dict entry will itself be a dict. pygments_settings = defaultdict(dict) - + # Create a dict of processing functions, and generic processing functions settings_func = dict() def set_kv_data(k, v): @@ -317,7 +399,7 @@ def set_kv_temp_data_if_not_default(k, v): def set_kv_data_fvextfile(k, v): # Error checking on TeX side should be enough, but be careful anyway try: - v = int(v) + v = int(v) except ValueError: print('* PythonTeX error') print(' Unable to parse package option fvextfile.') @@ -332,7 +414,7 @@ def set_kv_data_fvextfile(k, v): else: settings[k] = v def set_kv_pygments(k, v): - input_family, lexer_opts, options = v.replace(' ','').split('|') + family, lexer_opts, options = v.replace(' ','').split('|') lexer = None lex_dict = {} opt_dict = {} @@ -355,10 +437,10 @@ def set_kv_pygments(k, v): elif v in ('false', 'False'): v = False else: - k = option + k = options v = True opt_dict[k] = v - if input_family != ':GLOBAL': + if family != ':GLOBAL': if 'lexer' in pygments_settings[':GLOBAL']: lexer = pygments_settings[':GLOBAL']['lexer'] lex_dict.update(pygments_settings[':GLOBAL']['lexer_options']) @@ -367,12 +449,13 @@ def set_kv_pygments(k, v): opt_dict['style'] = 'default' opt_dict['commandprefix'] = 'PYG' + opt_dict['style'] if lexer is not None: - pygments_settings[input_family]['lexer'] = lexer - pygments_settings[input_family]['lexer_options'] = lex_dict - pygments_settings[input_family]['formatter_options'] = opt_dict + pygments_settings[family]['lexer'] = lexer + pygments_settings[family]['lexer_options'] = lex_dict + pygments_settings[family]['formatter_options'] = opt_dict settings_func['version'] = set_kv_data settings_func['outputdir'] = set_kv_data settings_func['workingdir'] = set_kv_data + settings_func['workingdirset'] = set_kv_data settings_func['gobble'] = set_kv_data settings_func['rerun'] = set_kv_temp_data_if_not_default settings_func['hashdependencies'] = set_kv_temp_data_if_not_default @@ -388,7 +471,7 @@ def set_kv_pygments(k, v): settings_func['pyconbanner'] = set_kv_data settings_func['pyconfilename'] = set_kv_data settings_func['depythontex'] = set_kv_data - + # Process settings for line in pytxsettings.split('\n'): if line: @@ -400,40 +483,39 @@ def set_kv_pygments(k, v): print(' Unknown option "' + key + '"') temp_data['warnings'] += 1 - # Check for compatility between the .pytxcode and the script + # Check for compatibility between the .pytxcode and the script if 'version' not in settings or settings['version'] != data['version']: - print('* PythonTeX warning') - print(' The version of the PythonTeX scripts does not match') - print(' the last code saved by the document--run LaTeX to create') - print(' an updated version. Attempting to proceed.') - sys.stdout.flush() - + print('* PythonTeX error') + print(' The version of the PythonTeX scripts does not match the last code') + print(' saved by the document--run LaTeX to create an updated version.\n') + sys.exit(1) + # Store all results that haven't already been stored. data['settings'] = settings data['pygments_settings'] = pygments_settings - + # Create a tuple of vital quantities that invalidate old saved data # Don't need to include outputdir, because if that changes, no old output # fvextfile could be checked on a case-by-case basis, which would result - # in faster output, but that would involve a good bit of additional + # in faster output, but that would involve a good bit of additional # logic, which probably isn't worth it for a feature that will rarely be # changed. - data['vitals'] = (data['version'], data['encoding'], + data['vitals'] = (data['version'], data['encoding'], settings['gobble'], settings['fvextfile']) - + # Create tuples of vital quantities data['code_vitals'] = (settings['workingdir'], settings['keeptemps'], settings['makestderr'], settings['stderrfilename']) data['cons_vitals'] = (settings['workingdir']) data['typeset_vitals'] = () - + # Pass any customizations to types for k in engine_dict: engine_dict[k].customize(pyfuture=settings['pyfuture'], pyconfuture=settings['pyconfuture'], pyconbanner=settings['pyconbanner'], pyconfilename=settings['pyconfilename']) - + # Store code # Do this last, so that Pygments settings are available if pytxcode.startswith('=>PYTHONTEX#'): @@ -445,41 +527,79 @@ def set_kv_pygments(k, v): +def set_upgrade_compatibility(data, old, temp_data): + ''' + When upgrading, modify settings to maintain backward compatibility when + possible and important + ''' + if (old['version'].startswith('v') and + not data['settings']['workingdirset'] and + data['settings']['outputdir'] != '.'): + old['compatibility'] = '0.13' + do_upgrade_compatibility(data, old, temp_data) + + +def do_upgrade_compatibility(data, old_data, temp_data): + if 'compatibility' in old_data: + c = old_data['compatibility'] + if (c == '0.13' and not data['settings']['workingdirset'] and + data['settings']['outputdir'] != '.'): + data['compatibility'] = c + data['settings']['workingdir'] = data['settings']['outputdir'] + msg = ''' + **** PythonTeX upgrade message **** + Beginning with v0.14, the default working directory is the document + directory rather than the output directory. PythonTeX has detected + that you have been using the output directory as the working directory. + It will continue to use the output directory for now. To keep your + current settings long-term and avoid seeing this message in the future, + add the following command to the preamble of your document, right after + the "\\usepackage{pythontex}": "\setpythontexworkingdir{}". + If you wish to continue with the new settings instead, simply delete + the file with extension .pkl in the output directory, and run PythonTeX. + **** End PythonTeX upgrade message **** + ''' + temp_data['upgrade_message'] = textwrap.dedent(msg) + + def get_old_data(data, old_data, temp_data): ''' - Load data from the last run, if it exists, into the dict old_data. - Determine the path to the PythonTeX scripts, either by using a previously + Load data from the last run, if it exists, into the dict old_data. + Determine the path to the PythonTeX scripts, either by using a previously found, saved path or via kpsewhich. - - The old data is used for determining when PythonTeX has been upgraded, - when any settings have changed, when code has changed (via hashes), and - what files may need to be cleaned up. The location of the PythonTeX - scripts is needed so that they can be imported by the scripts created by - PythonTeX. The location of the scripts is confirmed even if they were - previously located, to make sure that the path is still valid. Finding - the scripts depends on having a TeX installation that includes the + + The old data is used for determining when PythonTeX has been upgraded, + when any settings have changed, when code has changed (via hashes), and + what files may need to be cleaned up. The location of the PythonTeX + scripts is needed so that they can be imported by the scripts created by + PythonTeX. The location of the scripts is confirmed even if they were + previously located, to make sure that the path is still valid. Finding + the scripts depends on having a TeX installation that includes the Kpathsea library (TeX Live and MiKTeX, possibly others). - + All code that relies on old_data is written based on the assumption that - if old_data exists and has the current PythonTeX version, then it + if old_data exists and has the current PythonTeX version, then it contains all needed information. Thus, all code relying on old_data must - check that it was loaded and that it has the current version. If not, + check that it was loaded and that it has the current version. If not, code should adapt gracefully. ''' # Create a string containing the name of the data file - pythontex_data_file = os.path.join(data['settings']['outputdir'], 'pythontex_data.pkl') - + pythontex_data_file = os.path.expanduser(os.path.normcase(os.path.join(data['settings']['outputdir'], 'pythontex_data.pkl'))) + # Load the old data if it exists (read as binary pickle) if os.path.isfile(pythontex_data_file): f = open(pythontex_data_file, 'rb') old = pickle.load(f) f.close() - # Check for compabilility + # Check for compatibility if 'vitals' in old and data['vitals'] == old['vitals']: temp_data['loaded_old_data'] = True old_data.update(old) + do_upgrade_compatibility(data, old_data, temp_data) else: + if 'version' in old and old['version'] != data['version']: + set_upgrade_compatibility(data, old, temp_data) temp_data['loaded_old_data'] = False # Clean up all old files if 'files' in old: @@ -496,16 +616,17 @@ def get_old_data(data, old_data, temp_data): os.remove(f) else: temp_data['loaded_old_data'] = False - + # Set the utilspath + # Assume that if the utils aren't in the same location as + # `pythontex.py`, then they are somewhere else on `sys.path` that + # will always be available (for example, installed as a Python module), + # and thus specifying a path isn't necessary. if os.path.isfile(os.path.join(sys.path[0], 'pythontex_utils.py')): # Need the path with forward slashes, so escaping isn't necessary data['utilspath'] = sys.path[0].replace('\\', '/') else: - print('* PythonTeX error') - print(' Could not determine the utils path from sys.path[0]') - print(' The file "pythontex_utils.py" may be missing') - return sys.exit(1) + data['utilspath'] = '' @@ -518,35 +639,35 @@ def modified_dependencies(key, data, old_data, temp_data): old_dep_hash_dict = old_data['dependencies'][key] workingdir = data['settings']['workingdir'] for dep in old_dep_hash_dict.keys(): - # We need to know if the path is relative (based off the - # working directory) or absolute. We can't use - # os.path.isabs() alone for determining the distinction, + # We need to know if the path is relative (based off the + # working directory) or absolute. We can't use + # os.path.isabs() alone for determining the distinction, # because we must take into account the possibility of an # initial ~ (tilde) standing for the home directory. dep_file = os.path.expanduser(os.path.normcase(dep)) if not os.path.isabs(dep_file): - dep_file = os.path.join(workingdir, dep_file) + dep_file = os.path.expanduser(os.path.normcase(os.path.join(workingdir, dep_file))) if not os.path.isfile(dep_file): print('* PythonTeX error') print(' Cannot find dependency "' + dep + '"') print(' It belongs to ' + key.replace('#', ':')) print(' Relative paths to dependencies must be specified from the working directory.') temp_data['errors'] += 1 - # A removed dependency should trigger an error, but it - # shouldn't cause code to execute. Running the code - # again would just give more errors when it can't find - # the dependency. (There won't be issues when a - # dependency is added or removed, because that would - # involve modifying code, which would trigger + # A removed dependency should trigger an error, but it + # shouldn't cause code to execute. Running the code + # again would just give more errors when it can't find + # the dependency. (There won't be issues when a + # dependency is added or removed, because that would + # involve modifying code, which would trigger # re-execution.) elif hashdependencies: - # Read and hash the file in binary. Opening in text mode + # Read and hash the file in binary. Opening in text mode # would require an unnecessary decoding and encoding cycle. f = open(dep_file, 'rb') hasher = sha1() - hash = hasher(f.read()).hexdigest() + h = hasher(f.read()).hexdigest() f.close() - if hash != old_dep_hash_dict[dep][1]: + if h != old_dep_hash_dict[dep][1]: return True else: mtime = os.path.getmtime(dep_file) @@ -589,28 +710,28 @@ def should_rerun(hash, old_hash, old_exit_status, key, rerun, data, old_data, te def hash_all(data, temp_data, old_data, engine_dict): ''' Hash the code to see what has changed and needs to be updated. - - Save the hashes in hashdict. Create update_code, a list of bools - regarding whether code should be executed. Create update_pygments, a - list of bools determining what needs updated Pygments highlighting. - Update pygments_settings to account for Pygments (as opposed to PythonTeX) + + Save the hashes in hashdict. Create update_code, a list of bools + regarding whether code should be executed. Create update_pygments, a + list of bools determining what needs updated Pygments highlighting. + Update pygments_settings to account for Pygments (as opposed to PythonTeX) commands and environments. ''' - # Note that the PythonTeX information that accompanies code must be - # hashed in addition to the code itself; the code could stay the same, - # but its context or args could change, which might require that code be - # executed. All of the PythonTeX information is hashed except for the - # input line number. Context-dependent code is going too far if + # Note that the PythonTeX information that accompanies code must be + # hashed in addition to the code itself; the code could stay the same, + # but its context or args could change, which might require that code be + # executed. All of the PythonTeX information is hashed except for the + # input line number. Context-dependent code is going too far if # it depends on that. - + # Create variables to more easily access parts of data pytxcode = temp_data['pytxcode'] encoding = data['encoding'] loaded_old_data = temp_data['loaded_old_data'] rerun = temp_data['rerun'] pygments_settings = data['pygments_settings'] - + # Calculate cumulative hashes for all code that is executed # Calculate individual hashes for all code that will be typeset code_hasher = defaultdict(sha1) @@ -625,6 +746,7 @@ def hash_all(data, temp_data, old_data, engine_dict): if c.is_typeset: typeset_hasher[c.key_typeset].update(c.hashable_delims_typeset.encode(encoding)) typeset_hasher[c.key_typeset].update(code_encoded) + typeset_hasher[c.key_typeset].update(c.args_prettyprint.encode(encoding)) elif c.is_cons: cons_hasher[c.key_run].update(c.hashable_delims_run.encode(encoding)) code_encoded = c.code.encode(encoding) @@ -632,37 +754,39 @@ def hash_all(data, temp_data, old_data, engine_dict): if c.is_typeset: typeset_hasher[c.key_typeset].update(c.hashable_delims_typeset.encode(encoding)) typeset_hasher[c.key_typeset].update(code_encoded) + typeset_hasher[c.key_typeset].update(c.args_prettyprint.encode(encoding)) elif c.is_cc: cc_hasher[c.cc_type].update(c.hashable_delims_run.encode(encoding)) cc_hasher[c.cc_type].update(c.code.encode(encoding)) elif c.is_typeset: typeset_hasher[c.key_typeset].update(c.hashable_delims_typeset.encode(encoding)) typeset_hasher[c.key_typeset].update(c.code.encode(encoding)) - + typeset_hasher[c.key_typeset].update(c.args_prettyprint.encode(encoding)) + # Store hashes code_hash_dict = {} for key in code_hasher: - input_family = key.split('#', 1)[0] - code_hash_dict[key] = (code_hasher[key].hexdigest(), - cc_hasher[input_family].hexdigest(), - engine_dict[input_family].get_hash()) + family = key.split('#', 1)[0] + code_hash_dict[key] = (code_hasher[key].hexdigest(), + cc_hasher[family].hexdigest(), + engine_dict[family].get_hash()) data['code_hash_dict'] = code_hash_dict - + cons_hash_dict = {} for key in cons_hasher: - input_family = key.split('#', 1)[0] - cons_hash_dict[key] = (cons_hasher[key].hexdigest(), - cc_hasher[input_family].hexdigest(), - engine_dict[input_family].get_hash()) + family = key.split('#', 1)[0] + cons_hash_dict[key] = (cons_hasher[key].hexdigest(), + cc_hasher[family].hexdigest(), + engine_dict[family].get_hash()) data['cons_hash_dict'] = cons_hash_dict - + typeset_hash_dict = {} for key in typeset_hasher: typeset_hash_dict[key] = typeset_hasher[key].hexdigest() data['typeset_hash_dict'] = typeset_hash_dict - - + + # See what needs to be updated. # In the process, copy over macros and files that may be reused. code_update = {} @@ -689,27 +813,27 @@ def hash_all(data, temp_data, old_data, engine_dict): old_typeset_hash_dict = old_data['typeset_hash_dict'] old_pygments_settings = old_data['pygments_settings'] for s in pygments_settings: - if (s in old_pygments_settings and + if (s in old_pygments_settings and pygments_settings[s] == old_pygments_settings[s]): pygments_settings_changed[s] = False else: pygments_settings_changed[s] = True - # If old data was loaded (and thus is compatible) determine what has - # changed so that only + # If old data was loaded (and thus is compatible) determine what has + # changed so that only # modified code may be executed. Otherwise, execute everything. # We don't have to worry about checking for changes in pyfuture, because # custom code and default code are hashed. The treatment of keeptemps # could be made more efficient (if changed to 'none', just delete old temp - # files rather than running everything again), but given that it is + # files rather than running everything again), but given that it is # intended as a debugging aid, that probable isn't worth it. - # We don't have to worry about hashdependencies changing, because if it + # We don't have to worry about hashdependencies changing, because if it # does the hashes won't match (file contents vs. mtime) and thus code will # be re-executed. if loaded_old_data and data['code_vitals'] == old_data['code_vitals']: # Compare the hash values, and set which code needs to be run for key in code_hash_dict: - if (key in old_code_hash_dict and + if (key in old_code_hash_dict and not should_rerun(code_hash_dict[key], old_code_hash_dict[key], old_exit_status[key], key, rerun, data, old_data, temp_data)): code_update[key] = False macros[key] = old_macros[key] @@ -717,15 +841,15 @@ def hash_all(data, temp_data, old_data, engine_dict): dependencies[key] = old_dependencies[key] exit_status[key] = old_exit_status[key] else: - code_update[key] = True - else: + code_update[key] = True + else: for key in code_hash_dict: code_update[key] = True - + if loaded_old_data and data['cons_vitals'] == old_data['cons_vitals']: # Compare the hash values, and set which code needs to be run for key in cons_hash_dict: - if (key in old_cons_hash_dict and + if (key in old_cons_hash_dict and not should_rerun(cons_hash_dict[key], old_cons_hash_dict[key], old_exit_status[key], key, rerun, data, old_data, temp_data)): cons_update[key] = False macros[key] = old_macros[key] @@ -734,23 +858,26 @@ def hash_all(data, temp_data, old_data, engine_dict): dependencies[key] = old_dependencies[key] exit_status[key] = old_exit_status[key] else: - cons_update[key] = True - else: + cons_update[key] = True + else: for key in cons_hash_dict: cons_update[key] = True - + if loaded_old_data and data['typeset_vitals'] == old_data['typeset_vitals']: for key in typeset_hash_dict: - input_family = key.split('#', 1)[0] - if input_family in pygments_settings: - if (not pygments_settings_changed[input_family] and - key in old_typeset_hash_dict and - typeset_hash_dict[key] == old_typeset_hash_dict[key]): + family = key.split('#', 1)[0] + if family in pygments_settings: + if (not pygments_settings_changed[family] and + key in old_typeset_hash_dict and + typeset_hash_dict[key] == old_typeset_hash_dict[key] and + not modified_dependencies(key, data, old_data, temp_data)): pygments_update[key] = False if key in old_pygments_macros: pygments_macros[key] = old_pygments_macros[key] if key in old_pygments_files: pygments_files[key] = old_pygments_files[key] + if key in old_dependencies: + dependencies[key] = old_dependencies[key] else: pygments_update[key] = True else: @@ -768,8 +895,8 @@ def hash_all(data, temp_data, old_data, engine_dict): pygments_style_defs = old_data['pygments_style_defs'] else: for key in typeset_hash_dict: - input_family = key.split('#', 1)[0] - if input_family in pygments_settings: + family = key.split('#', 1)[0] + if family in pygments_settings: pygments_update[key] = True else: pygments_update[key] = False @@ -781,7 +908,7 @@ def hash_all(data, temp_data, old_data, engine_dict): for s in pygments_style_list: formatter = LatexFormatter(style=s, commandprefix='PYG'+s) pygments_style_defs[s] = formatter.get_style_defs() - + # Save to data temp_data['code_update'] = code_update temp_data['cons_update'] = cons_update @@ -795,9 +922,9 @@ def hash_all(data, temp_data, old_data, engine_dict): data['typeset_cache'] = typeset_cache data['dependencies'] = dependencies data['exit_status'] = exit_status - - - # Clean up for code that will be run again, and for code that no longer + + + # Clean up for code that will be run again, and for code that no longer # exists. if loaded_old_data: # Take care of code files @@ -827,8 +954,8 @@ def hash_all(data, temp_data, old_data, engine_dict): if os.path.isfile(f): os.remove(f) # Take care of old Pygments files - # The approach here is a little different since there isn't a - # Pygments-specific hash dict, but there is a Pygments-specific + # The approach here is a little different since there isn't a + # Pygments-specific hash dict, but there is a Pygments-specific # dict of lists of files. for key in pygments_update: if pygments_update[key] and key in old_pygments_files: @@ -866,23 +993,74 @@ def parse_code_write_scripts(data, temp_data, engine_dict): cons_update = temp_data['cons_update'] pygments_update = temp_data['pygments_update'] files = data['files'] - # We need to keep track of the last instance for each session, so - # that duplicates can be eliminated. Some LaTeX environments process - # their content multiple times and thus will create duplicates. We + debug = temp_data['debug'] + interactive = temp_data['interactive'] + + # Tweak the update dicts to work with debug command-line option. + # #### This should probably be refactored later, once the debug interface + # stabilizes + if debug is not None or interactive is not None: + if debug is not None: + arg = debug + else: + arg = interactive + for k in cons_update: + cons_update[k] = False + if ':' in arg: + # May need to refine in light of substitution of `:` -> `_` + # in session names? + arg_key = arg.replace(':', '#') + if arg_key not in code_update: + return sys.exit('Session {0} does not exist'.format(arg)) + else: + for k in code_update: + code_update[k] = False + code_update[arg_key] = True + if debug is not None: + temp_data['debug_key'] = arg_key + else: + temp_data['interactive_key'] = arg_key + else: + session_count_dict = defaultdict(list) + for k in code_update: + s = k.split('#')[1] + session_count_dict[s].append(k) + if arg not in session_count_dict: + if arg in cons_update: + return sys.exit('Console sessions are not currently supported for interactive mode.') + else: + return sys.exit('Session "{0}" does not exist.'.format(arg)) + elif len(session_count_dict[arg]) > 1: + return sys.exit('Ambiguous session name "{0}"; please specify ::'.format(arg)) + else: + for k in code_update: + code_update[k] = False + arg_key = session_count_dict[arg][0] + code_update[arg_key] = True + if debug is not None: + temp_data['debug_key'] = arg_key + else: + temp_data['interactive_key'] = arg_key + + + + # We need to keep track of the last instance for each session, so + # that duplicates can be eliminated. Some LaTeX environments process + # their content multiple times and thus will create duplicates. We # need to initialize everything at -1, since instances begin at zero. def negative_one(): return -1 last_instance = defaultdict(negative_one) for c in pytxcode: - if c.input_instance_int > last_instance[c.key_run]: - last_instance[c.key_run] = c.input_instance_int + if c.instance_int > last_instance[c.key_run]: + last_instance[c.key_run] = c.instance_int if c.is_code: if code_update[c.key_run]: code_dict[c.key_run].append(c) if c.is_typeset and pygments_update[c.key_typeset]: pygments_list.append(c) elif c.is_cons: - # Only append to Pygments if not run, since Pygments is + # Only append to Pygments if not run, since Pygments is # automatically taken care of during run for console content if cons_update[c.key_run]: cons_dict[c.key_run].append(c) @@ -895,7 +1073,7 @@ def negative_one(): cc_dict_begin[c.cc_type].append(c) else: cc_dict_end[c.cc_type].append(c) - + # Save temp_data['code_dict'] = code_dict temp_data['cc_dict_begin'] = cc_dict_begin @@ -908,22 +1086,63 @@ def negative_one(): # Also accumulate error indices for handling stderr code_index_dict = {} for key in code_dict: - input_family, input_session, input_restart = key.split('#') - fname = os.path.join(outputdir, input_family + '_' + input_session + '_' + input_restart + '.' + engine_dict[input_family].extension) + family, session, restart = key.split('#') + fname = os.path.join(outputdir, family + '_' + session + '_' + restart + '.' + engine_dict[family].extension) + # Want to keep track of files without expanding user, but need to + # expand user when actually writing files files[key].append(fname) - sessionfile = open(fname, 'w', encoding=encoding) - script, code_index = engine_dict[input_family].get_script(encoding, - utilspath, - workingdir, - cc_dict_begin[input_family], - code_dict[key], - cc_dict_end[input_family]) + sessionfile = open(os.path.expanduser(os.path.normcase(fname)), 'w', encoding=encoding) + script, code_index = engine_dict[family].get_script(encoding, + utilspath, + outputdir, + workingdir, + cc_dict_begin[family], + code_dict[key], + cc_dict_end[family], + debug, + interactive) for lines in script: sessionfile.write(lines) sessionfile.close() code_index_dict[key] = code_index temp_data['code_index_dict'] = code_index_dict + # Write synchronization file if in debug mode + if debug is not None: + # Might improve tracking/cleanup of syncdb files + key = temp_data['debug_key'] + family, session, restart = key.split('#') + basename = key.replace('#', '_') + syncdb_fname = os.path.join(outputdir, basename + '.' + engine_dict[family].extension + '.syncdb') + files[key].append(syncdb_fname) + # #### In future version, try to use currfile to get this information + # automatically via the .pytxcode + main_doc_fname = None + for ext in ('.tex', '.ltx', '.dtx'): + if os.path.isfile(data['raw_jobname'] + ext): + main_doc_fname = data['raw_jobname'] + ext + break + if not main_doc_fname: + return sys.exit('Could not determine extension for main file "{0}"'.format(data['raw_jobname'])) + main_code_fname = basename + '.' + engine_dict[family].extension + f = open(os.path.expanduser(os.path.normcase(syncdb_fname)), 'w', encoding='utf8') + f.write('{0},,{1},,\n'.format(main_code_fname, main_doc_fname)) + # All paths are relative to the main code file. So if there is ever + # an option for creating other code files, in other locations, then + # the relative paths to those files will need to be specified. + for e in code_index_dict[key].values(): + # #### Probably redo approach so this conversion isn't needed + if not e.input_file: + input_file = main_doc_fname + else: + input_file = e.input_file + if ',' in input_file or ',' in main_code_fname: + line = '"{0}",{1},"{2}",{3},{4}\n'.format(main_code_fname, e.lines_total+1, input_file, e.line_int, e.lines_input) + else: + line = '{0},{1},{2},{3},{4}\n'.format(main_code_fname, e.lines_total+1, input_file, e.line_int, e.lines_input) + f.write(line) + f.close() + @@ -935,8 +1154,9 @@ def do_multiprocessing(data, temp_data, old_data, engine_dict): keeptemps = data['settings']['keeptemps'] fvextfile = data['settings']['fvextfile'] pygments_settings = data['pygments_settings'] + jobs = temp_data['jobs'] verbose = temp_data['verbose'] - + code_dict = temp_data['code_dict'] cons_dict = temp_data['cons_dict'] cc_dict_begin = temp_data['cc_dict_begin'] @@ -949,123 +1169,218 @@ def do_multiprocessing(data, temp_data, old_data, engine_dict): pygments_files = data['pygments_files'] pygments_macros = data['pygments_macros'] typeset_cache = data['typeset_cache'] - + errors = temp_data['errors'] warnings = temp_data['warnings'] - + makestderr = data['settings']['makestderr'] stderrfilename = data['settings']['stderrfilename'] code_index_dict = temp_data['code_index_dict'] - + hashdependencies = temp_data['hashdependencies'] dependencies = data['dependencies'] exit_status = data['exit_status'] start_time = data['start_time'] - - - # Set maximum number of concurrent processes for multiprocessing - # Accoding to the docs, cpu_count() may raise an error - try: - max_processes = multiprocessing.cpu_count() - except NotImplementedError: - max_processes = 1 - pool = multiprocessing.Pool(max_processes) + debug = temp_data['debug'] + interactive = temp_data['interactive'] + + # If in debug or interactive mode, short-circuit the whole process + # #### This should probably be refactored later, once debugging is more + # mature + if debug is not None or interactive is not None: + import shlex + if debug is not None: + print('Entering debug mode for "{0}"\n'.format(debug) + '-'*20 + '\n') + key = temp_data['debug_key'] + else: + print('Entering interactive mode for "{0}"\n'.format(interactive) + '-'*20 + '\n') + key = temp_data['interactive_key'] + basename = key.replace('#', '_') + family, session, restart = key.split('#') + # #### Revise as debugging is expanded + if debug is not None and engine_dict[family].language != 'python': + return sys.exit('Currently, debug only supports Python') + if debug is not None: + # #### Eventually, should move to pythontex_engines.py and + # provide means for customization + command = '{python} {debug} {file}.py --interactive' + command = command.replace('{python}', interpreter_dict['python']) + command = command.replace('{debug}', '"{0}"'.format(os.path.join(sys.path[0], 'syncpdb.py'))) + else: + command = engine_dict[family].command + ' --interactive' + # Need to be in script directory so that pdb and any other tools that + # expect this will function correctly. + orig_cwd = os.getcwd() + if outputdir: + os.chdir(os.path.expanduser(os.path.normcase(outputdir))) + # Note that command is a string, which must be converted to list + # Must double-escape any backslashes so that they survive `shlex.split()` + script = basename + if os.path.isabs(os.path.expanduser(os.path.normcase(outputdir))): + script_full = os.path.expanduser(os.path.join(outputdir, basename)) + else: + script_full = os.path.expanduser(os.path.join(orig_cwd, outputdir, basename)) + if platform.system() == 'Windows': + script_full = script_full.replace('/', '\\') + else: + script_full = script_full.replace('\\', '/') + # `shlex.split()` only works with Unicode after 2.7.2 + if (sys.version_info.major == 2 and sys.version_info.micro < 3): + exec_cmd = shlex.split(bytes(command.format(file=script.replace('\\', '\\\\'), File=script_full.replace('\\', '\\\\')))) + exec_cmd = [unicode(elem) for elem in exec_cmd] + else: + exec_cmd = shlex.split(command.format(file=script.replace('\\', '\\\\'), File=script_full.replace('\\', '\\\\'))) + try: + proc = subprocess.Popen(exec_cmd) + except (OSError, FileNotFoundError) as e: + if platform.system() == 'Windows' and e.errno == 2: + # Batch files won't be found when called without extension. They + # would be found if `shell=True`, but then getting the right + # exit code is tricky. So we perform some `cmd` trickery that + # is essentially equivalent to `shell=True`, but gives correct + # exit codes. Note that `subprocess.Popen()` works with strings + # under Windows; a list is not required. + exec_cmd_string = ' '.join(exec_cmd) + exec_cmd_string = 'cmd /C "@echo off & call {0} & if errorlevel 1 exit 1"'.format(exec_cmd_string) + proc = subprocess.Popen(exec_cmd_string) + else: + raise + proc.wait() + os.chdir(orig_cwd) + # Do a basic update of pickled data + # This is only really needed for tracking the code file and the + # synchronization file (if it was created) + if temp_data['loaded_old_data'] and key in old_data['exit_status']: + exit_status[key] = old_data['exit_status'][key] + else: + exit_status[key] = (None, None) + if temp_data['loaded_old_data']: + data['last_new_file_time'] = old_data['last_new_file_time'] + else: + data['last_new_file_time'] = start_time + pythontex_data_file = os.path.expanduser(os.path.normcase(os.path.join(outputdir, 'pythontex_data.pkl'))) + f = open(pythontex_data_file, 'wb') + pickle.dump(data, f, -1) + f.close() + return + + + # Create a pool for multiprocessing. Set the maximum number of + # concurrent processes to a user-specified value for jobs. If the user + # has not specified a value, then it will be None, and + # multiprocessing.Pool() will use cpu_count(). + pool = multiprocessing.Pool(jobs) tasks = [] - + # If verbose, print a list of processes if verbose: - print('\n* PythonTeX will run the following processes:') - - # Add code processes. Note that everything placed in the codedict + print('\n* PythonTeX will run the following processes') + print(' with working directory {0}'.format(workingdir)) + print(' (maximum concurrent processes = {0})'.format(jobs)) + + # Add code processes. Note that everything placed in the codedict # needs to be executed, based on previous testing, except for custom code. for key in code_dict: - input_family = key.split('#')[0] + family = key.split('#')[0] # Uncomment the following for debugging, and comment out what follows - '''run_code(encoding, outputdir, workingdir, code_dict[key], - engine_dict[input_family].language, - engine_dict[input_family].command, - engine_dict[input_family].created, - engine_dict[input_family].extension, + '''run_code(encoding, outputdir, + workingdir, + cc_dict_begin[family], + code_dict[key], + cc_dict_end[family], + engine_dict[family].language, + engine_dict[family].commands, + engine_dict[family].created, + engine_dict[family].extension, makestderr, stderrfilename, code_index_dict[key], - engine_dict[input_family].errors, - engine_dict[input_family].warnings, - engine_dict[input_family].linenumbers, - engine_dict[input_family].lookbehind, - keeptemps, hashdependencies)''' - tasks.append(pool.apply_async(run_code, [encoding, outputdir, - workingdir, code_dict[key], - engine_dict[input_family].language, - engine_dict[input_family].command, - engine_dict[input_family].created, - engine_dict[input_family].extension, + engine_dict[family].errors, + engine_dict[family].warnings, + engine_dict[family].linenumbers, + engine_dict[family].lookbehind, + keeptemps, hashdependencies, + pygments_settings]))''' + tasks.append(pool.apply_async(run_code, [encoding, outputdir, + workingdir, + cc_dict_begin[family], + code_dict[key], + cc_dict_end[family], + engine_dict[family].language, + engine_dict[family].commands, + engine_dict[family].created, + engine_dict[family].extension, makestderr, stderrfilename, code_index_dict[key], - engine_dict[input_family].errors, - engine_dict[input_family].warnings, - engine_dict[input_family].linenumbers, - engine_dict[input_family].lookbehind, - keeptemps, hashdependencies])) + engine_dict[family].errors, + engine_dict[family].warnings, + engine_dict[family].linenumbers, + engine_dict[family].lookbehind, + keeptemps, hashdependencies, + pygments_settings])) if verbose: print(' - Code process ' + key.replace('#', ':')) - + # Add console processes for key in cons_dict: - input_family = key.split('#')[0] - if engine_dict[input_family].language.startswith('python'): - if input_family in pygments_settings: + family = key.split('#')[0] + if engine_dict[family].language.startswith('python'): + if family in pygments_settings: # Uncomment the following for debugging - '''python_console(jobname, encoding, outputdir, workingdir, - fvextfile, pygments_settings[input_family], - cc_dict_begin[input_family], cons_dict[key], - cc_dict_end[input_family], engine_dict[input_family].startup, - engine_dict[input_family].banner, - engine_dict[input_family].filename)''' + '''python_console(jobname, encoding, outputdir, workingdir, + fvextfile, pygments_settings[family], + cc_dict_begin[family], cons_dict[key], + cc_dict_end[family], engine_dict[family].startup, + engine_dict[family].banner, + engine_dict[family].filename)''' tasks.append(pool.apply_async(python_console, [jobname, encoding, outputdir, workingdir, fvextfile, - pygments_settings[input_family], - cc_dict_begin[input_family], + pygments_settings[family], + cc_dict_begin[family], cons_dict[key], - cc_dict_end[input_family], - engine_dict[input_family].startup, - engine_dict[input_family].banner, - engine_dict[input_family].filename])) + cc_dict_end[family], + engine_dict[family].startup, + engine_dict[family].banner, + engine_dict[family].filename])) else: tasks.append(pool.apply_async(python_console, [jobname, encoding, outputdir, workingdir, fvextfile, None, - cc_dict_begin[input_family], + cc_dict_begin[family], cons_dict[key], - cc_dict_end[input_family], - engine_dict[input_family].startup, - engine_dict[input_family].banner, - engine_dict[input_family].filename])) + cc_dict_end[family], + engine_dict[family].startup, + engine_dict[family].banner, + engine_dict[family].filename])) else: print('* PythonTeX error') print(' Currently, non-Python consoles are not supported') errors += 1 if verbose: print(' - Console process ' + key.replace('#', ':')) - + # Add a Pygments process if pygments_list: - tasks.append(pool.apply_async(do_pygments, [encoding, outputdir, + # Uncomment the following for debugging + # do_pygments(encoding, outputdir, fvextfile, pygments_list, + # pygments_settings, typeset_cache, hashdependencies) + tasks.append(pool.apply_async(do_pygments, [encoding, outputdir, fvextfile, pygments_list, pygments_settings, - typeset_cache])) + typeset_cache, + hashdependencies])) if verbose: print(' - Pygments process') - + # Execute the processes pool.close() pool.join() - + # Get the outputs of processes # Get the files and macros created. Get the number of errors and warnings - # produced. Get any messages returned. Get the exit_status, which is a + # produced. Get any messages returned. Get the exit_status, which is a # dictionary of code that failed and thus must be run again (its hash is # set to a null string). Keep track of whether there were any new files, # so that the last time of file creation in .pytxmcr can be updated. @@ -1083,7 +1398,7 @@ def do_multiprocessing(data, temp_data, old_data, engine_dict): errors += result['errors'] warnings += result['warnings'] exit_status[key] = (result['errors'], result['warnings']) - messages.extend(result['messages']) + messages.extend(result['messages']) elif result['process'] == 'console': key = result['key'] files[key].extend(result['files']) @@ -1104,19 +1419,21 @@ def do_multiprocessing(data, temp_data, old_data, engine_dict): if result['pygments_files'][k]: new_files = True break + for k, v in result['dependencies'].items(): + dependencies[k] = v pygments_macros.update(result['pygments_macros']) errors += result['errors'] warnings += result['warnings'] - messages.extend(result['messages']) - + messages.extend(result['messages']) + # Do a quick check to see if any dependencies were modified since the # beginning of the run. If so, reset them so they will run next time and # issue a warning unresolved_dependencies = False - unresolved_sessions= [] + unresolved_sessions = [] for key in dependencies: for dep, val in dependencies[key].items(): - if val[0] > start_time: + if val[0] is None or val[0] > start_time: unresolved_dependencies = True dependencies[key][dep] = (None, None) unresolved_sessions.append(key.replace('#', ':')) @@ -1127,11 +1444,11 @@ def do_multiprocessing(data, temp_data, old_data, engine_dict): for s in set(unresolved_sessions): print(' - ' + s) warnings += 1 - - + + # Save all content (only needs to be done if code was indeed run). # Save a commented-out time corresponding to the last time PythonTeX ran - # and created files, so that tools like latexmk can easily detect when + # and created files, so that tools like latexmk can easily detect when # another run is needed. if tasks: if new_files or not temp_data['loaded_old_data']: @@ -1139,28 +1456,33 @@ def do_multiprocessing(data, temp_data, old_data, engine_dict): else: last_new_file_time = old_data['last_new_file_time'] data['last_new_file_time'] = last_new_file_time - - macro_file = open(os.path.join(outputdir, jobname + '.pytxmcr'), 'w', encoding=encoding) + + macro_file = open(os.path.expanduser(os.path.normcase(os.path.join(outputdir, jobname + '.pytxmcr'))), 'w', encoding=encoding) macro_file.write('%Last time of file creation: ' + str(last_new_file_time) + '\n\n') for key in macros: macro_file.write(''.join(macros[key])) macro_file.close() - - pygments_macro_file = open(os.path.join(outputdir, jobname + '.pytxpyg'), 'w', encoding=encoding) + + pygments_macro_file = open(os.path.expanduser(os.path.normcase(os.path.join(outputdir, jobname + '.pytxpyg'))), 'w', encoding=encoding) # Only save Pygments styles that are used style_set = set([pygments_settings[k]['formatter_options']['style'] for k in pygments_settings if k != ':GLOBAL']) + if style_set: + from pygments.formatters import LatexFormatter + formatter = LatexFormatter(style='default', commandprefix='PYG') + PYG_style_defs = formatter.get_style_defs() + pygments_macro_file.write(PYG_style_defs) for key in pygments_style_defs: if key in style_set: pygments_macro_file.write(''.join(pygments_style_defs[key])) for key in pygments_macros: pygments_macro_file.write(''.join(pygments_macros[key])) pygments_macro_file.close() - - pythontex_data_file = os.path.join(outputdir, 'pythontex_data.pkl') + + pythontex_data_file = os.path.expanduser(os.path.normcase(os.path.join(outputdir, 'pythontex_data.pkl'))) f = open(pythontex_data_file, 'wb') pickle.dump(data, f, -1) f.close() - + # Print any errors and warnings. if messages: print('\n'.join(messages)) @@ -1173,18 +1495,19 @@ def do_multiprocessing(data, temp_data, old_data, engine_dict): -def run_code(encoding, outputdir, workingdir, code_list, language, command, - command_created, extension, makestderr, stderrfilename, - code_index, errorsig, warningsig, linesig, stderrlookbehind, - keeptemps, hashdependencies): +def run_code(encoding, outputdir, workingdir, + cc_begin_list, code_list, cc_end_list, language, commands, + command_created, extension, makestderr, stderrfilename, + code_index, errorsig, warningsig, linesig, stderrlookbehind, + keeptemps, hashdependencies, pygments_settings): ''' Function for multiprocessing code files ''' import shlex - + # Create what's needed for storing results - input_family = code_list[0].input_family - input_session = code_list[0].input_session + family = code_list[0].family + session = code_list[0].session key_run = code_list[0].key_run files = [] macros = [] @@ -1193,60 +1516,85 @@ def run_code(encoding, outputdir, workingdir, code_list, language, command, warnings = 0 unknowns = 0 messages = [] - - # Create message lists only for stderr, one for undelimited stderr and - # one for delimited, so it's easy to keep track of if there is any + + # Create message lists only for stderr, one for undelimited stderr and + # one for delimited, so it's easy to keep track of if there is any # stderr. These are added onto messages at the end. err_messages_ud = [] err_messages_d = [] - + # We need to let the user know we are switching code files # We check at the end to see if there were indeed any errors and warnings # and if not, clear messages. messages.append('\n---- Messages for ' + key_run.replace('#', ':') + ' ----') - + # Open files for stdout and stderr, run the code, then close the files basename = key_run.replace('#', '_') - out_file_name = os.path.join(outputdir, basename + '.out') - err_file_name = os.path.join(outputdir, basename + '.err') + out_file_name = os.path.expanduser(os.path.normcase(os.path.join(outputdir, basename + '.out'))) + err_file_name = os.path.expanduser(os.path.normcase(os.path.join(outputdir, basename + '.err'))) out_file = open(out_file_name, 'w', encoding=encoding) err_file = open(err_file_name, 'w', encoding=encoding) - # Note that command is a string, which must be converted to list - # Must double-escape any backslashes so that they survive `shlex.split()` - script = os.path.join(outputdir, basename) - # `shlex.split()` only works with Unicode after 2.7.2 - if (sys.version_info.major == 2 and sys.version_info.micro < 3): - exec_cmd = shlex.split(bytes(command.format(file=script.replace('\\', '\\\\')))) - exec_cmd = [unicode(elem) for elem in exec_cmd] + script = os.path.expanduser(os.path.join(outputdir, basename)) + if platform.system() == 'Windows': + script = script.replace('/', '\\') + else: + script = script.replace('\\', '/') + if os.path.isabs(script): + script_full = script else: - exec_cmd = shlex.split(command.format(file=script.replace('\\', '\\\\'))) - # Add any created files due to the command - # This needs to be done before attempts to execute, to prevent orphans + script_full = os.path.expanduser(os.path.join(os.getcwd(), outputdir, basename)) + if platform.system() == 'Windows': + script_full = script_full.replace('/', '\\') + else: + script_full = script_full.replace('\\', '/') + # #### Need to revise so that intermediate files can be detected and cleaned up for f in command_created: - files.append(f.format(file=script)) - try: - proc = subprocess.Popen(exec_cmd, stdout=out_file, stderr=err_file) - except WindowsError as e: - if e.errno == 2: - # Batch files won't be found when called without extension. They - # would be found if `shell=True`, but then getting the right - # exit code is tricky. So we perform some `cmd` trickery that - # is essentially equivalent to `shell=True`, but gives correct - # exit codes. Note that `subprocess.Popen()` works with strings - # under Windows; a list is not required. - exec_cmd_string = ' '.join(exec_cmd) - exec_cmd_string = 'cmd /C "@echo off & call {0} & if errorlevel 1 exit 1"'.format(exec_cmd_string) - proc = subprocess.Popen(exec_cmd_string, stdout=out_file, stderr=err_file) + files.append(f.format(file=script, File=script_full)) + for command in commands: + # Note that command is a string, which must be converted to list + # Must double-escape any backslashes so that they survive `shlex.split()` + # `shlex.split()` only works with Unicode after 2.7.2 + if (sys.version_info.major == 2 and sys.version_info.micro < 3): + exec_cmd = shlex.split(bytes(command.format(file=script.replace('\\', '\\\\'), File=script_full.replace('\\', '\\\\'), workingdir=workingdir.replace('\\', '\\\\')))) + exec_cmd = [unicode(elem) for elem in exec_cmd] else: - raise - - proc.wait() + if family != 'juliacon': + exec_cmd = shlex.split(command.format(file=script.replace('\\', '\\\\'), File=script_full.replace('\\', '\\\\'), workingdir=workingdir.replace('\\', '\\\\'))) + else: + exec_cmd = shlex.split(command.format(file=script.replace('\\', '/'), File=script_full.replace('\\', '/'), workingdir=workingdir.replace('\\', '/'))) + # Add any created files due to the command + # This needs to be done before attempts to execute, to prevent orphans + try: + if family != 'Rcon': + proc = subprocess.Popen(exec_cmd, stdout=out_file, stderr=err_file) + else: + proc = subprocess.Popen(exec_cmd, stdout=out_file, stderr=subprocess.STDOUT) + except OSError as e: + if e.errno == 2: + # Batch files won't be found when called without extension. They + # would be found if `shell=True`, but then getting the right + # exit code is tricky. So we perform some `cmd` trickery that + # is essentially equivalent to `shell=True`, but gives correct + # exit codes. Note that `subprocess.Popen()` works with strings + # under Windows; a list is not required. + exec_cmd_string = ' '.join(exec_cmd) + exec_cmd_string = 'cmd /C "@echo off & call {0} & if errorlevel 1 exit 1"'.format(exec_cmd_string) + if family != 'Rcon': + proc = subprocess.Popen(exec_cmd_string, stdout=out_file, stderr=err_file) + else: + proc = subprocess.Popen(exec_cmd_string, stdout=out_file, stderr=subprocess.STDOUT) + else: + raise + + proc.wait() + if proc.returncode != 0: + break out_file.close() err_file.close() - + # Process saved stdout into file(s) that are included in the TeX document. # - # Go through the saved output line by line, and save any printed content + # Go through the saved output line by line, and save any printed content # to its own file, named based on instance. # # The very end of the stdout lists dependencies, if any, so we start by @@ -1255,7 +1603,52 @@ def run_code(encoding, outputdir, workingdir, code_list, language, command, messages.append('* PythonTeX error') messages.append(' Missing output file for ' + key_run.replace('#', ':')) errors += 1 + elif family == 'juliacon' and proc.returncode == 1: + messages.append('* PythonTeX error') + messages.append(' Running code for Julia console failed') + with open(err_file_name, encoding='utf8') as f: + messages.append(f.read()) + errors += 1 else: + if family == 'juliacon': + with open(out_file_name.rsplit('.', 1)[0] + '.tex', 'r', encoding=encoding) as f: + tex_data_lines = f.readlines() + code_iter = itertools.chain(cc_begin_list, code_list, cc_end_list) + for n, line in enumerate(tex_data_lines): + if line.rstrip() == '\\begin{juliaterm}': + c = next(code_iter) + tex_data_lines[n] = '=>PYTHONTEX:STDOUT#{0}#code#\n'.format(c.instance) + if n != 0: + tex_data_lines[n-1] = '' + if line.rstrip() == '\\end{juliaterm}': + tex_data_lines[n] = '' + tex_data_lines.append('=>PYTHONTEX:DEPENDENCIES#\n=>PYTHONTEX:CREATED#\n') + with open(out_file_name, 'w', encoding=encoding) as f: + f.write(''.join(tex_data_lines)) + elif family == 'Rcon': + with open(out_file_name, 'r', encoding=encoding) as f: + stdout_lines = f.readlines() + for n, line in enumerate(stdout_lines): + if line.startswith('> =>PYTHONTEX:'): + stdout_lines[n] = line[2:] + elif '> write("=>PYTHONTEX:' in line: + if line.startswith('> write("=>PYTHONTEX:'): + stdout_lines[n] = '' + else: + # cat() and similar functions can result in the + # prompt not being at the start of a new line. In + # that case, preserve the prompt to accurately + # emulate the console. If there is a following + # console environment, this effectively amounts + # to adding an extra empty line (pressing ENTER) + # between the two. + stdout_lines[n] = line.split('write("=>PYTHONTEX:', 1)[0] + while stdout_lines and (stdout_lines[-1].startswith('>') and not stdout_lines[-1][1:].strip(' \n')): + stdout_lines.pop() + stdout_lines.append('=>PYTHONTEX:DEPENDENCIES#\n=>PYTHONTEX:CREATED#\n') + with open(out_file_name, 'w', encoding=encoding) as f: + f.write(''.join(stdout_lines)) + f = open(out_file_name, 'r', encoding=encoding) out = f.read() f.close() @@ -1267,39 +1660,42 @@ def run_code(encoding, outputdir, workingdir, code_list, language, command, valid_stdout = False if proc.returncode == 0: raise ValueError('Missing "created" and/or "dependencies" delims in stdout; invalid template?') - + if valid_stdout: # Add created files to created list for c in created.splitlines(): - files.append(c) - + if os.path.isabs(os.path.expanduser(os.path.normcase(c))): + files.append(c) + else: + files.append(os.path.join(workingdir, c)) + # Create a set of dependencies, to eliminate duplicates in the event # that there are any. This is mainly useful when dependencies are - # automatically determined (for example, through redefining open()), - # may be specified multiple times as a result, and are hashed (and + # automatically determined (for example, through redefining open()), + # may be specified multiple times as a result, and are hashed (and # of a large enough size that hashing time is non-negligible). deps = set([dep for dep in deps.splitlines()]) # Process dependencies; get mtimes and (if specified) hashes for dep in deps: dep_file = os.path.expanduser(os.path.normcase(dep)) if not os.path.isabs(dep_file): - dep_file = os.path.join(workingdir, dep_file) + dep_file = os.path.expanduser(os.path.normcase(os.path.join(workingdir, dep_file))) if not os.path.isfile(dep_file): - # If we can't find the file, we return a null hash and issue - # an error. We don't need to change the exit status. If the - # code does depend on the file, there will be a separate - # error when the code attempts to use the file. If the code - # doesn't really depend on the file, then the error will be - # raised again anyway the next time PythonTeX runs when the + # If we can't find the file, we return a null hash and issue + # an error. We don't need to change the exit status. If the + # code does depend on the file, there will be a separate + # error when the code attempts to use the file. If the code + # doesn't really depend on the file, then the error will be + # raised again anyway the next time PythonTeX runs when the # dependency is listed but not found. dependencies[dep] = (None, None) messages.append('* PythonTeX error') messages.append(' Cannot find dependency "' + dep + '"') messages.append(' It belongs to ' + key_run.replace('#', ':')) messages.append(' Relative paths to dependencies must be specified from the working directory.') - errors += 1 + errors += 1 elif hashdependencies: - # Read and hash the file in binary. Opening in text mode + # Read and hash the file in binary. Opening in text mode # would require an unnecessary decoding and encoding cycle. hasher = sha1() f = open(dep_file, 'rb') @@ -1308,26 +1704,56 @@ def run_code(encoding, outputdir, workingdir, code_list, language, command, dependencies[dep] = (os.path.getmtime(dep_file), hasher.hexdigest()) else: dependencies[dep] = (os.path.getmtime(dep_file), '') - + + if family in ('juliacon', 'Rcon'): + from pygments import highlight + from pygments.lexers import get_lexer_by_name + from pygments.formatters import LatexFormatter + formatter = dict() + lexer = dict() + for codetype in pygments_settings: + if codetype != ':GLOBAL': + p = pygments_settings[codetype]['formatter_options'].copy() + p['commandprefix'] = 'PYG' + formatter[codetype] = LatexFormatter(**p) + lexer[codetype] = get_lexer_by_name(pygments_settings[codetype]['lexer'], **p) for block in out.split('=>PYTHONTEX:STDOUT#')[1:]: if block: delims, content = block.split('#\n', 1) - if content: - input_instance, input_command = delims.split('#') - if input_instance.endswith('CC'): - messages.append('* PythonTeX warning') - messages.append(' Custom code for "' + input_family + '" attempted to print or write to stdout') - messages.append(' This is not supported; use a normal code command or environment') - messages.append(' The following content was written:') - messages.append('') - messages.extend([' ' + l for l in content.splitlines()]) - warnings += 1 - elif input_command == 'i': - content = r'\pytx@SVMCR{pytx@MCR@' + key_run.replace('#', '@') + '@' + input_instance + '}\n' + content.rstrip('\n') + '\\endpytx@SVMCR\n\n' + if content and not content.endswith('\n'): + # Content might not end with a newline. For example, + # Rcon with something like cat() as the last function. + content += '\n' + instance, command = delims.split('#') + if content or command in ('s', 'sub'): + if instance.endswith('CC'): + if family not in ('juliacon', 'Rcon'): + messages.append('* PythonTeX warning') + messages.append(' Custom code for "' + family + '" attempted to print or write to stdout') + messages.append(' This is not supported; use a normal code command or environment') + messages.append(' The following content was written:') + messages.append('') + messages.extend([' ' + l for l in content.splitlines()]) + warnings += 1 + elif command == 'i': + content = r'\pytx@SVMCR{pytx@MCR@' + key_run.replace('#', '@') + '@' + instance + '}\n' + content.rstrip('\n') + '\\endpytx@SVMCR\n\n' macros.append(content) else: - fname = os.path.join(outputdir, basename + '_' + input_instance + '.stdout') - f = open(fname, 'w', encoding=encoding) + fname = os.path.join(outputdir, basename + '_' + instance + '.stdout') + f = open(os.path.expanduser(os.path.normcase(fname)), 'w', encoding=encoding) + if command in ('s', 'sub'): + if content: + fields = [x.split('\n', 1)[1].rsplit('\n', 1)[0] for x in content.split('=>PYTHONTEX:FIELD_DELIM#')[1:]] + content = code_list[int(instance)].sub_template.format(*fields) + else: + # If no replacement fields, de-templatize + content = code_list[int(instance)].sub_template.replace('{{', '{').replace('}}', '}') + if command == 's': + # Remove newline added by printing, prevent + # LaTeX from adding a space after content + content = content.rsplit('\n', 1)[0] + '\\endinput\n' + if family in ('juliacon', 'Rcon'): + content = highlight(content, lexer[family], formatter[family]) f.write(content) f.close() files.append(fname) @@ -1337,6 +1763,8 @@ def run_code(encoding, outputdir, workingdir, code_list, language, command, messages.append('* PythonTeX error') messages.append(' Missing stderr file for ' + key_run.replace('#', ':')) errors += 1 + elif family in ('juliacon', 'Rcon'): + pass else: # Open error and code files. f = open(err_file_name, encoding=encoding) @@ -1358,23 +1786,25 @@ def run_code(encoding, outputdir, workingdir, code_list, language, command, # Create the full basename that will be replaced in stderr # We need two versions, one with the correct slashes for the OS, # and one with the opposite slashes. This is needed when a language - # doesn't obey the OS's slash convention in paths given in stderr. - # For example, Windows uses backslashes, but Ruby under Windows uses + # doesn't obey the OS's slash convention in paths given in stderr. + # For example, Windows uses backslashes, but Ruby under Windows uses # forward in paths given in stderr. - fullbasename_correct = os.path.join(outputdir, basename) + # #### Consider os.path.normcase(), making search case-insensitive + outputdir_exp = os.path.expanduser(outputdir) + fullbasename_correct = os.path.join(outputdir_exp, basename) if '\\' in fullbasename_correct: fullbasename_reslashed = fullbasename_correct.replace('\\', '/') else: fullbasename_reslashed = fullbasename_correct.replace('/', '\\') - + if err_ud: it = iter(code_index.items()) index_now = next(it) index_next = index_now start_errgobble = None for n, line in enumerate(err_ud): - if basename in line: - # Get the gobbleation. This is used to determine if + if basename in line and (family not in ('perlsix', 'psix') or '.p6:' in line or '.p6 line' in line): + # Get the gobbleation. This is used to determine if # other lines containing the basename are a continuation, # or separate messages. errgobble = match('(\s*)', line).groups()[0] @@ -1382,7 +1812,7 @@ def run_code(encoding, outputdir, workingdir, code_list, language, command, start_errgobble = errgobble # Only issue a message and track down the line numer if # this is indeed the start of a new message, rather than - # a continuation of an old message that happens to + # a continuation of an old message that happens to # contain the basename if errgobble == start_errgobble: # Determine the corresponding line number in the document @@ -1401,25 +1831,25 @@ def run_code(encoding, outputdir, workingdir, code_list, language, command, except: break if errlinenum > index_now[1].lines_total + index_now[1].lines_input: - doclinenum = str(index_now[1].input_line_int + index_now[1].lines_input) + doclinenum = str(index_now[1].line_int + index_now[1].lines_input) else: - doclinenum = str(index_now[1].input_line_int + errlinenum - index_now[1].lines_total - 1) + doclinenum = str(index_now[1].line_int + errlinenum - index_now[1].lines_total - 1) input_file = index_now[1].input_file else: doclinenum = '??' input_file = '??' - - # Try to determine if we are dealing with an error or a + + # Try to determine if we are dealing with an error or a # warning. found = False index = n if stderrlookbehind: while index >= 0: - # The order here is important. If a line matches - # both the error and warning patterns, default to + # The order here is important. If a line matches + # both the error and warning patterns, default to # error. past_line = err_ud[index] - if (index < n and basename in past_line): + if (index < n and basename in past_line and (family not in ('perlsix', 'psix') or '.p6:' in past_line or '.p6 line' in past_line)): break for pattern in warningsig: if pattern in past_line: @@ -1438,11 +1868,11 @@ def run_code(encoding, outputdir, workingdir, code_list, language, command, index -= 1 else: while index < len(err_ud): - # The order here is important. If a line matches - # both the error and warning patterns, default to + # The order here is important. If a line matches + # both the error and warning patterns, default to # error. future_line = err_ud[index] - if (index > n and basename in future_line and + if (index > n and basename in future_line and future_line.startswith(start_errgobble)): break for pattern in warningsig: @@ -1469,10 +1899,10 @@ def run_code(encoding, outputdir, workingdir, code_list, language, command, err_messages_ud.append('* PythonTeX stderr - {0} on line {1} in "{2}":'.format(alert_type, doclinenum, input_file)) else: err_messages_ud.append('* PythonTeX stderr - {0} on line {1}:'.format(alert_type, doclinenum)) - err_messages_ud.append(' ' + line.replace(outputdir, '').rstrip('\n')) + err_messages_ud.append(' ' + line.replace(outputdir_exp, '').rstrip('\n')) else: err_messages_ud.append(' ' + line.rstrip('\n')) - + # Create .stderr if makestderr and err_messages_ud: process = False @@ -1483,8 +1913,9 @@ def run_code(encoding, outputdir, workingdir, code_list, language, command, index_now_last = index_now index_next_last = index_next err_key_last_int = -1 + p6_sorry_search = False for n, line in enumerate(err_ud): - if basename in line: + if basename in line and (family not in ('perlsix', 'psix') or '.p6:' in line or '.p6 line' in line): # Determine the corresponding line number in the document found = False for pattern in linesig: @@ -1513,7 +1944,7 @@ def run_code(encoding, outputdir, workingdir, code_list, language, command, process = False else: process = True - if len(index_now[1].input_command) > 1: + if len(index_now[1].command) > 1: if errlinenum > index_now[1].lines_total + index_now[1].lines_input: codelinenum = str(index_now[1].lines_user + index_now[1].lines_input + 1) else: @@ -1521,13 +1952,14 @@ def run_code(encoding, outputdir, workingdir, code_list, language, command, else: codelinenum = '1' else: + errlinenum = '??' codelinenum = '??' messages.append('* PythonTeX error') messages.append(' Line number ' + str(errlinenum) + ' could not be synced with the document') messages.append(' Content from stderr is not delimited, and cannot be resolved') errors += 1 process = False - + if process: if int(index_now[0]) > err_key_last_int: err_key = basename + '_' + index_now[0] @@ -1540,15 +1972,44 @@ def run_code(encoding, outputdir, workingdir, code_list, language, command, if stderrfilename == 'full': line = line.replace(fullbasename, basename) elif stderrfilename == 'session': - line = line.replace(fullbasename, input_session) + line = line.replace(fullbasename, session) elif stderrfilename == 'genericfile': line = line.replace(fullbasename + '.' + extension, '') elif stderrfilename == 'genericscript': line = line.replace(fullbasename + '.' + extension, ' + + - -

PythonTeX allows you to run Python code from within LaTeX documents and automatically include the output. This document serves as an example of what is possible with PythonTeX.1

-

General Python interaction

+

Geoffrey M. Poore

+ +

General Python interaction

We can typeset code that is passed to Python, and bring back the results.

This can be simple. For example, print('Python says hi!') returns the following:

@@ -44,181 +226,184 @@

General Python interaction

Python says hi!

-

Python interaction can also be more complex. print(str(2**2**2) + r'\endinput') returns 16. In this case, the printed results include LaTeX code, which is correctly interpreted by LaTeX to ensure that there is not an extra space after the 16. Printed output is saved to a file and brought back in via \input, and the \endinput command stops input immediately, before LaTeX gets to the end of the line and inserts a space character there, after the 16.

+

Python interaction can also be more complex. print(str(2**2**2) + r'\endinput') returns 16. In this case, the printed result includes LaTeX code, which is correctly interpreted by LaTeX to ensure that there is not an extra space after the 16. Printed output is saved to a file and brought back in via \input, and the \endinput command stops input immediately, before LaTeX gets to the end of the line and inserts a space character there, after the 16.

Printing works, but as the last example demonstrates, you have to be careful about spacing if you have text immediately after the printed content. In that case, it’s usually best to assemble text within a PythonTeX environment and store the text in a variable. Then you can bring in the text later, using the \py command. The \py command brings in a string representation of its argument. First we create the text.

-
mytext = '$1 + 1 = {0}$'.format(1 + 1)
-

Then we bring it in: \(1 + 1 = 2\). The \py command can even bring in verbatim content, as long as you are using the pdfTeX or XeTeX engines. (Unfortunately, verbatim does not work with LuaTeX, due to a bug.2)

+
mytext = '$1 + 1 = {0}$'.format(1 + 1)
+

Then we bring it in: \(1 + 1 = 2\). The \py command can even bring in verbatim content.

We don’t have to typeset the code we’re executing. It can be hidden. And then we can access it later: This is a message from Python.

-

Pygments highlighting

+

It is also possible to perform variable substitution or string interpolation. The earlier result could be recreated: \(1 + 1 = 2\).

+

Pygments highlighting

PythonTeX supports syntax highlighting via Pygments. Any language supported by Pygments can be highlighted. Unicode is supported. Consider this snippet copied and pasted from a Python 3 interactive session. (Using random strings of Unicode for variable names is probably not a good idea, but PythonTeX will happily highlight it for you.)

-
>>> âæéöø = 123
->>> ßçñðŠ = 456
->>> âæéöø + ßçñðŠ
-579
+
>>> âæéöø = 123
+>>> ßçñðŠ = 456
+>>> âæéöø + ßçñðŠ
+579

There is also a Pygments command for inline use: \pygment.

-

Python console environment

+

Python console environment

PythonTeX includes an environment that emulates a Python interactive session. Commands are entered within the environment, each line is treated as input to an interactive session, and the result is typeset.

-
>>> x = 123
->>> y = 345
->>> z = x + y
->>> z
-468
->>> def f(expr):
-...     return(expr**4)
-... 
->>> f(x)
-228886641
->>> print('Python says hi from the console!')
-Python says hi from the console!
-

It is possible to refer to the values of console variables later on in inline contexts, using the \pycon command. For example, the value of \(z\) was 468.

-

Basic SymPy interaction

+
>>> x = 123
+>>> y = 345
+>>> z = x + y
+>>> z
+468
+>>> def f(expr):
+...     return(expr**4)
+... 
+>>> f(x)
+228886641
+>>> print('Python says hi from the console!')
+Python says hi from the console!
+

It is possible to refer to the values of console variables later on in inline contexts, using the \pycon command. For example, the value of \(z\) was 468.

+

Basic SymPy interaction

PythonTeX allows us to perform algebraic manipulations with SymPy and then properly typeset the results.

-

We create three variables, and define \(z\) in terms of the other two.

-
var('x, y, z')
-z = x + y
-

Now we can access what \(z\) is equal to:

-

\[z=x + y\]

+

We create three variables, and define \(z\) in terms of the other two.

+
var('x, y, z')
+z = x + y
+

Now we can access what \(z\) is equal to:

+

\[z=x + y\]

Many things are possible, including some very nice calculus.

-
f = x**3 + cos(x)**5
-g = Integral(f, x)
-

\[\int x^{3} + \cos^{5}{\left (x \right )}\, dx=\frac{1}{4} x^{4} + \frac{1}{5} \sin^{5}{\left (x \right )} - \frac{2}{3} \sin^{3}{\left (x \right )} + \sin{\left (x \right )}\]

+
f = x**3 + cos(x)**5
+g = Integral(f, x)
+

\[\int \left(x^{3} + \cos^{5}{\left(x \right)}\right)\, dx=\frac{x^{4}}{4} + \frac{\sin^{5}{\left(x \right)}}{5} - \frac{2 \sin^{3}{\left(x \right)}}{3} + \sin{\left(x \right)}\]

It’s easy to use arbitrary symbols in equations.

-
phi = Symbol(r'\phi')
-h = Integral(exp(-phi**2), (phi, 0, oo))
-

\[\int_{0}^{\infty} e^{- \phi^{2}}\, d\phi=\frac{1}{2} \sqrt{\pi}\]

-

Plots with matplotlib

+
phi = Symbol(r'\phi')
+h = Integral(exp(-phi**2), (phi, 0, oo))
+

\[\int\limits_{0}^{\infty} e^{- \phi^{2}}\, d\phi=\frac{\sqrt{\pi}}{2}\]

+

Plots with matplotlib

We can create plots with matplotlib, perfectly matching the plot fonts with the document fonts. No more searching for the code that created a figure!

+

It is possible to pass page dimensions and similar contextual information from the LaTeX side to the Python side. If you want your figures to be, for example, a particular fraction of the page width, you can pass the value of \textwidth to the Python side, and use it in creating your figures. See \setpythontexcontext in the main documentation for details.

You may want to use matplotlib’s PGF backend when creating plots.

-
rc('text', usetex=True)
-rc('font', family='serif')
-rc('font', size=10.0)
-rc('legend', fontsize=10.0)
-rc('font', weight='normal')
-x = linspace(0, 10)
-figure(figsize=(4, 2.5))
-plot(x, sin(x), label='$\sin(x)$')
-xlabel(r'$x\mathrm{-axis}$')
-ylabel(r'$y\mathrm{-axis}$')
-legend(loc='lower right')
-savefig('myplot.png', bbox_inches='tight')
+
rc('text', usetex=True)
+rc('font', family='serif')
+rc('font', size=10.0)
+rc('legend', fontsize=10.0)
+rc('font', weight='normal')
+x = linspace(0, 10)
+figure(figsize=(4, 2.5))
+plot(x, sin(x), label='$\sin(x)$')
+xlabel(r'$x\mathrm{-axis}$')
+ylabel(r'$y\mathrm{-axis}$')
+legend(loc='lower right')
+savefig('myplot.png', bbox_inches='tight')
+

image

-

Basic pylab interaction

-
from scipy.integrate import quad
-myintegral = quad(lambda x: e**-x**2, 0, inf)[0]
-

\[\int_0^\infty e^{-x^2}\,dx = 0.886226925453\]

-

An automated derivative and integral table

+
+

Basic pylab interaction

+
from scipy.integrate import quad
+myintegral = quad(lambda x: e**-x**2, 0, inf)[0]
+

\[\int_0^\infty e^{-x^2}\,dx = 0.886226925452758\]

+

An automated derivative and integral table

PythonTeX allows some amazing document automation, such as this derivative and integral table. Try typing that by hand, fast!

-
from re import sub
-
-var('x')
-
-# Create a list of functions to include in the table
-funcs = ['sin(x)', 'cos(x)', 'tan(x)', 
-         'sin(x)**2', 'cos(x)**2', 'tan(x)**2', 
-         'asin(x)', 'acos(x)', 'atan(x)', 
-         'sinh(x)', 'cosh(x)', 'tanh(x)']
-
-print(r'\begin{align*}')
-
-for func in funcs:
-    # Put in some vertical space when switching to arc and hyperbolic funcs
-    if func == 'asin(x)' or func == 'sinh(x)':
-        print(r'&\\')
-    myderiv = 'Derivative(' + func + ', x)'
-    myint = 'Integral(' + func + ', x)'
-    print(latex(eval(myderiv)) + '&=' +
-            latex(eval(myderiv + '.doit()')) + r'\quad & \quad')
-    print(latex(eval(myint)) + '&=' +
-            latex(eval(myint+'.doit()')) + r'\\')
-print(r'\end{align*}')
-

\[\begin{aligned} -\frac{\partial}{\partial x} \sin{\left (x \right )}&=\cos{\left (x \right )}\quad & \quad -\int \sin{\left (x \right )}\, dx&=- \cos{\left (x \right )}\\ -\frac{\partial}{\partial x} \cos{\left (x \right )}&=- \sin{\left (x \right )}\quad & \quad -\int \cos{\left (x \right )}\, dx&=\sin{\left (x \right )}\\ -\frac{\partial}{\partial x} \tan{\left (x \right )}&=\tan^{2}{\left (x \right )} + 1\quad & \quad -\int \tan{\left (x \right )}\, dx&=- \frac{1}{2} \log{\left (\sin^{2}{\left (x \right )} -1 \right )}\\ -\frac{\partial}{\partial x} \sin^{2}{\left (x \right )}&=2 \sin{\left (x \right )} \cos{\left (x \right )}\quad & \quad -\int \sin^{2}{\left (x \right )}\, dx&=\frac{1}{2} x - \frac{1}{2} \sin{\left (x \right )} \cos{\left (x \right )}\\ -\frac{\partial}{\partial x} \cos^{2}{\left (x \right )}&=- 2 \sin{\left (x \right )} \cos{\left (x \right )}\quad & \quad -\int \cos^{2}{\left (x \right )}\, dx&=\frac{1}{2} x + \frac{1}{2} \sin{\left (x \right )} \cos{\left (x \right )}\\ -\frac{\partial}{\partial x} \tan^{2}{\left (x \right )}&=\left(2 \tan^{2}{\left (x \right )} + 2\right) \tan{\left (x \right )}\quad & \quad -\int \tan^{2}{\left (x \right )}\, dx&=- x + \frac{\sin{\left (x \right )}}{\cos{\left (x \right )}}\\ +

from re import sub
+
+var('x')
+
+# Create a list of functions to include in the table
+funcs = ['sin(x)', 'cos(x)', 'tan(x)', 
+         'sin(x)**2', 'cos(x)**2', 'tan(x)**2', 
+         'asin(x)', 'acos(x)', 'atan(x)', 
+         'sinh(x)', 'cosh(x)', 'tanh(x)']
+
+print(r'\begin{align*}')
+
+for func in funcs:
+    # Put in some vertical space when switching to arc and hyperbolic funcs
+    if func == 'asin(x)' or func == 'sinh(x)':
+        print(r'&\\')
+    myderiv = 'Derivative(' + func + ', x)'
+    myint = 'Integral(' + func + ', x)'
+    print(latex(eval(myderiv)) + '&=' +
+            latex(eval(myderiv + '.doit()')) + r'\quad & \quad')
+    print(latex(eval(myint)) + '&=' +
+            latex(eval(myint+'.doit()')) + r'\\')
+print(r'\end{align*}')
+

\[\begin{aligned} +\frac{d}{d x} \sin{\left(x \right)}&=\cos{\left(x \right)}\quad & \quad +\int \sin{\left(x \right)}\, dx&=- \cos{\left(x \right)}\\ +\frac{d}{d x} \cos{\left(x \right)}&=- \sin{\left(x \right)}\quad & \quad +\int \cos{\left(x \right)}\, dx&=\sin{\left(x \right)}\\ +\frac{d}{d x} \tan{\left(x \right)}&=\tan^{2}{\left(x \right)} + 1\quad & \quad +\int \tan{\left(x \right)}\, dx&=- \log{\left(\cos{\left(x \right)} \right)}\\ +\frac{d}{d x} \sin^{2}{\left(x \right)}&=2 \sin{\left(x \right)} \cos{\left(x \right)}\quad & \quad +\int \sin^{2}{\left(x \right)}\, dx&=\frac{x}{2} - \frac{\sin{\left(x \right)} \cos{\left(x \right)}}{2}\\ +\frac{d}{d x} \cos^{2}{\left(x \right)}&=- 2 \sin{\left(x \right)} \cos{\left(x \right)}\quad & \quad +\int \cos^{2}{\left(x \right)}\, dx&=\frac{x}{2} + \frac{\sin{\left(x \right)} \cos{\left(x \right)}}{2}\\ +\frac{d}{d x} \tan^{2}{\left(x \right)}&=\left(2 \tan^{2}{\left(x \right)} + 2\right) \tan{\left(x \right)}\quad & \quad +\int \tan^{2}{\left(x \right)}\, dx&=- x + \frac{\sin{\left(x \right)}}{\cos{\left(x \right)}}\\ &\\ -\frac{\partial}{\partial x} \operatorname{asin}{\left (x \right )}&=\frac{1}{\sqrt{- x^{2} + 1}}\quad & \quad -\int \operatorname{asin}{\left (x \right )}\, dx&=x \operatorname{asin}{\left (x \right )} + \sqrt{- x^{2} + 1}\\ -\frac{\partial}{\partial x} \operatorname{acos}{\left (x \right )}&=- \frac{1}{\sqrt{- x^{2} + 1}}\quad & \quad -\int \operatorname{acos}{\left (x \right )}\, dx&=x \operatorname{acos}{\left (x \right )} - \sqrt{- x^{2} + 1}\\ -\frac{\partial}{\partial x} \operatorname{atan}{\left (x \right )}&=\frac{1}{x^{2} + 1}\quad & \quad -\int \operatorname{atan}{\left (x \right )}\, dx&=x \operatorname{atan}{\left (x \right )} - \frac{1}{2} \log{\left (x^{2} + 1 \right )}\\ +\frac{d}{d x} \operatorname{asin}{\left(x \right)}&=\frac{1}{\sqrt{1 - x^{2}}}\quad & \quad +\int \operatorname{asin}{\left(x \right)}\, dx&=x \operatorname{asin}{\left(x \right)} + \sqrt{1 - x^{2}}\\ +\frac{d}{d x} \operatorname{acos}{\left(x \right)}&=- \frac{1}{\sqrt{1 - x^{2}}}\quad & \quad +\int \operatorname{acos}{\left(x \right)}\, dx&=x \operatorname{acos}{\left(x \right)} - \sqrt{1 - x^{2}}\\ +\frac{d}{d x} \operatorname{atan}{\left(x \right)}&=\frac{1}{x^{2} + 1}\quad & \quad +\int \operatorname{atan}{\left(x \right)}\, dx&=x \operatorname{atan}{\left(x \right)} - \frac{\log{\left(x^{2} + 1 \right)}}{2}\\ &\\ -\frac{\partial}{\partial x} \sinh{\left (x \right )}&=\cosh{\left (x \right )}\quad & \quad -\int \sinh{\left (x \right )}\, dx&=\cosh{\left (x \right )}\\ -\frac{\partial}{\partial x} \cosh{\left (x \right )}&=\sinh{\left (x \right )}\quad & \quad -\int \cosh{\left (x \right )}\, dx&=\sinh{\left (x \right )}\\ -\frac{\partial}{\partial x} \tanh{\left (x \right )}&=- \tanh^{2}{\left (x \right )} + 1\quad & \quad -\int \tanh{\left (x \right )}\, dx&=- x - \log{\left (\tanh{\left (x \right )} -1 \right )}\\\end{aligned}\]

-

Step-by-step solutions

+\frac{d}{d x} \sinh{\left(x \right)}&=\cosh{\left(x \right)}\quad & \quad +\int \sinh{\left(x \right)}\, dx&=\cosh{\left(x \right)}\\ +\frac{d}{d x} \cosh{\left(x \right)}&=\sinh{\left(x \right)}\quad & \quad +\int \cosh{\left(x \right)}\, dx&=\sinh{\left(x \right)}\\ +\frac{d}{d x} \tanh{\left(x \right)}&=1 - \tanh^{2}{\left(x \right)}\quad & \quad +\int \tanh{\left(x \right)}\, dx&=x - \log{\left(\tanh{\left(x \right)} + 1 \right)}\\\end{aligned}\]

+

Step-by-step solutions

Using SymPy, it is possible to typeset step-by-step solutions. In this particular case, we also use the mdframed package to place a colored background behind our code.

-
x, y, z = symbols('x,y,z')
-f = Symbol('f(x,y,z)')
-
-# Define limits of integration
-x_llim = 0
-x_ulim = 2
-y_llim = 0
-y_ulim = 3
-z_llim = 0
-z_ulim = 4
-
-print(r'\begin{align*}')
-
-# Notice how I define f as a symbol, then later as an actual function
-left = Integral(f, (x, x_llim, x_ulim), (y, y_llim, y_ulim), (z, z_llim, z_ulim))
-f = x*y + y*sin(z) + cos(x+y)
-right = Integral(f, (x, x_llim, x_ulim), (y, y_llim, y_ulim), (z, z_llim, z_ulim))
-print(latex(left) + '&=' + latex(right) + r'\\')
-
-# For each step, I move limits from an outer integral to an inner, evaluated 
-# integral until the outer integral is no longer needed
-right = Integral(Integral(f, (z, z_llim, z_ulim)).doit(), (x, x_llim, x_ulim),
-                 (y, y_llim, y_ulim))
-print('&=' + latex(right) + r'\\')
-
-right = Integral(Integral(f, (z, z_llim, z_ulim), (y, y_llim, y_ulim)).doit(),
-                 (x, x_llim, x_ulim))
-print('&=' + latex(right) + r'\\')
-
-right = Integral(f, (z, z_llim, z_ulim), (y, y_llim, y_ulim), 
-                 (x, x_llim, x_ulim)).doit()
-print('&=' + latex(right) + r'\\')
-
-print('&=' + latex(N(right)) + r'\\')
-
-print(r'\end{align*}')
-

\[\begin{aligned} -\int_{0}^{4}\int_{0}^{3}\int_{0}^{2} f(x,y,z)\, dx\, dy\, dz&=\int_{0}^{4}\int_{0}^{3}\int_{0}^{2} x y + y \sin{\left (z \right )} + \cos{\left (x + y \right )}\, dx\, dy\, dz\\ -&=\int_{0}^{3}\int_{0}^{2} 4 x y - y \cos{\left (4 \right )} + y + 4 \cos{\left (x + y \right )}\, dx\, dy\\ -&=\int_{0}^{2} 18 x - 4 \sin{\left (x \right )} + 4 \sin{\left (x + 3 \right )} - \frac{9}{2} \cos{\left (4 \right )} + \frac{9}{2}\, dx\\ -&=4 \cos{\left (3 \right )} + 4 \cos{\left (2 \right )} - 4 \cos{\left (5 \right )} - 9 \cos{\left (4 \right )} + 41\\ +

x, y, z = symbols('x,y,z')
+f = Symbol('f(x,y,z)')
+
+# Define limits of integration
+x_llim = 0
+x_ulim = 2
+y_llim = 0
+y_ulim = 3
+z_llim = 0
+z_ulim = 4
+
+print(r'\begin{align*}')
+
+# Notice how I define f as a symbol, then later as an actual function
+left = Integral(f, (x, x_llim, x_ulim), (y, y_llim, y_ulim), (z, z_llim, z_ulim))
+f = x*y + y*sin(z) + cos(x+y)
+right = Integral(f, (x, x_llim, x_ulim), (y, y_llim, y_ulim), (z, z_llim, z_ulim))
+print(latex(left) + '&=' + latex(right) + r'\\')
+
+# For each step, I move limits from an outer integral to an inner, evaluated 
+# integral until the outer integral is no longer needed
+right = Integral(Integral(f, (z, z_llim, z_ulim)).doit(), (x, x_llim, x_ulim),
+                 (y, y_llim, y_ulim))
+print('&=' + latex(right) + r'\\')
+
+right = Integral(Integral(f, (z, z_llim, z_ulim), (y, y_llim, y_ulim)).doit(),
+                 (x, x_llim, x_ulim))
+print('&=' + latex(right) + r'\\')
+
+right = Integral(f, (z, z_llim, z_ulim), (y, y_llim, y_ulim), 
+                 (x, x_llim, x_ulim)).doit()
+print('&=' + latex(right) + r'\\')
+
+print('&=' + latex(N(right)) + r'\\')
+
+print(r'\end{align*}')
+

\[\begin{aligned} +\int\limits_{0}^{4}\int\limits_{0}^{3}\int\limits_{0}^{2} f(x,y,z)\, dx\, dy\, dz&=\int\limits_{0}^{4}\int\limits_{0}^{3}\int\limits_{0}^{2} \left(x y + y \sin{\left(z \right)} + \cos{\left(x + y \right)}\right)\, dx\, dy\, dz\\ +&=\int\limits_{0}^{3}\int\limits_{0}^{2} \left(4 x y - y \cos{\left(4 \right)} + y + 4 \cos{\left(x + y \right)}\right)\, dx\, dy\\ +&=\int\limits_{0}^{2} \left(18 x - 4 \sin{\left(x \right)} + 4 \sin{\left(x + 3 \right)} - \frac{9 \cos{\left(4 \right)}}{2} + \frac{9}{2}\right)\, dx\\ +&=4 \cos{\left(3 \right)} + 4 \cos{\left(2 \right)} - 4 \cos{\left(5 \right)} - 9 \cos{\left(4 \right)} + 41\\ &=40.1235865133293\\\end{aligned}\]

-

Including stderr

+

Including stderr

PythonTeX allows code to be typset next to the stderr it produces. This requires the package option makestderr.

-
x = 123
-y = 345
-z = x + y +
+
x = 123
+y = 345
+z = x + y +

This code causes a syntax error:

  File "py_errorsession_9.py", line 3
     z = x + y +
               ^
 SyntaxError: invalid syntax

The package option stderrfilename allows the file name that appears in the error message to be customized.

-
+
    -

  1. Since PythonTeX runs Python code (and potentially other code) on your computer, documents using PythonTeX have a greater potential for security risks than do standard LaTeX documents. You should only compile PythonTeX documents from sources you trust.↩

    2. -

  2. http://tracker.luatex.org/view.php?id=733↩

    3. +

  3. Since PythonTeX runs Python code (and potentially other code) on your computer, documents using PythonTeX have a greater potential for security risks than do standard LaTeX documents. You should only compile PythonTeX documents from sources you trust.↩︎

-
+ diff --git a/pythontex_gallery/pythontex_gallery.pdf b/pythontex_gallery/pythontex_gallery.pdf index 66ba25e..41f7c98 100644 Binary files a/pythontex_gallery/pythontex_gallery.pdf and b/pythontex_gallery/pythontex_gallery.pdf differ diff --git a/pythontex_gallery/pythontex_gallery.tex b/pythontex_gallery/pythontex_gallery.tex index 29feb22..10295a0 100644 --- a/pythontex_gallery/pythontex_gallery.tex +++ b/pythontex_gallery/pythontex_gallery.tex @@ -4,19 +4,19 @@ % Detect pdftex/xetex/luatex, and load appropriate font packages. % This is inspired by the approach in the iftex package. % pdftex: -\expandafter\ifx\csname pdfmatch\endcsname\relax +\ifx\pdfmatch\undefined \else \usepackage[T1]{fontenc} \usepackage[utf8]{inputenc} \fi % xetex: -\expandafter\ifx\csname XeTeXinterchartoks\endcsname\relax +\ifx\XeTeXinterchartoks\undefined \else \usepackage{fontspec} \defaultfontfeatures{Ligatures=TeX} \fi % luatex: -\expandafter\ifx\csname directlua\endcsname\relax +\ifx\directlua\undefined \else \usepackage{fontspec} \fi @@ -69,16 +69,18 @@ \section{General Python interaction} \printpythontex[verb] \end{quote} -Python interaction can also be more complex. \pyb{print(str(2**2**2) + r'\endinput')} returns \printpythontex. In this case, the printed results include \LaTeX\ code, which is correctly interpreted by \LaTeX\ to ensure that there is not an extra space after the 16. Printed output is saved to a file and brought back in via \verb|\input|, and the \verb|\endinput| command stops input immediately, before \LaTeX\ gets to the end of the line and inserts a space character there, after the 16. +Python interaction can also be more complex. \pyb{print(str(2**2**2) + r'\endinput')} returns \printpythontex. In this case, the printed result includes \LaTeX\ code, which is correctly interpreted by \LaTeX\ to ensure that there is not an extra space after the 16. Printed output is saved to a file and brought back in via \verb|\input|, and the \verb|\endinput| command stops input immediately, before \LaTeX\ gets to the end of the line and inserts a space character there, after the 16. Printing works, but as the last example demonstrates, you have to be careful about spacing if you have text immediately after the printed content. In that case, it's usually best to assemble text within a \pytex\ environment and store the text in a variable. Then you can bring in the text later, using the \pygment{latex}{\py} command. The \pygment{latex}{\py} command brings in a string representation of its argument. First we create the text. \begin{pyblock} mytext = '$1 + 1 = {0}$'.format(1 + 1) \end{pyblock} -Then we bring it in: \py{mytext}. The \pygment{latex}{\py} command can even bring in verbatim content, as long as you are using the pdfTeX or XeTeX engines. (Unfortunately, verbatim does not work with LuaTeX, due to a bug.\footnote{\url{http://tracker.luatex.org/view.php?id=733}}) +Then we bring it in: \py{mytext}. The \pygment{latex}{\py} command can even bring in verbatim content. We don't have to typeset the code we're executing. It can be hidden.\pyc{mystring=r'\textbf{This is a message from Python}'} And then we can access it later: \py{mystring}. +It is also possible to perform variable substitution or string interpolation. The earlier result could be recreated: \pys{$1 + 1 = !{1+1}$}. + \section{Pygments highlighting} @@ -152,6 +154,8 @@ \section{Plots with matplotlib} We can create plots with matplotlib, perfectly matching the plot fonts with the document fonts. No more searching for the code that created a figure! +It is possible to pass page dimensions and similar contextual information from the \LaTeX\ side to the Python side. If you want your figures to be, for example, a particular fraction of the page width, you can pass the value of \pygment{latex}{\textwidth} to the Python side, and use it in creating your figures. See \pygment{latex}{\setpythontexcontext} in the main documentation for details. + You may want to use matplotlib's PGF backend when creating plots. \begin{pylabblock} @@ -162,7 +166,7 @@ \section{Plots with matplotlib} rc('font', weight='normal') x = linspace(0, 10) figure(figsize=(4, 2.5)) -plot(x, sin(x), label='$\sin(x)$') +plot(x, sin(x), label=r'$\sin(x)$') xlabel(r'$x\mathrm{-axis}$') ylabel(r'$y\mathrm{-axis}$') legend(loc='lower right') diff --git a/pythontex_quickstart/pythontex_quickstart.pdf b/pythontex_quickstart/pythontex_quickstart.pdf index cacc158..c47904c 100644 Binary files a/pythontex_quickstart/pythontex_quickstart.pdf and b/pythontex_quickstart/pythontex_quickstart.pdf differ diff --git a/pythontex_quickstart/pythontex_quickstart.tex b/pythontex_quickstart/pythontex_quickstart.tex index 41d4f43..fb3c8ef 100644 --- a/pythontex_quickstart/pythontex_quickstart.tex +++ b/pythontex_quickstart/pythontex_quickstart.tex @@ -1,5 +1,28 @@ \documentclass[twocolumn]{article} +% Engine-specific settings +% Detect pdftex/xetex/luatex, and load appropriate font packages. +% This is inspired by the approach in the iftex package. +% pdftex: +\ifx\pdfmatch\undefined +\else + \usepackage[T1]{fontenc} + \usepackage[utf8]{inputenc} +\fi +% xetex: +\ifx\XeTeXinterchartoks\undefined +\else + \usepackage{fontspec} + \defaultfontfeatures{Ligatures=TeX} +\fi +% luatex: +\ifx\directlua\undefined +\else + \usepackage{fontspec} +\fi +% End engine-specific settings +\usepackage{lmodern} + \usepackage[margin=1in]{geometry} \usepackage{microtype} @@ -7,8 +30,6 @@ \usepackage[svgnames]{xcolor} -\usepackage{nopageno} - \usepackage{hyperref} \hypersetup{colorlinks=true,urlcolor=Green,linkcolor=blue} @@ -28,25 +49,42 @@ +\section*{Installing} + +\pytex\ requires Python 2.7 or 3.2+. When using \pytex\ with LyX, be aware that LyX may try to use its own version of Python; you may need to reconfigure LyX to use other Python installations. + +\pytex\ is included in TeX Live and MiKTeX. It may be installed via the package manager. + +A Python installation script is included with the package. It should be able to install the package in most situations. Depending on the configuration of your system, you may have to run the installation script with administrative priviliges. + +Detailed installation information is available in the main documentation, \texttt{pythontex.pdf}. + \section*{Compiling} -Compiling a document that uses \pytex\ involves three steps: run latex, run \texttt{pythontex.py}, and finally run latex again. You may wish to create a symlink or launching wrapper for \texttt{pythontex.py}, if one was not created during installation. \pytex\ is compatible with the pdfLaTeX, XeLaTeX, and LuaLaTeX engines. There are minor engine-specific differences. +Compiling a document that uses \pytex\ involves three steps: run \LaTeX, run \texttt{pythontex.py}, and finally run \LaTeX\ again. You may wish to create a symlink or launching wrapper for \texttt{pythontex.py}, if one was not created during installation. \pytex\ is compatible with the pdfTeX, XeTeX, and LuaTeX engines, so you can use \texttt{latex}, \texttt{pdflatex}, \texttt{xelatex}, or \texttt{lualatex}. + +The last two compile steps are \emph{only} necessary when code needs to be executed or highlighted. Otherwise, the document may be compiled just like a normal \LaTeX\ document; all output is cached. + +\pytex\ is compatible with \texttt{latexmk}. Details for configuring \texttt{latexmk} are provided in the main documentation. -\section*{Commands} -\pygment{latex}{\py} returns a string representation of its argument. For example, \pygment{latex}{\py{2 + 4**2}} produces \py{2 + 4**2}, and \pygment{latex}{\py{'ABC'.lower()}} produces \py{'ABC'.lower()}. \pygment{latex}{\py}'s argument can be delimited by curly braces, or by a matched pair of other characters (just like \pygment{latex}{\verb}). +\section*{Basic commands} + +\pygment{latex}{\py} returns a string representation of its argument. For example, \pygment{latex}{\py{2 + 4**2}} produces ``\py{2 + 4**2}'', and \pygment{latex}{\py{'ABC'.lower()}} produces ``\py{'ABC'.lower()}''. \pygment{latex}{\py}'s argument can be delimited by curly braces, or by a matched pair of other characters (just like \pygment{latex}{\verb}). \pygment{latex}{\pyc} executes code. By default, anything that is printed is automatically included in the document (see \texttt{autoprint}/\texttt{autostdout} in the main documentation). For example, \pygment{latex}{\pyc{var = 2}} \pyc{var = 2} creates a variable, and then its value may be accessed later via \pygment{latex}{\py{var}}: \py{var}. -\pygment{latex}{\pyb} executes and typesets code. For example, \pygment{latex}{\pyb{var = 2}} typesets \pyb{var = 2} in addition to creating the variable. If anything is printed, it is not automatically included, but can be accessed via \pygment{latex}{\printpythontex} and \pygment{latex}{\stdoutpythontex}. +\pygment{latex}{\pyb} executes and typesets code. For example, \pygment{latex}{\pyb{var = 2}} typesets \pyb{var = 2} in addition to creating the variable. If anything is printed, it is not automatically included, but can be accessed via \pygment{latex}{\printpythontex} or \pygment{latex}{\stdoutpythontex}. + +\pygment{latex}{\pyv} only typesets code; nothing is executed. For example, \pygment{latex}{\pyv{var = 2}} produces \pyv{var = 2}. -\pygment{latex}{\pyv} only typesets code. For example, \pygment{latex}{\pyv{var = 2}} produces \pyv{var = 2}. +\pygment{latex}{\pys} performs variable substitution or string interpolation on code. Substitution fields are denoted by \verb|!{...}|; details about escaping are provided in the main documentation. For example, using the pre-existing variable \pyv{var}, \pygment{latex}{\pys{\verb|var = !{var}|}} yields \pys{\verb|var = !{var}|}. -\section*{Environments} +\section*{Basic environments} -There are \texttt{pycode}, \texttt{pyblock}, and \texttt{pyverbatim} environments, which are the environment equivalents of \pygment{latex}{\pyc}, \pygment{latex}{\pyb}, and \pygment{latex}{\pyv}. For example, +There are \texttt{pycode}, \texttt{pyblock}, \texttt{pyverbatim}, and \texttt{pysub} environments, which are the environment equivalents of \pygment{latex}{\pyc}, \pygment{latex}{\pyb}, \pygment{latex}{\pyv}, and \pygment{latex}{\pys}. For example, \begin{pygments}{text} \begin{pycode} print(r'\begin{center}') @@ -61,6 +99,21 @@ \section*{Environments} print(r'\end{center}') \end{pycode} +The \pygment{latex}{\begin} and \pygment{latex}{\end} of an environment should be on lines by themselves. Code in environments may be indented; see the \texttt{gobble} option in the main documentation for more details. + + +\section*{More commands/environments} + +All commands and environments described so far have names beginning with \texttt{py}. There are equivalent commands and environments that begin with \texttt{sympy}; these automatically include +\begin{pygments}{python} +from sympy import * +\end{pygments} +There are also equivalent commands and environments that begin with \texttt{pylab}; these automatically use matplotlib's \texttt{pylab} module via +\begin{pygments}{python} +from pylab import * +\end{pygments} +The \texttt{sympy} and \texttt{pylab} commands and environments execute code in separate sessions from the \texttt{py} commands and environments. This can make it easier to avoid namespace conflicts. + There is also a \texttt{pyconsole} environment that emulates a Python interactive console. For example, \begin{pygments}{text} \begin{pyconsole} @@ -73,21 +126,68 @@ \section*{Environments} var = 1 + 1 var \end{pyconsole} +Console variable values may be accessed inline via the \pygment{latex}{\pycon} command. More console information is available in the main documentation. +\section*{Working with Python 2} +\pytex\ supports both Python 2 and 3. Under Python 2, imports from \pygment{python}{__future__} will work so long as they are the first user-entered code in a given session. \pytex\ imports most things from \pygment{python}{__future__} by default. To control what is automatically imported, see the \texttt{pyfuture} and \texttt{pyconfuture} package options in the main documentation. -The \pygment{latex}{\begin} and \pygment{latex}{\end} of an environment should be on lines by themselves. Code in environments may be indented; see the \texttt{gobble} option in the documentation for more details. + +\section*{Support for additional languages} + +\pytex\ also provides support for additional languages. Currently, Ruby, Julia, Octave, Sage, Bash, and Rust support is included. To enable commands and environments for these language, see the \texttt{usefamily} package option in the main documentation. + +Language support is provided via a template system; in most cases, a new language can be added with about 100 lines of template code---and basic support can require less than 20 lines. If you would like support for a new language, please open an issue at \href{https://github.com/gpoore/pythontex}{GitHub}. The main documentation also contains a summary of the process for adding languages. \section*{Macro programming} -\pytex\ commands can be used inside other commands in macro programming. They will usually work fine, but curly braces should be used as delimiters and special \LaTeX\ characters such as \texttt{\%} and \texttt{\#} should be avoided in the Python code. \pytex\ environments cannot be used inside \LaTeX\ commands, due to the way \LaTeX\ deals with verbatim content and catcodes. +\pytex\ commands can be used inside other commands in macro programming. They will usually work fine, but curly braces should be used as delimiters and special \LaTeX\ characters such as \texttt{\%} and \texttt{\#} should be avoided in the Python code. These limitations can be removed by passing arguments verbatim or through catcode trickery. \pytex\ environments cannot normally be used inside \LaTeX\ commands, due to the way \LaTeX\ deals with verbatim content and catcodes. \section*{Additional features} -\pytex\ provides many additional features. The working and output directories can be specified. The user can determine when code is executed with the package option \texttt{rerun}, based on factors such as modification and exit status. By default, all commands and environments run in a single session, providing continuity. Commands and environments accept an optional argument that specifies the session in which the code is executed; sessions run in parallel. \pytex\ provides a utilities class that is always imported into each session. The utilities class provides methods for tracking dependencies and automatically cleaning up created files. +\pytex\ provides many additional features. The working and output directories can be specified via \pygment{latex}{\setpythontexworkingdir} and \pygment{latex}{\setpythontexoutputdir}. The user can determine when code is executed with the package option \texttt{rerun}, selecting factors such as modification and exit status. By default, all commands and environments with the same base name (\texttt{py}, \texttt{sympy}, \texttt{pylab}, etc.) run in a single session, providing continuity. Commands and environments accept an optional argument that specifies the session in which the code is executed; sessions run in parallel. \pytex\ provides a utilities class that is always imported into each session. The utilities class provides methods for tracking dependencies and automatically cleaning up created files. The utilities class also allows information such as page width to be passed from the \TeX\ side to Python/other languages. See the main documentation for additional information. \pytex\ also provides the \texttt{depythontex} utility, which creates a copy of a document in which all \pytex\ commands and environments have been replaced by their output. The resulting document is more suitable for journal submission, sharing, and conversion to other document formats. +Code may be run in interactive mode on the command line via the \verb|--interactive| and \verb|--debug| options. This is primarily useful for working with interactive debuggers. + + +\section*{Customizing typesetting} + +\pytex\ typesets code using the \texttt{fancyvrb} package and the \texttt{fvextra} package that extends \texttt{fancyvrb}. There is a \pygment{latex}{\setpythontexfv} command for setting \pytex-specific \texttt{fancyvrb} and \texttt{fvextra} options. The normal \pygment{latex}{\fvset} works as well for document-wide settings. \pytex\ environments take a second optional argument that consists of \texttt{fancyvrb} and \texttt{fvextra} settings. This can be used to customize automatic line breaking or line highlighting for a single environment. + + + +\section*{Unicode support} + +\pytex\ supports Unicode under all \LaTeX\ engines. For example, consider the following example from Python: +\begin{pygments}{python} +my_string = '¥ § ß Ğ Ð Ñ Ö þ ø' +\end{pygments} +This requires some engine-specific packages. Typical packages are listed below. + +\begin{itemize} +\item pdfLaTeX: +\begin{pygments}{latex} +\usepackage[T1]{fontenc} +\usepackage[utf8]{inputenc} +\end{pygments} +\item LuaLaTeX: +\begin{pygments}{latex} +\usepackage{fontspec} +\end{pygments} +\item XeLaTeX: +\begin{pygments}{latex} +\usepackage{fontspec} +\defaultfontfeatures{Ligatures=TeX} +\end{pygments} +\end{itemize} + +If you are using Python 2, you will also need to specify that you are using Unicode. You may want +\begin{pygments}{python} +from __future__ import unicode_literals +\end{pygments} +at the beginning of your Python code. Or you can just load the \pytex\ package with the option \verb|pyfuture=all|, which will import \verb|unicode_literals| automatically. \end{document} diff --git a/test/R/R_test.tex b/test/R/R_test.tex new file mode 100644 index 0000000..233c3d0 --- /dev/null +++ b/test/R/R_test.tex @@ -0,0 +1,103 @@ +\documentclass[11pt]{article} + +% Engine-specific settings +% pdftex: +\ifcsname pdfmatch\endcsname + \usepackage[T1]{fontenc} + \usepackage[utf8]{inputenc} +\fi +% xetex: +\ifcsname XeTeXinterchartoks\endcsname + \usepackage{fontspec} + \defaultfontfeatures{Ligatures=TeX} +\fi +% luatex: +\ifcsname directlua\endcsname + \usepackage{fontspec} +\fi +% End engine-specific settings + +\usepackage{lmodern} +\usepackage{amssymb,amsmath} +\usepackage{graphicx} +\usepackage{fullpage} +\usepackage[keeptemps=all, makestderr, usefamily={R}]{pythontex} + + +\begin{document} + + + +\section*{R} + +\subsection*{Commands} + +\R{2^8} + +\Rc{write(2^16, stdout())} + +\Rb{cat(2^32)} + +\printpythontex + +\Rv{cat(2^32)} + +\Rs{\LaTeX\ and then \textcolor{blue}{!{"R"}} and back to \LaTeX.} + + +\subsection*{Environments} + +Code: +\begin{Rcode} +cat("A string.", " ") +cat(2^8) +\end{Rcode} + +Block: +\begin{Rblock} +cat("A string.", " ") +cat(2^8) +\end{Rblock} + +\printpythontex + +Verbatim: +\begin{Rverbatim} +cat("A string.", " ") +cat(2^8) +\end{Rverbatim} + +Sub: +\begin{Rsub} +\LaTeX\ and then \textcolor{blue}{!{"R"}} and back to \LaTeX. +\end{Rsub} + + +\section*{R stderr} + + +\begin{Rblock}[err1][numbers=left] +# Comment +s <- "R a +\end{Rblock} + +\stderrpythontex[][breaklines, breakafter=\\/] + +\begin{Rblock}[err2][numbers=left] +1++ +\end{Rblock} + +\stderrpythontex[][breaklines, breakafter=\\/] + +\begin{Rblock}[err3][numbers=left] +# Comment +# Another comment +1 + "ab" +\end{Rblock} + +\stderrpythontex[][breaklines, breakafter=\\/] + + + +\end{document} + diff --git a/test/R/Rcon_test.tex b/test/R/Rcon_test.tex new file mode 100644 index 0000000..8e17411 --- /dev/null +++ b/test/R/Rcon_test.tex @@ -0,0 +1,79 @@ +\documentclass[11pt]{article} + +% Engine-specific settings +% pdftex: +\ifcsname pdfmatch\endcsname + \usepackage[T1]{fontenc} + \usepackage[utf8]{inputenc} +\fi +% xetex: +\ifcsname XeTeXinterchartoks\endcsname + \usepackage{fontspec} + \defaultfontfeatures{Ligatures=TeX} +\fi +% luatex: +\ifcsname directlua\endcsname + \usepackage{fontspec} +\fi +% End engine-specific settings + +\usepackage{lmodern} +\usepackage{amssymb,amsmath} +\usepackage{graphicx} +\usepackage{fullpage} +\usepackage[keeptemps=all, makestderr, usefamily={Rcon}]{pythontex} + + +\begin{document} + + +\section*{R Console} + + +\subsection*{Basics} + +Console math and printing. + +\begin{Rconsole} +1+4 +print("Some text ...") +\end{Rconsole} + +Test \pygment{r}{cat()} without a trailing newline \verb|\n|. This should produce output followed immediately on the same line by the prompt and the next command. +\begin{Rconsole} +cat("Some", "text", "...") +cat("And", "some", "more", "...") +\end{Rconsole} + +After a final \pygment{r}{cat()} without a trailing newline \verb|\n|, the next environment should start normally, with the prompt at the beginning of the line. + +\begin{Rconsole} +write("stdout", stdout()) +write("stderr", stderr()) +\end{Rconsole} + + +\subsection*{Continuity between environments} + +Set a variable. +\begin{Rconsole} +x <- 2^12 +\end{Rconsole} + +Retrieve variable value. +\begin{Rconsole} +x +\end{Rconsole} + + +\subsection*{Continue after errors} + +\begin{Rconsole} +1+"a" +3*6 +\end{Rconsole} + + + +\end{document} + diff --git a/test/bash/bash_test.tex b/test/bash/bash_test.tex new file mode 100644 index 0000000..8b9496e --- /dev/null +++ b/test/bash/bash_test.tex @@ -0,0 +1,58 @@ +\documentclass[11pt]{article} + +% Engine-specific settings +% pdftex: +\ifcsname pdfmatch\endcsname + \usepackage[T1]{fontenc} + \usepackage[utf8]{inputenc} +\fi +% xetex: +\ifcsname XeTeXinterchartoks\endcsname + \usepackage{fontspec} + \defaultfontfeatures{Ligatures=TeX} +\fi +% luatex: +\ifcsname directlua\endcsname + \usepackage{fontspec} +\fi +% End engine-specific settings + +\usepackage{lmodern} +\usepackage{amssymb,amsmath} +\usepackage{graphicx} +\usepackage{fullpage} +\usepackage[keeptemps=all, makestderr, usefamily=bash]{pythontex} + +\begin{document} + + +\section*{Bash} + +Inline: \bash{echo "Hello from bash!"} + +Code environment: +\begin{bashcode} +echo "More from bash." +\end{bashcode} + +Block environment: +\begin{bashblock} +ls -a +\end{bashblock} +Printed output: +\printpythontex[verbatim] + +Verbatim: +\begin{bashverbatim} +echo "More from bash." +\end{bashverbatim} + + +Sub environment: +\begin{bashsub} +Some text \textcolor{blue}{!{"followed by things from bash"}} and then more text. +\end{bashsub} + +Again, with command: \bashs{Some text \textcolor{blue}{!{"followed by more things from bash"}} and then more text.} + +\end{document} diff --git a/test/javascript/javascript_test.tex b/test/javascript/javascript_test.tex new file mode 100644 index 0000000..e281a08 --- /dev/null +++ b/test/javascript/javascript_test.tex @@ -0,0 +1,103 @@ +\documentclass[11pt]{article} + +% Engine-specific settings +% pdftex: +\ifcsname pdfmatch\endcsname + \usepackage[T1]{fontenc} + \usepackage[utf8]{inputenc} +\fi +% xetex: +\ifcsname XeTeXinterchartoks\endcsname + \usepackage{fontspec} + \defaultfontfeatures{Ligatures=TeX} +\fi +% luatex: +\ifcsname directlua\endcsname + \usepackage{fontspec} +\fi +% End engine-specific settings + +\usepackage{lmodern} +\usepackage{amssymb,amsmath} +\usepackage{graphicx} +\usepackage{fullpage} +\usepackage[keeptemps=all, makestderr, usefamily={javascript}]{pythontex} + + +\begin{document} + + + +\section*{JavaScript} + +\subsection*{Commands} + +\javascript{2**8} + +\javascriptc{console.log(2**16);} + +\javascriptb{console.log(2**32);} + +\printpythontex + +\javascriptv{console.log(2**32);} + +\javascripts{\LaTeX\ and then \textcolor{blue}{!{"JavaScript"}} and back to \LaTeX.} + + +\subsection*{Environments} + +Code: +\begin{javascriptcode} +console.log("A string." + " "); +console.log(2**8); +\end{javascriptcode} + +Block: +\begin{javascriptblock} +console.log("A string." + " "); +console.log(2**8); +\end{javascriptblock} + +\printpythontex + +Verbatim: +\begin{javascriptverbatim} +console.log("A string." + " "); +console.log(2**8); +\end{javascriptverbatim} + +Sub: +\begin{javascriptsub} +\LaTeX\ and then \textcolor{blue}{!{"JavaScript"}} and back to \LaTeX. +\end{javascriptsub} + + +\section*{JavaScript stderr} + + +\begin{javascriptblock}[err1][numbers=left] +// Comment +s = "javascript a +\end{javascriptblock} + +\stderrpythontex[][breaklines, breakafter=\\/] + +\begin{javascriptblock}[err2][numbers=left] +1+; +\end{javascriptblock} + +\stderrpythontex[][breaklines, breakafter=\\/] + +\begin{javascriptblock}[err3][numbers=left] +// Comment +// Another comment +"a" "b"; +\end{javascriptblock} + +\stderrpythontex[][breaklines, breakafter=\\/] + + + +\end{document} + diff --git a/test/julia/julia_test.tex b/test/julia/julia_test.tex new file mode 100644 index 0000000..612b9e5 --- /dev/null +++ b/test/julia/julia_test.tex @@ -0,0 +1,104 @@ +\documentclass[11pt]{article} + +% Engine-specific settings +% pdftex: +\ifcsname pdfmatch\endcsname + \usepackage[T1]{fontenc} + \usepackage[utf8]{inputenc} +\fi +% xetex: +\ifcsname XeTeXinterchartoks\endcsname + \usepackage{fontspec} + \defaultfontfeatures{Ligatures=TeX} +\fi +% luatex: +\ifcsname directlua\endcsname + \usepackage{fontspec} +\fi +% End engine-specific settings + +\usepackage{lmodern} +\usepackage{amssymb,amsmath} +\usepackage{graphicx} +\usepackage{fullpage} +\usepackage[keeptemps=all, makestderr, usefamily={julia}]{pythontex} + + +\begin{document} + + + +\section*{Julia} + +\subsection*{Commands} + +\julia{2^8} + +\juliac{println(2^16)} + +\juliab{println(2^32)} + +\printpythontex + +\juliav{println(2^32)} + +\julias{\LaTeX\ and then \textcolor{blue}{!{"Julia"}} and back to \LaTeX.} + + +\subsection*{Environments} + +Code: +\begin{juliacode} +println("A string.") +println(2^8) +\end{juliacode} + +Block: +\begin{juliablock} +println("A string.") +println(2^8) +\end{juliablock} + +\printpythontex + +Verbatim: +\begin{juliaverbatim} +println("A string.") +println(2^8) +\end{juliaverbatim} + +Sub: +\begin{juliasub} +\LaTeX\ and then \textcolor{blue}{!{"Julia"}} and back to \LaTeX. +\end{juliasub} + + +\section*{Julia stderr} + + +\begin{juliablock}[err1][numbers=left] +# Comment +s = """ + Julia a +\end{juliablock} + +\stderrpythontex[][breaklines, breakafter=\\/] + +\begin{juliablock}[err2][numbers=left] +1+ +\end{juliablock} + +\stderrpythontex[][breaklines, breakafter=\\/] + +\begin{juliablock}[err3][numbers=left] +# Comment +# Another comment +1 + "ab" +\end{juliablock} + +\stderrpythontex[][breaklines, breakafter=\\/] + + + +\end{document} + diff --git a/test/julia/juliacon_test.tex b/test/julia/juliacon_test.tex new file mode 100644 index 0000000..7a8138e --- /dev/null +++ b/test/julia/juliacon_test.tex @@ -0,0 +1,76 @@ +\documentclass[11pt]{article} + +% Engine-specific settings +% pdftex: +\ifcsname pdfmatch\endcsname + \usepackage[T1]{fontenc} + \usepackage[utf8]{inputenc} +\fi +% xetex: +\ifcsname XeTeXinterchartoks\endcsname + \usepackage{fontspec} + \defaultfontfeatures{Ligatures=TeX} +\fi +% luatex: +\ifcsname directlua\endcsname + \usepackage{fontspec} +\fi +% End engine-specific settings + +\usepackage{lmodern} +\usepackage{amssymb,amsmath} +\usepackage{graphicx} +\usepackage{fullpage} +\usepackage[keeptemps=all, makestderr, usefamily={juliacon,julia}]{pythontex} + + +\begin{document} + + +\section*{Julia Console} + + +\subsection*{Basics} + +Console math and printing. + +\begin{juliaconsole} +1+4 +println("Some text ...") +\end{juliaconsole} + +Test code that generates output without a trailing newline \verb|\n|. Since there is always a newline before the \verb|julia>| prompt, this should affect vertical layout, but not line breaks. +\begin{juliaconsole} +print("Some text ...") +print("And some more ...") +\end{juliaconsole} + +The next environment should start normally, with the prompt at the beginning of the line. +\begin{juliaconsole} +println("After ...") +\end{juliaconsole} + + + +\subsection*{Continuity between environments} + +Set a variable. +\begin{juliaconsole} +x = 2^12 +\end{juliaconsole} + +Retrieve variable value. +\begin{juliaconsole} +x +\end{juliaconsole} + + +\subsection*{Continue after errors} + +\begin{juliaconsole}[][breaklines] +1+"a" +3*6 +\end{juliaconsole} + + +\end{document} diff --git a/test/octave/octave_test.tex b/test/octave/octave_test.tex new file mode 100644 index 0000000..109b972 --- /dev/null +++ b/test/octave/octave_test.tex @@ -0,0 +1,102 @@ +\documentclass[11pt]{article} + +% Engine-specific settings +% pdftex: +\ifcsname pdfmatch\endcsname + \usepackage[T1]{fontenc} + \usepackage[utf8]{inputenc} +\fi +% xetex: +\ifcsname XeTeXinterchartoks\endcsname + \usepackage{fontspec} + \defaultfontfeatures{Ligatures=TeX} +\fi +% luatex: +\ifcsname directlua\endcsname + \usepackage{fontspec} +\fi +% End engine-specific settings + +\usepackage{lmodern} +\usepackage{amssymb,amsmath} +\usepackage{graphicx} +\usepackage{fullpage} +\usepackage[keeptemps=all, makestderr, usefamily={octave}]{pythontex} + +\begin{document} + + + +\section*{Octave} + +\subsection*{Commands} + +\octave{strrep(strrep(pwd(), '\', '/'), '_', '\_')} + +\octavec{disp(num2str(2^8))} + +\octaveb{disp(num2str(2^16))} + +\printpythontex + +\octavev{disp(num2str(2^32))} + + + +\subsection*{Environments} + +Code: +\begin{octavecode} +disp("Octave!") +disp(2^8) +\end{octavecode} + +Block: +\begin{octaveblock} +disp("Octave!") +disp(2^8) +\end{octaveblock} + +\printpythontex + +Verbatim: +\begin{octaveverbatim} +disp("Octave!") +disp(2^8) +\end{octaveverbatim} + +Sub: +\begin{octavesub} +In \LaTeX\ and then \textcolor{blue}{!{strcat(["Octave calculations $2^8=", num2str(2^8), "$"])}} and then back in \LaTeX. +\end{octavesub} + + + +\section*{Octave stderr} + + +\begin{octaveblock}[err1][numbers=left] +% Comment +s = "Octave a +\end{octaveblock} + +\stderrpythontex + +\begin{octaveblock}[err2][numbers=left] +1+ +\end{octaveblock} + +\stderrpythontex + +\begin{octaveblock}[err3][numbers=left] +% Comment +% Another comment +1 + qrst +\end{octaveblock} + +\stderrpythontex + + + +\end{document} + diff --git a/test/perl/perl_test.tex b/test/perl/perl_test.tex new file mode 100644 index 0000000..f89303d --- /dev/null +++ b/test/perl/perl_test.tex @@ -0,0 +1,103 @@ +\documentclass[11pt]{article} + +% Engine-specific settings +% pdftex: +\ifcsname pdfmatch\endcsname + \usepackage[T1]{fontenc} + \usepackage[utf8]{inputenc} +\fi +% xetex: +\ifcsname XeTeXinterchartoks\endcsname + \usepackage{fontspec} + \defaultfontfeatures{Ligatures=TeX} +\fi +% luatex: +\ifcsname directlua\endcsname + \usepackage{fontspec} +\fi +% End engine-specific settings + +\usepackage{lmodern} +\usepackage{amssymb,amsmath} +\usepackage{graphicx} +\usepackage{fullpage} +\usepackage[keeptemps=all, makestderr, usefamily={perl}]{pythontex} + + +\begin{document} + + + +\section*{Perl} + +\subsection*{Commands} + +\perl{2**8} + +\perlc{print 2**16;} + +\perlb{print 2**32;} + +\printpythontex + +\perlv{print 2**32;} + +\perls{\LaTeX\ and then \textcolor{blue}{!{"Perl"}} and back to \LaTeX.} + + +\subsection*{Environments} + +Code: +\begin{perlcode} +print "A string." . " "; +print 2**8; +\end{perlcode} + +Block: +\begin{perlblock} +print "A string." . " "; +print 2**8; +\end{perlblock} + +\printpythontex + +Verbatim: +\begin{perlverbatim} +print "A string." . " "; +print 2**8; +\end{perlverbatim} + +Sub: +\begin{perlsub} +\LaTeX\ and then \textcolor{blue}{!{"Perl"}} and back to \LaTeX. +\end{perlsub} + + +\section*{Perl stderr} + + +\begin{perlblock}[err1][numbers=left] +# Comment +my $s = "Perl a +\end{perlblock} + +\stderrpythontex[][breaklines, breakafter=\\/] + +\begin{perlblock}[err2][numbers=left] +1+; +\end{perlblock} + +\stderrpythontex[][breaklines, breakafter=\\/] + +\begin{perlblock}[err3][numbers=left] +# Comment +# Another comment +"a" "b"; +\end{perlblock} + +\stderrpythontex[][breaklines, breakafter=\\/] + + + +\end{document} + diff --git a/test/perl6/perl6_test.tex b/test/perl6/perl6_test.tex new file mode 100644 index 0000000..d0f1ac4 --- /dev/null +++ b/test/perl6/perl6_test.tex @@ -0,0 +1,127 @@ +\documentclass[11pt]{article} + +% Engine-specific settings +% pdftex: +\ifcsname pdfmatch\endcsname + \usepackage[T1]{fontenc} + \usepackage[utf8]{inputenc} +\fi +% xetex: +\ifcsname XeTeXinterchartoks\endcsname + \usepackage{fontspec} + \defaultfontfeatures{Ligatures=TeX} +\fi +% luatex: +\ifcsname directlua\endcsname + \usepackage{fontspec} +\fi +% End engine-specific settings + +\usepackage{lmodern} +\usepackage{amssymb,amsmath} +\usepackage{graphicx} +\usepackage{fullpage} +\usepackage[keeptemps=all, makestderr, usefamily={perlsix}]{pythontex} + + +\begin{document} + + + +\section*{Perl 6} + +\subsection*{Commands} + +\perlsix{2**8} + +\perlsixc{put 2**16;} + +\perlsixb{put 2**32;} + +\printpythontex + +\perlsixv{put 2**32;} + +\perlsixs{\LaTeX\ and then \textcolor{blue}{!{"Perl 6"}} and back to \LaTeX.} + + +\subsection*{Environments} + +Code: +\begin{perlsixcode} +put "A string. " ~ "More."; +put 2**8; +\end{perlsixcode} + +Block: +\begin{perlsixblock} +put "A string. " ~ "More."; +put 2**8; +\end{perlsixblock} + +\printpythontex + +Verbatim: +\begin{perlsixverbatim} +put "A string. " ~ "More."; +put 2**8; +\end{perlsixverbatim} + +Sub: +\begin{perlsixsub} +\LaTeX\ and then \textcolor{blue}{!{"Perl 6"}} and back to \LaTeX. +\end{perlsixsub} + + +\section*{Perl 6 stderr} + + +\begin{perlsixblock}[err1][numbers=left] +# Comment +my $s = "Perl a +\end{perlsixblock} + +\stderrpythontex[][breaklines, breakafter=\\/] + + +\begin{perlsixblock}[err2][numbers=left] +1+; +\end{perlsixblock} + +\stderrpythontex[][breaklines, breakafter=\\/] + + +\begin{perlsixblock}[err3][numbers=left] +# Comment +# Another comment +"a" "b"; +\end{perlsixblock} + +\stderrpythontex[][breaklines, breakafter=\\/] + + +\begin{perlsixblock}[err4][numbers=left] +# Comment +my $idx = "text"; +my @arr = <1, 2, 3>; +my $var = @arr[$idx]; +\end{perlsixblock} + +\stderrpythontex[][breaklines, breakafter=\\/] + + +\begin{perlsixblock}[err5][numbers=left] +sub nums {1 ... 1000} + +for nums { + say "Current element: $_"; +} +\end{perlsixblock} + +\stderrpythontex[][breaklines, breakafter=\\/] + + + + +\end{document} + diff --git a/test/ruby/ruby_test.tex b/test/ruby/ruby_test.tex new file mode 100644 index 0000000..dd4b3e8 --- /dev/null +++ b/test/ruby/ruby_test.tex @@ -0,0 +1,119 @@ +\documentclass[11pt]{article} + +% Engine-specific settings +% pdftex: +\ifcsname pdfmatch\endcsname + \usepackage[T1]{fontenc} + \usepackage[utf8]{inputenc} +\fi +% xetex: +\ifcsname XeTeXinterchartoks\endcsname + \usepackage{fontspec} + \defaultfontfeatures{Ligatures=TeX} +\fi +% luatex: +\ifcsname directlua\endcsname + \usepackage{fontspec} +\fi +% End engine-specific settings + +\usepackage{lmodern} +\usepackage{amssymb,amsmath} +\usepackage{graphicx} +\usepackage{fullpage} +\usepackage[keeptemps=all, makestderr, usefamily={ruby, rb}]{pythontex} + +\begin{document} + + +\section*{Ruby (\texttt{ruby})} + +\ruby{33*33} + +Code: +\begin{rubycode} +puts "Hello from Ruby!" +\end{rubycode} + +Block: +\begin{rubyblock} +puts "ABC" +\end{rubyblock} +Printed output: \printpythontex + +Verbatim: +\begin{rubyverbatim} +puts "Hello from Ruby! +\end{rubyverbatim} + +Sub: +\begin{rubysub} +In \LaTeX\textcolor{blue}{!{"...then Ruby..."}}and back in \LaTeX! +\end{rubysub} + + + +\section*{Ruby (\texttt{rb})} + +\rb{33*33} + +Code: +\begin{rbcode} +puts "Hello from Ruby!" +\end{rbcode} + +Block: +\begin{rbblock} +puts "ABC" +\end{rbblock} +Printed output: \printpythontex + +Verbatim: +\begin{rbverbatim} +puts "Hello from Ruby! +\end{rbverbatim} + +Sub: +\begin{rbsub} +In \LaTeX\textcolor{blue}{!{"...then Ruby..."}}and back in \LaTeX! +\end{rbsub} + + + +\subsection*{Errors} + +\setpythontexfv{numbers=left, firstnumber=last} + +\begin{rubyblock}[err1] +# Comment +s = <<-QQ Python a +\end{rubyblock} + +Should get no STDERR following this, because the sync system isn't yet sophisticated enough: \stderrpythontex + +\begin{rubyblock}[err1] +# Comment +#s = 'Python a +\end{rubyblock} + +\stderrpythontex + + +\begin{rubyblock}[err2][numbers=left] +1+ +\end{rubyblock} + +\stderrpythontex + + +\begin{rubyblock}[err3][numbers=left] +# Hmm +# Ah +1+'ab' +\end{rubyblock} + +\stderrpythontex + + +\end{document} + diff --git a/test/rust/rust_test.tex b/test/rust/rust_test.tex new file mode 100644 index 0000000..1823b4f --- /dev/null +++ b/test/rust/rust_test.tex @@ -0,0 +1,131 @@ +\documentclass[11pt]{article} + +% Engine-specific settings +% pdftex: +\ifcsname pdfmatch\endcsname + \usepackage[T1]{fontenc} + \usepackage[utf8]{inputenc} +\fi +% xetex: +\ifcsname XeTeXinterchartoks\endcsname + \usepackage{fontspec} + \defaultfontfeatures{Ligatures=TeX} +\fi +% luatex: +\ifcsname directlua\endcsname + \usepackage{fontspec} +\fi +% End engine-specific settings + +\usepackage{lmodern} +\usepackage{amssymb,amsmath} +\usepackage{graphicx} +\usepackage{fullpage} +\usepackage[keeptemps=all, makestderr, usefamily={rust, rs}]{pythontex} +\fvset{breaklines} + +\begin{document} + + +\section*{Rust (\texttt{rust})} + +Inline: \rust{format!("$3 + 5 = {}$", 3+5)}. \rusts{$4 + 6 = !{4+6}$}. + + +\begin{rustcode} +println!("Hello from Rust!"); +println!("Running command family ``{}'', session ``{}'', restart ``{}''.", rstex.family, rstex.session, rstex.restart); +\end{rustcode} + + + +\begin{rustblock} +println!("{}\\endinput", 2.0_f64.powf(8.0_f64)); +\end{rustblock} + +Printed output: \printpythontex. + +\begin{rustsub} +\color{blue} +\begin{Verbatim} +2.0_f64.powf(8.0_f64) = !{2.0_f64.powf(8.0_f64)} +\end{Verbatim} +\end{rustsub} + + +\section*{Rust (\texttt{rs})} + +Inline: \rs{format!("$3 + 5 = {}$", 3+5)}. \rss{$4 + 6 = !{4+6}$}. + + +\begin{rscode} +println!("Hello from Rust!"); +println!("Running command family ``{}'', session ``{}'', restart ``{}''.", rstex.family, rstex.session, rstex.restart); +\end{rscode} + + + +\begin{rsblock} +println!("{}\\endinput", 2.0_f64.powf(8.0_f64)); +\end{rsblock} + +Printed output: \printpythontex. + +\begin{rssub} +\color{blue} +\begin{Verbatim} +2.0_f64.powf(8.0_f64) = !{2.0_f64.powf(8.0_f64)} +\end{Verbatim} +\end{rssub} + + + +\section*{Errors} + +\begin{rustblock}[error] +println!("{}\\endinput", 2.0_f64.powf(8.0_f64); +\end{rustblock} + +\stderrpythontex + + + +\section*{File tracking} + +\begin{rustblock}[file_io] +// Open and write to two files, to check created file tracking +let mut my_file1 = io::BufWriter::new(rstex.open("my_file1.txt", open_mode::W) + .unwrap()); +writeln!(my_file1, "This is a text file.").unwrap(); +writeln!(my_file1, "Some more text.").unwrap(); +writeln!(my_file1, "3 + 2 = {}", 3 + 2).unwrap(); +drop(my_file1); + +let mut my_file2 = io::BufWriter::new(rstex.open("my_file2.txt", open_mode::W) + .unwrap()); +writeln!(my_file2, "This is another text file.").unwrap(); +writeln!(my_file2, "Some more text for another text file.").unwrap(); +writeln!(my_file2, "5 + 8 = {}", 5 + 8).unwrap(); +drop(my_file2); + + +// Open and read from LaTeX file, to check dependency tracking +let in_file = "rust_test.tex"; +println!("Reading from file \"{}\"...", in_file); +for (n, line) in io::BufReader::new(rstex.open(in_file, open_mode::R) + .unwrap()) + .lines() + .map(Result::unwrap) + .enumerate() +{ + if n < 8 { + println!("line {}: {}", n + 1, line); + } +} +\end{rustblock} + +Text written to stdout: + +\printpythontex[verbatim] + +\end{document}