-
Notifications
You must be signed in to change notification settings - Fork 60
Expand file tree
/
Copy pathtesting.html
More file actions
392 lines (369 loc) · 28.6 KB
/
testing.html
File metadata and controls
392 lines (369 loc) · 28.6 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="X-UA-Compatible" content="IE=Edge" />
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>Testing — Matplotlib 3.4.3 documentation</title>
<link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
<link rel="stylesheet" href="../_static/mpl.css?v3.4.3-2-ge48e4acce0" type="text/css" />
<link rel="stylesheet" type="text/css" href="../_static/graphviz.css" />
<link rel="stylesheet" type="text/css" href="../_static/plot_directive.css" />
<link rel="stylesheet" type="text/css" href="../_static/copybutton.css" />
<link rel="stylesheet" type="text/css" href="../_static/gallery.css" />
<link rel="stylesheet" type="text/css" href="../_static/gallery-binder.css" />
<link rel="stylesheet" type="text/css" href="../_static/gallery-dataframe.css" />
<link rel="stylesheet" type="text/css" href="../_static/gallery-rendered-html.css" />
<script id="documentation_options" data-url_root="../" src="../_static/documentation_options.js"></script>
<script src="../_static/jquery.js"></script>
<script src="../_static/underscore.js"></script>
<script src="../_static/doctools.js"></script>
<script src="../_static/clipboard.min.js"></script>
<script src="../_static/copybutton.js"></script>
<script async="async" src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.7/latest.js?config=TeX-AMS-MML_HTMLorMML"></script>
<link rel="search" type="application/opensearchdescription+xml"
title="Search within Matplotlib 3.4.3 documentation"
href="../_static/opensearch.xml"/>
<link rel="shortcut icon" href="../_static/favicon.ico"/>
<link rel="index" title="Index" href="../genindex.html" />
<link rel="search" title="Search" href="../search.html" />
<link rel="next" title="Writing documentation" href="documenting_mpl.html" />
<link rel="prev" title="Setting up Matplotlib for development" href="development_setup.html" />
<link rel="top" title="Matplotlib 3.4.3 documentation" href="#" />
<link rel="canonical" href="https://matplotlib.org/stable/devel/testing.html" />
<link rel="stylesheet" href="../_static/custom.css" type="text/css" />
<meta name="viewport" content="width=device-width, initial-scale=0.9, maximum-scale=0.9" />
<script data-domain="matplotlib.org" defer="defer" src="https://views.scientific-python.org/js/script.js"></script>
</head><body>
<div id="unreleased-message"> You are reading an old version of the documentation (v3.4.3). For the latest version see <a href="https://matplotlib.org/stable/devel/testing.html">https://matplotlib.org/stable/devel/testing.html</a></div>
<!--
<div id="annc-banner">
</div>
-->
<div style="background-color: white; text-align: left; padding: 10px 10px 15px 15px; position: relative;">
<a href="../index.html">
<div style="float: left; position: absolute; width: 496px; bottom: 0; padding-bottom: 24px"><span style="float: right; color: #789; background: white">Version 3.4.3</span></div>
<img src="../_static/logo2_compressed.svg" height="125px" border="0" alt="matplotlib"/></a>
<!-- The "Fork me on github" ribbon -->
<div id="forkongithub"><a href="https://github.com/matplotlib/matplotlib">Fork me on GitHub</a></div>
</div>
<nav class="main-nav">
<ul>
<li><a href="../users/installing.html">Installation</a></li>
<li><a href="../contents.html">Documentation</a></li>
<li><a href="../gallery/index.html">Examples</a></li>
<li><a href="../tutorials/index.html">Tutorials</a></li>
<li><a href="index.html">Contributing</a></li>
<li class="nav-right">
<form class="search" action="../search.html" method="get">
<input type="text" name="q" aria-labelledby="searchlabel" placeholder="Search" autocomplete="off" autocorrect="off" autocapitalize="off" spellcheck="false"/>
</form>
</li>
</ul>
</nav>
<div class="related" role="navigation" aria-label="related navigation">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="../genindex.html" title="General Index"
accesskey="I">index</a></li>
<li class="right" >
<a href="../py-modindex.html" title="Python Module Index"
>modules</a> |</li>
<li class="right" >
<a href="documenting_mpl.html" title="Writing documentation"
accesskey="N">next</a> |</li>
<li class="right" >
<a href="development_setup.html" title="Setting up Matplotlib for development"
accesskey="P">previous</a> |</li>
<li><a href="../index.html">home</a>| </li>
<li><a href="../contents.html">contents</a> »</li>
<li class="nav-item nav-item-1"><a href="index.html" accesskey="U">The Matplotlib Developers' Guide</a> »</li>
<li class="nav-item nav-item-this"><a href="">Testing</a></li>
</ul>
</div>
<div class="sphinxsidebar" role="navigation" aria-label="main navigation">
<div class="sphinxsidebarwrapper">
<h3><a href="../contents.html">Table of Contents</a></h3>
<ul>
<li><a class="reference internal" href="#">Testing</a><ul>
<li><a class="reference internal" href="#requirements">Requirements</a></li>
<li><a class="reference internal" href="#running-the-tests">Running the tests</a></li>
<li><a class="reference internal" href="#writing-a-simple-test">Writing a simple test</a></li>
<li><a class="reference internal" href="#random-data-in-tests">Random data in tests</a></li>
<li><a class="reference internal" href="#writing-an-image-comparison-test">Writing an image comparison test</a></li>
<li><a class="reference internal" href="#creating-a-new-module-in-matplotlib-tests">Creating a new module in matplotlib.tests</a></li>
<li><a class="reference internal" href="#using-github-actions-for-ci">Using GitHub Actions for CI</a></li>
<li><a class="reference internal" href="#using-tox">Using tox</a></li>
<li><a class="reference internal" href="#building-old-versions-of-matplotlib">Building old versions of Matplotlib</a></li>
<li><a class="reference internal" href="#testing-released-versions-of-matplotlib">Testing released versions of Matplotlib</a><ul>
<li><a class="reference internal" href="#install-additional-dependencies">Install additional dependencies</a></li>
<li><a class="reference internal" href="#obtain-the-reference-images">Obtain the reference images</a></li>
<li><a class="reference internal" href="#run-the-tests">Run the tests</a></li>
</ul>
</li>
</ul>
</li>
</ul>
<div id="sidebar-pagesource" role="note" aria-label="source link"
style="margin-top: 1.5em; padding-top: 0.1em; border-top: 1px solid #86989b">
<a href="../_sources/devel/testing.rst.txt"
style="color: #c0c0c0" rel="nofollow">Show Page Source</a>
</div>
</div>
</div>
<div class="document">
<div class="documentwrapper">
<div class="bodywrapper">
<div class="body" role="main">
<div class="section" id="testing">
<span id="id1"></span><h1>Testing<a class="headerlink" href="#testing" title="Permalink to this headline">¶</a></h1>
<p>Matplotlib uses the <a class="reference external" href="http://doc.pytest.org/en/latest/">pytest</a> framework.</p>
<p>The tests are in <code class="file docutils literal notranslate"><span class="pre">lib/matplotlib/tests</span></code>, and customizations to the pytest
testing infrastructure are in <a class="reference internal" href="../api/testing_api.html#module-matplotlib.testing" title="matplotlib.testing"><code class="xref py py-mod docutils literal notranslate"><span class="pre">matplotlib.testing</span></code></a>.</p>
<div class="section" id="requirements">
<span id="testing-requirements"></span><h2>Requirements<a class="headerlink" href="#requirements" title="Permalink to this headline">¶</a></h2>
<p>To run the tests you will need to
<a class="reference internal" href="development_setup.html#installing-for-devs"><span class="std std-ref">set up Matplotlib for development</span></a>. Note in
particular the <a class="reference internal" href="development_setup.html#test-dependencies"><span class="std std-ref">additional dependencies</span></a> for testing.</p>
<div class="admonition note">
<p class="first admonition-title">Note</p>
<p>We will assume that you want to run the tests in a development setup.</p>
<p class="last">While you can run the tests against a regular installed version of
Matplotlib, this is a far less common use case. You still need the
<a class="reference internal" href="development_setup.html#test-dependencies"><span class="std std-ref">additional dependencies</span></a> for testing.
You have to additionally get the reference images from the repository,
because they are not distributed with pre-built Matplotlib packages.</p>
</div>
</div>
<div class="section" id="running-the-tests">
<h2>Running the tests<a class="headerlink" href="#running-the-tests" title="Permalink to this headline">¶</a></h2>
<p>In the root directory of your development repository run:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">python</span> <span class="o">-</span><span class="n">m</span> <span class="n">pytest</span>
</pre></div>
</div>
<p>pytest can be configured via a lot of <a class="reference external" href="http://doc.pytest.org/en/latest/usage.html">command-line parameters</a>. Some
particularly useful ones are:</p>
<table border="1" class="docutils align-default">
<colgroup>
<col width="46%" />
<col width="54%" />
</colgroup>
<tbody valign="top">
<tr class="row-odd"><td><code class="docutils literal notranslate"><span class="pre">-v</span></code> or <code class="docutils literal notranslate"><span class="pre">--verbose</span></code></td>
<td>Be more verbose</td>
</tr>
<tr class="row-even"><td><code class="docutils literal notranslate"><span class="pre">-n</span> <span class="pre">NUM</span></code></td>
<td>Run tests in parallel over NUM
processes (requires <a class="reference external" href="https://pypi.org/project/pytest-xdist/">pytest-xdist</a>)</td>
</tr>
<tr class="row-odd"><td><code class="docutils literal notranslate"><span class="pre">--capture=no</span></code> or <code class="docutils literal notranslate"><span class="pre">-s</span></code></td>
<td>Do not capture stdout</td>
</tr>
</tbody>
</table>
<p>To run a single test from the command line, you can provide a file path,
optionally followed by the function separated by two colons, e.g., (tests do
not need to be installed, but Matplotlib should be):</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">pytest</span> <span class="n">lib</span><span class="o">/</span><span class="n">matplotlib</span><span class="o">/</span><span class="n">tests</span><span class="o">/</span><span class="n">test_simplification</span><span class="o">.</span><span class="n">py</span><span class="p">::</span><span class="n">test_clipping</span>
</pre></div>
</div>
<p>An alternative implementation that does not look at command line arguments
and works from within Python is to run the tests from the Matplotlib library
function <a class="reference internal" href="../api/testing_api.html#matplotlib.test" title="matplotlib.test"><code class="xref py py-func docutils literal notranslate"><span class="pre">matplotlib.test()</span></code></a>:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">matplotlib</span>
<span class="n">matplotlib</span><span class="o">.</span><span class="n">test</span><span class="p">()</span>
</pre></div>
</div>
</div>
<div class="section" id="writing-a-simple-test">
<h2>Writing a simple test<a class="headerlink" href="#writing-a-simple-test" title="Permalink to this headline">¶</a></h2>
<p>Many elements of Matplotlib can be tested using standard tests. For
example, here is a test from <code class="file docutils literal notranslate"><span class="pre">matplotlib/tests/test_basic.py</span></code>:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">def</span> <span class="nf">test_simple</span><span class="p">():</span>
<span class="sd">"""</span>
<span class="sd"> very simple example test</span>
<span class="sd"> """</span>
<span class="k">assert</span> <span class="mi">1</span> <span class="o">+</span> <span class="mi">1</span> <span class="o">==</span> <span class="mi">2</span>
</pre></div>
</div>
<p>Pytest determines which functions are tests by searching for files whose names
begin with <code class="docutils literal notranslate"><span class="pre">"test_"</span></code> and then within those files for functions beginning with
<code class="docutils literal notranslate"><span class="pre">"test"</span></code> or classes beginning with <code class="docutils literal notranslate"><span class="pre">"Test"</span></code>.</p>
<p>Some tests have internal side effects that need to be cleaned up after their
execution (such as created figures or modified <a class="reference internal" href="../api/matplotlib_configuration_api.html#matplotlib.rcParams" title="matplotlib.rcParams"><code class="xref py py-obj docutils literal notranslate"><span class="pre">rcParams</span></code></a>). The pytest fixture
<code class="docutils literal notranslate"><span class="pre">matplotlib.testing.conftest.mpl_test_settings</span></code> will automatically clean
these up; there is no need to do anything further.</p>
</div>
<div class="section" id="random-data-in-tests">
<h2>Random data in tests<a class="headerlink" href="#random-data-in-tests" title="Permalink to this headline">¶</a></h2>
<p>Random data is a very convenient way to generate data for examples,
however the randomness is problematic for testing (as the tests
must be deterministic!). To work around this set the seed in each test.
For numpy use:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span>
<span class="n">np</span><span class="o">.</span><span class="n">random</span><span class="o">.</span><span class="n">seed</span><span class="p">(</span><span class="mi">19680801</span><span class="p">)</span>
</pre></div>
</div>
<p>The seed is John Hunter's birthday.</p>
</div>
<div class="section" id="writing-an-image-comparison-test">
<h2>Writing an image comparison test<a class="headerlink" href="#writing-an-image-comparison-test" title="Permalink to this headline">¶</a></h2>
<p>Writing an image-based test is only slightly more difficult than a simple
test. The main consideration is that you must specify the "baseline", or
expected, images in the <a class="reference internal" href="../api/testing_api.html#matplotlib.testing.decorators.image_comparison" title="matplotlib.testing.decorators.image_comparison"><code class="xref py py-obj docutils literal notranslate"><span class="pre">image_comparison</span></code></a>
decorator. For example, this test generates a single image and automatically
tests it:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="kn">from</span> <span class="nn">matplotlib.testing.decorators</span> <span class="kn">import</span> <span class="n">image_comparison</span>
<span class="kn">import</span> <span class="nn">matplotlib.pyplot</span> <span class="k">as</span> <span class="nn">plt</span>
<span class="nd">@image_comparison</span><span class="p">(</span><span class="n">baseline_images</span><span class="o">=</span><span class="p">[</span><span class="s1">'line_dashes'</span><span class="p">],</span> <span class="n">remove_text</span><span class="o">=</span><span class="kc">True</span><span class="p">,</span>
<span class="n">extensions</span><span class="o">=</span><span class="p">[</span><span class="s1">'png'</span><span class="p">])</span>
<span class="k">def</span> <span class="nf">test_line_dashes</span><span class="p">():</span>
<span class="n">fig</span><span class="p">,</span> <span class="n">ax</span> <span class="o">=</span> <span class="n">plt</span><span class="o">.</span><span class="n">subplots</span><span class="p">()</span>
<span class="n">ax</span><span class="o">.</span><span class="n">plot</span><span class="p">(</span><span class="nb">range</span><span class="p">(</span><span class="mi">10</span><span class="p">),</span> <span class="n">linestyle</span><span class="o">=</span><span class="p">(</span><span class="mi">0</span><span class="p">,</span> <span class="p">(</span><span class="mi">3</span><span class="p">,</span> <span class="mi">3</span><span class="p">)),</span> <span class="n">lw</span><span class="o">=</span><span class="mi">5</span><span class="p">)</span>
</pre></div>
</div>
<p>The first time this test is run, there will be no baseline image to compare
against, so the test will fail. Copy the output images (in this case
<code class="file docutils literal notranslate"><span class="pre">result_images/test_lines/test_line_dashes.png</span></code>) to the correct
subdirectory of <code class="file docutils literal notranslate"><span class="pre">baseline_images</span></code> tree in the source directory (in this
case <code class="file docutils literal notranslate"><span class="pre">lib/matplotlib/tests/baseline_images/test_lines</span></code>). Put this new
file under source code revision control (with <code class="docutils literal notranslate"><span class="pre">git</span> <span class="pre">add</span></code>). When rerunning
the tests, they should now pass.</p>
<p>Baseline images take a lot of space in the Matplotlib repository.
An alternative approach for image comparison tests is to use the
<a class="reference internal" href="../api/testing_api.html#matplotlib.testing.decorators.check_figures_equal" title="matplotlib.testing.decorators.check_figures_equal"><code class="xref py py-obj docutils literal notranslate"><span class="pre">check_figures_equal</span></code></a> decorator, which should be
used to decorate a function taking two <a class="reference internal" href="../api/figure_api.html#matplotlib.figure.Figure" title="matplotlib.figure.Figure"><code class="xref py py-obj docutils literal notranslate"><span class="pre">Figure</span></code></a> parameters and draws the same
images on the figures using two different methods (the tested method and the
baseline method). The decorator will arrange for setting up the figures and
then collect the drawn results and compare them.</p>
<p>See the documentation of <a class="reference internal" href="../api/testing_api.html#matplotlib.testing.decorators.image_comparison" title="matplotlib.testing.decorators.image_comparison"><code class="xref py py-obj docutils literal notranslate"><span class="pre">image_comparison</span></code></a> and
<a class="reference internal" href="../api/testing_api.html#matplotlib.testing.decorators.check_figures_equal" title="matplotlib.testing.decorators.check_figures_equal"><code class="xref py py-obj docutils literal notranslate"><span class="pre">check_figures_equal</span></code></a> for additional information
about their use.</p>
</div>
<div class="section" id="creating-a-new-module-in-matplotlib-tests">
<h2>Creating a new module in matplotlib.tests<a class="headerlink" href="#creating-a-new-module-in-matplotlib-tests" title="Permalink to this headline">¶</a></h2>
<p>We try to keep the tests categorized by the primary module they are
testing. For example, the tests related to the <code class="docutils literal notranslate"><span class="pre">mathtext.py</span></code> module
are in <code class="docutils literal notranslate"><span class="pre">test_mathtext.py</span></code>.</p>
</div>
<div class="section" id="using-github-actions-for-ci">
<h2>Using GitHub Actions for CI<a class="headerlink" href="#using-github-actions-for-ci" title="Permalink to this headline">¶</a></h2>
<p><a class="reference external" href="https://docs.github.com/en/actions">GitHub Actions</a> is a hosted CI system
"in the cloud".</p>
<p>GitHub Actions is configured to receive notifications of new commits to GitHub
repos and to run builds or tests when it sees these new commits. It looks for a
YAML files in <code class="docutils literal notranslate"><span class="pre">.github/workflows</span></code> to see how to test the project.</p>
<p>GitHub Actions is already enabled for the <a class="reference external" href="https://github.com/matplotlib/matplotlib/">main Matplotlib GitHub repository</a> -- for example, see <a class="reference external" href="https://github.com/matplotlib/matplotlib/actions?query=workflow%3ATests">the Tests
workflows</a>.</p>
<p>GitHub Actions should be automatically enabled for your personal Matplotlib
fork once the YAML workflow files are in it. It generally isn't necessary to
look at these workflows, since any pull request submitted against the main
Matplotlib repository will be tested.</p>
<p>You can see the GitHub Actions results at
<a class="reference external" href="https://github.com/your_GitHub_user_name/matplotlib/actions">https://github.com/your_GitHub_user_name/matplotlib/actions</a> -- here's <a class="reference external" href="https://github.com/QuLogic/matplotlib/actions">an
example</a>.</p>
</div>
<div class="section" id="using-tox">
<h2>Using tox<a class="headerlink" href="#using-tox" title="Permalink to this headline">¶</a></h2>
<p><a class="reference external" href="https://tox.readthedocs.io/en/latest/">Tox</a> is a tool for running tests
against multiple Python environments, including multiple versions of Python
(e.g., 3.6, 3.7) and even different Python implementations altogether
(e.g., CPython, PyPy, Jython, etc.), as long as all these versions are
available on your system's $PATH (consider using your system package manager,
e.g. apt-get, yum, or Homebrew, to install them).</p>
<p>tox makes it easy to determine if your working copy introduced any
regressions before submitting a pull request. Here's how to use it:</p>
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>$ pip install tox
$ tox
</pre></div>
</div>
<p>You can also run tox on a subset of environments:</p>
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>$ tox -e py37,py38
</pre></div>
</div>
<p>Tox processes everything serially so it can take a long time to test
several environments. To speed it up, you might try using a new,
parallelized version of tox called <code class="docutils literal notranslate"><span class="pre">detox</span></code>. Give this a try:</p>
<div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>$ pip install -U -i http://pypi.testrun.org detox
$ detox
</pre></div>
</div>
<p>Tox is configured using a file called <code class="docutils literal notranslate"><span class="pre">tox.ini</span></code>. You may need to
edit this file if you want to add new environments to test (e.g.,
<code class="docutils literal notranslate"><span class="pre">py33</span></code>) or if you want to tweak the dependencies or the way the
tests are run. For more info on the <code class="docutils literal notranslate"><span class="pre">tox.ini</span></code> file, see the <a class="reference external" href="https://tox.readthedocs.io/en/latest/config.html">Tox
Configuration Specification</a>.</p>
</div>
<div class="section" id="building-old-versions-of-matplotlib">
<h2>Building old versions of Matplotlib<a class="headerlink" href="#building-old-versions-of-matplotlib" title="Permalink to this headline">¶</a></h2>
<p>When running a <code class="docutils literal notranslate"><span class="pre">git</span> <span class="pre">bisect</span></code> to see which commit introduced a certain bug,
you may (rarely) need to build very old versions of Matplotlib. The following
constraints need to be taken into account:</p>
<ul class="simple">
<li>Matplotlib 1.3 (or earlier) requires numpy 1.8 (or earlier).</li>
</ul>
</div>
<div class="section" id="testing-released-versions-of-matplotlib">
<h2>Testing released versions of Matplotlib<a class="headerlink" href="#testing-released-versions-of-matplotlib" title="Permalink to this headline">¶</a></h2>
<p>Running the tests on an installation of a released version (e.g. PyPI package
or conda package) also requires additional setup.</p>
<div class="admonition note">
<p class="first admonition-title">Note</p>
<p class="last">For an end-user, there is usually no need to run the tests on released
versions of Matplotlib. Official releases are tested before publishing.</p>
</div>
<div class="section" id="install-additional-dependencies">
<h3>Install additional dependencies<a class="headerlink" href="#install-additional-dependencies" title="Permalink to this headline">¶</a></h3>
<p>Install the <a class="reference internal" href="development_setup.html#test-dependencies"><span class="std std-ref">additional dependencies for testing</span></a>.</p>
</div>
<div class="section" id="obtain-the-reference-images">
<h3>Obtain the reference images<a class="headerlink" href="#obtain-the-reference-images" title="Permalink to this headline">¶</a></h3>
<p>Many tests compare the plot result against reference images. The reference
images are not part of the regular packaged versions (pip wheels or conda
packages). If you want to run tests with reference images, you need to obtain
the reference images matching the version of Matplotlib you want to test.</p>
<p>To do so, either download the matching source distribution
<code class="docutils literal notranslate"><span class="pre">matplotlib-X.Y.Z.tar.gz</span></code> from <a class="reference external" href="https://pypi.org/project/matplotlib/">PyPI</a>
or alternatively, clone the git repository and <code class="docutils literal notranslate"><span class="pre">git</span> <span class="pre">checkout</span> <span class="pre">vX.Y.Z</span></code>. Copy
the folder <code class="file docutils literal notranslate"><span class="pre">lib/matplotlib/tests/baseline_images</span></code> to the folder
<code class="file docutils literal notranslate"><span class="pre">matplotlib/tests</span></code> of your the matplotlib installation to test.
The correct target folder can be found using:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">python</span> <span class="o">-</span><span class="n">c</span> <span class="s2">"import matplotlib.tests; print(matplotlib.tests.__file__.rsplit('/', 1)[0])"</span>
</pre></div>
</div>
<p>An analogous copying of <code class="file docutils literal notranslate"><span class="pre">lib/mpl_toolkits/tests/baseline_images</span></code>
is necessary for testing the <a class="reference internal" href="../api/toolkits/index.html#toolkits"><span class="std std-ref">Toolkits</span></a>.</p>
</div>
<div class="section" id="run-the-tests">
<h3>Run the tests<a class="headerlink" href="#run-the-tests" title="Permalink to this headline">¶</a></h3>
<p>To run the all the tests on your installed version of Matplotlib:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">python</span> <span class="o">-</span><span class="n">m</span> <span class="n">pytest</span> <span class="o">--</span><span class="n">pyargs</span> <span class="n">matplotlib</span><span class="o">.</span><span class="n">tests</span>
</pre></div>
</div>
<p>The test discovery scope can be narrowed to single test modules or even single
functions:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">python</span> <span class="o">-</span><span class="n">m</span> <span class="n">pytest</span> <span class="o">--</span><span class="n">pyargs</span> <span class="n">matplotlib</span><span class="o">.</span><span class="n">tests</span><span class="o">.</span><span class="n">test_simplification</span><span class="o">.</span><span class="n">py</span><span class="p">::</span><span class="n">test_clipping</span>
</pre></div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
<div class="clearer"></div>
</div>
<footer>
<div class="footer">
© Copyright 2002 - 2012 John Hunter, Darren Dale, Eric Firing, Michael Droettboom and the Matplotlib development team; 2012 - 2021 The Matplotlib development team.
<br />
Last updated on Aug 13, 2021.
Created using
<a href="http://sphinx-doc.org/">Sphinx</a> 3.4.3.
Doc version v3.4.3-2-ge48e4acce0.
</div>
</footer>
</body>
</html>