Skip to content

Commit 9a1bfbe

Browse files
kushalkolarkushalkolar
authored
simplify screenshot testing (#447)
* rgb and rmse funcs work * increase screenshot rmse threshold to 0.025 * drop py3.9, add mac actions * workflow name * skip mac CI for now * just use numpy to convert RGBA -> RGB * rgb for nb tests * forgot to remove rgba screenshot saving for nb * forgot to add rmse to nb test utils * more tweaks * allow setting rmse tolerance for nb tests * use slightly higher tolerance for iw nb tests * proper way to set tolerance for nb
1 parent af23b58 commit 9a1bfbe

File tree

78 files changed

+552
-175
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

78 files changed

+552
-175
lines changed

‎.github/workflows/ci.yml‎

Lines changed: 86 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -46,15 +46,13 @@ jobs:
4646
make html SPHINXOPTS="-W --keep-going"
4747
4848
test-build-full:
49-
name: Test examples, env with notebook and glfw
49+
name: Test Linux, notebook + glfw
5050
runs-on: bigmem
5151
if: ${{ !github.event.pull_request.draft }}
5252
strategy:
5353
fail-fast: false
5454
matrix:
5555
include:
56-
- name: Test py39
57-
pyversion: '3.9'
5856
- name: Test py310
5957
pyversion: '3.10'
6058
- name: Test py311
@@ -103,15 +101,13 @@ jobs:
103101
examples/notebooks/diffs
104102
105103
test-build-desktop:
106-
name: Test examples, env with only glfw
104+
name: Test Linux, only glfw
107105
runs-on: bigmem
108106
if: ${{ !github.event.pull_request.draft }}
109107
strategy:
110108
fail-fast: false
111109
matrix:
112110
include:
113-
- name: Test py39
114-
pyversion: '3.9'
115111
- name: Test py310
116112
pyversion: '3.10'
117113
- name: Test py311
@@ -156,3 +152,87 @@ jobs:
156152
name: screenshot-diffs
157153
path: |
158154
examples/desktop/diffs
155+
156+
# test-build-full-mac:
157+
# name: Test Mac, notebook + glfw
158+
# runs-on: macos-14
159+
# if: ${{ !github.event.pull_request.draft }}
160+
# strategy:
161+
# fail-fast: false
162+
# matrix:
163+
# include:
164+
# - name: Test py310
165+
# pyversion: '3.10'
166+
# - name: Test py311
167+
# pyversion: '3.11'
168+
# - name: Test py312
169+
# pyversion: '3.12'
170+
# steps:
171+
# - uses: actions/checkout@v3
172+
# with:
173+
# lfs: true
174+
# - name: Set up Python
175+
# uses: actions/setup-python@v3
176+
# with:
177+
# python-version: ${{ matrix.pyversion }}
178+
# - name: Install dev dependencies
179+
# run: |
180+
# python -m pip install --upgrade pip setuptools
181+
# # remove pygfx from install_requires, we install using pygfx@main
182+
# pip install -e ".["tests"]"
183+
# pip install git+https://github.com/pygfx/pygfx.git@main
184+
# - name: Show wgpu backend
185+
# run:
186+
# python -c "from examples.tests.testutils import wgpu_backend; print(wgpu_backend)"
187+
# - name: Test examples
188+
# run: |
189+
# pytest -v examples
190+
# pytest --nbmake examples/notebooks/
191+
# - uses: actions/upload-artifact@v3
192+
# if: ${{ failure() }}
193+
# with:
194+
# name: screenshot-diffs
195+
# path: |
196+
# examples/desktop/diffs
197+
# examples/notebooks/diffs
198+
#
199+
# test-build-glfw-mac:
200+
# name: Test Mac, glfw
201+
# runs-on: macos-14
202+
# if: ${{ !github.event.pull_request.draft }}
203+
# strategy:
204+
# fail-fast: false
205+
# matrix:
206+
# include:
207+
# - name: Test py310
208+
# pyversion: '3.10'
209+
# - name: Test py311
210+
# pyversion: '3.11'
211+
# - name: Test py312
212+
# pyversion: '3.12'
213+
# steps:
214+
# - uses: actions/checkout@v3
215+
# with:
216+
# lfs: true
217+
# - name: Set up Python
218+
# uses: actions/setup-python@v3
219+
# with:
220+
# python-version: ${{ matrix.pyversion }}
221+
# - name: Install dev dependencies
222+
# run: |
223+
# python -m pip install --upgrade pip setuptools
224+
# # remove pygfx from install_requires, we install using pygfx@main
225+
# pip install -e ".["tests-desktop"]"
226+
# pip install git+https://github.com/pygfx/pygfx.git@main
227+
# - name: Show wgpu backend
228+
# run:
229+
# python -c "from examples.tests.testutils import wgpu_backend; print(wgpu_backend)"
230+
# - name: Test examples
231+
# run: |
232+
# pytest -v examples
233+
# - uses: actions/upload-artifact@v3
234+
# if: ${{ failure() }}
235+
# with:
236+
# name: screenshot-diffs
237+
# path: |
238+
# examples/desktop/diffs

‎.github/workflows/screenshots.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -48,7 +48,7 @@ jobs:
4848
run: |
4949
# regenerate screenshots
5050
REGENERATE_SCREENSHOTS=1 pytest -v examples
51-
REGENERATE_SCREENSHOTS=1 pytest --nbmake examples/notebooks/
51+
FASTPLOTLIB_NB_TESTS=1 REGENERATE_SCREENSHOTS=1 pytest --nbmake examples/notebooks/
5252
- uses: actions/upload-artifact@v3
5353
if: always()
5454
with:

‎examples/desktop/screenshots/gridplot.png‎

Lines changed: 2 additions & 2 deletions
Loading

‎examples/desktop/screenshots/gridplot_non_square.png‎

Lines changed: 2 additions & 2 deletions
Loading

‎examples/desktop/screenshots/heatmap.png‎

Lines changed: 2 additions & 2 deletions
Loading

‎examples/desktop/screenshots/heatmap_cmap.png‎

Lines changed: 2 additions & 2 deletions
Loading

‎examples/desktop/screenshots/heatmap_data.png‎

Lines changed: 2 additions & 2 deletions
Loading

‎examples/desktop/screenshots/heatmap_vmin_vmax.png‎

Lines changed: 2 additions & 2 deletions
Loading

‎examples/desktop/screenshots/image_cmap.png‎

Lines changed: 2 additions & 2 deletions
Loading

‎examples/desktop/screenshots/image_rgb.png‎

Lines changed: 2 additions & 2 deletions
Loading

0 commit comments

Comments
 (0)