Skip to content

test: modernize tests to use pytest - #2192

Merged
taddes merged 21 commits into
masterfrom
test/update-to-pytest-fixtures-STOR-518
Apr 30, 2026
Merged

taddes merged 21 commits into
masterfrom
test/update-to-pytest-fixtures-STOR-518

Conversation

@taddes

@taddes taddes commented Apr 6, 2026 •

Copy link
Copy Markdown
Collaborator

Description

Our integration and e2e tests are a hodgepodge of class-based unitTest (circa 2010 Python) and the newer pytest. This means we are leaving a lot of features on the table, like parallelized test runs and simpler maintenance with fixtures. This task updates our runs to use Pytest fully.

Testing

Our E2E & integration test runs

Issue(s)

Closes STOR-518.

@taddes taddes self-assigned this Apr 6, 2026
@taddes
taddes force-pushed the test/update-to-pytest-fixtures-STOR-518 branch 2 times, most recently from 912d942 to 7e97e86 Compare April 21, 2026 20:55
@taddes taddes changed the title [WIP] test: modernize tests to use pytest test: modernize tests to use pytest Apr 21, 2026
@taddes
taddes force-pushed the test/update-to-pytest-fixtures-STOR-518 branch 3 times, most recently from 365703d to b8d22a3 Compare April 23, 2026 10:35
@taddes
taddes marked this pull request as ready for review April 23, 2026 12:01
@taddes
taddes force-pushed the test/update-to-pytest-fixtures-STOR-518 branch 3 times, most recently from 814e8c7 to fc77303 Compare April 27, 2026 08:57
@taddes
taddes requested review from pjenvey April 27, 2026 10:21
MYSQL_PASSWORD: test
healthcheck:
test: ["CMD-SHELL", "mysqladmin -uroot -p$${MYSQL_ROOT_PASSWORD} version"]
test: ["CMD-SHELL", "mysqladmin -h 127.0.0.1 -P 3306 -uroot -p$${MYSQL_ROOT_PASSWORD} ping"]

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was done due to some recurring timeout issues. Found that forcing the health check to use TCP instead and increasing the max retry fixed it.

mysqladmin version without -h 127.0.0.1 connects via Unix socket, not TCP. So the healthcheck passes (socket is ready) before port 3306 is accepting TCP connections. Seemed we were getting 'Connection refused' on TCP. This race exists in the MySQL e2e too, but that job has a lighter stack without the spanner emulator competing for resources, so MySQL is fully ready before syncserver even starts. With spanner 1.5.52 being heavier, everything starts simultaneously and hits the race, I think.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see some discussion of this here on this issue, so this makes sense. Let's apply it to the other couple mysql health checks in our compose files.

There's another comment there claiming a 3s timeout as being "crucial", who knows if that's true/still the case but we could also bump to that from our 2s as well.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah good call, I even bumped the postgres ones up by a second so it's 3s across the board, unless a reason not to. Wonder if some of the intermittent issues are at all related to this as that link gives pretty compelling reasons.

@taddes
taddes force-pushed the test/update-to-pytest-fixtures-STOR-518 branch from fc77303 to 2e4fdc5 Compare April 27, 2026 18:04
MYSQL_PASSWORD: test
healthcheck:
test: ["CMD-SHELL", "mysqladmin -uroot -p$${MYSQL_ROOT_PASSWORD} version"]
test: ["CMD-SHELL", "mysqladmin -h 127.0.0.1 -P 3306 -uroot -p$${MYSQL_ROOT_PASSWORD} ping"]

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see some discussion of this here on this issue, so this makes sense. Let's apply it to the other couple mysql health checks in our compose files.

There's another comment there claiming a 3s timeout as being "crucial", who knows if that's true/still the case but we could also bump to that from our 2s as well.

# This Source Code Form is subject to the terms of the Mozilla Public
# License, v. 2.0. If a copy of the MPL was not distributed with this file,
# You can obtain one at http://mozilla.org/MPL/2.0/.
"""Fixtures and helpers for tokenserver integration tests.

@pjenvey pjenvey Apr 27, 2026 •

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This module replaces most of test_support.py I think, should that be killed now (or the non fixture functions could move there, but again I won't block on that)

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's land this if it's not a block and when I rebase with the large test_support and test_storage PR, I can remove that and/or move whatever standalone functions out

FXA_METRICS_HASH_SECRET = os.environ.get("SYNC_MASTER_SECRET", "secret0")


# ── DB-mode helper ───────────────────────────────────────────────────────────

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI my preference would be to only keep fixtures in this file with the other functions living elsewhere but I won't block on that.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, as above: let's land this if it's not a block and when I rebase with the large test_support and test_storage PR, I can remove that and/or move whatever standalone functions out.

@taddes
taddes force-pushed the test/update-to-pytest-fixtures-STOR-518 branch 2 times, most recently from 3808203 to 6a913e6 Compare April 30, 2026 15:46
@taddes
taddes merged commit 4283c36 into master Apr 30, 2026
26 checks passed
@taddes
taddes deleted the test/update-to-pytest-fixtures-STOR-518 branch April 30, 2026 20:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants