Skip to content

Every Model Target test which needs an access token is xfailed because the OAuth2 credentials are revoked #3441

Description

@adamtheturtle

Follow-up to #3440.

What happens

The Model Target Web API credentials in the committed secrets archive are no longer accepted by real Vuforia. get_access_token in tests/mock_vws/fixtures/model_target_prepared_requests.py handles this with an imperative pytest.xfail:

if (
    backend == VuforiaBackend.REAL
    and response.status_code == HTTPStatus.UNAUTHORIZED
    and response.json() == {"error": "invalid_client"}
):
    pytest.xfail(reason=...)

So the whole authenticated half of the verified fake suite reports as xfailed and CI stays green. Note that xfail_strict = true in pyproject.toml does not help here: it only applies to the pytest.mark.xfail marker, not to an imperative pytest.xfail() call, which can never XPASS.

What I checked

Decrypting secrets.tar.gpg and requesting a token directly, today:

POST https://vws.vuforia.com/oauth2/token   (grant_type=client_credentials)
401  {"error":"invalid_client"}

The values are well formed — client ID 21 characters, secret 33, no stray whitespace — and all 100 ci_secrets/vuforia_secrets_*.env files share the same MODEL_TARGET_VUFORIA_CLIENT_ID, so this is revocation rather than a bad file. The VWS credentials in the same archive still work, so the archive itself is fine.

Running the real-Vuforia parametrisations of tests/mock_vws/test_model_target_web_api.py gives an exact split:

102 passed, 102 xfailed
Verified against real Vuforia (102) Not verified (102)
TestAuthorizationHeader::test_invalid_bearer_token (88) TestInvalidJson::test_body_not_json_object (40)
TestAuthorizationHeader::test_missing (8) TestErrorResponses::test_invalid_dataset_request (33)
TestAuthentication::test_invalid_oauth2_token_request (3) TestInvalidJson::test_wrong_content_type (8)
TestErrorResponses::test_invalid_basic_auth_header (2) TestInvalidJson::test_invalid_json (8)
TestInvalidJson::test_body_not_utf_8 (8)
TestErrorResponses::test_unknown_dataset (3)
TestStandardDataset::test_create_status_and_delete (1)
TestStandardDataset::test_create_with_cad_data_blob (1)

The pattern is clean: everything which deliberately sends a bad token or bad basic auth is genuinely verified, because it never needs a working one. Everything which needs a working token is not.

Why it matters

Every dataset creation validation rule the mock has asserts a Vuforia error string — error.expected.jsstring, error.expected.validenum, element is required — and none of those assertions currently run against Vuforia. The tests still pass against the mock, which means they assert that the mock agrees with itself.

This is load-bearing for the open Model Target work. #3193, #3194, #3195, #3196 and #3192 all ask for observed behaviour, and none of them can be closed honestly while the credentials are revoked. #3440 is a concrete instance: it had to assume the existing 400 / ERROR / Invalid Json: ... shape extends to a body which is not valid UTF-8, and had to leave the /oauth2/token lenient-decode behaviour mock-only, because neither could be observed.

Suggested resolution

Rotate the Model Target Web API credentials and regenerate the archive. admin/create_secrets_files.py already does this via vws_web_tools.get_model_target_web_api_details, which drives Selenium against the developer portal and creates a fresh OAuth2 client.

Consider also making the failure louder than an xfail. A single session-scoped check which errors once with "Model Target credentials are revoked, rotate them" would be more visible than 102 individually xfailed tests, and would not look like intentional coverage.

Acceptance criteria

  • The Model Target credentials in secrets.tar.gpg return a token from POST /oauth2/token.
  • The real-Vuforia parametrisations of tests/mock_vws/test_model_target_web_api.py report 204 passed, 0 xfailed, with the mock corrected wherever an observed response differs.
  • The two behaviours Reject Model Target request bodies which are not valid UTF-8 #3440 could not observe are confirmed or corrected: the response to a dataset creation body which is not valid UTF-8, and the response to an OAuth2 token request whose form body is not valid UTF-8. If the latter matches the mock, TestMockOnlyErrors::test_oauth2_token_body_not_utf_8 moves into a verified fake class and the note in docs/source/differences-to-vws.rst is removed.
  • The revoked-credential pytest.xfail branch in get_access_token is removed, or replaced by a loud single failure.

Dependencies

Rotation needs a Vuforia developer portal login, which is not stored in the repository. #3202 also needs credentials regenerated, against an Enterprise account for the advanced scope; if that account is available, doing both in one pass would close this and unblock that at the same time.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions