Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,7 @@ jobs:
- tests/mock_vws/test_invalid_json.py::TestInvalidJSON::test_invalid_json
- tests/mock_vws/test_invalid_json.py::TestInvalidJSON::test_invalid_json_with_skewed_time
- tests/mock_vws/test_target_list.py
- tests/mock_vws/test_reco_counts_report.py
- tests/mock_vws/test_target_raters.py
- tests/mock_vws/test_target_summary.py
- tests/mock_vws/test_unexpected_json.py
Expand Down
32 changes: 32 additions & 0 deletions docs/source/differences-to-vws.rst
Original file line number Diff line number Diff line change
Expand Up @@ -201,6 +201,38 @@ Two Model Target Web API error paths remain mock-only in ``tests/mock_vws/test_m
Downloads of still-processing datasets are mock-only because exercising the path against real Vuforia would require creating a dataset on every test run; the mock drives the processing window deterministically.
Advanced-dataset creation with more than 20 models is mock-only because the available test account lacks the advanced-dataset scope and real Vuforia rejects the request with a 403 before validating model counts.

Reco counts reports
-------------------

The mock does not count recognitions, so a generated reco counts report
contains only the ``target_id,reco_count`` header row.
The mock returns the same report for the current month and the previous month.

The mock does not use the database ID in the request path.
It uses the database which matches the request's server keys, and it accepts
any database ID.
Real Vuforia returns a 401 response for a request which is signed with valid
server keys but which names a database ID that those keys do not belong to.

Real Vuforia returns a presigned URL for cloud storage, and the report takes
between a few seconds and one hour to generate.
The mock returns a URL served by the mock itself, without the query
parameters of a presigned URL, and the report takes
:paramref:`~mock_vws.MockVWS.processing_time_seconds` seconds to generate.
The URL returned by the Flask and Docker mock is built from the
:envvar:`VWS_BASE_URL` environment variable.
As with real Vuforia, the URL returns a 404 response until the report is
ready, and it requires no authorization.

The whole endpoint is mock-only in
``tests/mock_vws/test_reco_counts_report.py``, because the test credentials do
not include a database ID and so a request cannot be made which real Vuforia
authenticates.
Nothing about it has been verified against real Vuforia: not the ``Fail``
result code returned for a ``month`` which is not in the ``YYYY-mm`` form or
which is neither the current month nor the previous month, not the columns of
the CSV report, and not the headers of either response.

Header cases
------------

Expand Down
7 changes: 7 additions & 0 deletions docs/source/docker.rst
Original file line number Diff line number Diff line change
Expand Up @@ -138,6 +138,13 @@ VWS container

Default: ``2.0``

.. envvar:: VWS_BASE_URL

The base URL which clients use to reach the VWS container.
The download URL of a reco counts report is built from this URL.

Default: ``https://vws.vuforia.com``

.. envvar:: DUPLICATES_IMAGE_MATCHER

The matcher to use for the duplicates endpoint.
Expand Down
1 change: 1 addition & 0 deletions newsfragments/reco-counts-report.change
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Add the reco counts report endpoint, and a download URL for the generated CSV report.
4 changes: 4 additions & 0 deletions spelling_private_dict.txt
Original file line number Diff line number Diff line change
@@ -1,8 +1,10 @@
CSV
KiB
MPixel
MiB
MissingSchema
OAuth
Reco
Ubuntu
VuMark
admin
Expand Down Expand Up @@ -76,12 +78,14 @@ pdict
plugins
png
pragma
presigned
processable
pyrefly
pyright
pytest
readme
readthedocs
reco
recognitions
refactoring
regex
Expand Down
60 changes: 58 additions & 2 deletions src/mock_vws/_flask_server/vws.py
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,10 @@
from mock_vws._model_target_web_api import (
oauth2_token as model_target_oauth2_token,
)
from mock_vws._reco_counts_web_api import create_reco_counts_report
from mock_vws._reco_counts_web_api import (
download_reco_counts_report as download_report,
)
from mock_vws._services_validators import run_services_validators
from mock_vws._services_validators.exceptions import (
FailError,
Expand Down Expand Up @@ -93,6 +97,9 @@ class VWSSettings(BaseSettings):
target_manager_base_url: str
processing_time_seconds: float = 2.0
vws_host: str = ""
# The base URL which clients use to reach this application.
# Generated reco counts reports are served from this URL.
vws_base_url: str = "https://vws.vuforia.com"
duplicates_image_matcher: _ImageMatcherChoice = (
_ImageMatcherChoice.STRUCTURAL_SIMILARITY
)
Expand Down Expand Up @@ -189,11 +196,16 @@ def validate_request() -> None:

The VuMark endpoint does its own validation because it needs to
authenticate against both cloud and VuMark databases.

Reco counts report downloads stand in for presigned URLs, which are not
authorized with VWS credentials.
"""
if request.endpoint == "generate_vumark_instance":
return
if request.path == "/oauth2/token" or request.path.startswith(
"/modeltargets/",
if (
request.path == "/oauth2/token"
or request.path.startswith("/modeltargets/")
or request.path.startswith("/reports/recoCounts/")
):
return
run_services_validators(
Expand Down Expand Up @@ -385,6 +397,50 @@ def delete_advanced_model_target_dataset(dataset_uuid: str) -> Response:
)


@VWS_FLASK_APP.route(
rule="/imagetargets/databases/<string:database_id>/reports/recoCounts",
methods=[HTTPMethod.POST],
)
@beartype
def reco_counts_report(database_id: str) -> Response:
"""Request a reco counts report for a database.

Fake implementation of
https://developer.vuforia.com/library/web-api/cloud-targets-web-services-api
"""
# The mock authenticates with the request's server keys, so the database
# ID in the path is not used.
del database_id
settings = VWSSettings.model_validate(obj={})
return _to_flask_response(
api_response=create_reco_counts_report(
request_body=request.data,
target_manager=TARGET_MANAGER,
generation_time_seconds=settings.processing_time_seconds,
base_url=settings.vws_base_url.rstrip("/"),
),
)


@VWS_FLASK_APP.route(
rule="/reports/recoCounts/<string:report_id>",
methods=[HTTPMethod.GET],
)
@beartype
def download_reco_counts_report(report_id: str) -> Response:
"""Download a generated reco counts report.

This stands in for the presigned URL which real Vuforia returns, so it
does not require any authorization.
"""
return _to_flask_response(
api_response=download_report(
target_manager=TARGET_MANAGER,
report_id=report_id,
),
)


@VWS_FLASK_APP.route(rule="/targets", methods=[HTTPMethod.POST])
@beartype
def add_target() -> Response:
Expand Down
9 changes: 9 additions & 0 deletions src/mock_vws/_mock_common.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,15 @@

from beartype import beartype

# A database ID as it appears in the path of a reco counts report request.
DATABASE_ID_PATTERN = "[A-Za-z0-9_-]+"
# The path of the endpoint which requests a reco counts report.
RECO_COUNTS_REPORT_PATH_PATTERN = (
f"/imagetargets/databases/{DATABASE_ID_PATTERN}/reports/recoCounts"
)
# The path which stands in for a reco counts report presigned URL.
RECO_COUNTS_DOWNLOAD_PATH_PATTERN = "/reports/recoCounts/[A-Za-z0-9]+"


@beartype
class MissingSchemeError(Exception):
Expand Down
163 changes: 163 additions & 0 deletions src/mock_vws/_reco_counts_web_api.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,163 @@
"""A fake implementation of the Vuforia reco counts report endpoints."""

import datetime
import email.utils
import json
import logging
import re
import uuid
from http import HTTPStatus
from typing import Any
from zoneinfo import ZoneInfo

from beartype import beartype

from mock_vws._constants import ResultCodes
from mock_vws._mock_common import json_dump
from mock_vws._services_validators.exceptions import FailError
from mock_vws.reco_counts import RecoCountsReport
from mock_vws.target_manager import TargetManager

_ResponseType = tuple[int, dict[str, str], str | bytes]
_LOGGER = logging.getLogger(name=__name__)
_MONTH_PATTERN = re.compile(pattern=r"[0-9]{4}-[0-9]{2}")


@beartype
def _headers(*, content_type: str, content_length: int) -> dict[str, str]:
"""Return response headers which match other VWS endpoints."""
date = email.utils.formatdate(timeval=None, localtime=False, usegmt=True)
return {
"Connection": "keep-alive",
"Content-Length": str(object=content_length),
"Content-Type": content_type,
"Date": date,
"server": "envoy",
"x-envoy-upstream-service-time": "5",
"strict-transport-security": "max-age=31536000",
"x-aws-region": "us-east-2, us-west-2",
"x-content-type-options": "nosniff",
}


@beartype
def _download_headers(
*, content_type: str, content_length: int
) -> dict[str, str]:
"""Return response headers for a report download.

Real Vuforia serves reports from cloud storage, so these do not match the
headers of the VWS API.
"""
date = email.utils.formatdate(timeval=None, localtime=False, usegmt=True)
return {
"Content-Length": str(object=content_length),
"Content-Type": content_type,
"Date": date,
}


@beartype
def _months_in_range() -> set[str]:
"""Return the months which a report can be requested for.

Only the current month and the previous month can be requested.
"""
now = datetime.datetime.now(tz=ZoneInfo(key="UTC"))
first_of_month = now.replace(day=1)
last_of_previous_month = first_of_month - datetime.timedelta(days=1)
return {
now.strftime(format="%Y-%m"),
last_of_previous_month.strftime(format="%Y-%m"),
}


@beartype
def create_reco_counts_report(
*,
request_body: bytes,
target_manager: TargetManager,
generation_time_seconds: float,
base_url: str,
) -> _ResponseType:
"""Request a reco counts report for a database.

Args:
request_body: The body of the request.
target_manager: The target manager which stores generated reports.
generation_time_seconds: The number of seconds before a generated
report is available to download.
base_url: The base URL to serve the generated report from.

Returns:
A response which includes a URL to download the report from.

Raises:
FailError: The given month is not a month in the ``YYYY-mm`` form
which the report can be requested for.
"""
request_json: dict[str, Any] = json.loads(s=request_body)
month = request_json["month"]
if not isinstance(month, str) or not _MONTH_PATTERN.fullmatch(
string=month,
):
_LOGGER.warning(msg='The given "month" is not in the YYYY-mm form.')
raise FailError(status_code=HTTPStatus.BAD_REQUEST)

if month not in _months_in_range():
_LOGGER.warning(
msg=(
'The given "month" is not the current month or the previous '
"month."
),
)
raise FailError(status_code=HTTPStatus.BAD_REQUEST)

report = RecoCountsReport(
generation_time_seconds=generation_time_seconds,
)
target_manager.add_reco_counts_report(reco_counts_report=report)

body = {
"result_code": ResultCodes.SUCCESS.value,
"transaction_id": uuid.uuid4().hex,
"presigned_url": f"{base_url}/reports/recoCounts/{report.uuid_}",
}
body_json = json_dump(body=body)
headers = _headers(
content_type="application/json",
content_length=len(body_json),
)
return HTTPStatus.OK, headers, body_json


@beartype
def download_reco_counts_report(
*,
target_manager: TargetManager,
report_id: str,
) -> _ResponseType:
"""Download a generated reco counts report.

Args:
target_manager: The target manager which stores generated reports.
report_id: The identifier of the report to download.

Returns:
The CSV content of the report, or a 404 response while the report is
not ready.
"""
report = target_manager.reco_counts_reports.get(report_id)
if report is None or not report.is_available:
return (
HTTPStatus.NOT_FOUND,
_download_headers(content_type="text/plain", content_length=0),
"",
)

body = report.csv_content
headers = _download_headers(
content_type="text/csv",
content_length=len(body),
)
return HTTPStatus.OK, headers, body
Loading