Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: gooddata/gooddata-python-sdk
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: rel/1.69.0
Choose a base ref
...
head repository: gooddata/gooddata-python-sdk
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: master
Choose a head ref
Checking mergeability… Don’t worry, you can still create the pull request.
  • 11 commits
  • 33 files changed
  • 6 contributors

Commits on Jun 25, 2026

  1. feat(eval): conversation tracking and --preserve-failed flag

    Add conversation_id and response_id fields to ChatResult for tracing
    eval conversations back to the server. Failed conversations can be
    preserved for post-mortem inspection via --preserve-failed.
    
    Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
    macekond and claude committed Jun 25, 2026
    Configuration menu
    Copy the full SHA
    63a79b6 View commit details
    Browse the repository at this point in the history

Commits on Jun 27, 2026

  1. fix(eval): fix search_tool correctness always scoring 0%

    _args_match checked emit_widget (renamed to user_requested_search in the
    tool schema) and limit (optional, server-side default). Both mismatched on
    every real model call, so tool_correctness was always False regardless of
    whether the model used the right keywords and object types.
    
    Fix: evaluate only keywords (case-insensitive) and object_types — the two
    fields that actually determine whether the search was semantically correct.
    Tomkess committed Jun 27, 2026
    Configuration menu
    Copy the full SHA
    2ebc8b0 View commit details
    Browse the repository at this point in the history

Commits on Jun 30, 2026

  1. build(deps): bump dev tooling (ty, ruff, pre-commit, tox, vcrpy, moto)

    Bump development and test tooling to current releases:
    - ty 0.0.40 -> 0.0.55
    - ruff 0.15.15 -> 0.15.20
    - pre-commit 4.0.1 -> 4.6.0
    - tox 4.32.0 -> 4.56.1, tox-uv 1.29.0 -> 1.35.2
    - vcrpy 8.0.0 -> 8.2.1 (root + sdk/pandas/fdw test groups)
    - moto 5.1.22 -> 5.2.2 (re-lock within existing >=5.1.6 spec)
    
    ty 0.0.55's improved inference flags the cast in MetricValueFilter.description
    as redundant, so drop it (and the now-unused typing.cast import). All other
    runtime deps left capped as-is; urllib3 stays pinned to the OpenAPI generator.
    
    Validated: make lint, format, type-check green; sdk/pandas/fdw/pipelines tests pass.
    
    jira: trivial
    risk: low
    hkad98 committed Jun 30, 2026
    Configuration menu
    Copy the full SHA
    e64214d View commit details
    Browse the repository at this point in the history

Commits on Jul 1, 2026

  1. chore(eval): fix lint violations flagged by ruff

    Apply ruff autofixes surfaced by `make lint-fix` across gooddata-eval:
    remove unused imports and re-sort import blocks, and add the missing
    `# noqa: PLC0415` to the deliberately function-local `timezone as _tz`
    imports (matching the `_dt` line directly above them).
    
    No behavior change. These violations went unnoticed because gooddata-eval
    is not yet wired into the test/lint harness (see follow-up commit).
    
    jira: trivial
    risk: nonprod
    hkad98 committed Jul 1, 2026
    Configuration menu
    Copy the full SHA
    7f97db7 View commit details
    Browse the repository at this point in the history
  2. test(eval): wire gooddata-eval into tox and fix stale unit tests

    gooddata-eval was the only package without a tox.ini, so its tests never
    ran via `make test`/CI. As a result the suite silently drifted from the
    source as the package was refactored (notably "move all agentic evaluation
    logic into gooddata_eval SDK"), accumulating 6 deterministic failures.
    
    Add a tox.ini mirroring the other packages, with `extras = llm-judge` so
    the optional `openai` dependency is installed (9 tests import it). Add
    pytest-json-report to the test group since the shared pytest command emits
    a JSON report.
    
    Update the 6 stale tests to match current, intentional source behavior:
    - langfuse test_kind inference now returns "vis_agentic" for visualization
      expected_output
    - JSON report run keys are provider-prefixed ("Provider/model") to stay
      collision-free across providers
    - ItemReport now requires a `question` field
    - run_agentic_visualization only deletes conversations it created; a
      caller-supplied initial_conversation_id is left intact
    
    All 216 eval tests pass via tox.
    
    jira: trivial
    risk: nonprod
    hkad98 committed Jul 1, 2026
    Configuration menu
    Copy the full SHA
    ecd1f66 View commit details
    Browse the repository at this point in the history
  3. build(deps): declare jinja2 explicitly for docs-scripts tests

    scripts/docs/python_ref_builder.py imports jinja2, but jinja2 was never
    declared in the workspace test group -- it was only present transitively.
    Bumping the dev tooling pruned that transitive provider, breaking the
    docs-scripts-tests CI job (uv sync --group test --locked; make
    test-docs-scripts) with "ModuleNotFoundError: No module named 'jinja2'".
    
    Add jinja2~=3.1 (matching scripts/script-requirements.txt) to the test
    group alongside the other scripts/docs dependencies so the synced env
    provides it.
    
    jira: trivial
    risk: nonprod
    hkad98 committed Jul 1, 2026
    Configuration menu
    Copy the full SHA
    10ea9f2 View commit details
    Browse the repository at this point in the history
  4. fix(eval): harden _args_match against malformed tool-call JSON

    parsed_arguments() returns raw model-emitted JSON, so a bad tool call
    like {"keywords":[1]} or mixed-type object_types would raise on
    .lower()/sorted() and abort the whole eval run instead of scoring
    tool_correctness=False. Add _normalize_str_list to drop non-string
    entries defensively; valid comparisons (incl. case-insensitive keyword
    match) are unchanged.
    
    Addresses CodeRabbit review on PR #1675.
    
    Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
    Tomkess and claude committed Jul 1, 2026
    Configuration menu
    Copy the full SHA
    f847679 View commit details
    Browse the repository at this point in the history
  5. Merge pull request #1672 from macekond/feat/eval-conversation-tracking

    feat(eval): conversation tracking and --preserve-failed flag
    hkad98 authored Jul 1, 2026
    Configuration menu
    Copy the full SHA
    bb355b3 View commit details
    Browse the repository at this point in the history

Commits on Jul 2, 2026

  1. Merge pull request #1676 from hkad98/jkd/deps

    build(deps): bump dev tooling + wire gooddata-eval into tox
    hkad98 authored Jul 2, 2026
    Configuration menu
    Copy the full SHA
    39d7676 View commit details
    Browse the repository at this point in the history
  2. docs(eval): clarify _args_match comments per review

    Address review nits on search_tool _args_match:
    - reword _normalize_str_list comment: dropping non-string entries prevents
      a crash, it does not force a mismatch (surviving strings still compare)
    - note that object_types is compared case-sensitively on purpose (controlled
      ObjectType StrEnum values emitted verbatim)
    
    No behavior change. keywords/object_types are declared list[str] in the
    search_objects schema, so string-collapse false-negatives cannot occur.
    
    JIRA: TRIVIAL
    risk: nonprod
    Tomkess committed Jul 2, 2026
    Configuration menu
    Copy the full SHA
    53dc5d2 View commit details
    Browse the repository at this point in the history

Commits on Jul 3, 2026

  1. Merge pull request #1675 from gooddata/fix/eval-search-tool-correctness

    fix(eval): fix search_tool correctness always scoring 0%
    Tomkess authored Jul 3, 2026
    Configuration menu
    Copy the full SHA
    2a72b14 View commit details
    Browse the repository at this point in the history
Loading