… changes

- test_agentic_visualization: caller-supplied conv is intentionally not
  deleted; update assertions to match the documented policy
- test_langfuse_source: {"visualization": {...}} now infers "vis_agentic",
  not "visualization"; update expected test_kind
- test_cli: JSON run keys now use "provider_name/model" format; update
  assertions in end-to-end and multi-model tests

JIRA: TRIVIAL
risk: nonprod

…rage

Discovery pass over gooddata-eval: expand fixtures to one item per
test_kind, add a cross-kind end-to-end runner test, winner-selection
assertions, a FEAT-002 dataset-contract test, and broad coverage for the
multi-turn agentic system (agentic_runner dispatch, _langfuse helpers,
evaluate_agentic_* orchestrators, simulated-response generators, and the
alert_skill/conversation pure helpers).

All tests run offline (OpenAI, Langfuse, and SSE mocked).
Total line coverage 69% -> 81%; 198 -> 274 tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Add ConversationStep and ChatResult.steps/reasoning_steps so the SSE parser
records every event the agent emits per turn (text, reasoning, tool calls,
tool results, visualizations) in arrival order, not just the final text and a
reasoning count. Consumed by the synthetic conversations generator (FEAT-002).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Two bugs causing empty detail:{} for agentic_conversation items:

1. _activated_skills() read "skills" key from set_skills args; the actual
   tool sends "skill_names" — so skill routing was always False.

2. run_agentic_items() never set item_report.best_detail for agentic kinds.
   agentic_conversation now calls run_agentic_conversation() directly and
   returns a detail dict (conversation_success, full_skill_coverage,
   per-turn skill_routing/output_present/output_correct); the runner sets
   best_detail and pass_at_k from that dict instead of the raise-on-fail path.

JIRA: TRIVIAL
risk: nonprod

…-up routing

#1 max_clarification_turns: default was 20 (caused 462s timeout on GPT
conv-002). Now defaults to 3, exposed as --max-clarification-turns CLI flag
threaded through RunConfig -> run_agentic_items -> run_agentic_conversation.

#2 skill_routing leniency for follow-up turns: turn 0 requires explicit
set_skills (strict); turns 1+ check output_present only — the model carries
skill state from the prior turn, consistent with ToolSandbox milestone
evaluation and COMPASS conversation-level scoring. TurnResult gains turn_index
field; skill_success property branches on it.

JIRA: TRIVIAL
risk: nonprod

The old check fired on "?" or "please" anywhere in the response, causing
false positives on normal completions like "I'll create the metric now,
please wait." — injecting a simulated reply that confused the model and
prevented it from producing the expected output.

New rule: only trigger if the last non-empty line ends with "?". Genuine
clarification requests end the response with a question; completions that
happen to contain "please" or a mid-sentence "?" do not.

JIRA: TRIVIAL
risk: nonprod

_is_asking_clarification was checking only the very last non-empty line.
When models respond with a clarification question followed by bullet-point
options (e.g. 'Which metric?\n- A\n- B'), the last line is a bullet item
and the check returned False, causing the loop to break prematurely.

Fix: walk backwards skipping bullet/numbered-list lines; the first non-list
line is the effective question line and is checked for a trailing '?'.

Also expose all_tool_calls and text_response in TurnResult and the JSON
report for turn-level diagnostics.

JIRA: TRIVIAL
risk: nonprod

The previous bullet-skipping heuristic failed when the model follows a
question with a non-list continuation sentence on the last line (e.g.
'Which version?\n\nIf you choose (2), tell me the ID.').

Since we only call _is_asking_clarification when the expected output has NOT
been produced, any '?' in the response reliably indicates the model is
awaiting user input rather than being a rhetorical flourish in a complete
answer.

JIRA: TRIVIAL
risk: nonprod

…uestion

Previously the loop broke when output was absent and the text had no '?'.
Models like GPT-5.2 sometimes produce a confirmation statement ('confirm OK')
rather than a question, then create the metric only if given another turn.

Change: always use a clarification slot when output is absent and budget
allows — if the response has '?' send a contextual simulated reply, otherwise
send the neutral 'Please proceed and complete the task.' nudge.

JIRA: TRIVIAL
risk: nonprod

Two optional provenance fields so eval results carry enough context to
know which workspace and skill category a conversation was testing —
without having to cross-reference the original YAML archive.

JIRA: TRIVIAL
risk: nonprod

--skip-ids accepts a file of item IDs (one per line) to exclude from
the current run — enables incremental re-runs after partial failures
without re-evaluating already-completed items.

main() now auto-loads a .env file via python-dotenv when present so
API keys do not need to be exported manually in each shell session.
python-dotenv added to the llm-judge optional dependency group.

…ualization config flags

anomaly_detection, clustering, and forecasting no longer have dedicated
execute_* tool calls. Output presence and correctness are now checked
via the config flags (anomaly_detection_enabled / clustering_enabled /
forecast_enabled) on create_adhoc_visualization.

_check_viz_skill_activated and _check_viz_skill_metric helpers
centralise that logic so _check_output_present and _check_output_correct
both resolve through the same path.

Also tightens _get_sim_user_response: removes the OpenAI GPT fallback
(which prolonged clarification loops with vague replies) and replaces it
with deterministic, metric-ref-anchored responses for each skill type.
key_driver_analysis and what_if_analysis now also include date attribute
guidance when available.

…ation loops

Pass exact {metric/id} and {label/id} URI identifiers into the sim-user
prompt instead of display names so the agent can match the option from
its list without guessing. System prompt now tells the user to respond
with the identifier directly. Temperature lowered to 0.3 for more
deterministic option selection.

… guardrail eval

agentic_runner: treat SSE 502 (gen-ai max_iterations reached) as a
scored FAIL rather than an infra error so it counts in pass rate stats.

metric_skill: nudge agent with an explicit create_metric instruction
when it returns a text-only MAQL description without calling the tool.

alert_skill: normalize expected trigger through _TRIGGER_DISPLAY_TO_API
before comparing — fixes false mismatches when the fixture uses display
names for the expected value.

runner: synthesize a cyber_policy refusal response for guardrail items
so the LLM judge can score the rejection as PASS instead of erroring out.

models: make CreatedVisualization.type optional to handle API responses
that omit the field.

_args_match checked emit_widget (renamed to user_requested_search in the
tool schema) and limit (optional, server-side default). Both mismatched on
every real model call, so tool_correctness was always False regardless of
whether the model used the right keywords and object types.

Fix: evaluate only keywords (case-insensitive) and object_types — the two
fields that actually determine whether the search was semantically correct.