Python: Honor replace usage snapshots in Foundry hosting by Mahdi-Hasan · Pull Request #10205 · microsoft/agent-framework

Mehedi Hasan (Mahdi-Hasan) · 2026-11-09T13:59:47Z

Motivation & Context

Foundry hosting reports the wrong token usage when an agent streams cumulative usage snapshots. _OutputItemTracker added every usage update, even ones with merge_mode="replace". Two snapshots of 100/10 and 250/25 were reported as 350/35 instead of 250/25.

Description & Review Guide

What are the major changes?
- In _OutputItemTracker.handle, a replace usage update now replaces the tracked usage. append updates are still added.
- This is the same rule as Content._add_usage_content and AgentResponse.from_updates.
What is the impact of these changes?
- Only the usage in the final response.completed event changes, and only when the agent sends replace updates.
- Agents that send only append updates get the same totals as before.
What do you want reviewers to focus on?
- The replaced value is a deepcopy, so the tracker never holds a reference to the update's usage_details.

Related Issue

Fixes #10189

AI Assistance

No material AI assistance was used.
This is an AI-assisted contribution. I reviewed, understood, and verified all submitted content and accept responsibility for it.

AI assistance details: implementation, tests, and this description.

Contribution Checklist

The code builds clean without any errors or warnings
All unit tests pass, and I have added new tests where possible
The PR follows the Contribution Guidelines
This PR links to an agreed issue with no competing open PR, or the Related Issue section documents a trivial-change or repository-automation exception.
This is not a breaking change. If it is a breaking change, add the breaking change label (or add "[BREAKING]" to the title prefix, before or after any language prefix) — a workflow keeps the label and title prefix in sync automatically.

🤖 Generated with Claude Code

_OutputItemTracker added every usage update with add_usage_details, even when the update had merge_mode="replace". Two cumulative snapshots of 100/10 and 250/25 were reported as 350/35. Replace the tracked usage for replace updates and keep adding append updates, the same rule as Content._add_usage_content and AgentResponse.from_updates. Fixes microsoft#10189 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Copilot

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

Copilot

🟡 Changes recommended

A moderate active finding remains because the added test does not verify that replacement usage details are copied rather than retained by reference.

1 open issue

Test that replacement usage is copied · New

_{Review effort: Lite}

* Add agent-framework #10205; bump OSS count to 61 PRs - Foundry hosting fix for replace usage snapshots (microsoft/agent-framework#10205) - Projects stay at 19: agent-framework is already listed Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Add Nov 9 PRs; bump OSS counts to 70 PRs / 21 projects - quran.com-frontend-next #3368-#3370 - New projects: quranic-universal-library #1021-#1023, hadith-api #144-#146 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

The merge-mode test only checked totals, so it would still pass if the tracker kept the update's usage_details by reference. Mutate the update after handle() and check the tracker's usage is unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Eduard van Valkenburg (eavanvalkenburg) · 2026-11-10T10:26:02Z

-            self._usage_details = add_usage_details(self._usage_details, content.usage_details)
+            # A "replace" update is a cumulative snapshot, so it supersedes earlier usage instead of adding to it.
+            if content.merge_mode == "replace":
+                self._usage_details = deepcopy(content.usage_details)


Mehedi Hasan (@Mahdi-Hasan), this replaces the tracker-wide usage slot, so a later executor's snapshot erases usage already accumulated from earlier agents in a hosted workflow. For example, agent A reporting 1000/100 followed by agent B reporting 50/5 produces 50/5 instead of 1050/105. Can we scope replacement to the same logical agent/response stream and add a multi-executor regression?

Fixed in d68981f. The tracker now keeps usage per (author_name, response_id) stream. A replace snapshot overwrites only its own stream, and the reported usage is the sum over all streams. A re-run of the same agent gets a new response_id, so both runs count.

Added TestWorkflowAgentHosting::test_replace_usage_from_each_executor_is_summed: two chained agents report 1000/100 and 50/5, and the response reports 1050/105. Before the fix it reported 50/5. There is also a tracker-level test, test_output_item_tracker_scopes_replace_usage_to_its_stream.

A hosted workflow sends every executor's updates through one _OutputItemTracker. A replace snapshot from a later agent erased usage already tracked for earlier agents: 1000/100 then 50/5 gave 50/5 instead of 1050/105. Track usage per (author_name, response_id) stream. A replace update overwrites only its own stream and the reported usage is the sum over all streams. Add a tracker test and a two-executor workflow regression. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Mehedi Hasan (Mahdi-Hasan) requested review from SergeyMenshykh, Darren Cohen (dargilco), Eduard van Valkenburg (eavanvalkenburg), Jose Alvarez (jpalvarezl), Roger Barreto (rogerbarreto), sophia-ramsey and westey (westey-m) as code owners November 9, 2026 13:59

Copilot AI balanced review requested due to automatic review settings November 9, 2026 13:59

Mehedi Hasan (Mahdi-Hasan) temporarily deployed to github-app-auth November 9, 2026 13:59 — with GitHub Actions Inactive

Copilot AI reviewed Nov 9, 2026

Mehedi Hasan (Mahdi-Hasan) temporarily deployed to github-app-auth November 9, 2026 14:00 — with GitHub Actions Inactive

agent-framework-automation Bot added the python Usage: [Issues, PRs], Target: Python label Nov 9, 2026

Mehedi Hasan (Mahdi-Hasan) requested a lite review from Copilot November 9, 2026 14:03

Copilot started reviewing on behalf of Mehedi Hasan (Mahdi-Hasan) November 9, 2026 14:04

View session

Copilot AI reviewed Nov 9, 2026

View reviewed changes

Comment thread python/packages/foundry_hosting/agent_framework_foundry_hosting/_responses.py Resolved

Mehedi Hasan (Mahdi-Hasan) temporarily deployed to github-app-auth November 9, 2026 14:12 — with GitHub Actions Inactive

Eduard van Valkenburg (eavanvalkenburg) reviewed Nov 10, 2026

View reviewed changes

Mehedi Hasan (Mahdi-Hasan) temporarily deployed to github-app-auth November 10, 2026 11:34 — with GitHub Actions Inactive

Navigation Menu

Repository navigation

Python: Honor replace usage snapshots in Foundry hosting - #10205

Python: Honor replace usage snapshots in Foundry hosting#10205
Mehedi Hasan (Mahdi-Hasan) wants to merge 3 commits into
microsoft:mainfrom
Mahdi-Hasan:fix/10189-hosting-usage-replace

Mehedi Hasan (Mahdi-Hasan) commented Nov 9, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Eduard van Valkenburg (eavanvalkenburg) Nov 10, 2026

Uh oh!

Mehedi Hasan (Mahdi-Hasan) Nov 10, 2026

Uh oh!

This branch was previously deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

Mehedi Hasan (Mahdi-Hasan) commented Nov 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation & Context

Description & Review Guide

Related Issue

AI Assistance

Contribution Checklist

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

Uh oh!

Eduard van Valkenburg (eavanvalkenburg) Nov 10, 2026

Choose a reason for hiding this comment

Uh oh!

Mehedi Hasan (Mahdi-Hasan) Nov 10, 2026

Choose a reason for hiding this comment

Uh oh!

This branch was previously deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Mehedi Hasan (Mahdi-Hasan) commented Nov 9, 2026 •

edited

Loading