feat(desktop): Display stored totals for Tokens and Cost in Desktop Session Context#28887
Open
OpeOginni wants to merge 2 commits into
Open
feat(desktop): Display stored totals for Tokens and Cost in Desktop Session Context#28887OpeOginni wants to merge 2 commits into
OpeOginni wants to merge 2 commits into
Conversation
Contributor
Author
|
Would need to think of having a sort of COUNT query to get the number of total messages, including user and assistant messages. But this is a server side thing. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Issue for this PR
Closes #28836
Type of change
What does this PR do?
Problem
Session cost and token totals were computed from loaded messages only, so they changed when scrolling up to load older messages.
Solution
Use the stored session totals (
session.costandsession.tokens) already provided by the server.SessionContextUsageandSessionContextTabnow read cost and token totals frominfo()?.cost/info()?.tokenssession-context-metrics.tssimplified:getSessionContext()now only computes context-window usage from the latest assistant messageIssues is that we don't have nice server side storage for data such as the total messages in a session, so this is currently still using the total rendered messages for its calculation.
How did you verify your code works?
Ran the dev desktop with a large session and this had the complete cost and tokens, without needing me to scroll all the way to load in new messages.
Screenshots / recordings
Checklist