Skip to content

Web Lab 2: Add project files as context to AI tutor#68567

Merged
molly-moen merged 19 commits into
stagingfrom
molly/files-context-weblab2
Sep 26, 2025
Merged

Web Lab 2: Add project files as context to AI tutor#68567
molly-moen merged 19 commits into
stagingfrom
molly/files-context-weblab2

Conversation

@molly-moen

Copy link
Copy Markdown
Contributor

This is an initial implementation (UI subject to change) of allowing users to add text-based project files to AI Tutor as context. This feature will only be available in Web Lab 2 for now. The way the feature works is:

  • In web lab 2, on all text files a new file option is available, "Add to AI Tutor Chat"
  • If you click the add option, the file will appear in the chat editor, similar to how a multi-modal file appears (I did some minor refactoring of the multimodal previewer to support this).
  • You can remove an added file from the chat editor via an 'x' button.
  • The file contents/relevant metadata are added to AI Chat redux in a new property, userAddedContext. This was discussed on slack. We need to do multiple operations on the files from within the aichat domain, so it made sense to put it here, and it could be used in other situations, not just Web Lab 2.
  • If the user adds a file that they already added to the current chat context, it will be replaced.
  • On message send, the file contents will be included as part of the hidden context as the part of the code to focus on, the file name will appear as a chip in the history, and the userAddedContext will be cleared for the next message.

Screenshots

Pop-up menu option for text-based file

Screenshot 2025-09-25 at 1 10 01 PM

No pop-up menu option for an image

Screenshot 2025-09-25 at 1 10 10 PM

File chip in message editor

With multi-modal upload

Screenshot 2025-09-25 at 1 10 53 PM

Without multi-modal upload

Screenshot 2025-09-25 at 1 10 41 PM

File chip in message history

Screenshot 2025-09-25 at 1 10 27 PM

Links

Testing story

I confirmed this all works in Web Lab 2 but appears nowhere else, and does not cause issues for AI tutor in python lab or AI Chat.

Follow-up work

Exactly how we prompt the AI on how to focus on this section of code may need to be tweaked. Right now it works fine for the first message, but if you attach a new file to your second message and don't clear the context, it tends to still focus on the first file.

Next up I'm going to work on adding code snippets as context, and will look more at how to best tweak the prompt we are using here.

PR Creation Checklist:

  • Tests provide adequate coverage
  • Privacy impacts have been documented
  • Security impacts have been documented
  • Code is well-commented
  • New features are translatable or updates will not break translations
  • Relevant documentation has been added or updated
  • User impact is well-understood and desirable
  • Follow-up work items (including potential tech debt) are tracked and linked

@molly-moen molly-moen requested review from a team, cnbrenci and edcodedotorg September 25, 2025 20:23

@fisher-alice fisher-alice left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great! Left some minor comments/questions. Nice job!

Comment thread apps/src/aichat/views/assets/staged-files-preview.module.scss
}
dispatch(
addItemToUserAddedContext({
displayName: fullFilename,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Curious why you have both displayName and filename - they seem like duplicates.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They are now, but when we add code snippets the display name for those will be something like my_file.html (6-8)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh gotcha - that make sense. Thanks!

clickHandler: () => openRenameFilePrompt({fileId: file.id}),
},
{
condition: enableUserAddedContext(appName, file),

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if we should display this option if the user is not in the AI tutor tab.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe a question for @dmcavoy

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What option is this referring to?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alice is wondering if we should show "add to AI Tutor Chat" as an option on the file dropdown if the ai tutor tab isn't open. I think it makes sense for now to always show it.

Comment thread apps/src/aichat/redux/thunks/submitChatContents.ts Outdated
@@ -1,4 +1,5 @@
{
"addToAiTutorContext": "Add to AI Tutor Chat",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we still need to add loc strings?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure, but I'm going to keep doing it until there's an official announcement 😄

@edcodedotorg

edcodedotorg commented Sep 26, 2025

Copy link
Copy Markdown
Contributor

This looks like a great addition and something I could see going tutor-wide pretty soon! With that in mind and in the interest of keeping the shared plumbing clean (especially things like AiTutorContextHelper that have actually been designed to be shared amongst various use-cases) I think use-case specific methods like getHiddenContextCallbackWebLab2 are a sign that we just need to give the shared plumbing a little love to keep up with the changing requirements.

As discussed offline, I have prepped a stacked PR to better integrate this with AiTutorContextHelper and the intended direction there. I kept the description short in the interest of time, but there are various reasons behind this approach that we can discuss more whenever you have time. Let me know if you have any questions/concerns in the meantime!

@cnbrenci cnbrenci left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚢

@molly-moen molly-moen merged commit 2c1a7b7 into staging Sep 26, 2025
6 checks passed
@molly-moen molly-moen deleted the molly/files-context-weblab2 branch September 26, 2025 23:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants