Skip to content

Defaulting to test wide timeout for reset progress for flaky test multiple_choice_contained_levels.feature#61512

Merged
vijayamanohararaj merged 2 commits into
stagingfrom
vijaya/multichoice_reset_prog
Oct 4, 2024
Merged

Defaulting to test wide timeout for reset progress for flaky test multiple_choice_contained_levels.feature#61512
vijayamanohararaj merged 2 commits into
stagingfrom
vijaya/multichoice_reset_prog

Conversation

@vijayamanohararaj

Copy link
Copy Markdown
Contributor

There are two root causes for flakiness in multiple_choice_contained_levels.feature,

  1. There is no progress bar in the header
  2. The choice continues to be checked after user clicks "Delete Answer"

This PR focuses on #2 above. When looking at a few different test runs, the API call to POST /delete_predict_level_progress takes under 100 ms in successful cases, while in case of failures, the call takes over 2 seconds and in some cases the test exists before the call completes.

While trying this scenario manually, I noticed a fairly long delay performing this action on test.

Fix: The test currently has an upper limit of 5 seconds for the action to complete and UI reflect the change. This change would remove the upper limit and have this step also fall back to the test wide timeout.

There is a follow up here worth pursuing if this helps but results in a large latency. Added that in this https://codedotorg.atlassian.net/browse/TEACH-1383 to track.

Links

JIRA https://codedotorg.atlassian.net/browse/TEACH-1383

Testing story

Drone

Deployment strategy

Regular DTP

Follow-up work

Privacy

Security

Caching

PR Checklist:

  • Tests provide adequate coverage
  • Privacy and Security impacts have been assessed
  • Code is well-commented
  • New features are translatable or updates will not break translations
  • Relevant documentation has been added or updated
  • User impact is well-understood and desirable
  • Pull Request is labeled appropriately
  • Follow-up work items (including potential tech debt) are tracked and linked

@vijayamanohararaj vijayamanohararaj marked this pull request as ready for review October 3, 2024 14:16
@vijayamanohararaj vijayamanohararaj requested review from a team and lfryemason and removed request for a team October 3, 2024 14:17

@lfryemason lfryemason left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One thing to consider, otherwise looks good

@vijayamanohararaj vijayamanohararaj merged commit 0767f38 into staging Oct 4, 2024
@vijayamanohararaj vijayamanohararaj deleted the vijaya/multichoice_reset_prog branch October 4, 2024 03:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants