Flaky eyes test Teacher dashboard assessments tab survey submissions#56711
Conversation
|
|
||
| def jquery_is_element_visible(selector) | ||
| "return $(#{selector.dump}).is(':visible') && $(#{selector.dump}).css('visibility') !== 'hidden';" | ||
| "return #{jquery_is_defined} && $(#{selector.dump}).is(':visible') && $(#{selector.dump}).css('visibility') !== 'hidden';" |
There was a problem hiding this comment.
I don't think we need to test that jquery is defined when verifying that an element is visible. It will cause this to return false.
Instead, the act of loading a page should wait for jquery to become available. So I think we can remove this check.
There was a problem hiding this comment.
The exception is thrown after step #2 (detailed in PR description), where a jQuery seems to have been executed successfully, but some page reload seems to have happened since leading to the exception. I think adding the additional check for $ increases our resilience in general, but don't think it would address this specific behavior we are seeing.
There was a problem hiding this comment.
These tests are also executed as part of drone and seem to work working successfully. So, the overall condition is working as expected.
I always figured this happened when the browser going to a new page (this is consistent with one of the videos you posted -- didn't check all of them). But, I also always wondered why this check didn't work as-is, so I'm super excited to have a more robust check for jquery to be present! |
c754040 to
7a07e9b
Compare
|
Hi folks, I like the creativity here, but I suspect that the issue is something different. Here are my observations:
my recommendation would be to revert the changes in this PR, and update this call as follows: -And I press the first "#uitest-save-section-changes" element
+And I press the first "#uitest-save-section-changes" element to load a new page |
Thank you @davidsbailey for taking a look and providing detailed insights. This makes a lot of sense! My bad on overseeing that the page had not transitioned. Updated the PR to wait for page load. |
Eyes test
Firefox_teacher_tools_teacher_dashboard_teacher_dashboard_assessments2_Assessments tab survey submissionsintermittently fails withReferenceError: $ is not definederror when waiting for an element on teacher dashboard page.Root cause analysis:
Spot checked multiple failure examples (https://app.saucelabs.com/tests/770b726d37fe4e8d9c9be7d3ab95d017#308, https://app.saucelabs.com/tests/9b28115386bd480f8adfa517907b6a89#280) of this test in sauce labs and noticed the below pattern,
Checking for jQuery loaded
"return (typeof jQuery !== 'undefined');"returnstrueChecking for element visibility
"return $(\"#classroom-sections\").is(':visible') && $(\"#classroom-sections\").css('visibility') !== 'hidden';"returnsfalse(this happens 0 or more times)Same call to check for element visibility throws error that jQuery isn't loaded.
Looking at the screen shots for these steps, we see that when step #2 is executing, it is still in the section set up page and not transitioned to the dashboard. So, the jQuery load check isn't happening on the correct page leading to this issue.
Fix
Wait for the new page to load after hitting save button.
Links
JIRA https://codedotorg.atlassian.net/browse/TEACH-830?atlOrigin=eyJpIjoiZjUwZTc3NmRiNDVkNDNlMWExNDlmNDEzZGU4NDQ3ZGYiLCJwIjoiaiJ9
Testing story
Validated the teacher dashboard test locally. Relying on drone to catch any issues this change causes on other tests.
Deployment strategy
Regular DTP
Follow-up work
None at this time. Curious if folks think we should follow up this change in other jQueries if this improves reliability
Privacy
N/A
Security
N/A
Caching
N/A
PR Checklist: