Fix features/contractEditorSections.feature to account for "unfilled example" dialog by bcjordan · Pull Request #3544 · code-dot-org/code-dot-org

bcjordan · 2015-08-14T00:42:05Z

#3501 added example checking on close of the contract editor. Some of our tests had empty examples which lead to an unfilled example dialog showing up.

This dismisses the dialog in the tests where this resulted in failures.

Before

Popup

Test output

brian:ui brian$ ui && bundle exec ./runner.rb -d localhost.studio.code.org:3000 -l -m -f features/contractEditorSections.feature 
Starting UI tests for UnknownBrowser
cucumber features/contractEditorSections.feature -t ~@eyes -t ~@skip -t ~@pegasus_db_access -S
UI tests for UnknownBrowser failed (0:59 minutes)
0 succeeded.  1 failed.  Test count: 1.  Total duration: 74.38 seconds.  Average test duration: 74.38 seconds.
brian:ui brian$ cat error.log 
2015-08-13 17:25:05 -0700

---
:browser: local
Using the default profile...
@dashboard_db_access
Feature: Contract Editor section configuration and manipulation

  Background:                                                                      # features/contractEditorSections.feature:4
    Given I am on "http://learn.code.org/"                                         # step_definitions/steps.rb:18
    And I am a student                                                             # step_definitions/steps.rb:364
    And I am on "http://learn.code.org/s/algebra/stage/7/puzzle/4?noautoplay=true" # step_definitions/steps.rb:18
    And I rotate to landscape                                                      # step_definitions/steps.rb:70
    And I close the dialog                                                         # step_definitions/steps.rb:29
    Then element "#runButton" is visible                                           # step_definitions/steps.rb:214
    And I open the blockly category with ID "7"                                    # step_definitions/steps.rb:129
    And I press the SVG text "Create a Function"                                   # step_definitions/steps.rb:115
    And I wait to see "#modalEditorClose"                                          # step_definitions/steps.rb:23

  Scenario: Examples should be hidden when specified in contract editor            # features/contractEditorSections.feature:15
    And examples are visible                                                       # step_definitions/contract_editor_steps.rb:38
    And the "Contract and Purpose Statement" contract editor header is visible     # step_definitions/contract_editor_steps.rb:24
    And the "Examples" contract editor header is visible                           # step_definitions/contract_editor_steps.rb:24
    And the "Definition" contract editor header is visible                         # step_definitions/contract_editor_steps.rb:24
    And I press "modalEditorClose"                                                 # step_definitions/steps.rb:81
    And I configure the contract editor to disable examples                        # step_definitions/contract_editor_steps.rb:16
    And I open the blockly category with ID "7"                                    # step_definitions/steps.rb:129
    And I press the SVG text "Create a Function"                                   # step_definitions/steps.rb:115
    And I wait to see "#modalEditorClose"                                          # step_definitions/steps.rb:23
    And there are no visible examples                                              # step_definitions/contract_editor_steps.rb:34

      expected: false
           got: true

      (compared using ==)
       (RSpec::Expectations::ExpectationNotMetError)
      ./step_definitions/contract_editor_steps.rb:35:in `/^there are no visible examples$/'
      features/contractEditorSections.feature:27:in `And there are no visible examples'
    And the "Contract and Purpose Statement" contract editor header is visible     # step_definitions/contract_editor_steps.rb:24
    And the "Examples" contract editor header isn't visible                        # step_definitions/contract_editor_steps.rb:29
    And the "Definition" contract editor header is visible                         # step_definitions/contract_editor_steps.rb:24
    And I open the blockly category with ID "f"                                    # step_definitions/steps.rb:129
    And I press the edit button on a function call named "something"               # step_definitions/steps.rb:144
    And there are no visible examples                                              # step_definitions/contract_editor_steps.rb:34
    And the "Examples" contract editor header isn't visible                        # step_definitions/contract_editor_steps.rb:29

  Scenario: Expanding / collapsing sections should persist across function openings # features/contractEditorSections.feature:37
    Then element ".contractEditor" is visible                                       # step_definitions/steps.rb:214
    When I press the contract editor header "Contract and Purpose Statement"        # step_definitions/contract_editor_steps.rb:20
    Then element ".contractEditor" is hidden                                        # step_definitions/steps.rb:220
    And I press "modalEditorClose"                                                  # step_definitions/steps.rb:81
    And I open the blockly category with ID "7"                                     # step_definitions/steps.rb:129
    And I press the SVG text "Create a Function"                                    # step_definitions/steps.rb:115
    And I wait to see "#modalEditorClose"                                           # step_definitions/steps.rb:23
    Then element ".contractEditor" is hidden                                        # step_definitions/steps.rb:220

  Scenario: Collapsing all sections, re-opening editor and expanding examples should not show function definition # features/contractEditorSections.feature:50
    When I press the contract editor header "Contract and Purpose Statement"                                      # step_definitions/contract_editor_steps.rb:20
    And I press the contract editor header "Examples"                                                             # step_definitions/contract_editor_steps.rb:20
    And I press the contract editor header "Definition"                                                           # step_definitions/contract_editor_steps.rb:20
    And the function editor definition block is not visible                                                       # step_definitions/contract_editor_steps.rb:46
    And I press "modalEditorClose"                                                                                # step_definitions/steps.rb:81
    And I open the blockly category with ID "7"                                                                   # step_definitions/steps.rb:129
    And I press the SVG text "Create a Function"                                                                  # step_definitions/steps.rb:115
    And I wait to see "#modalEditorClose"                                                                         # step_definitions/steps.rb:23
    And there are no visible examples                                                                             # step_definitions/contract_editor_steps.rb:34
    And I press the contract editor header "Examples"                                                             # step_definitions/contract_editor_steps.rb:20
    And examples are visible                                                                                      # step_definitions/contract_editor_steps.rb:38
    And the function editor definition block is not visible                                                       # step_definitions/contract_editor_steps.rb:46
    And I press the contract editor header "Definition"                                                           # step_definitions/contract_editor_steps.rb:20
    And the function editor definition block is visible                                                           # step_definitions/contract_editor_steps.rb:42
    And only one functional definition block is visible                                                           # step_definitions/contract_editor_steps.rb:50

  Scenario: Opening a variable then a function should show proper headers          # features/contractEditorSections.feature:70
    And I press "modalEditorClose"                                                 # step_definitions/steps.rb:81
    And I open the blockly category with ID "6"                                    # step_definitions/steps.rb:129
    And I press the SVG text "Create a Variable"                                   # step_definitions/steps.rb:115
    And the "Definition" contract editor header is invisible                       # step_definitions/contract_editor_steps.rb:29

      expected: true
           got: false

      (compared using ==)
       (RSpec::Expectations::ExpectationNotMetError)
      ./step_definitions/contract_editor_steps.rb:31:in `/^the "([^"]*)" contract editor header (?:isn't |is not |is in)visible$/'
      features/contractEditorSections.feature:75:in `And the "Definition" contract editor header is invisible'
    And the "Contract and Purpose Statement" contract editor header is invisible   # step_definitions/contract_editor_steps.rb:29
    And I wait to see "#modalEditorClose"                                          # step_definitions/steps.rb:23
    And I press "modalEditorClose"                                                 # step_definitions/steps.rb:81
    And I open the blockly category with ID "7"                                    # step_definitions/steps.rb:129
    And I press the SVG text "Create a Function"                                   # step_definitions/steps.rb:115
    And I wait to see "#modalEditorClose"                                          # step_definitions/steps.rb:23
    And the "Definition" contract editor header is visible                         # step_definitions/contract_editor_steps.rb:24
    And the "Examples" contract editor header is visible                           # step_definitions/contract_editor_steps.rb:24
    And the "Contract and Purpose Statement" contract editor header is visible     # step_definitions/contract_editor_steps.rb:24
    And element ".contractEditor" is visible                                       # step_definitions/steps.rb:214

Failing Scenarios:
cucumber features/contractEditorSections.feature:15 # Scenario: Examples should be hidden when specified in contract editor
cucumber features/contractEditorSections.feature:70 # Scenario: Opening a variable then a function should show proper headers

4 scenarios (2 failed, 2 passed)
90 steps (2 failed, 17 skipped, 71 passed)
0m38.341s

After

brian:ui brian$ ui && bundle exec ./runner.rb -d localhost.studio.code.org:3000 -l -m -f features/contractEditorSections.feature 
Starting UI tests for UnknownBrowser
cucumber features/contractEditorSections.feature -t ~@eyes -t ~@skip -t ~@pegasus_db_access -S
UI tests for UnknownBrowser succeeded (1:23 minutes)
1 succeeded.  0 failed.  Test count: 1.  Total duration: 97.77 seconds.  Average test duration: 97.77 seconds.
brian:ui brian$ g

…xample dialog

islemaster · 2015-08-14T00:50:33Z

Should we just configure the contract editor to disable examples before we press "modalEditorClose"?

Does this modal animate on close, and/or should we use "And I close the dialog" for this step as well? (It should apply to anything with an x-close button)

Could these scenarios ever fail to show that pop-up, as they seemed to do on my local machine? Should we fail in that case?

Re: 1, I like that idea for these, I believe there is a toggle that can be set after initialization
Re: 2, I don't think it should animate like the instructions dialog.
Re: 3, could it be that your local apps or blockly-core code is out of date?

Re: 3, probably.

islemaster · 2015-08-14T00:57:05Z

Looks like we should also add this change to contractEditor.feature to fix the eyes tests.

Test started failing because the level changed out from under it. Copied a version of the level from a few days ago (a passing one) into the allthethings script so it won't change in the future. Updated the test to pass (doesn't need the cancel button presses anymore). Aliased involved blocks so when it _does_ break again, it will be easier to read what it was trying to do.

bcjordan · 2015-08-14T01:59:41Z

Hm, so I think I like this PR's approach overall better than the somewhat confusing-feeling on/off switching in #3545

The one thing we do lose with the level no-examples change is eyes coverage of example block layout, but there's decent coverage of the eyes But I think that'd be better to add as an explicit separate test later.

🚢 it!

islemaster · 2015-08-14T02:06:18Z

👍

Fix features/contractEditorSections.feature to account for "unfilled example" dialog

Fix features/contractEditorSections.feature to account for unfilled e…

f1403be

…xample dialog

islemaster reviewed Aug 14, 2015
View reviewed changes

bcjordan added a commit that referenced this pull request Aug 14, 2015

Merge pull request #3544 from code-dot-org/fix_contract_ui_failures

05af77c

Fix features/contractEditorSections.feature to account for "unfilled example" dialog

bcjordan merged commit 05af77c into staging Aug 14, 2015

bcjordan deleted the fix_contract_ui_failures branch August 14, 2015 02:06

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix features/contractEditorSections.feature to account for "unfilled example" dialog#3544

Fix features/contractEditorSections.feature to account for "unfilled example" dialog#3544
bcjordan merged 2 commits into
stagingfrom
fix_contract_ui_failures

bcjordan commented Aug 14, 2015

Uh oh!

islemaster Aug 14, 2015

Uh oh!

bcjordan Aug 14, 2015

Uh oh!

islemaster Aug 14, 2015

Uh oh!

islemaster commented Aug 14, 2015

Uh oh!

bcjordan commented Aug 14, 2015

Uh oh!

islemaster commented Aug 14, 2015

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

bcjordan commented Aug 14, 2015

Before

Popup

Test output

After

Uh oh!

islemaster Aug 14, 2015

Choose a reason for hiding this comment

Uh oh!

bcjordan Aug 14, 2015

Choose a reason for hiding this comment

Uh oh!

islemaster Aug 14, 2015

Choose a reason for hiding this comment

Uh oh!

islemaster commented Aug 14, 2015

Uh oh!

bcjordan commented Aug 14, 2015

Uh oh!

islemaster commented Aug 14, 2015

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants