Automated testing for Chrome

I'd very much like to add automated Chrome testing, similar to our Firefox automated test: Start up a Chrome profile with the extension loaded, open a sample URL, and verify that it got rewritten to HTTPS. Some of the groundwork is in run-chromium.sh, but I'm not sure how to proceed with the rest of the test: in particular how to detect once the page is fully loaded and shut down. If you have some ideas, please contribute!

Might be messy, though looks like some good info in:

travis-ci/travis-ci#272

I've already done something like that for a Chrome extension I've created using Selenium and ChromeDriver. The first thing you'll need to do is to install ChromeDriver itself. If you're on Linux, you'll need to build it manually. I have an executable file I can send you if you'd like. Afterwards, install Selenium. And of course you need to have Google Chrome installed.

Selenium scripts can be coded using either Java or Python, I use Python. Here's an example of loading a Chrome browser with an extension using Python.

import time
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.support import expected_conditions
from selenium.webdriver.common.by import By

chromeOps = webdriver.ChromeOptions()
chromeOps.add_extension('Source.crx')

driver = webdriver.Chrome('/usr/bin/chromedriver', chrome_options = chromeOps)  # Optional argument, if not specified will search path.
driver.get('https://google.com')

wait = WebDriverWait(driver, 20)
wait.until(expected_conditions.visibility_of_element_located((By.ID, 'hplogo')))

#Page Loaded

time.sleep(5) # delays for 5 seconds
driver.quit()

I'm willing to contribute but I'm new here, I don't know what exactly do we need to test.

I guess the intention would be to do the testing on travis

I don't think Travis supports ChromeDriver

@reedy

@reedy is right, the ideal is run automated tests on Travis CI, but according to travis-ci/travis-ci#272, Chrome won't run on Travis. Even so, I think it's worthwhile to have an automated test that developers can run on their workstations, so I greatly appreciate the offer, @TheNavigat!

The most important test I'd like to do is to start up Chrome, navigate to an example rewritten site, e.g. http://libssh.org/robots.txt, wait for the document to load, then verify the document's URL is https://libssh.org/robots.txt. Once we've got that running we can look at more tests, for instance opening the icon menu, ensuring the correct list of applicable rules is there, and enabling / disabling rules.

@jsha

@jsha That's the exact ticket I was looking at as well when I mentioned that Travis won't work.

And I'll get that done in a while.

import time
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.support import expected_conditions
from selenium.webdriver.common.by import By

chromeOps = webdriver.ChromeOptions()
chromeOps.add_extension('Source.crx') #Source.crx here is HTTPS Everywhere's .crx file

driver = webdriver.Chrome('/usr/bin/chromedriver', chrome_options = chromeOps)  # Optional argument, if not specified will search path.
driver.get('http://libssh.org/robots.txt')

#Page Loaded

if driver.current_url.startswith('https'):
    print "HTTP to HTTPS redirection successful"
elif(driver.current_url.startswith('http')):
    print "HTTP to HTTPS redirection failed"

time.sleep(5) # delays for 5 seconds
driver.quit()

There you go. I'm not sure where I should commit that.

Nice! Is the script synchronous, so the page will definitely be loaded by the time we check the URL?

This should go in a new directory test/, with a comment at the top about how to get set up to run it. Probably run-chromium.sh should be renamed to test-chromium.sh, and do the package build, then run this script with the newly build crx. Similar to test.sh, test-chromium.sh should retain a --justrun flag that starts an instance with the newly built extension.

Okay, got it.

One more thing. The new directory test/ should be created at the top directory?

Yep! You'll see there's already https-everywhere-tests, which I may move under test/ later. That directory contains a secondary Firefox extension that loads and tests the main FF one.

I forgot to mention that yes, the script is synchronous.

#2623

This issue should be closed.

Let's keep it open for a bit -- until we get travis integration.

So, since I'm not familiar with Travis, does it automatically run test.sh, and if any script exits with a code other than 0, it reports that something's failed?

I'll start with moving things first, following the features you've listed, but I don't want anything to cause Travis reporting false errors because that will basically ruin everything.

Yep -- that's correct.

Take a look at https://github.com/EFForg/https-everywhere/blob/master/.travis.yml

And the results https://travis-ci.org/EFForg/https-everywhere

Perfect. I'll do it right now.

I'd be grateful if you have enough time to check the other PR, since there's actually a conflict hanging there...

Here. #2731

http://pastebin.com/dK0b6duN
http://pastebin.com/uPAC00k7

We might need to convert to docker in the near future.

@TheNavigat

@TheNavigat Pretty sure we're running on the Docker infra (which is what sudo: false enables).

Take a look at #2779 -- and then I think we can close this issue. Awesome work! 👍

@jsha

Maybe we can leave it open for a bit longer, so that @jsha can let us know what other tests need to be added.

Besides, I still need to do a little bit more cleanup.

There's one more bit of cleanup I'd like to do, move https-everywhere-checker to /test/rules. I'm not sure how to do that, because https-everywhere-checker is an independent repo. Any help, please?

@semenko

@semenko Anything?

Don't have time right now (I just quickly review tiny PRs when bored / waiting for science).

Oh... Okay...

@mnordhoff

Thanks to @mnordhoff I made some progress

#2889 moves https-everywhere-checker to test/rules. It crashes because coverage.checker.config includes a wrong path for platform_certs, which is corrected by that PR. jsha/https-everywhere-checker#2

Once https-everywhere-checker's PR is merged, I need to update the commit version in test/rules repo, and then we're good to go.

On a side-note. https-everywhere-checker's branch in .gitmodules should be updates, not coverage. That's where the commit we're currently using (pre #2889 PR) exists. Not sure if I should fix this myself, maybe whoever wrote the file has a point setting it to coverage, and not updates.

Okay so both PRs are merged, I think that's the cleanest the structure can be.

I think now's the time to wait for people to suggest more tests to add to the script.

Mostly fixed! Whoo!

Navigation Menu

Repository navigation

Automated testing for Chrome #2230

7 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Automated testing for Chrome #2230

Description

Activity

semenko commented on Jul 15, 2015

TheNavigat commented on Aug 20, 2015

reedy commented on Aug 20, 2015

TheNavigat commented on Aug 20, 2015

jsha commented on Aug 20, 2015

TheNavigat commented on Aug 20, 2015

TheNavigat commented on Aug 20, 2015

jsha commented on Aug 20, 2015

TheNavigat commented on Aug 20, 2015

jsha commented on Aug 20, 2015

TheNavigat commented on Aug 20, 2015

TheNavigat commented on Aug 21, 2015

TheNavigat commented on Aug 29, 2015

semenko commented on Aug 29, 2015

7 remaining items

TheNavigat commented on Aug 30, 2015

semenko commented on Aug 30, 2015

TheNavigat commented on Aug 30, 2015

TheNavigat commented on Aug 30, 2015

TheNavigat commented on Sep 5, 2015

semenko commented on Sep 6, 2015

TheNavigat commented on Sep 6, 2015

TheNavigat commented on Sep 10, 2015

TheNavigat commented on Sep 13, 2015

semenko commented on Sep 13, 2015

TheNavigat commented on Sep 13, 2015

TheNavigat commented on Sep 14, 2015

TheNavigat commented on Sep 16, 2015

semenko commented on Dec 17, 2015

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions