Skip to content

Add initial test framework with TS#638

Open
zbettenbuk wants to merge 4 commits intoprerender:masterfrom
zbettenbuk:refactor-for-tests
Open

Add initial test framework with TS#638
zbettenbuk wants to merge 4 commits intoprerender:masterfrom
zbettenbuk:refactor-for-tests

Conversation

@zbettenbuk
Copy link
Copy Markdown
Contributor

This PR adds the foundations of a test framework where we can send requests through the Prerender server and compare the results to an expected output in a lenient way.

The PR is quite large, but has atomic commits, so it's easy to review it commit-by-commit:

pin dependencies: Some initial cleanup of the dependencies before refactor
add programmatic start-stop: Adds the possibility to programmatically start and stop the Prerender server without side effects (almost... needs some more refactor to be truly side effect-less). Also removes the process.exit calls that makes tests difficult. I did the refactor the least intrusive way I possibly could and tried to use the same coding style. My thinking is that any bigger refactor to that part should only come when there are adequate tests available.
fix leaked timeouts: Fixes two setTimeout calls that were blocking the server to properly exit on request.
add initial tests with TS: This adds the actual test suite. Only tests a few flows for now, but it's quite easy to extend.

@thoop
Copy link
Copy Markdown
Contributor

thoop commented May 25, 2020

This looks great! I'm excited to be able to run some tests here to verify things look correct across different versions of Prerender and different versions of Chrome. I'll add a few minor comments as I look though things more closely. Thanks for doing this!

Copy link
Copy Markdown
Contributor

@thoop thoop left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the package-lock.json is out of sync after this change and will need to be updated. Shouldn't be an issue but we'll want to make sure those match if we pin these.

lib/server.js Outdated
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do like this better for being able to start/stop for tests, but since the lib/index.js used to call app.listen and then server.start() would actually start the Chrome server, does that make this a breaking change requiring a major version bump? I'd argue no one would actually include lib/index.js without calling server.start() since that would have been useless, but just a thought.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a very good point, thanks! Actually we can still keep it backwards compatible, if i'm not mistaken. The lib/index.js is not an entry point of the app, so it may be good as is now. However the ./index.js is a de-facto entry point, where I added the server.start() invocation now. And the server.js still does what we need, if I didn't mess it up :)

What do you think?

@thoop
Copy link
Copy Markdown
Contributor

thoop commented May 25, 2020

This is super awesome. I like how the tests are set up. Simple and clear with the input/output files.

The only thing I'd change would just be a few function names in the tests to be a bit more verbose to more easily understand at a glance:

    it('should respond properly to a simple non-js file', function(done) {
        const fileName = 'basic-1.html';

        new PrerenderServerInvoker(server.address, testInstance.address).matchFiles(fileName, testFiles.get(fileName)).then(match => {
            assert.equal(match, true);
            done();
        }, done).catch(done);
    });

Since you are starting an express server to serve the input files from a URL, I feel like it reads a little easier like:

    it('should respond properly to a simple non-js file', function(done) {
        const urlName = 'basic-1.html';

        new PrerenderServerInvoker(server.address, testInstance.address).renderURLAndVerifyMatch(urlName, expectedOutputFiles.get(urlName)).then(match => {
            assert.equal(match, true);
            done();
        }, done).catch(done);
    });

It doesn't have to be exactly that, but something like that. Thoughts?

@thoop
Copy link
Copy Markdown
Contributor

thoop commented May 25, 2020

It looks like this doesn't run on Node v10 (I tested with 10.8.0) due to:

  Server - simple
(node:20304) UnhandledPromiseRejectionWarning: TypeError: fs_1.default.promises.opendir is not a function
    at Function.<anonymous> (/Users/todd/projects/prerender/test/utils/OutputLoader.ts:9:41)

Looks like that should be an easy change to get working with Node v10.

@zbettenbuk
Copy link
Copy Markdown
Contributor Author

It looks like this doesn't run on Node v10 (I tested with 10.8.0) due to:

  Server - simple
(node:20304) UnhandledPromiseRejectionWarning: TypeError: fs_1.default.promises.opendir is not a function
    at Function.<anonymous> (/Users/todd/projects/prerender/test/utils/OutputLoader.ts:9:41)

Looks like that should be an easy change to get working with Node v10.

Right, sorry for that. I was using v14 and forgot to test on v10. Fixed now.

@zbettenbuk
Copy link
Copy Markdown
Contributor Author

I think the package-lock.json is out of sync after this change and will need to be updated. Shouldn't be an issue but we'll want to make sure those match if we pin these.

I double checked package-lock.json and I couldn't find any discrepancies there. Even deleted it and made npm regenerate it from scratch, and still no changes on my side. I used npm version 6.2.0 which seems to be the bundled npm for node 10.8.0. Can you please give me some more info about what seems to be out of sync on your side?

@zbettenbuk
Copy link
Copy Markdown
Contributor Author

renderURLAndVerifyMatch

Good call! I did a similar structure now. I still kept the test server URL to be appended automatically to remoteFilePath in the renderTestURLAndVerifyMatch, otherwise we needed to append the server address every time we invoke that call and I didn't want to do that (hence the ...TestURL... fragment in the name).

What do you think?

@zbettenbuk zbettenbuk force-pushed the refactor-for-tests branch from e908d85 to 4abd93e Compare May 31, 2020 14:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants