Skip to content

Try out CircleCI.#21751

Merged
DanielRosenwasser merged 2 commits into
microsoft:masterfrom
FelicianoTech:circleci
Feb 16, 2018
Merged

Try out CircleCI.#21751
DanielRosenwasser merged 2 commits into
microsoft:masterfrom
FelicianoTech:circleci

Conversation

@FelicianoTech
Copy link
Copy Markdown
Contributor

@FelicianoTech FelicianoTech commented Feb 7, 2018

  • You've signed the CLA

P.S. The sign in link on https://cla.opensource.microsoft.com/ is white text on a white background so it was hard to find.


I can see that this repo is being build with multiple CI providers in mind. I created a config for CircleCI 2.0 and wanted to demo that it brings with it reduced build times.

Node.js Version Travis CI CircleCI
9 10min 30sec 6min 15sec
8 9min 40sec 7min 23sec
6 14min 28sec 9min 28sec

Here's my fork on CircleCI: https://circleci.com/workflow-run/a2a9cad4-53d9-4238-a78c-3f20d6ff03a9

CircleCI is free just like Travis CI for something like TypeScript and clearly has better performance. Something to consider. Also, this PR shouldn't be merged as is. There's a circleci branch mentioned in the config that I put there for my testing purposes. Depending on your maintenance schedule, we could probably use regex for the release branches. If there's any questions on additional customization, please let me know.


Sidenote, you can see from this table, regardless of the provider, that Node.js has more or less been building faster with each new major release. That's encouraging.

cc @DanielRosenwasser

@msftclas
Copy link
Copy Markdown

msftclas commented Feb 7, 2018

CLA assistant check
All CLA requirements met.

@weswigham
Copy link
Copy Markdown
Member

No promises or anything, but I do have some comments about questions about what circle can do and our travis usage.

We use travis's cron feature to run our user-suite tests (we read an environment variable in the test runner to see if they should be included) once a day (this is configured on travis's web ui, which is why there's nothing in the config indicating it). Does circle ci have an equivalent feature?
Also, AFAIK, circle auto-caches node_modules dependencies and things between builds? Is there a way to purge this cache? We don't pin many of our dependencies (and prefer to be know when we break), but sometimes if we do get broken/break something, we may publish an updated dep (if we control it), and rerun a build - in travis we usually need to purge the cache when we do this.

Comment thread .circleci/config.yml Outdated

base: &base
environment:
- workerCount: 3
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, this number is tailored for maximal speed on travis. Set this to the maximum number of concurrent threads circleci will allow and actually benefit from the parallelism of (we had to measure travis with a bunch of different numbers before we landed on 3 - on a local machine we default to the number of logical processors available, which is usually optimal).

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense. I'll take a look later today.

@FelicianoTech
Copy link
Copy Markdown
Contributor Author

Sure - so regarding Cron, CircleCI 2.0 does have this feature, we call it scheduled builds. If you can provide me more information about how you're going about that as well as the cron syntax you're using, I can update my PR with the CircleCI equivalent.

Regarding caching, CircleCI 1.0 would have automatically cached node_modules. CircleCI 2.0, which is what I'm using in my PR is more explicit. This PR does not cache anything except the Docker images in which the environment runs. That is, the raw Docker image, not anything in it as a result of the build.

If you did want to use cache, the new cache works on setting a cache key. Which can be based off node_modules, a hash of packages.json, the Git cache, or an external resource. We can discuss that more if curious.

@weswigham
Copy link
Copy Markdown
Member

weswigham commented Feb 8, 2018

This code is the place where we change our behavior for a cron build - we inspect an environment variable, as might be expected. (Otherwise we expect to be run same as a normal build)

@FelicianoTech
Copy link
Copy Markdown
Contributor Author

FelicianoTech commented Feb 8, 2018

@weswigham Hey so my new commit addresses your comments from yesterday.

Cron

I added the config for Scheduled Workflows which is our equivalent to Travis's Cron. This is a UI changed I had to do here. In CircleCI, I created a new context with the name nightlies and added the TRAVIS_EVENT_TYPE variable with a value of cron. This will cause builds to trigger at 12:00am PST a.k.a. 8:00am UTC.

workerCount

I did some research. Here's a comparison of CircleCI build times based on workerCount (asterisk means avg of two builds):

# of Workers Node.js v9 (MM:SS) Node.js v8 (MM:SS) Node.js v6 (MM:SS)
3 6:15 7:23 9:28
4 5:52* 6:15* 8:44*
5 5:52 6:22 failed
6 5:47 5:30 failed
7 5:26* 6:33* failed
8 5:35* 5:17* failed
9 7:09* 5:57* failed
10 5:48 6:02 failed
11 5:47 5:38 failed

So this tells me that a workerCount of 8 is most efficient for these builds. However, the Node.js v6 builds fail with any workerCount above 4. So either, we stick to 4 or I adjust the config to set the workerCount to 8 for Node.js v9 and Node.js v8 and 4 for Node.js v6. With a workerCount of 8, we're looking at a total build time reduction of over 50% when compared to Travis, for Node.js v9 and v8.

Caching

Nothing with regard to npm/Yarn packages are being cached at all in my PR.

Again, all of these builds can be found here: https://circleci.com/gh/felicianotech/TypeScript/tree/circleci

Please let me know if there's any further questions or comments.

@weswigham
Copy link
Copy Markdown
Member

weswigham commented Feb 13, 2018

@FelicianoTech Is there some way we could get the custom log folds we had on travis back into the circle output (travis parsed the console output for some special identifiers to create folded sections with durations)? We've been using travis-fold and it's facilities or custom formatted strings to clean up/compact the output the way we like. TBH it's the only thing I can think of that's missing.

@FelicianoTech
Copy link
Copy Markdown
Contributor Author

FelicianoTech commented Feb 13, 2018

In short, no.

Travis CI's log folding feature we don't have an equivalent to. The closest thing I can think of, is in the UI we automatically "fold" the "Steps" that are defined in the CircleCI config file. We could separate npm test for example into its own step so that it could be "folded" separately from the commands before it. So this gets you part of the way there.

I believe the travis-fold npm module is being used to indicate custom commands to fold in your build output correct? That we wouldn't be able to replicate right now.

Personally, I have written a CircleCI Chrome extension that works pretty well and part of my goals for that is to test new UI ideas for CircleCI.com and GitHub.com. I've opened an Issue to investigate adding fold support via the extension in the future: FelicianoTech/pointless#26

@weswigham
Copy link
Copy Markdown
Member

I believe the travis-fold npm module is being used to indicate custom commands to fold in your build output correct? That we wouldn't be able to replicate right now.

Not custom commands, just specific sentinel text surrounding the desired fold; that's why you can have multiple folds in a single command's output (as we do).

@FelicianoTech
Copy link
Copy Markdown
Contributor Author

Ah okay. Well we don't have an alternative to travis-fold. The rest of the features we do and with that the decreased build time.

@DanielRosenwasser
Copy link
Copy Markdown
Member

🎉 we're going to give this a shot! Thanks a ton for the PR Ricardo 😃

@DanielRosenwasser DanielRosenwasser merged commit 4b34c42 into microsoft:master Feb 16, 2018
@DanielRosenwasser
Copy link
Copy Markdown
Member

I'm having trouble adding it to the repo 😕. I think this might be a Microsoft-wide permissions issue... Figuring it out with Wesley offline.

@FelicianoTech
Copy link
Copy Markdown
Contributor Author

FelicianoTech commented Feb 17, 2018

That's great news @DanielRosenwasser. Here's what needs to be done now:

  • CircleCI needs to be added for this repo (which is free) (if there's any issues, let me or Support know)
  • An org-wide context needs to be setup called nightlies with a variable called TRAVIS_EVENT_TYPE with the value of cron. Instructions here.
  • PR Remove unneeded circleci branch in CircleCi config. #22017 should be merged because this PR was merged before those lines were removed.

@DanielRosenwasser
Copy link
Copy Markdown
Member

DanielRosenwasser commented Mar 9, 2018

Hey @FelicianoTech, we reached out to Microsoft's GitHub admins and have CircleCI running on master! Problem: we're not seeing how to enable this for our PRs within Apps & Services for our repo's settings. Any idea if this is another permissions issue?

image

@FelicianoTech
Copy link
Copy Markdown
Contributor Author

It's all done on CircleCI.com. Add the project, then go to settings.

@weswigham
Copy link
Copy Markdown
Member

We have build forked pull requests on, but still aren't getting status badges from circle on PRs:
image

@FelicianoTech
Copy link
Copy Markdown
Contributor Author

FelicianoTech commented Mar 9, 2018

@weswigham Right now builds will only run for branches listed in lines 5-8 of the config:

        - master
        - release-2.5
        - release-2.6
        - release-2.7

That would need to be loosened if you'd like other branches to also run.

@weswigham
Copy link
Copy Markdown
Member

@FelicianoTech that whitelist also applies to forks? We want to build all PR-associated code (which is typically in a fork, but not always), plus master and our release branches (so we test post-merge).

@FelicianoTech
Copy link
Copy Markdown
Contributor Author

Yes. That specifies branches, regardless of where they come from.

It might make more sense to just build all branches then? Or maybe just specify what not to build?

@weswigham
Copy link
Copy Markdown
Member

But then random branches we make on the repo that don't have open PRs would get builds. And swapping to only branches that have open PRs could miss building a (likely unintentional) direct push to master or a release branch (though is probably the best compromise...).

@willfrew
Copy link
Copy Markdown

willfrew commented Apr 5, 2018

@FelicianoTech This is the best 'sales pitch' I've ever seen 👏

@FelicianoTech FelicianoTech deleted the circleci branch April 19, 2018 18:02
@microsoft microsoft locked and limited conversation to collaborators Jul 30, 2018
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants