Conversation
This patch is the first in the series of introducing awesome-typescript-loader support where it sets up a proxy channel for the "after-compile" plugin. "watch-run" doesn't really work yet and is needed for the loader, that'd be handled in the next patch. APIs supported during "after-compile": - compilation.addError(String|Error) -> void - compilation.addFileDependency(String|Array.<String>) -> void Note on shared state: since the mapping between threads and plugins can be 1:N, we need to make sure that a plugin executor is called at most once for any one thread. This is only achievable by tracking some shared state between all plugin instances (that is scoped to the ThreadPool, really).
Bugfixes
--------
- It seems that we were creating a foreground worker _everytime_ a
"watch-run" plugin is applied... this is now fixed.
Internal Changes
----------------
- ThreadPool now itself performs the plugin binding (to "after-compile")
so the plugins do not have to track any shared state themselves. Now
it's safe to have multiple plugins use the same thread-pool and we'll
have only one plugin handler installed per thread pool, as should be.
FakeCompiler Changes
--------------------
Added the following new APIs to the fake compiler:
- `purgeErrors(Object) -> void`:
Purge compilation errors that match some attributes, like
"loaderSource"
- `addModuleError(Error, Module) -> void`:
Add an error that is bound to a module. We use the module.id, this
might change to only accept the id.
- `purgeModuleErrors(Module, Object) -> void`:
Like `purgeErrors` but at the module level.
- `addAsset(assetPath: String, { source: String, size: Number }) ->
void`:
Register an asset to the compilation. This goes to
compilation.assets[assetPath].
"after-compile" plugin:
The first parameter, "compilation", now exposes:
- id: this is happypack-exclusive and is guaranteed to change between
compilations in the same process (e.g in --watch mode) or in different
processes (different builds). This ID can be used by loaders like
ts-loader to perform things at most once per compilation.
- modules: maps to a skimmed-down version of "compilation.modules" that
includes resource, errors, warnings, and static stuff
- errors: maps to a serializable version of "compilation.errors"
"watch-run" plugin:
The first parameter, "watching", now exposes:
- "mtimes": what is found inside
`watching.compiler.watchFileSystem.watcher.mtimes` ...
FakeLoaderContext Changes
-------------------------
- A new API "emitModuleError" for emitting errors at the module level
(this maps to this._module.errors on the foreground loader)
- A new API "writeModuleMetaAttribute" for defining stuff on the
foreground loader's "this._module.meta" map
We'll just treat this as a special case because it's the most common and loader authors should be responsible enough not to add cyclic properties to an error, really. This *will* bite us later but until then.
|
One thing I noticed is that "warming up" a typescript loader takes a considerable amount of time (~700ms in my samples). In the regular webpack build, using either loader, it took around ~800ms to build the example, which contains only 3 sources. With happypack, there's about ~100ms overhead for the entire build, so it takes 900ms if you're using a single thread, and ~700ms for every consecutive thread. It's very unfortunate. Compiling the example using happypack in 2 threads takes somewhere between 1500-1700ms (using either loader - it's almost the same.) I digged a bit into In my opinion, this should either be solved at the TypeScript level (e.g. have This should be doable because I checked Sublime Text's TS plugin and it seems to have a tsserver instance in the background to do the diagnostics/type-checking. TL;DR - don't expect fascinating results unless your builds take ~15 seconds or longer already. |
|
It might be useful if you guys can pull in some TypeScript experts to this thread because this is out of my scope. |
|
CC @jbrantly @s-panferov @DanielRosenwasser is this 'warmup' that @amireh mentions a typescript issue? |
|
Also, is there any large open-source TypeScript project that can be built with webpack that we (I) can test on? |
|
angular2-webpack-starter is a good one. |
|
I could not get either to work. I first installed these packaged Then I copied the And then tried to compile. Your fork of awesome-typescript-loader was 40 seconds longer than not using happypack (68 sec instead of 28 sec). It completed Using your fork of ts-loader I got: Looks like i'm doing something wrong because I can't figure out how to get either to compile. FYI I cannot get my project to build using |
|
😱 that is not exciting news. I didn't get the chance to check out the angular project yet so I can't verify, but will take a look as soon as I can. Also, more work was done on |
Sorry, I just saw this message. This might just be the initial parse/bind time (parsing & binding |
|
@DanielRosenwasser is this a tsconfig option that can be used? |
|
It should be (in the nightlies) - check out microsoft/TypeScript#8735 |
|
@amireh I will post my project source code up at my next opportunity and post the link here. |
|
Hey @amireh, here is the source code for the project I'm working on, and trying to get HappyPack working with typescript. https://drive.google.com/open?id=0B6I99RX-ZH6gRzRqa2FtY2R0OTg To run, follow these commands Install will take about 5 minutes usually. To view the app, go to The only view of the Angular2 app you'll have access to is the login screen, but that should be fine since you're working on compilation stuff and not the app components. Webpack config is located in |
|
Hey @amireh , just wondering if this is still on your mental to do list as a possibility? |
I've asked webpack core for help on this, to see how webpack may be able to help typescript loaders do their job without the use of internal objects, see this discussion if you're interested. That's what's making it very difficult to support them in happypack. I didn't do additional work on this PR because I'm not sure the approach is feasible at all; we might end up with worse performance than without happypack like you've seen so far or it will make things a maintenance nightmare on my end. I'm hoping that I could get some clues out of that discussion which would enable us to apply fixes in happypack ahead of time until webpack core officially supports this use case (this won't be until webpack 3 I believe so it's still some time away.) So, yes, it's still on the to-do list and I hope it gets moving soon. :) |
|
@amireh Any new updates on this? Our build times using webpack is about 120s... we would love to use Happypack to speed things up. |
|
@abergs I'd love to help out in this but given how complicated ts loaders are, and the fact that it's a hit/miss thing (i don't really know if it can be solved at happypack's layer at all) it's going to take a lot of time to figure this out, and I don't have that time. If you're willing to sponsor the work, maybe I can investigate but otherwise I'm afraid there isn't much to do. Did you guys try the suggestion of using the ts loader only for transpiling without the error reporting? I've heard that that works with happypack (and then you can use your editor or a separate linter to receive program-wide reports.) That may be a viable alternative? |
|
Hey @amireh - Thanks for getting back to me. I'm now indeed using awesome-typescript-loader only for transpiling but have yet to try HappyPack. |
Things to do/check:
ts-loader support:
--watchmode--watchmodetranspileOnlycompiler option--bailawesome-typescript-loader support:
--watchmode--watchmode--bailTo try out these changes, use the
happypack-supportbranches in my forks of those repositories. For example, to install the happypack-supporting version ofts-loader:You also need to clone this PR branch.