Skip to content

fix: avoid string copy when processing input source-map#10885

Merged
existentialism merged 3 commits intobabel:masterfrom
JLHwung:fix-10875
Dec 18, 2019
Merged

fix: avoid string copy when processing input source-map#10885
existentialism merged 3 commits intobabel:masterfrom
JLHwung:fix-10875

Conversation

@JLHwung
Copy link
Copy Markdown
Contributor

@JLHwung JLHwung commented Dec 17, 2019

Q                       A
Fixed Issues? Fixes #10875
Patch: Bug Fix? Yes
Tests Added + Pass? No test is added, it is covered by current test suite
License MIT

This PR fixes a performance regression introduced in #10623.

// fromMapFileComment requires the whole comment block
`//${lastComment}`,

The lastComment was copied into a new string, prefixed by // and later unwrapped by convert-source-map.
https://github.com/thlorenz/convert-source-map/blob/9e9a7a9b652c30878c9c9aa591d861a9fdf61a7e/index.js#L31

This approach introduced significant memory footprint when there are many files with input source-map, i.e. the test repo mentioned in #10875. Here we use convertSourceMap.fromJSON to avoid unnecessary string copy.

@artisb45 Thank you for the reproduction repo, it helps me a lot! You can test flight this fix by downloading and overwrite node_modules/@babel/core/lib/transformation/normalize-file.js with this file. We are expected to land it in a patch release if review is good.

@JLHwung JLHwung added PR: Bug Fix 🐛 A type of pull request used for our changelog categories Priority: High area: sourcemaps labels Dec 17, 2019
// eslint-disable-next-line max-len
const INLINE_SOURCEMAP_REGEX = /^[@#]\s+sourceMappingURL=data:(?:application|text)\/json;(?:charset[:=]\S+?;)?base64,(?:.*)$/;
const EXTERNAL_SOURCEMAP_REGEX = /^[@#][ \t]+sourceMappingURL=(?:[^\s'"`]+?)[ \t]*$/;
const EXTERNAL_SOURCEMAP_REGEX = /^[@#][ \t]+sourceMappingURL=(?<url>[^\s'"`]+?)[ \t]*$/;
Copy link
Copy Markdown
Contributor Author

@JLHwung JLHwung Dec 17, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dogfooding named-capturing-regex here.

The file size is bloated by helpers (6678 B -> 11276 B) but I think it is okay since named-capturing-regex is supported on Node.js 10+, which means we can get rid of the helpers in v8.

I have changed my mind.

Comment thread packages/babel-core/src/transformation/normalize-file.js Outdated
Copy link
Copy Markdown
Member

@nicolo-ribaudo nicolo-ribaudo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with @jridgewell suggestion. Apart from that, ✔️

Co-Authored-By: Justin Ridgewell <justin@ridgewell.name>
@existentialism existentialism merged commit b3c7df9 into babel:master Dec 18, 2019
@existentialism
Copy link
Copy Markdown
Member

Derp, missed the flow errors... will push a PR fix in the morning if no one gets to it first!

@abonckus
Copy link
Copy Markdown

abonckus commented Dec 18, 2019

Hello, thanks for such a quick response to this issue @JLHwung ! However I tried to replace the file you provided and it did not seem to do anything. Not sure if I'm doing it wrong, but I'm just supposed to replace the file at the path right? I'm using the same repo I linked in the issue. Also tried rolling back before ejection and then replacing - same result.

@nicolo-ribaudo
Copy link
Copy Markdown
Member

#10885 (comment)

Ops, I didn't see your comment 😅

@JLHwung
Copy link
Copy Markdown
Contributor Author

JLHwung commented Dec 18, 2019

@artisb45 Oops, you may need clear the babel-loader cache first because it is an ad-hoc approach.

# Clear babel-loader cache
rm -rf node_modules/.cache/babel-loader
cp normalize-file.js node_modules/@babel/core/lib/transformation/normalize-file.js
npm run build

It went well on my local environment. You can also wait for the upcoming 7.7.7 release.

Update:

#10875 is not fixed: #10875 (comment)

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

area: sourcemaps outdated A closed issue/PR that is archived due to age. Recommended to make a new issue PR: Bug Fix 🐛 A type of pull request used for our changelog categories Priority: High

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Very high memory usage compiling mozilla/PDF.js

5 participants