fix(deps): update undici to 4.x by janpio · Pull Request #8842 · prisma/prisma

janpio · 2021-08-20T15:56:21Z

Co-authored-by: Joël Galeran <Jolg42@users.noreply.github.com>

janpio · 2021-08-20T16:23:36Z

Available for manual testing as 2.30.0-integration-undici-4.2

Jolg42 · 2021-08-23T07:47:59Z

Note: we founds that undici was using Blob https://github.com/nodejs/undici/search?q=require%28%27buffer%27%29&type=code which was only added with Node v15.7.0 https://nodejs.org/docs/latest-v16.x/api/buffer.html#buffer_class_blob (Node PR nodejs/node#36811)

We need to make sure this doesn't impact our supported versions of Node (12, 14)

Jolg42

Thank you for pushing this through the door, it might very well be useful for other wasm projects 💚

Jolg42 · 2022-01-12T08:07:30Z

This PR is current blocked because the following action job times out CI / client (library, ubuntu-latest, 16) (pull_request) Cancelled after 25m

Some investigation might be needed there first, what I found earlier was that something gets really slow during tests.
Example:
https://github.com/prisma/prisma/runs/4608802724?check_suite_focus=true#step:9:1493

 PASS src/__tests__/integration/happy/binary/test.ts (34.847 s)
  ✓ binary (34030 ms)

vs on main branch

https://github.com/prisma/prisma/runs/4605713109?check_suite_focus=true#step:9:1047
PASS src/__tests__/integration/happy/binary/test.ts (5.121 s)
  ✓ binary (4215 ms)

@millsp saw the heap size getting big too, so that might be linked.

juanzgc · 2022-03-13T15:34:04Z

Is there any news on the state of this PR and whether it is still being blocked 😇 ?

I believe this PR would fix the following Issue: #6899
Which is currently preventing me from using Prisma in Turborepo as a package

vincent-thomas · 2022-03-17T15:02:23Z

I have an issue with _http_common and I think this pr is going to fix that problem! Error here:

ERROR in ./apps/thunderbolt/prisma/client/runtime/index.js 27999:21-44
Module not found: Error: Can't resolve '_http_common' in '[path]\Codebase\apps\[app_name]\prisma\client\runtime'

…ci-4

Jolg42 · 2022-03-23T14:06:09Z

+/**
+ * Example
+ */
+// const inlineUndiciWasm = replaceWithPlugin([
+//   [
+//     /(await WebAssembly\.compile\().*?'(.*?)'\)\)\)/g,
+//     async (regex, contents) => {
+//       for (const match of contents.matchAll(regex)) {
+//         if (match[2].includes('simd') === false) {
+//           // we only bundle lhttp wasm files that are not simd compiled
+//           const engineCoreDir = resolve.sync('@prisma/engine-core')
+//           const undiciPackage = resolve.sync('undici/package.json', { basedir: engineCoreDir })
+//           const lhttpWasmPath = path.join(path.dirname(undiciPackage), 'lib', match[2])
+//           const wasmContents = (await fs.promises.readFile(lhttpWasmPath)).toString('base64')
+//           const inlineWasm = `${match[1]}(Buffer.from("${wasmContents}", "base64")))`
+
+//           contents = contents.replace(match[0], inlineWasm)
+//         }
+//       }
+
+//       return contents
+//     },
+//   ],
+// ])


What is this example for @millsp ?

It used to show how to inline wasm when the wasm file is not directly imported. However, in the mean time undici changed how they load the wasm and now puts the wasm contents in a .js file instead of doing a readFileSync so this is not needed anymore and works well when bundled.

Should we remove this then in a follow up PR?

We can probably trim it a bit, but I'd like to keep an example for how to use replaceWithPlugin.

Jolg42 · 2022-03-23T14:08:16Z

+// because undici lazily loads llhttp wasm which bloats the memory
+// TODO: hopefully replace with `import` but that causes segfaults
+const undici = () => require('undici')


So that's our current workaround for the segfaults for the Binary? any idea why is this happening?

I initially wanted to do () => import('undici') and I could not understand why this could cause segfaults in jest and only on CI. So I swapped for () => require('undici').

Worth opening a PR again to see if it still happens, and if so maybe point it out to Undici/Node ppl?

I tried this today. I made undici not to load lazily like it does by re-wrapping it in () => . Somehow, their lazy loading of the wasm (ie. on module load) was also causing memory leaks in jest. It would make sense to debug where it segfaults when doing import instead of require so that we can report to them.

michalkvasnicak and others added 12 commits June 29, 2021 17:09

fix(deps): update unidici

fea217c

build: copy undici wasm files to dist

e79239e

build: copy llhttp

4cc8333

fix: mark _http_common external

b92f97d

chore: remove fs-extra

206c0ab

Merge remote-tracking branch 'origin/main' into pr/michalkvasnicak/7931

3644418

Delete undici.ts

c6b0286

Delete build.js

6ddf052

add runtime/llhttp to package.json

487d6a8

copy wasm to generator-build as well

22b805c

remove unused import

3a2df2c

fix(deps): update undici to 4.4.6 (#8831)

6079f14

Co-authored-by: Joël Galeran <Jolg42@users.noreply.github.com>

janpio requested review from Jolg42, millsp and williamluke4 as code owners August 20, 2021 15:56

janpio mentioned this pull request Aug 20, 2021

fix(deps): update undici #8823

Closed

janpio changed the title ~~fix(deps): update unidici~~ fix(deps): update unidici to 4.x Aug 20, 2021

janpio changed the title ~~fix(deps): update unidici to 4.x~~ fix(deps): update undici to 4.x Aug 20, 2021

Merge branch 'main' into integration/undici-4

50b79d6

millsp reviewed Aug 20, 2021

View reviewed changes

Comment thread packages/engine-core/package.json Outdated

millsp reviewed Aug 20, 2021

View reviewed changes

Comment thread packages/cli/helpers/build.js Outdated

millsp reviewed Aug 20, 2021

View reviewed changes

Comment thread packages/client/helpers/build.js Outdated

millsp mentioned this pull request Aug 20, 2021

fix(deps): update undici to 4.0.0 #7931

Closed

Jolg42 mentioned this pull request Aug 23, 2021

Prisma Client (Binary): upgrade to undici@4 #7539

Closed

Jolg42 linked an issue Aug 23, 2021 that may be closed by this pull request

Prisma Client (Binary): upgrade to undici@4 #7539

Closed

janpio commented Aug 23, 2021

View reviewed changes

Comment thread packages/engine-core/src/binary/Connection.ts Outdated

janpio commented Aug 23, 2021

View reviewed changes

Comment thread pnpm-lock.yaml

janpio commented Aug 23, 2021

View reviewed changes

Comment thread packages/cli/helpers/build.js Outdated

janpio commented Dec 22, 2021

View reviewed changes

Comment thread packages/migrate/src/__tests__/DbCommand.test.ts

janpio commented Dec 22, 2021

View reviewed changes

Comment thread packages/migrate/src/__tests__/MigrateDeploy.test.ts

janpio commented Dec 22, 2021

View reviewed changes

Comment thread pnpm-lock.yaml

millsp added 2 commits December 22, 2021 15:55

chore(engine-core): bump undici

4dd5fb0

chore: cleanup

a811b01

aqrln approved these changes Dec 22, 2021

View reviewed changes

Comment thread packages/client/helpers/build.ts Outdated

Comment thread packages/client/helpers/build.ts Outdated

Comment thread packages/client/helpers/build.ts Outdated

Jolg42 reviewed Dec 22, 2021

View reviewed changes

Comment thread helpers/compile/plugins/replaceWithPlugin.ts

Jolg42 approved these changes Dec 22, 2021

View reviewed changes

smasala approved these changes Jan 4, 2022

View reviewed changes

Merge branch 'main' into integration/undici-4

ac83530

smasala approved these changes Jan 9, 2022

View reviewed changes

Jolg42 mentioned this pull request Jan 10, 2022

fix(deps): update dependency undici to v4 #11086

Closed

1 task

juanzgc mentioned this pull request Mar 18, 2022

I get "Can't resolve '_http_common'" when I use NestJS with monorepo (webpack) #6899

Closed

Merge branch 'main' of github.com:prisma/prisma into integration/undi…

fb7d30d

…ci-4

millsp removed this from the 3.8.0 milestone Mar 22, 2022

millsp force-pushed the integration/undici-4 branch from 4a1e395 to fb7d30d Compare March 23, 2022 09:20

millsp added 5 commits March 23, 2022 11:11

fix(client): unlazify undici

c2a8ed8

fix(client): defer undici loading

347dd06

fix(client): load unidici with require

fcc6f06

chore(client): cleanup

6b860b3

docs(client): add comment

6c82dc0

millsp added this to the 3.12.0 milestone Mar 23, 2022

millsp merged commit 6fc821f into main Mar 23, 2022

millsp deleted the integration/undici-4 branch March 23, 2022 13:51

Jolg42 reviewed Mar 23, 2022

View reviewed changes

lucasbogerd mentioned this pull request Apr 5, 2022

Prisma Client doesn't work outside of node_modules when using custom output dir #8326

Closed

Conversation

janpio commented Aug 20, 2021 • edited by millsp Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

janpio commented Aug 20, 2021

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Jolg42 commented Aug 23, 2021

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Jolg42 left a comment

Choose a reason for hiding this comment

Uh oh!

Jolg42 commented Jan 12, 2022

Uh oh!

juanzgc commented Mar 13, 2022

Uh oh!

vincent-thomas commented Mar 17, 2022

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

12 participants

janpio commented Aug 20, 2021 •

edited by millsp

Loading