Skip to content

Commit 9ecf077

Browse files
nicktrnericallam
nicktrn
and
ericallam
authored
v3: small features and fixes for prod infra (triggerdotdev#968)
* add checkpoint restore events * fix retries.enabledInDev * fix tsconfig paths * fix provider build and dev * update kubernetes provider and fix builds again * upgrade prod base to node 20 * update infra publish workflow * rethrow k8s errors after logging * shorten index container names * fix error type assertion * rename type assertion * remove resource limits for now * add missing run id on create * add push to deploy command for self-hosting * checkpointing fixes * update coordinator image * ensure valid registry login * delete checkpoint archive after successful push * log options on error * structured logs for socket connections * fix structured log merge * exit process after checkpointing * update restore pull secret name * append shortcode to restore names * log handler payload * disable post start lifecycle hook * pass in coordinator host via volume * replace dapi with taskinfo * add missing restore label * don't restart restored containers * remove init container from create * atomic post-completion checkpoints * switch to run id for container names * improve wait accuracy * measure basic checkpoint perf * always log disconnect reason * use system clock to end wait spans * checkpoint readiness and cancel signals * restore from checkpoint events and fix statuses * remove attempt id env var * restore dependencies from events * reconnect wip * lifecycle hooks are back * fix hooks and improve reconnect * make docker send postStart hook * only checkpoint for retry if large delay * fix a few more resume issues * lifecycle hook fixes * skip connection handler when waiting for post start hook --------- Co-authored-by: Eric Allam <eric@trigger.dev>
1 parent b890a64 commit 9ecf077

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

48 files changed

+2270
-1001
lines changed

.github/workflows/publish-dev.yml renamed to .github/workflows/publish-infra.yml

Lines changed: 19 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,11 @@
1-
name: "🚢 Publish Container Images (dev)"
1+
name: "🚢 Publish Infra Images"
22

33
on:
44
push:
55
tags:
6-
- "dev-*"
6+
- "infra-dev-*"
7+
- "infra-test-*"
8+
- "infra-prod-*"
79
paths:
810
- ".github/workflows/publish.yml"
911
- "packages/**"
@@ -44,12 +46,21 @@ jobs:
4446
steps:
4547
- uses: actions/checkout@v4
4648

47-
- name: Generate build ID
49+
- name: Generate image reference
4850
id: prep
51+
# WARNING: This step expects the workflow to have been triggered by a specific tag format of: infra-${env}-*
4952
run: |
53+
env=$(echo ${{ github.ref_name }} | cut -d- -f2)
5054
sha=${GITHUB_SHA::7}
5155
ts=$(date +%s)
52-
echo "BUILD_ID=${{ matrix.package }}-${sha}-${ts}" >> "$GITHUB_OUTPUT"
56+
if [[ "${{ matrix.package }}" == *-provider ]]; then
57+
provider_type=$(echo ${{ matrix.package }} | cut -d- -f1)
58+
repository=provider/${provider_type}
59+
else
60+
repository=${{ matrix.package }}
61+
fi
62+
echo "IMAGE_TAG=${env}-${sha}-${ts}" >> "$GITHUB_OUTPUT"
63+
echo "REPOSITORY=${repository}" >> "$GITHUB_OUTPUT"
5364
5465
- name: Set up Docker Buildx
5566
uses: docker/setup-buildx-action@v3
@@ -63,7 +74,7 @@ jobs:
6374

6475
- name: 🚢 Build Container Image
6576
run: |
66-
docker build -t dev_image -f ./apps/${{ matrix.package }}/Containerfile .
77+
docker build -t infra_image -f ./apps/${{ matrix.package }}/Containerfile .
6778
6879
# ..to push image
6980
- name: 🐙 Login to GitHub Container Registry
@@ -75,9 +86,9 @@ jobs:
7586

7687
- name: 🐙 Push to GitHub Container Registry
7788
run: |
78-
docker tag dev_image $REGISTRY/$REPOSITORY:$IMAGE_TAG
89+
docker tag infra_image $REGISTRY/$REPOSITORY:$IMAGE_TAG
7990
docker push $REGISTRY/$REPOSITORY:$IMAGE_TAG
8091
env:
8192
REGISTRY: ghcr.io/triggerdotdev
82-
REPOSITORY: dev
83-
IMAGE_TAG: ${{ steps.prep.outputs.BUILD_ID }}
93+
REPOSITORY: ${{ steps.prep.outputs.REPOSITORY }}
94+
IMAGE_TAG: ${{ steps.prep.outputs.IMAGE_TAG }}

‎apps/coordinator/Containerfile‎

Lines changed: 1 addition & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# syntax=docker/dockerfile:labs
22

3-
FROM node:18.18.2-bullseye-slim@sha256:21479df46c3173ee0cefc6b264928e10239152c4f74df872ca9369be01a245b7 AS node-18
3+
FROM node:18-bullseye-slim@sha256:a4edd54dcfdcacc8a4100fee71498e8671d99556a1acf5614539214a70092426 AS node-18
44

55
WORKDIR /app
66

@@ -35,7 +35,6 @@ COPY --from=pruner --chown=node:node /app/out/full/ .
3535
COPY --from=dev-deps --chown=node:node /app/ .
3636
COPY --chown=node:node turbo.json turbo.json
3737

38-
RUN pnpm run -r --filter '@trigger.dev/core*' build
3938
RUN pnpm run -r --filter coordinator build:bundle
4039

4140
FROM alpine AS cri-tools
@@ -58,6 +57,4 @@ COPY --from=builder --chown=node:node /app/apps/coordinator/dist/index.mjs ./ind
5857

5958
EXPOSE 8000
6059

61-
USER node
62-
6360
CMD [ "/usr/bin/dumb-init", "--", "/usr/local/bin/node", "./index.mjs" ]

‎apps/coordinator/package.json‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@
66
"main": "dist/index.cjs",
77
"scripts": {
88
"build": "npm run build:bundle",
9-
"build:bundle": "esbuild src/index.ts --bundle --outfile=dist/index.mjs --platform=node --format=esm --target=esnext --banner:js=\"const require = createRequire(import.meta.url);\"",
9+
"build:bundle": "esbuild src/index.ts --bundle --outfile=dist/index.mjs --platform=node --format=esm --target=esnext --banner:js=\"import { createRequire } from 'module';const require = createRequire(import.meta.url);\"",
1010
"build:image": "docker build -f Containerfile . -t coordinator",
1111
"dev": "tsx --no-warnings=ExperimentalWarning --require dotenv/config --watch src/index.ts",
1212
"start": "tsx src/index.ts",
@@ -19,6 +19,7 @@
1919
"@trigger.dev/core": "workspace:*",
2020
"@trigger.dev/core-apps": "workspace:*",
2121
"execa": "^8.0.1",
22+
"nanoid": "^5.0.6",
2223
"prom-client": "^15.1.0",
2324
"socket.io": "^4.7.4",
2425
"socket.io-client": "^4.7.4"

0 commit comments

Comments
 (0)