diff --git a/docs/diagrams.md b/docs/diagrams.md new file mode 100644 index 0000000..ab77f8d --- /dev/null +++ b/docs/diagrams.md @@ -0,0 +1,155 @@ +# escld diagrams + +Rendered copies live in [`docs/images/`](images/). The prose behind each +diagram: [ARCHITECTURE.md](ARCHITECTURE.md), [WS_SFU.md](WS_SFU.md), +[RTMP.md](RTMP.md), [INFRASTRUCTURE.md](INFRASTRUCTURE.md). + +## System architecture + +Eight services around shared Postgres and a few purpose-specific stores. +Production shapes are shown (ALB, SQS, MSK, S3/CloudFront, Cognito); locally, +`docker-compose.yaml` emulates DynamoDB and SQS and runs real Postgres, Redis, +Elasticsearch and Kafka, but still uses the hosted Cognito pool. + + + +```mermaid +flowchart TB + U(["Browser"]) + ENC(["OBS / ffmpeg encoder"]) + + subgraph EDGE["Edge · auth"] + direction LR + CF["CloudFrontSPA · media · HLS"] + ALB["ALB/api/* · /analytics/* · /hls/*"] + COG["CognitoJWT · admin/moderator groups"] + end + + subgraph SYNC["Request path"] + direction LR + FE["frontendReact 19 · Vite"] + BE["backendJava 25 · Spring Boot 4"] + AN["analyticsNode · trending API"] + SFU["ws-sfuRust · Socket.IO · mediasoupchat + WebRTC calls"] + RT["rtmpRust · RTMP → HLS"] + end + + subgraph ASYNC["Async"] + direction LR + SQT[["SQS transcode-jobs"]] + SQP[["SQS post-events"]] + K[["Kafka / MSKwarehouse · live.*"]] + W["workerffmpeg transcode"] + FW["feed-workerembed · index · fan-out"] + BQ["bq-sink → BigQuery"] + end + + subgraph DATA["Stores"] + direction LR + PG[("Postgresusers · posts · outbox")] + DDB[("DynamoDBfeeds · graph · chats")] + ES[("Elasticsearch 9")] + RD[("Rediscache · pub/sub")] + S3[("S3media · HLS")] + end + + U --> CF --> FE + U -- "REST + JWT" --> ALB + U -- "Socket.IO · WebRTC UDP" --> SFU + U -. "sign in" .-> COG + ENC -- "RTMP :1935" --> RT + ALB --> BE & AN + + BE --> SQT --> W + BE --> SQP --> FW + BE -- "outbox relay" --> K --> BQ + K -- "live.started/ended" --> SFU + RT -- "live.ended" --> K + + BE --> PG & DDB & ES & RD + RD -- "analytics-events" --> AN + W --> S3 + FW --> ES & DDB + SFU --> DDB + RT --> PG & S3 + S3 --> CF +``` + +## Video call setup (mediasoup via Socket.IO) + +All signaling is custom Socket.IO events; media flows over UDP directly to +the SFU. One mediasoup Router per conversation, created on the first join. + + + +```mermaid +sequenceDiagram + autonumber + participant A as Alice (browser) + participant S as ws-sfu + participant B as Bob (browser) + participant D as DynamoDB conversations + + A->>S: connect (Cognito access token) + S->>S: verify JWT (JWKS) → Postgres user → Identity + A->>S: call:join {conversationId} + S->>D: is Alice a participant? + S-->>A: router RTP capabilities, existing producers, recording state + A->>S: call:createTransport (send) → call:connectTransport (DTLS) + A->>S: call:produce (mic, camera) + S-->>B: call:newProducer + B->>S: call:createTransport (recv) → call:connectTransport + B->>S: call:consume (Alice's producer) + S-->>B: consumer (paused) + B->>S: call:resumeConsumer + A-)S: RTP over UDP 40000-40019 + S-)B: forwarded RTP (SFU, no mixing) + A->>S: call:pauseProducer (mute) + S-->>B: call:producerStateChanged + B->>S: call:leave + S-->>A: call:peerLeft (room torn down when empty) +``` + +## Going live: RTMP → HLS pipeline + + + +```mermaid +flowchart LR + subgraph APP["In the app"] + K1["POST /api/v1/live/stream-key"] + K2["POST /api/v1/live/streamscreates LIVE post"] + end + + ENC["OBS / ffmpegrtmp://host:1935/live/<streamKey>"] + + subgraph RTMP["rtmp service (Rust, tokio)"] + HS["handshakeC0/C1 → S0/S1/S2 → C2"] + CMD["connect · createStream · publishAMF0 / AMF3"] + AUTH{"stream key validand stream announced?"} + FLV["re-mux A/V messagesto FLV tags"] + FF["ffmpeg -c copyfMP4 HLS, 1s segments"] + DISK[("local HLS dir")] + SYNC["s3_syncpoll every 100ms"] + HTTP["/hls/* fallback route"] + end + + PG[("Postgresusers.stream_key · posts")] + S3[("S3 live/<key>/")] + CF["CloudFrontm3u8 TTL ≤ 1s"] + V(["ViewersHLS player"]) + K[["Kafka live.ended"]] + SFU["ws-sfufeed:liveEnded push"] + + K1 --> K2 + K2 --> PG + ENC --> HS --> CMD --> AUTH + AUTH -- "lookup" --> PG + AUTH -- "reject" --> X(["connection closed"]) + AUTH -- "ok" --> FLV --> FF --> DISK + DISK --> SYNC --> S3 --> CF --> V + DISK --> HTTP + CMD -. "encoder disconnects" .-> END["mark post ENDEDSIGINT ffmpeg → #EXT-X-ENDLISTfinal S3 sync"] + END --> PG + END --> K --> SFU +``` diff --git a/docs/images/app-login.png b/docs/images/app-login.png new file mode 100644 index 0000000..fcc2b6e Binary files /dev/null and b/docs/images/app-login.png differ diff --git a/docs/images/app-sign-up.png b/docs/images/app-sign-up.png new file mode 100644 index 0000000..699fe20 Binary files /dev/null and b/docs/images/app-sign-up.png differ diff --git a/docs/images/diagram-architecture.png b/docs/images/diagram-architecture.png new file mode 100644 index 0000000..92bb98b Binary files /dev/null and b/docs/images/diagram-architecture.png differ diff --git a/docs/images/diagram-architecture.svg b/docs/images/diagram-architecture.svg new file mode 100644 index 0000000..6e16879 --- /dev/null +++ b/docs/images/diagram-architecture.svg @@ -0,0 +1,99 @@ +StoresAsyncRequest pathEdge · authREST + JWTSocket.IO · WebRTC UDPsign inRTMP :1935outbox relaylive.started/endedlive.endedanalytics-eventsBrowserOBS / ffmpeg encoderCloudFrontSPA · media · HLSALB/api/* · /analytics/* · /hls/*CognitoJWT · admin/moderator groupsfrontendReact 19 · VitebackendJava 25 · Spring Boot 4analyticsNode · trending APIws-sfuRust · Socket.IO · mediasoupchat + WebRTC callsrtmpRust · RTMP → HLSSQS transcode-jobsSQS post-eventsKafka / MSKwarehouse · live.*workerffmpeg transcodefeed-workerembed · index · fan-outbq-sink → BigQueryPostgresusers · posts · outboxDynamoDBfeeds · graph · chatsElasticsearch 9Rediscache · pub/subS3media · HLS \ No newline at end of file diff --git a/docs/images/diagram-call-sequence.png b/docs/images/diagram-call-sequence.png new file mode 100644 index 0000000..d207b08 Binary files /dev/null and b/docs/images/diagram-call-sequence.png differ diff --git a/docs/images/diagram-call-sequence.svg b/docs/images/diagram-call-sequence.svg new file mode 100644 index 0000000..120d6b1 --- /dev/null +++ b/docs/images/diagram-call-sequence.svg @@ -0,0 +1,99 @@ +DynamoDB conversationsBob (browser)ws-sfuAlice (browser)DynamoDB conversationsBob (browser)ws-sfuAlice (browser)connect (Cognito access token)1verify JWT (JWKS) → Postgres user → Identity2call:join {conversationId}3is Alice a participant?4router RTP capabilities, existing producers, recording state5call:createTransport (send) → call:connectTransport (DTLS)6call:produce (mic, camera)7call:newProducer8call:createTransport (recv) → call:connectTransport9call:consume (Alice's producer)10consumer (paused)11call:resumeConsumer12RTP over UDP 40000-4001913forwarded RTP (SFU, no mixing)14call:pauseProducer (mute)15call:producerStateChanged16call:leave17call:peerLeft (room torn down when empty)18 \ No newline at end of file diff --git a/docs/images/diagram-rtmp-hls.png b/docs/images/diagram-rtmp-hls.png new file mode 100644 index 0000000..9e5b238 Binary files /dev/null and b/docs/images/diagram-rtmp-hls.png differ diff --git a/docs/images/diagram-rtmp-hls.svg b/docs/images/diagram-rtmp-hls.svg new file mode 100644 index 0000000..92b44f5 --- /dev/null +++ b/docs/images/diagram-rtmp-hls.svg @@ -0,0 +1,99 @@ +rtmp service (Rust, tokio)In the applookuprejectokencoder disconnectsPOST /api/v1/live/stream-keyPOST /api/v1/live/streamscreates LIVE postOBS / ffmpegrtmp://host:1935/live/<streamKey>handshakeC0/C1 → S0/S1/S2 → C2connect · createStream · publishAMF0 / AMF3stream key validand stream announced?re-mux A/V messagesto FLV tagsffmpeg -c copyfMP4 HLS, 1s segmentslocal HLS dirs3_syncpoll every 100ms/hls/* fallback routePostgresusers.stream_key · postsS3 live/<key>/CloudFrontm3u8 TTL ≤ 1sViewersHLS playerKafka live.endedws-sfufeed:liveEnded pushconnection closedmark post ENDEDSIGINT ffmpeg → #EXT-X-ENDLISTfinal S3 sync \ No newline at end of file diff --git a/docs/images/terminal-compose-ps.png b/docs/images/terminal-compose-ps.png new file mode 100644 index 0000000..4affaef Binary files /dev/null and b/docs/images/terminal-compose-ps.png differ
Stores
Async
Request path
Edge · auth
REST + JWT
Socket.IO · WebRTC UDP
sign in
RTMP :1935
outbox relay
live.started/ended
live.ended
analytics-events
Browser
OBS / ffmpeg encoder
CloudFrontSPA · media · HLS
ALB/api/* · /analytics/* · /hls/*
CognitoJWT · admin/moderator groups
frontendReact 19 · Vite
backendJava 25 · Spring Boot 4
analyticsNode · trending API
ws-sfuRust · Socket.IO · mediasoupchat + WebRTC calls
rtmpRust · RTMP → HLS
SQS transcode-jobs
SQS post-events
Kafka / MSKwarehouse · live.*
workerffmpeg transcode
feed-workerembed · index · fan-out
bq-sink → BigQuery
Postgresusers · posts · outbox
DynamoDBfeeds · graph · chats
Elasticsearch 9
Rediscache · pub/sub
S3media · HLS
rtmp service (Rust, tokio)
In the app
lookup
reject
ok
encoder disconnects
POST /api/v1/live/stream-key
POST /api/v1/live/streamscreates LIVE post
OBS / ffmpegrtmp://host:1935/live/<streamKey>
handshakeC0/C1 → S0/S1/S2 → C2
connect · createStream · publishAMF0 / AMF3
stream key validand stream announced?
re-mux A/V messagesto FLV tags
ffmpeg -c copyfMP4 HLS, 1s segments
local HLS dir
s3_syncpoll every 100ms
/hls/* fallback route
Postgresusers.stream_key · posts
S3 live/<key>/
CloudFrontm3u8 TTL ≤ 1s
ViewersHLS player
Kafka live.ended
ws-sfufeed:liveEnded push
connection closed
mark post ENDEDSIGINT ffmpeg → #EXT-X-ENDLISTfinal S3 sync