diff --git a/docs/diagrams.md b/docs/diagrams.md new file mode 100644 index 0000000..ab77f8d --- /dev/null +++ b/docs/diagrams.md @@ -0,0 +1,155 @@ +# escld diagrams + +Rendered copies live in [`docs/images/`](images/). The prose behind each +diagram: [ARCHITECTURE.md](ARCHITECTURE.md), [WS_SFU.md](WS_SFU.md), +[RTMP.md](RTMP.md), [INFRASTRUCTURE.md](INFRASTRUCTURE.md). + +## System architecture + +Eight services around shared Postgres and a few purpose-specific stores. +Production shapes are shown (ALB, SQS, MSK, S3/CloudFront, Cognito); locally, +`docker-compose.yaml` emulates DynamoDB and SQS and runs real Postgres, Redis, +Elasticsearch and Kafka, but still uses the hosted Cognito pool. + +![escld architecture](images/diagram-architecture.svg) + +```mermaid +flowchart TB + U(["Browser"]) + ENC(["OBS / ffmpeg encoder"]) + + subgraph EDGE["Edge · auth"] + direction LR + CF["CloudFront
SPA · media · HLS"] + ALB["ALB
/api/* · /analytics/* · /hls/*"] + COG["Cognito
JWT · admin/moderator groups"] + end + + subgraph SYNC["Request path"] + direction LR + FE["frontend
React 19 · Vite"] + BE["backend
Java 25 · Spring Boot 4"] + AN["analytics
Node · trending API"] + SFU["ws-sfu
Rust · Socket.IO · mediasoup
chat + WebRTC calls"] + RT["rtmp
Rust · RTMP → HLS"] + end + + subgraph ASYNC["Async"] + direction LR + SQT[["SQS transcode-jobs"]] + SQP[["SQS post-events"]] + K[["Kafka / MSK
warehouse · live.*"]] + W["worker
ffmpeg transcode"] + FW["feed-worker
embed · index · fan-out"] + BQ["bq-sink → BigQuery"] + end + + subgraph DATA["Stores"] + direction LR + PG[("Postgres
users · posts · outbox")] + DDB[("DynamoDB
feeds · graph · chats")] + ES[("Elasticsearch 9")] + RD[("Redis
cache · pub/sub")] + S3[("S3
media · HLS")] + end + + U --> CF --> FE + U -- "REST + JWT" --> ALB + U -- "Socket.IO · WebRTC UDP" --> SFU + U -. "sign in" .-> COG + ENC -- "RTMP :1935" --> RT + ALB --> BE & AN + + BE --> SQT --> W + BE --> SQP --> FW + BE -- "outbox relay" --> K --> BQ + K -- "live.started/ended" --> SFU + RT -- "live.ended" --> K + + BE --> PG & DDB & ES & RD + RD -- "analytics-events" --> AN + W --> S3 + FW --> ES & DDB + SFU --> DDB + RT --> PG & S3 + S3 --> CF +``` + +## Video call setup (mediasoup via Socket.IO) + +All signaling is custom Socket.IO events; media flows over UDP directly to +the SFU. One mediasoup Router per conversation, created on the first join. + +![call flow](images/diagram-call-sequence.svg) + +```mermaid +sequenceDiagram + autonumber + participant A as Alice (browser) + participant S as ws-sfu + participant B as Bob (browser) + participant D as DynamoDB conversations + + A->>S: connect (Cognito access token) + S->>S: verify JWT (JWKS) → Postgres user → Identity + A->>S: call:join {conversationId} + S->>D: is Alice a participant? + S-->>A: router RTP capabilities, existing producers, recording state + A->>S: call:createTransport (send) → call:connectTransport (DTLS) + A->>S: call:produce (mic, camera) + S-->>B: call:newProducer + B->>S: call:createTransport (recv) → call:connectTransport + B->>S: call:consume (Alice's producer) + S-->>B: consumer (paused) + B->>S: call:resumeConsumer + A-)S: RTP over UDP 40000-40019 + S-)B: forwarded RTP (SFU, no mixing) + A->>S: call:pauseProducer (mute) + S-->>B: call:producerStateChanged + B->>S: call:leave + S-->>A: call:peerLeft (room torn down when empty) +``` + +## Going live: RTMP → HLS pipeline + +![rtmp pipeline](images/diagram-rtmp-hls.svg) + +```mermaid +flowchart LR + subgraph APP["In the app"] + K1["POST /api/v1/live/stream-key"] + K2["POST /api/v1/live/streams
creates LIVE post"] + end + + ENC["OBS / ffmpeg
rtmp://host:1935/live/<streamKey>"] + + subgraph RTMP["rtmp service (Rust, tokio)"] + HS["handshake
C0/C1 → S0/S1/S2 → C2"] + CMD["connect · createStream · publish
AMF0 / AMF3"] + AUTH{"stream key valid
and stream announced?"} + FLV["re-mux A/V messages
to FLV tags"] + FF["ffmpeg -c copy
fMP4 HLS, 1s segments"] + DISK[("local HLS dir")] + SYNC["s3_sync
poll every 100ms"] + HTTP["/hls/* fallback route"] + end + + PG[("Postgres
users.stream_key · posts")] + S3[("S3 live/<key>/")] + CF["CloudFront
m3u8 TTL ≤ 1s"] + V(["Viewers
HLS player"]) + K[["Kafka live.ended"]] + SFU["ws-sfu
feed:liveEnded push"] + + K1 --> K2 + K2 --> PG + ENC --> HS --> CMD --> AUTH + AUTH -- "lookup" --> PG + AUTH -- "reject" --> X(["connection closed"]) + AUTH -- "ok" --> FLV --> FF --> DISK + DISK --> SYNC --> S3 --> CF --> V + DISK --> HTTP + CMD -. "encoder disconnects" .-> END["mark post ENDED
SIGINT ffmpeg → #EXT-X-ENDLIST
final S3 sync"] + END --> PG + END --> K --> SFU +``` diff --git a/docs/images/app-login.png b/docs/images/app-login.png new file mode 100644 index 0000000..fcc2b6e Binary files /dev/null and b/docs/images/app-login.png differ diff --git a/docs/images/app-sign-up.png b/docs/images/app-sign-up.png new file mode 100644 index 0000000..699fe20 Binary files /dev/null and b/docs/images/app-sign-up.png differ diff --git a/docs/images/diagram-architecture.png b/docs/images/diagram-architecture.png new file mode 100644 index 0000000..92bb98b Binary files /dev/null and b/docs/images/diagram-architecture.png differ diff --git a/docs/images/diagram-architecture.svg b/docs/images/diagram-architecture.svg new file mode 100644 index 0000000..6e16879 --- /dev/null +++ b/docs/images/diagram-architecture.svg @@ -0,0 +1,99 @@ +

Stores

Async

Request path

Edge · auth

REST + JWT

Socket.IO · WebRTC UDP

sign in

RTMP :1935

outbox relay

live.started/ended

live.ended

analytics-events

Browser

OBS / ffmpeg encoder

CloudFront
SPA · media · HLS

ALB
/api/* · /analytics/* · /hls/*

Cognito
JWT · admin/moderator groups

frontend
React 19 · Vite

backend
Java 25 · Spring Boot 4

analytics
Node · trending API

ws-sfu
Rust · Socket.IO · mediasoup
chat + WebRTC calls

rtmp
Rust · RTMP → HLS

SQS transcode-jobs

SQS post-events

Kafka / MSK
warehouse · live.*

worker
ffmpeg transcode

feed-worker
embed · index · fan-out

bq-sink → BigQuery

Postgres
users · posts · outbox

DynamoDB
feeds · graph · chats

Elasticsearch 9

Redis
cache · pub/sub

S3
media · HLS

\ No newline at end of file diff --git a/docs/images/diagram-call-sequence.png b/docs/images/diagram-call-sequence.png new file mode 100644 index 0000000..d207b08 Binary files /dev/null and b/docs/images/diagram-call-sequence.png differ diff --git a/docs/images/diagram-call-sequence.svg b/docs/images/diagram-call-sequence.svg new file mode 100644 index 0000000..120d6b1 --- /dev/null +++ b/docs/images/diagram-call-sequence.svg @@ -0,0 +1,99 @@ +DynamoDB conversationsBob (browser)ws-sfuAlice (browser)DynamoDB conversationsBob (browser)ws-sfuAlice (browser)connect (Cognito access token)1verify JWT (JWKS) → Postgres user → Identity2call:join {conversationId}3is Alice a participant?4router RTP capabilities, existing producers, recording state5call:createTransport (send) → call:connectTransport (DTLS)6call:produce (mic, camera)7call:newProducer8call:createTransport (recv) → call:connectTransport9call:consume (Alice's producer)10consumer (paused)11call:resumeConsumer12RTP over UDP 40000-4001913forwarded RTP (SFU, no mixing)14call:pauseProducer (mute)15call:producerStateChanged16call:leave17call:peerLeft (room torn down when empty)18 \ No newline at end of file diff --git a/docs/images/diagram-rtmp-hls.png b/docs/images/diagram-rtmp-hls.png new file mode 100644 index 0000000..9e5b238 Binary files /dev/null and b/docs/images/diagram-rtmp-hls.png differ diff --git a/docs/images/diagram-rtmp-hls.svg b/docs/images/diagram-rtmp-hls.svg new file mode 100644 index 0000000..92b44f5 --- /dev/null +++ b/docs/images/diagram-rtmp-hls.svg @@ -0,0 +1,99 @@ +

rtmp service (Rust, tokio)

In the app

lookup

reject

ok

encoder disconnects

POST /api/v1/live/stream-key

POST /api/v1/live/streams
creates LIVE post

OBS / ffmpeg
rtmp://host:1935/live/<streamKey>

handshake
C0/C1 → S0/S1/S2 → C2

connect · createStream · publish
AMF0 / AMF3

stream key valid
and stream announced?

re-mux A/V messages
to FLV tags

ffmpeg -c copy
fMP4 HLS, 1s segments

local HLS dir

s3_sync
poll every 100ms

/hls/* fallback route

Postgres
users.stream_key · posts

S3 live/<key>/

CloudFront
m3u8 TTL ≤ 1s

Viewers
HLS player

Kafka live.ended

ws-sfu
feed:liveEnded push

connection closed

mark post ENDED
SIGINT ffmpeg → #EXT-X-ENDLIST
final S3 sync

\ No newline at end of file diff --git a/docs/images/terminal-compose-ps.png b/docs/images/terminal-compose-ps.png new file mode 100644 index 0000000..4affaef Binary files /dev/null and b/docs/images/terminal-compose-ps.png differ