Skip to content

Latest commit

 

History

History
 
 

README.md

Sim Helm Chart

Deploy Sim — the open-source AI workspace where teams build, deploy, and manage AI agents — on Kubernetes.

  • Chart version: see Chart.yaml
  • App version: tracks the upstream Sim release
  • Kubernetes: 1.25+
  • License: Apache-2.0

TL;DR

# Generate required secrets
export BETTER_AUTH_SECRET=$(openssl rand -hex 32)
export ENCRYPTION_KEY=$(openssl rand -hex 32)
export INTERNAL_API_SECRET=$(openssl rand -hex 32)
export CRON_SECRET=$(openssl rand -hex 32)
export POSTGRES_PASSWORD=$(openssl rand -base64 24 | tr -d '/+=')

# Install from this repository
helm install sim ./helm/sim \
  --namespace sim --create-namespace \
  --set app.env.BETTER_AUTH_SECRET="$BETTER_AUTH_SECRET" \
  --set app.env.ENCRYPTION_KEY="$ENCRYPTION_KEY" \
  --set app.env.INTERNAL_API_SECRET="$INTERNAL_API_SECRET" \
  --set app.env.CRON_SECRET="$CRON_SECRET" \
  --set postgresql.auth.password="$POSTGRES_PASSWORD"

After install, follow the on-screen NOTES.txt to reach the app.


Introduction

This chart deploys the Sim platform on a Kubernetes cluster using the Helm package manager. A default install includes:

  • app — the Sim Next.js web application (Deployment).
  • realtime — the WebSocket service for live workflow updates (Deployment).
  • postgresql — an in-cluster pgvector/pgvector Postgres (StatefulSet, with a headless Service for stable per-pod DNS).
  • migrations — an init container on the app Deployment that applies database migrations before each app pod starts.
  • cronjobs — scheduled jobs for workflow schedule execution, inbox/calendar/drive polling (Gmail, Outlook, Calendar, Drive, Sheets, IMAP, RSS), workspace event and HubSpot webhook polling, outbox processing, subscription renewal, billing-seat and inbox-entitlement reconciliation, time-pause/resume polling, data drains, and connector syncs.
  • serviceaccount — a dedicated ServiceAccount with automountServiceAccountToken: false.

Optional components (off by default):

  • copilot — the Sim Copilot service plus its own Postgres StatefulSet.
  • ollama — local LLM inference, with optional NVIDIA GPU support.
  • pii — Presidio PII redaction service (analyzer + anonymizer) for the Guardrails PII block and log redaction. See PII redaction.
  • telemetry — OpenTelemetry Collector wired to Jaeger / Prometheus / OTLP backends.
  • ingress — NGINX-style Ingress for the app and realtime services.
  • networkPolicy — east-west and egress isolation (blocks cloud metadata endpoints by default).
  • hpa — HorizontalPodAutoscaler for app and realtime.
  • podDisruptionBudget — auto-activates when replicaCount > 1.
  • servicemonitor — Prometheus Operator integration.

Prerequisites

Requirement Version / Notes
Kubernetes 1.25+ (Chart.yaml enforces kubeVersion: ">=1.25.0-0")
Helm 3.8+
StorageClass A default StorageClass that supports ReadWriteOnce PVCs (for Postgres, Ollama). Set global.storageClass to pick a non-default class.
Ingress controller Only if ingress.enabled=true. The chart's defaults assume nginx.
cert-manager Only if you want auto-issued TLS certificates. See cert-manager docs.
metrics-server Only if autoscaling.enabled=true (HPA needs metrics).
External Secrets Operator Only if externalSecrets.enabled=true. See ESO docs.
Prometheus Operator Only if monitoring.serviceMonitor.enabled=true.
Namespace PSS labels Recommended: pod-security.kubernetes.io/enforce=restricted. The chart's pod and container security contexts are PSS-restricted by default.

Generate required secrets

Sim will not start without these. Generate them once and feed them via --set, an existing Kubernetes Secret, or External Secrets Operator.

# Application secrets (32 bytes hex each)
openssl rand -hex 32   # BETTER_AUTH_SECRET    - signs auth JWTs
openssl rand -hex 32   # ENCRYPTION_KEY        - encrypts sensitive env vars
openssl rand -hex 32   # INTERNAL_API_SECRET   - service-to-service auth
openssl rand -hex 32   # CRON_SECRET           - required if cronjobs.enabled (default true)
openssl rand -hex 32   # API_ENCRYPTION_KEY    - optional; encrypts user API keys at rest

# Postgres password
openssl rand -base64 24 | tr -d '/+='

If you set app.secrets.existingSecret.enabled=true and point at a pre-created Secret, you do not also pass these via --set — pick one path.


Installing the chart

From this repository

helm install sim ./helm/sim \
  --namespace sim --create-namespace \
  --set app.env.BETTER_AUTH_SECRET="$BETTER_AUTH_SECRET" \
  --set app.env.ENCRYPTION_KEY="$ENCRYPTION_KEY" \
  --set app.env.INTERNAL_API_SECRET="$INTERNAL_API_SECRET" \
  --set app.env.CRON_SECRET="$CRON_SECRET" \
  --set postgresql.auth.password="$POSTGRES_PASSWORD"

With a values file

helm install sim ./helm/sim \
  --namespace sim --create-namespace \
  --values my-values.yaml

Run helm template ./helm/sim --values my-values.yaml | less first to see what will be applied.

Validate the install

helm install sim ./helm/sim --dry-run --debug \
  --values my-values.yaml \
  --set app.env.BETTER_AUTH_SECRET=$(openssl rand -hex 16) \
  --set app.env.ENCRYPTION_KEY=$(openssl rand -hex 16) \
  --set app.env.INTERNAL_API_SECRET=$(openssl rand -hex 16) \
  --set app.env.CRON_SECRET=$(openssl rand -hex 16) \
  --set postgresql.auth.password=$(openssl rand -base64 12 | tr -d '/+=')

Upgrading

helm upgrade sim ./helm/sim --namespace sim --values my-values.yaml

Uninstalling

helm uninstall sim --namespace sim

PVCs are not deleted by helm uninstall. If you want to wipe data too:

# WARNING: this destroys all Postgres, Ollama, and shared-storage data.
kubectl delete pvc --namespace sim \
  -l app.kubernetes.io/instance=sim

# Or list and delete by name
kubectl get pvc --namespace sim
kubectl delete pvc <pvc-name> --namespace sim

# Then delete the namespace if you're done with it
kubectl delete namespace sim

Examples

Pre-built values files for common scenarios live in helm/sim/examples/. Each file has a header explaining when to use it and any prerequisites.

File When to use
values-development.yaml Local dev / kind / minikube. Minimal resources, no TLS.
values-production.yaml Generic production: HA, network policy, autoscaling, monitoring.
values-aws.yaml EKS — EBS GP3 storage, ALB ingress, IRSA-friendly.
values-gcp.yaml GKE — Persistent Disk storage, GCP managed certs, Workload Identity.
values-azure.yaml AKS — managed-csi storage, NGINX ingress, GPU node pools.
values-external-db.yaml Production with a managed Postgres (RDS, Cloud SQL, Azure DB).
values-external-secrets.yaml Sync secrets from Vault / AWS SM / Azure KV / GCP SM via External Secrets Operator.
values-existing-secret.yaml GitOps / Sealed Secrets / SOPS — reference pre-created Kubernetes Secrets.
values-copilot.yaml Enables the Copilot service + its Postgres StatefulSet.
values-whitelabeled.yaml Custom branding (logo, name, support links).

Use one with:

helm install sim ./helm/sim \
  --namespace sim --create-namespace \
  --values ./helm/sim/examples/values-production.yaml \
  --set app.env.BETTER_AUTH_SECRET="$BETTER_AUTH_SECRET" \
  --set app.env.ENCRYPTION_KEY="$ENCRYPTION_KEY" \
  --set app.env.INTERNAL_API_SECRET="$INTERNAL_API_SECRET" \
  --set postgresql.auth.password="$POSTGRES_PASSWORD"

Parameters

This chart is intentionally configurable. Rather than maintain a hand-curated parameter table (which would drift), read the canonical sources:

# Print all values with comments and defaults
helm show values ./helm/sim

# Print the JSON Schema (used by `helm install` to validate your values)
cat ./helm/sim/values.schema.json

values.yaml is heavily commented; each top-level section explains what it controls and which sub-keys are required vs optional. For per-cloud examples and idiomatic overrides, see examples/.


Production checklist

Before installing in production, confirm each of the following:

  • High availability — scale app.replicaCount > 1. The chart auto-creates a PodDisruptionBudget with maxUnavailable: "25%". Set podDisruptionBudget.minAvailable instead for a stricter policy.

  • Pinned images — override image.tag (or image.digest) with an explicit version. Do not rely on the chart's default tag in production.

  • Secrets management — provide secrets via External Secrets Operator (ESO) or pre-created Kubernetes Secrets. Never commit secrets to values.yaml.

  • TLS / Ingress — set the cert-manager.io/cluster-issuer annotation on the ingress and tune proxy-body-size / proxy-read-timeout for your workload. See commented examples in values.yaml.

  • Network policy egress — review networkPolicy.egressExceptCidrs. Defaults block cloud metadata endpoints (169.254.169.254/32, 169.254.170.2/32); add your cluster's API server CIDR for stronger isolation. Custom egress rules go in networkPolicy.egress (a list).

    Every datastore you run outside the chart needs its own egress rule. The default policy allows HTTPS (443) plus the bundled Postgres and Redis by pod selector — nothing else on a non-443 port. So a managed Postgres, a managed Redis, or any REDIS_URL you supply through a Secret is reachable only if you add a rule for it. This bites hardest when the URL comes from a Secret, because the chart cannot see the host and cannot generate the rule for you:

    networkPolicy:
      enabled: true
      egress:
        - to:
            - ipBlock:
                cidr: 10.0.0.0/16   # your VPC / managed-service subnet
          ports:
            - protocol: TCP
              port: 6379           # managed Redis
            - protocol: TCP
              port: 5432           # managed Postgres

    If you would rather not maintain CIDR lists, networkPolicy.allowExternalEgress: true drops the port restriction entirely while still blocking the cloud metadata endpoints. It defaults to false — this chart is deliberately stricter than the common chart default of unrestricted egress.

  • Network policy ingressnetworkPolicy.ingressFrom defaults to [{}] (an empty peer selector), which allows ingress traffic from any pod in the cluster, not just your ingress controller. This is a deliberate simple default, not a locked-down one. On a shared or multi-tenant cluster, scope it down, e.g. to the ingress-nginx namespace:

    networkPolicy:
      ingressFrom:
        - namespaceSelector:
            matchLabels:
              kubernetes.io/metadata.name: ingress-nginx
  • Namespace hardening — label the install namespace with Pod Security Standards restricted enforcement (pod-security.kubernetes.io/enforce=restricted). All workloads set runAsNonRoot, drop all Linux capabilities, disable privilege escalation, and set seccompProfile: RuntimeDefault — the four controls the Restricted profile requires. readOnlyRootFilesystem is intentionally not defaulted anywhere (Postgres/Ollama genuinely need a writable root; the stateless services — realtime, pii, copilot — could tolerate it but aren't pre-wired with a /tmp emptyDir). If your policy requires it, set <component>.securityContext.readOnlyRootFilesystem: true and mount an emptyDir at /tmp yourself via extraVolumes/extraVolumeMounts.

  • Env validation — keys under app.env, realtime.env, and copilot.env are passed through to the application and validated at startup. The JSON Schema intentionally does not enforce additionalProperties: false (would break custom user envs), so typos like OPENA_API_KEY (instead of OPENAI_API_KEY) surface as missing-key errors at runtime, not at helm install time. Review your env block carefully.

  • Set public URLsapp.env.NEXT_PUBLIC_APP_URL and app.env.BETTER_AUTH_URL must match your public origin (e.g. https://sim.example.com). Leaving them as localhost breaks sign-in.


Secrets

The chart supports three ways to provide secrets, in increasing order of production-readiness:

1. Inline --set (dev / dry-run only)

helm install sim ./helm/sim --set app.env.BETTER_AUTH_SECRET=...

Discouraged for production — values land in helm get values output.

2. Pre-existing Kubernetes Secret

Create the Secret first, then reference it:

kubectl create secret generic sim-app-secrets --namespace sim \
  --from-literal=BETTER_AUTH_SECRET=$(openssl rand -hex 32) \
  --from-literal=ENCRYPTION_KEY=$(openssl rand -hex 32) \
  --from-literal=INTERNAL_API_SECRET=$(openssl rand -hex 32) \
  --from-literal=CRON_SECRET=$(openssl rand -hex 32)

kubectl create secret generic sim-postgres-secret --namespace sim \
  --from-literal=POSTGRES_PASSWORD=$(openssl rand -base64 24 | tr -d '/+=')
app:
  secrets:
    existingSecret:
      enabled: true
      name: sim-app-secrets

postgresql:
  auth:
    existingSecret:
      enabled: true
      name: sim-postgres-secret   # must contain the password under the key POSTGRES_PASSWORD

See examples/values-existing-secret.yaml.

3. External Secrets Operator (recommended)

Sync from Azure Key Vault, AWS Secrets Manager, HashiCorp Vault, or GCP Secret Manager. Install ESO once, create a ClusterSecretStore, then:

externalSecrets:
  enabled: true
  refreshInterval: 1h
  secretStoreRef:
    name: my-secret-store
    kind: ClusterSecretStore
  remoteRefs:
    app:
      BETTER_AUTH_SECRET: sim/app/better-auth-secret
      ENCRYPTION_KEY: sim/app/encryption-key
      INTERNAL_API_SECRET: sim/app/internal-api-secret
    postgresql:
      password: sim/postgresql/password
    # Only needed when copilot.enabled=true and copilot.server.secret.create=true.
    # Every non-empty copilot.server.env key must have a matching entry here —
    # template rendering fails with a clear message naming the missing key otherwise.
    copilot:
      AGENT_API_DB_ENCRYPTION_KEY: sim/copilot/agent-api-db-encryption-key
      INTERNAL_API_SECRET: sim/copilot/internal-api-secret
      LICENSE_KEY: sim/copilot/license-key
      SIM_BASE_URL: sim/copilot/sim-base-url
      SIM_AGENT_API_KEY: sim/copilot/sim-agent-api-key
      REDIS_URL: sim/copilot/redis-url
      OPENAI_API_KEY_1: sim/copilot/openai-api-key

See examples/values-external-secrets.yaml.


Persistence

Postgres, Ollama, and any configured sharedStorage.volumes[] use PersistentVolumeClaims. PVCs survive helm uninstall — see Uninstalling for full cleanup.

Component Default size Access mode Storage class
postgresql 10Gi ReadWriteOnce global.storageClass
copilot.postgresql 10Gi ReadWriteOnce global.storageClass
ollama 100Gi ReadWriteOnce global.storageClass
sharedStorage.volumes[] user-defined ReadWriteMany recommended sharedStorage.storageClass

For production, use a StorageClass with reclaimPolicy: Retain on database volumes.


Security

The chart applies Pod Security Standards restricted defaults to every workload:

  • runAsNonRoot: true
  • allowPrivilegeEscalation: false
  • capabilities.drop: [ALL]
  • seccompProfile.type: RuntimeDefault

User-supplied securityContext values are merged with the defaults — your values win, but you don't have to repeat the defaults.

Other security features:

  • automountServiceAccountToken: false on the ServiceAccount and every pod.
  • Every value in app.env and realtime.env is written to a chart-managed Secret and mounted via envFrom: secretRef — no values are inlined on the container spec. This eliminates a sensitivity classifier (no static list of "secret" keys to maintain) and ensures new provider keys can never accidentally leak into pod manifests. Two categories are inlined on the container instead: chart-computed values (DATABASE_URL, SOCKET_SERVER_URL, OLLAMA_URL, PII_URL) and operational defaults under app.envDefaults / realtime.envDefaults (rate limits, timeouts, IVM tunables, feature-flag defaults, branding defaults, http://localhost:3000 URL fallbacks). Operational defaults are non-sensitive by design — moving them out of app.env keeps the Secret small and means External Secrets Operator users only have to map the keys they actually set, not every chart default. A value placed in app.env always wins over the same key in app.envDefaults (the template skips the inline default when an override exists).
  • Optional networkPolicy.enabled=true enforces east-west isolation and blocks cloud metadata endpoints in egress.

Autoscaling

autoscaling:
  enabled: true
  minReplicas: 2
  maxReplicas: 20
  targetCPUUtilizationPercentage: 70
  targetMemoryUtilizationPercentage: 80

When autoscaling.enabled=true, the chart omits spec.replicas from the Deployment so the HPA owns replica count. Requires metrics-server in the cluster. The realtime Deployment gets the same HPA unless autoscaling.realtime.enabled=false — scale realtime past one replica only with REDIS_URL set (Socket.IO Redis adapter), or cross-pod collaboration events are dropped.


Monitoring

monitoring:
  serviceMonitor:
    enabled: true
    interval: 30s

Requires the Prometheus Operator CRDs. Scrapes /metrics on the app and realtime services — note the default images do not currently expose a /metrics endpoint, so enable this only with a build that does.


PII redaction

Sim can redact personally identifiable information using a Presidio service (analyzer + anonymizer combined into one image listening on port 5001). Enable it with:

pii:
  enabled: true

When enabled, the chart deploys it as a standalone <release>-pii Deployment + Service and auto-wires PII_URL on the app to the in-cluster service. The service bundles five large spaCy models (en/es/it/pl/fi, ~2.2GB), so the first start takes ~3 minutes while models load — the startupProbe allows for this. Size the pii.resources for at least ~4Gi memory.

This alone powers the Guardrails PII block and on-demand masking. To additionally turn on automatic log redaction (the org/workspace data-retention scrub), you must:

app:
  env:
    PII_REDACTION: "true"
    # The log-redaction path calls the app's own /api/guardrails/mask-batch,
    # which must be reachable from inside the cluster. Set this to the in-cluster
    # app Service URL (NOT the public ingress, which usually isn't hairpin-reachable).
    INTERNAL_API_BASE_URL: "http://<release>-app.<namespace>.svc.cluster.local:3000"

Without a cluster-reachable INTERNAL_API_BASE_URL (it falls back to NEXT_PUBLIC_APP_URL), the redaction path fails closed — it scrubs affected fields to [REDACTION_FAILED] rather than leaking, but redaction won't actually run.

The PII image is published at ghcr.io/simstudioai/pii (multi-arch). If you mirror images into a private registry, retag it alongside the app/realtime/migrations images.


Troubleshooting

Error: execution error at (sim/templates/...): app.env.BETTER_AUTH_SECRET is required for production deployment

You ran helm install without setting required secrets. Generate them and pass with --set:

helm install sim ./helm/sim \
  --set app.env.BETTER_AUTH_SECRET=$(openssl rand -hex 32) \
  --set app.env.ENCRYPTION_KEY=$(openssl rand -hex 32) \
  --set app.env.INTERNAL_API_SECRET=$(openssl rand -hex 32) \
  --set postgresql.auth.password=$(openssl rand -base64 24 | tr -d '/+=')

App pods stuck in CrashLoopBackOff

kubectl logs --namespace sim deploy/sim-app --tail 200

Common causes:

  • NEXT_PUBLIC_APP_URL still set to http://localhost:3000 in a clustered deploy → set it to your public origin.
  • DATABASE_URL not reachable → check the Postgres pod is running and postgresql.auth.password matches.
  • Missing migration → check kubectl logs deploy/sim-app -c migrations (migrations run as an init container on the app pod).

Image pull errors (ErrImagePull / ImagePullBackOff)

  • You pushed Sim to a private registry but haven't configured pull secrets. Set global.imagePullSecrets and global.imageRegistry.
  • You overrode image.tag to a tag that doesn't exist in the registry. helm get values sim and verify.

Postgres pod Pending

kubectl describe pvc --namespace sim

Almost always one of:

  • No default StorageClass → set global.storageClass.
  • No PV provisioner → install one (e.g. EBS CSI on EKS, local-path-provisioner for dev).
  • StorageClass exists but doesn't support ReadWriteOnce → pick another class.

Ingress not routing

kubectl get ingress --namespace sim
kubectl describe ingress --namespace sim
  • Ingress controller not installed → install ingress-nginx or similar.
  • ingress.className doesn't match your controller → set it to your installed class.
  • DNS not pointed at the ingress's external IP / LoadBalancer.

Get logs from each component

kubectl --namespace sim logs -f deployment/sim-app
kubectl --namespace sim logs -f deployment/sim-realtime
kubectl --namespace sim logs -f statefulset/sim-postgresql
kubectl --namespace sim logs deploy/sim-app -c migrations

Upgrading to 1.5.0

Two changes alter behavior on an existing release. Neither requires action, but read both.

  • Free-tier plan limits are no longer preset. app.envDefaults previously shipped RATE_LIMIT_FREE_SYNC, RATE_LIMIT_FREE_ASYNC, EXECUTION_TIMEOUT_FREE, EXECUTION_TIMEOUT_ASYNC_FREE, FREE_TABLES_LIMIT: 3, and FREE_TABLE_ROWS_LIMIT: 1000. With billing disabled the application treats these as opt-in — unset means unlimited — so presetting them imposed hosted-plan caps on self-hosted deployments and diverged from Docker Compose, which presets nothing. They are now commented out. On upgrade, these limits stop being enforced. To keep them, set the keys explicitly under app.env. An explicitly set value has always taken precedence and is unaffected.

  • Redis is now bundled (redis.enabled: true), matching the Docker Compose stack. Redis backs pub/sub and the Socket.IO adapter, and multi-replica deployments silently drop cross-pod events without it.

    An existing REDIS_URL always wins, wherever it comes from — no action needed on upgrade. The bundled URL ships as a ConfigMap listed before the app Secret in envFrom. Kubernetes resolves duplicate keys by letting the last source win, so a REDIS_URL in your chart-managed Secret, a pre-created existingSecret, or one synced by External Secrets overrides the bundled value — the chart never has to read it. The bundled Redis simply fills the gap when nothing else provides a URL.

    Set app.env.REDIS_URL to skip the bundled Deployment entirely (no unused pod), or redis.enabled: false to opt out.

Upgrading to 1.2.0

  • appVersion (the default image tag when image.tag is unset) is now v0.7.44 — the previous 0.6.73 referenced a tag that does not exist on GHCR, so an unpinned default install could not pull images. Production installs should still pin image.tag explicitly.
  • externalSecrets.apiVersion now defaults to "v1" — current External Secrets Operator releases no longer serve v1beta1 (removed upstream in 2026). Set externalSecrets.apiVersion: "v1beta1" only if you still run ESO < 0.17.
  • values.schema.json now declares every top-level key and rejects unknown top-level keys, so a typo like networkPolciy: fails fast at install time instead of being silently ignored. If an upgrade suddenly fails schema validation, check your values file for stray top-level keys.
  • The opt-in telemetry collector no longer ships a Prometheus scrape config for the app/realtime services (they expose no /metrics endpoint); OTLP ingestion is unchanged.

Upgrading to 1.1.0

No action is required for working configurations. Notes:

  • Pods for app and realtime roll once on upgrade (their rollout checksum now also covers the ExternalSecret manifest, fixing missed rollouts in ESO mode).
  • Two values keys that were never consumed by any template were removed: app.secrets.existingSecret.keys and *.existingSecret.passwordKey. Existing secrets must use the standard key names (BETTER_AUTH_SECRET, ..., POSTGRES_PASSWORD, EXTERNAL_DB_PASSWORD); leftover keys in your values file are ignored, not rejected.
  • telemetry.jaeger now exports over OTLP (otlp/jaeger) — point telemetry.jaeger.endpoint at Jaeger's OTLP gRPC port (4317). The previous jaeger exporter did not exist in the pinned collector image, so any prior jaeger-enabled config was already failing at collector startup.

Support


License

Apache-2.0 © Sim. See LICENSE.