Skip to content

Commit 7876dc5

Browse files
johnstcnjohnstcn
authored
feat(helm/provisioner): support deploying multiple provisioners in same namespace (coder#15637)
Fixes coder#15437 - Adds support for `coder.serviceAccount.disableCreate` (originally added to `helm/coder` in coder#14817). - Adds documentation and examples in `helm/provisioner/README.md` on deploying multiple provisioners in the same namespace leveraging `nameOverride`.
1 parent e6506f0 commit 7876dc5

11 files changed

Lines changed: 439 additions & 2 deletions

‎helm/coder/values.yaml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -113,7 +113,7 @@ coder:
113113
annotations: {}
114114
# coder.serviceAccount.name -- The service account name
115115
name: coder
116-
# coder.serviceAccount.name -- Whether to create the service account or use existing service account
116+
# coder.serviceAccount.disableCreate -- Whether to create the service account or use existing service account.
117117
disableCreate: false
118118

119119
# coder.securityContext -- Fields related to the container's security

‎helm/provisioner/README.md‎

Lines changed: 114 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -32,5 +32,118 @@ coder:
3232
value: "0.0.0.0:2112"
3333
replicaCount: 10
3434
provisionerDaemon:
35-
pskSecretName: "coder-provisioner-psk"
35+
keySecretName: "coder-provisionerd-key"
36+
keySecretKey: "provisionerd-key"
3637
```
38+
39+
## Specific Examples
40+
41+
Below are some common specific use-cases when deploying a Coder provisioner.
42+
43+
### Set Labels and Annotations
44+
45+
If you need to set deployment- or pod-level labels and annotations, set `coder.{annotations,labels}` or `coder.{podAnnotations,podLabels}`.
46+
47+
Example:
48+
49+
```yaml
50+
coder:
51+
# ...
52+
annotations:
53+
com.coder/annotation/foo: bar
54+
com.coder/annotation/baz: qux
55+
labels:
56+
com.coder/label/foo: bar
57+
com.coder/label/baz: qux
58+
podAnnotations:
59+
com.coder/podAnnotation/foo: bar
60+
com.coder/podAnnotation/baz: qux
61+
podLabels:
62+
com.coder/podLabel/foo: bar
63+
com.coder/podLabel/baz: qux
64+
```
65+
66+
### Additional Templates
67+
68+
You can include extra Kubernetes manifests in `extraTemplates`.
69+
70+
The below example will also create a `ConfigMap` along with the Helm release:
71+
72+
```yaml
73+
coder:
74+
# ...
75+
provisionerDaemon:
76+
# ...
77+
extraTemplates:
78+
- |
79+
apiVersion: v1
80+
kind: ConfigMap
81+
metadata:
82+
name: some-config
83+
namespace: {{ .Release.Namespace }}
84+
data:
85+
key: some-value
86+
```
87+
88+
### Disable Service Account Creation
89+
90+
### Deploying multiple provisioners in the same namespace
91+
92+
To deploy multiple provisioners in the same namespace, set the following values explicitly to avoid conflicts:
93+
94+
- `nameOverride`: controls the name of the provisioner deployment
95+
- `serviceAccount.name`: controls the name of the service account.
96+
97+
Note that `nameOverride` does not apply to `extraTemplates`, as illustrated below:
98+
99+
```yaml
100+
coder:
101+
# ...
102+
serviceAccount:
103+
name: other-coder-provisioner
104+
provisionerDaemon:
105+
# ...
106+
nameOverride: "other-coder-provisioner"
107+
extraTemplates:
108+
- |
109+
apiVersion: v1
110+
kind: ConfigMap
111+
metadata:
112+
name: some-other-config
113+
namespace: {{ .Release.Namespace }}
114+
data:
115+
key: some-other-value
116+
```
117+
118+
If you wish to deploy a second provisioner that references an existing service account, you can do so as follows:
119+
120+
- Set `coder.serviceAccount.disableCreate=true` to disable service account creation,
121+
- Set `coder.serviceAccount.workspacePerms=false` to disable creation of a role and role binding,
122+
- Set `coder.serviceAccount.nameOverride` to the name of an existing service account.
123+
124+
See below for a concrete example:
125+
126+
```yaml
127+
coder:
128+
# ...
129+
serviceAccount:
130+
name: preexisting-service-account
131+
disableCreate: true
132+
workspacePerms: false
133+
provisionerDaemon:
134+
# ...
135+
nameOverride: "other-coder-provisioner"
136+
```
137+
138+
# Testing
139+
140+
The test suite for this chart lives in `./tests/chart_test.go`.
141+
142+
Each test case runs `helm template` against the corresponding `test_case.yaml`, and compares the output with that of the corresponding `test_case.golden` in `./tests/testdata`.
143+
If `expectedError` is not empty for that specific test case, no corresponding `.golden` file is required.
144+
145+
To add a new test case:
146+
147+
- Create an appropriately named `.yaml` file in `testdata/` along with a corresponding `.golden` file, if required.
148+
- Add the test case to the array in `chart_test.go`, setting `name` to the name of the files you added previously (without the extension). If appropriate, set `expectedError`.
149+
- Run the tests and ensure that no regressions in existing test cases occur: `go test ./tests`.

‎helm/provisioner/templates/coder.yaml‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,7 @@
11
---
2+
{{- if not .Values.coder.serviceAccount.disableCreate }}
23
{{ include "libcoder.serviceaccount" (list . "coder.serviceaccount") }}
4+
{{- end }}
35

46
---
57
{{ include "libcoder.deployment" (list . "coder.deployment") }}

‎helm/provisioner/tests/chart_test.go‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -78,6 +78,18 @@ var testCases = []testCase{
7878
name: "extra_templates",
7979
expectedError: "",
8080
},
81+
{
82+
name: "sa_disabled",
83+
expectedError: "",
84+
},
85+
{
86+
name: "name_override",
87+
expectedError: "",
88+
},
89+
{
90+
name: "name_override_existing_sa",
91+
expectedError: "",
92+
},
8193
}
8294

8395
type testCase struct {

‎helm/provisioner/tests/testdata/name_override.golden‎

Lines changed: 144 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,144 @@
1+
---
2+
# Source: coder-provisioner/templates/coder.yaml
3+
apiVersion: v1
4+
kind: ServiceAccount
5+
metadata:
6+
annotations: {}
7+
labels:
8+
app.kubernetes.io/instance: release-name
9+
app.kubernetes.io/managed-by: Helm
10+
app.kubernetes.io/name: other-coder-provisioner
11+
app.kubernetes.io/part-of: other-coder-provisioner
12+
app.kubernetes.io/version: 0.1.0
13+
helm.sh/chart: coder-provisioner-0.1.0
14+
name: other-coder-provisioner
15+
---
16+
# Source: coder-provisioner/templates/extra-templates.yaml
17+
apiVersion: v1
18+
kind: ConfigMap
19+
metadata:
20+
name: some-config
21+
namespace: default
22+
data:
23+
key: some-value
24+
---
25+
# Source: coder-provisioner/templates/rbac.yaml
26+
apiVersion: rbac.authorization.k8s.io/v1
27+
kind: Role
28+
metadata:
29+
name: other-coder-provisioner-workspace-perms
30+
rules:
31+
- apiGroups: [""]
32+
resources: ["pods"]
33+
verbs:
34+
- create
35+
- delete
36+
- deletecollection
37+
- get
38+
- list
39+
- patch
40+
- update
41+
- watch
42+
- apiGroups: [""]
43+
resources: ["persistentvolumeclaims"]
44+
verbs:
45+
- create
46+
- delete
47+
- deletecollection
48+
- get
49+
- list
50+
- patch
51+
- update
52+
- watch
53+
- apiGroups:
54+
- apps
55+
resources:
56+
- deployments
57+
verbs:
58+
- create
59+
- delete
60+
- deletecollection
61+
- get
62+
- list
63+
- patch
64+
- update
65+
- watch
66+
---
67+
# Source: coder-provisioner/templates/rbac.yaml
68+
apiVersion: rbac.authorization.k8s.io/v1
69+
kind: RoleBinding
70+
metadata:
71+
name: "other-coder-provisioner"
72+
subjects:
73+
- kind: ServiceAccount
74+
name: "other-coder-provisioner"
75+
roleRef:
76+
apiGroup: rbac.authorization.k8s.io
77+
kind: Role
78+
name: other-coder-provisioner-workspace-perms
79+
---
80+
# Source: coder-provisioner/templates/coder.yaml
81+
apiVersion: apps/v1
82+
kind: Deployment
83+
metadata:
84+
annotations: {}
85+
labels:
86+
app.kubernetes.io/instance: release-name
87+
app.kubernetes.io/managed-by: Helm
88+
app.kubernetes.io/name: other-coder-provisioner
89+
app.kubernetes.io/part-of: other-coder-provisioner
90+
app.kubernetes.io/version: 0.1.0
91+
helm.sh/chart: coder-provisioner-0.1.0
92+
name: other-coder-provisioner
93+
spec:
94+
replicas: 1
95+
selector:
96+
matchLabels:
97+
app.kubernetes.io/instance: release-name
98+
app.kubernetes.io/name: other-coder-provisioner
99+
template:
100+
metadata:
101+
annotations: {}
102+
labels:
103+
app.kubernetes.io/instance: release-name
104+
app.kubernetes.io/managed-by: Helm
105+
app.kubernetes.io/name: other-coder-provisioner
106+
app.kubernetes.io/part-of: other-coder-provisioner
107+
app.kubernetes.io/version: 0.1.0
108+
helm.sh/chart: coder-provisioner-0.1.0
109+
spec:
110+
containers:
111+
- args:
112+
- provisionerd
113+
- start
114+
command:
115+
- /opt/coder
116+
env:
117+
- name: CODER_PROMETHEUS_ADDRESS
118+
value: 0.0.0.0:2112
119+
- name: CODER_PROVISIONER_DAEMON_PSK
120+
valueFrom:
121+
secretKeyRef:
122+
key: psk
123+
name: coder-provisioner-psk
124+
- name: CODER_URL
125+
value: http://coder.default.svc.cluster.local
126+
image: ghcr.io/coder/coder:latest
127+
imagePullPolicy: IfNotPresent
128+
lifecycle: {}
129+
name: coder
130+
ports: null
131+
resources: {}
132+
securityContext:
133+
allowPrivilegeEscalation: false
134+
readOnlyRootFilesystem: null
135+
runAsGroup: 1000
136+
runAsNonRoot: true
137+
runAsUser: 1000
138+
seccompProfile:
139+
type: RuntimeDefault
140+
volumeMounts: []
141+
restartPolicy: Always
142+
serviceAccountName: other-coder-provisioner
143+
terminationGracePeriodSeconds: 600
144+
volumes: []

‎helm/provisioner/tests/testdata/name_override.yaml‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
coder:
2+
image:
3+
tag: latest
4+
serviceAccount:
5+
name: other-coder-provisioner
6+
nameOverride: "other-coder-provisioner"
7+
# Note that extraTemplates does not respect nameOverride.
8+
extraTemplates:
9+
- |
10+
apiVersion: v1
11+
kind: ConfigMap
12+
metadata:
13+
name: some-config
14+
namespace: {{ .Release.Namespace }}
15+
data:
16+
key: some-value

‎helm/provisioner/tests/testdata/name_override_existing_sa.golden‎

Lines changed: 67 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,67 @@
1+
---
2+
# Source: coder-provisioner/templates/coder.yaml
3+
---
4+
apiVersion: apps/v1
5+
kind: Deployment
6+
metadata:
7+
annotations: {}
8+
labels:
9+
app.kubernetes.io/instance: release-name
10+
app.kubernetes.io/managed-by: Helm
11+
app.kubernetes.io/name: other-coder-provisioner
12+
app.kubernetes.io/part-of: other-coder-provisioner
13+
app.kubernetes.io/version: 0.1.0
14+
helm.sh/chart: coder-provisioner-0.1.0
15+
name: other-coder-provisioner
16+
spec:
17+
replicas: 1
18+
selector:
19+
matchLabels:
20+
app.kubernetes.io/instance: release-name
21+
app.kubernetes.io/name: other-coder-provisioner
22+
template:
23+
metadata:
24+
annotations: {}
25+
labels:
26+
app.kubernetes.io/instance: release-name
27+
app.kubernetes.io/managed-by: Helm
28+
app.kubernetes.io/name: other-coder-provisioner
29+
app.kubernetes.io/part-of: other-coder-provisioner
30+
app.kubernetes.io/version: 0.1.0
31+
helm.sh/chart: coder-provisioner-0.1.0
32+
spec:
33+
containers:
34+
- args:
35+
- provisionerd
36+
- start
37+
command:
38+
- /opt/coder
39+
env:
40+
- name: CODER_PROMETHEUS_ADDRESS
41+
value: 0.0.0.0:2112
42+
- name: CODER_PROVISIONER_DAEMON_PSK
43+
valueFrom:
44+
secretKeyRef:
45+
key: psk
46+
name: coder-provisioner-psk
47+
- name: CODER_URL
48+
value: http://coder.default.svc.cluster.local
49+
image: ghcr.io/coder/coder:latest
50+
imagePullPolicy: IfNotPresent
51+
lifecycle: {}
52+
name: coder
53+
ports: null
54+
resources: {}
55+
securityContext:
56+
allowPrivilegeEscalation: false
57+
readOnlyRootFilesystem: null
58+
runAsGroup: 1000
59+
runAsNonRoot: true
60+
runAsUser: 1000
61+
seccompProfile:
62+
type: RuntimeDefault
63+
volumeMounts: []
64+
restartPolicy: Always
65+
serviceAccountName: existing-coder-provisioner-serviceaccount
66+
terminationGracePeriodSeconds: 600
67+
volumes: []

‎helm/provisioner/tests/testdata/name_override_existing_sa.yaml‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
coder:
2+
image:
3+
tag: latest
4+
serviceAccount:
5+
name: "existing-coder-provisioner-serviceaccount"
6+
disableCreate: true
7+
workspacePerms: false
8+
nameOverride: "other-coder-provisioner"

0 commit comments

Comments
 (0)