Fork Notice:

This repo is a fork of the Leaseweb's maitained cloudstack-csi-driver, which is in-turn a fork of Apalia's cloudstack-csi-driver

This repository provides a Container Storage Interface (CSI) plugin for Apache CloudStack.

• Minimal Kubernetes version: v1.25

• The Kubernetes cluster must run in CloudStack. Tested only in a KVM zone.

• A disk offering with custom size must be available, with type "shared".

• In order to match the Kubernetes node and the CloudStack instance, they should both have the same name. If not, it is also possible to use cloud-init instance metadata to get the instance name: if the node has cloud-init enabled, metadata will be available in /run/cloud-init/instance-data.json; you should then make sure that /run/cloud-init/ is mounted from the node.

• Kubernetes nodes must be in the Root domain, and be created by the CloudStack account whose credentials are used in configuration.

Create the CloudStack configuration file cloud-config.

It should have the following format, defined for the CloudStack Kubernetes Provider:

[Global]
api-url = <CloudStack API URL>
api-key = <CloudStack API Key>
secret-key = <CloudStack API Secret>
ssl-no-verify = <Disable SSL certificate validation: true or false (optional)>

Create a secret named cloudstack-secret in namespace kube-system:

kubectl create secret generic \
  --namespace kube-system \
  --from-file ./cloud-config \
  cloudstack-secret

If you have also deployed the CloudStack Kubernetes Provider, you may use the same secret for both tools.

kubectl apply -f https://github.com/shapeblue/cloudstack-csi-driver/releases/latest/download/manifest.yaml

A storage class can be created manually: see example.

The provisioner value must be csi.cloudstack.apache.org.

The volumeBindingMode must be WaitForFirstConsumer, in order to delay the binding and provisioning of a PersistentVolume until a Pod using the PersistentVolumeClaim is created. It enables the provisioning of volumes in respect to topology constraints (e.g. volume in the right zone).

The storage class must also have a parameter named csi.cloudstack.apache.org/disk-offering-id whose value is the CloudStack disk offering ID.

The tool cloudstack-csi-sc-syncer may also be used to synchronize CloudStack disk offerings to Kubernetes storage classes.

More info...

Note: The VolumeSnapshot CRDs (CustomResourceDefinitions) of version 8.3.0 are installed in this deployment. If you use a different version, please ensure compatibility with your Kubernetes cluster and CSI sidecars.

// TODO: Should we have the crds locally or manually install it from:

kubectl apply -f https://raw.githubusercontent.com/kubernetes-csi/external-snapshotter/v8.3.0/client/config/crd/snapshot.storage.k8s.io_volumesnapshotclasses.yaml
kubectl apply -f https://raw.githubusercontent.com/kubernetes-csi/external-snapshotter/v8.3.0/client/config/crd/snapshot.storage.k8s.io_volumesnapshots.yaml
kubectl apply -f https://raw.githubusercontent.com/kubernetes-csi/external-snapshotter/v8.3.0/client/config/crd/snapshot.storage.k8s.io_volumesnapshotcontents.yaml

Example:

kubectl apply -f ./examples/k8s/pvc.yaml
kubectl apply -f ./examples/k8s/pod.yaml

To build the driver binary:

make build-cloudstack-csi-driver

To build the container images:

make container

For Volume snapshots to be created, the following configurations need to be applied:

kubectl aplly -f 00-snapshot-crds.yaml     # Installs the VolumeSnapshotClass, VolumeSnapshotContent and VolumeSnapshtot CRDs
volume-snapshot-class.yaml                 # Defines VolumeSnapshotClass for CloudStack CSI driver

Once the CRDs are installed, the snapshot can be taken by applying:

kubectl apply ./examples/k8s/snapshot/snapshot.yaml

In order to take the snapshot of a volume, persistentVolumeClaimName should be set to the right PVC name that is bound to the volume whose snapshot is to be taken.

You can check CloudStack volume snapshots if the snapshot was successfully created. If for any reason there was an issue, it can be investgated by checking the logs of the cloudstack-csi-controller pods: cloudstack-csi-controller, csi-snapshotter and snapshot-controller containers

kubectl logs -f <cloudstack-csi-controller pod_name> -n kube-system # defaults to tailing logs of cloudstack-csi-controller
kubectl logs -f <cloudstack-csi-controller pod_name> -n kube-system -c csi-snapshotter
kubectl logs -f <cloudstack-csi-controller pod_name> -n kube-system -c snapshot-controller

To restore a volume snapshot:

1. Restore a snapshot and Use it in a pod

• Create a PVC from the snapshot - for example ./examples/k8s/snapshot/pvc-from-snapshot.yaml
• Apply the configuration:

kubectl apply -f ./examples/k8s/snapshot/pvc-from-snapshot.yaml

• Create a pod that uses the restored PVC; example pod config ./examples/k8s/snapshot/restore-pod.yaml

kubectl apply -f ./examples/k8s/snapshot/restore-pod.yaml

2. To restore a snapshot when using a deployment Update the deployment to point to the restored PVC

spec:
  volumes:
    - name: app-volume
      persistentVolumeClaim:
        claimName: pvc-from-snapshot

• The CSI external-provisioner (a container in the cloudstack-csi-controller pod) sees the new PVC and notices it references a snapshot
• The CSI driver's CreateVolume method is called with a VolumeContentSource that contains the snapshot ID
• The CSI driver creates a new volume from the snapshot (using the CloudStack's createVolume API)
• The new volume is now available as a PV (persistent volume) and is bound to the new PVC
• The volume is NOT attached to any node just by restoring from a snapshot, the volume is only attached to a node when a Pod that uses the new PVC is scheduled on a node
• The CSI driver's ControllerPublishVolume and NodePublishVolume methods are called to attach and mount the volume to the node where the Pod is running

Hence to debug any issues during restoring a snapshot, check the logs of the cloudstack-csi-controller, external-provisioner containers

kubectl logs -f <cloudstack-csi-controller pod_name> -n kube-system # defaults to tailing logs of cloudstack-csi-controller
kubectl logs -f <cloudstack-csi-controller pod_name> -n kube-system -c external-provisioner

• CloudStack Kubernetes Provider - Kubernetes Cloud Controller Manager for Apache CloudStack
• CloudStack documentation on storage
• CSI (Container Storage Interface) specification

Copyright 2021 Apalia SAS

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

        http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.