An overview of the various sections of the Replicated YAML.

Persistent Volumes

Kubernetes applications often rely on persistent volumes (PVs) and persistent volume claims (PVCs) to manage data. When Replicated creates a Kubernetes appliance, a custom storage class named default is automatically created that will be available to the application for persistent volumes provisioned by Rook. Replicated will set the storageClassName on all PersistentVolumeClaims in your application, allowing customers to use an alternative provisioner, such as standard on GKE.

Fine-Grained Provisioniong

Add the replicated.com/no-rewrite-storage-class annotation to any PVC or StatefulSet’s volumeClaimTemplate to prevent Replicated from rewriting the storageClassName of that volume. You will need to ensure that an alternative storageClassName is set and a provisioner for that class is running in the customer’s environment.

Shared Filesystem

Known Issues

Replicated 2.36.0 to 2.38.2 had a race condition that could cause Pods to run after failing to mount the shared filesystem. Affected Pods would be writing to the ephemeral container filesystem rather than the shared filesystem.

Rook is not able to provision PVCs with an access mode of ReadWriteMany, but it does support a shared filesystem that can be mounted as a flexVolume. This can be enabled in the kubernetes section of your yaml.

    enabled: true
    - /subdir1

Templating is supported on the enabled and mount_paths properties.

Once enabled, you can mount the shared fileystem in any of your pods, either from the root or at a subpath as shown in this example:

#kind: scheduler-kubernetes
apiVersion: apps/v1
kind: Deployment
  name: my-deployment
  replicas: 2
      app: my-deployment
        app: my-deployment
      - name: ubuntu
        image: index.docker.io/ubuntu:16.04
        - /bin/sh
        - -c
        - 'sleep 3600'
        - name: shared
          mountPath: /var/lib/shared
      - name: shared
          driver: ceph.rook.io/rook
          fsType: ceph
            fsName: rook-shared-fs
            clusterNamespace: rook-ceph
            path: /subdir1 #optional

When mounting a subdirectory as shown in the example above, you must add the path you are mounting to kubernetes.shared_fs.mount_paths. Paths within the shared filesystem can be included in snapshots by adding them to the backup section of your yaml.

Shared Filesystem InitContainer

Replicated injects an initContainer into any pods that mount the shared filesystem to verify the mount succeeded. This will be the first initContainer in the pod. In the case of a failed mount, the initContainer will force delete the pod with grace-period=0. In order to delete the pod, Replicated binds a Role, pod-deleter, to the default service account in the application namespace that allows “delete” on pods in that namespace. This will cause kubelet to rebuild the pod sandbox and retry the mount. This will prevent the application from losing data by inadvertently writing to the ephemeral container filesystem instead of the shared filesystem.


For more information on using Persistent Volumes with Kubernetes, see the Kubernetes Documentation.

For information on snapshotting Kubernetes volumes, see the Replicated snapshots documentation for Kubernetes.

For more information on managing the storage needs of Kubernetes in customer environments, see Managing Storage.