COS-lite-k8s docs - Set up microceph

Table of Contents

MicroCeph is a lightweight way of deploying a Ceph cluster with a focus on reduced ops that can be used with MicroK8s.

Single-node MicroCeph installation

Install MicroCeph

Install the most recent stable release of MicroCeph:

sudo snap install microceph

Next, as MicroCeph creator recommend, prevent the software from being auto-updated:

sudo snap refresh --hold microceph

Allowing the snap to be auto-updated can lead to unintended consequences. In enterprise environments especially, it is better to research the ramifications of software changes before those changes are implemented.

We need to bootstrap the Ceph cluster:

sudo microceph cluster bootstrap

At this point we can check the status of the cluster and query the list of available disks that should be empty. The disk status is queried with:

sudo microceph.ceph status

Its output should look like:

  cluster:
    id:     9539a8ee-825a-462a-94fa-15613c09cab1
    health: HEALTH_WARN
            mon charm-dev-juju-34 is low on available space

  services:
    mon: 1 daemons, quorum charm-dev-juju-34 (age 8s)
    mgr: charm-dev-juju-34(active, since 3s)
    osd: 0 osds: 0 up, 0 in

  data:
    pools:   0 pools, 0 pgs
    objects: 0 objects, 0 B
    usage:   0 B used, 0 B / 0 B avail
    pgs:

Add storage

Three OSDs will be required to form a minimal Ceph cluster. In a production system, typically we would assign a physical block device to an OSD. However for this tutorial, we will make use of file backed OSDs for simplicity.

Add the three file-backed OSDs to the cluster by using the disk add command. In the example, three 4GiB files are being created:

sudo microceph disk add loop,4G,3

Connect MicroCeph to MicroK8s

Enable rook-ceph plugin by executing:

sudo microk8s enable rook-ceph

See more: rook-ceph addon

As we have already setup MicroCeph having it managed by rook is done with just:

sudo microk8s connect-external-ceph

At the end of this process you should have a storage class ready to use:

NAME       PROVISIONER                  RECLAIMPOLICY   VOLUMEBINDINGMODE   ALLOWVOLUMEEXPANSION   AGE
ceph-rbd   rook-ceph.rbd.csi.ceph.com   Delete          Immediate           true                   1s

See also

References


Last updated a month ago.