OpenNebula Frontend with Ceph Datastore

OpenNebula LogoThe quick guide provide users with step by step guide to configure OpenNebula Frontend with Ceph storage for its datastore.

Prerequisite

  • OpenNebula Frontend is installed as per installation guide in OpenNebula Documentation
  • Ceph Monitors (ceph-mon) is accessible from OpenNebula Frontend. Firewall TCP Ports 6789 – 6999 are opened.
  • ceph.conf and ceph client key (eg ceph.client.oneadmin.key) is available – should be provided by your Storage/Ceph Administrator.
  • Able to resolve Ceph Monitors’s hostname (eg: mon01, mon02, mon03) in OpenNebula Frontend. If required, you may put the entries in /etc/hosts file.
  • A new Ceph pool (eg: opennebula-pool) is created Ceph.

Software Version

In this guide, we will be using the following software / package version.

  • Ubuntu 14.04.x (Trusty)
  • Ceph 0.94 (Hammer)
  • OpenNebula 4.12 (Cotton Candy)

Pre-Installation

Install Ceph repo key

root@opennebula-frontend:~# wget -q -O- 'https://ceph.com/git/?p=ceph.git;a=blob_plain;f=keys/release.asc' | apt-key add -

Enable Ceph repo

root@opennebula-frontend:~# echo "deb http://ceph.com/debian-hammer/ trusty main" > /etc/apt/sources.list.d/ceph.list

Update and Upgrade the packages (if available)

root@opennebula-frontend:~# apt-get update
root@opennebula-frontend:~# apt-get dist-upgrade

Ceph client installation

Install Ceph client packages

root@opennebula-frontend:~# apt-get install ceph ceph-common

Place the ceph.conf and ceph client key (eg ceph.client.oneadmin.key) in /etc/ceph directory.

root@opennebula-frontend:~# ls /etc/ceph/
ceph.conf   ceph.client.oneadmin.key

Ceph configuration

All the following commands after this should be run as “oneadmin” user.

root@opennebula-frontend:~# su - oneadmin 
oneadmin@opennebula-frontend:~#

Prepare a file (eg: secret.xml) with the following content. This file will be used in all OpenNebula Nodes.

<secret ephemeral='no' private='no'>
  <uuid>a7675a71-9323-4545-bd1f-63ca86f22953</uuid>
  <usage type='ceph'>
    <name>client.oneadmin secret</name>
  </usage>
</secret>

Note: The uuid can be replaced by the output of running the uuidgen tool.

Create a new datastore definition file (eg: ceph-ds) with the following content.

NAME = "ceph-ds"
DS_MAD = ceph
TM_MAD = ceph
DISK_TYPE = RBD
POOL_NAME = opennebula-pool
BRIDGE_LIST = opennebula-frontend
CEPH_HOST = "mon01:6789 mon02:6789 mon03:6789"
CEPH_USER = "oneadmin"
CEPH_SECRET = "a7675a71-9323-4545-bd1f-63ca86f22953"
RBD_FORMAT = 2

Create the datastore

oneadmin@opennebula-frontend:~# onedatastore create ceph-ds

Verify the server able to connect to Ceph storage and access to the pool

oneadmin@opennebula-frontend:~# rbd --id oneadmin ls -l -p opennebula-pool

Reboot the server to complete the installation

oneadmin@opennebula-frontend:~# exit
root@opennebula-frontend:~# reboot