OpenNebula Node on Ubuntu 14.04 with KVM & Ceph

OpenNebula LogoThis guide is to provide users with step by step guide to install OpenNebula 4.12 using Ubuntu 14.04 as the operating system, KVM as the hypervisor and Ceph as Datastore.

Prerequisite

  • OpenNebula Frontend installation is completed with Ceph Datastore. Refer here for more details.
  • Ceph Monitors (ceph-mon) is accessible from OpenNebula Node. Firewall TCP Ports 6789 – 6999 are opened.
  • Able to resolve Ceph Monitors’s hostname (eg: mon01, mon02, mon03) in OpenNebula Node. If required, you may put the entries in /etc/hosts file.

Software Version

In this guide, we will be using the following software / package version.

  • Ubuntu 14.04.x (Trusty)
  • Ceph 0.94 (Hammer)
  • OpenNebula 4.12 (Cotton Candy)

Pre-Installation

Disable and remove the AppArmor package

root@opennebula-node:~# service apparmor teardown
root@opennebula-node:~# service apparmor stop
root@opennebula-node:~# apt-get purge apparmor

Install Ceph & OpenNebula Repo key

root@opennebula-node:~# wget -q -O- 'https://ceph.com/git/?p=ceph.git;a=blob_plain;f=keys/release.asc' | apt-key add -
root@opennebula-node:~# wget -q -O- http://downloads.opennebula.org/repo/Ubuntu/repo.key | apt-key add -

Enable Ceph & OpenNebula Repo

root@opennebula-node:~# echo "deb http://ceph.com/debian-hammer/ trusty main" > /etc/apt/sources.list.d/ceph.list
root@opennebula-node:~# echo "deb http://downloads.opennebula.org/repo/4.12/Ubuntu/14.04/ stable opennebula" > /etc/apt/sources.list.d/opennebula.list

Update and Upgrade the packages (if available)

root@opennebula-node:~# apt-get update
root@opennebula-node:~# apt-get dist-upgrade

Reboot the server to make new kernel (if available) take effects

root@opennebula-node:~# reboot

Ceph & KVM installation

Install Ceph & KVM packages

root@opennebula-node:~# apt-get install ceph ceph-common opennebula-node nfs-common bridge-utils qemu-system qemu-utils qemu-kvm

Copy Ceph configuration files from OpenNebula Frontend server

root@opennebula-node:~# scp user@opennebula-frontend:/etc/ceph/* /etc/ceph/

Login as oneadmin user and execute the following commands

root@opennebula-node:~# su - oneadmin
oneadmin@opennebula-node:~$ scp oneadmin@opennebula-frontend:~/.ssh/* .ssh/
oneadmin@opennebula-node:~$ scp oneadmin@opennebula-frontend:/var/lib/one/secret.xml /var/lib/one/
oneadmin@opennebula-node:~$ virsh secret-define /var/lib/one/secret.xml
oneadmin@opennebula-node:~$ virsh secret-set-value --secret a7675a71-9323-4545-bd1f-63ca86f22953 --base64 $(cat /etc/ceph/ceph.client.oneadmin.key)
oneadmin@opennebula-node:~$ exit

Verify the server able to connect to Ceph storage and access to the pool

root@opennebula-node:~# rbd --id oneadmin ls -l -p opennebula-pool

Note: Replace the “opennebula-pool” with the name of the pool you created when setup OpenNebula Frontend

Reboot the server to complete the installation

root@opennebula-node:~# reboot