= HOWTO: Install OpenStack on Debian GNU/Linux testing (wheezy) =

== Nova ==

This howto aims to provide guidelines to install & set up a multi-node Openstack-Compute (aka Nova) environment.

This environment will include 3 hosts :

 * 2 compute nodes:
  * compute1: pubnet@eth0=10.142.6.31 / privnet@eth1=192.168.66.1
  * compute2: pubnet@eth0=10.142.6.32 / privnet@eth1=192.168.66.2
 * 1 master/proxy/controller node (named controller in the following):
  * controller: pubnet@eth0=10.142.6.33 / privnet@eth1=192.168.66.100

Choices:
 * Virtualization technology: kvm/libvirt
 * Networking mode: VlanManger + multi_host

Services on compute* nodes :
 * puppet agent
 * nova-compute
 * nova-network
 * nova-api (metadata only)

On controller node:
 * puppet master
 * puppet agent
 * mysql database
 * keystone
 * glance (local storage)
 * nova-api
 * nova-scheduler
 * nova-novncproxy

===  PREREQUISITES ===

Things to prepare beforehand :

 * Machines :
  * They should have two network interfaces to ensure security. If only one interface is used the private part is more exposed to attacks coming from the public part.
   * a "public" one to communicate with the outside world
   * a "private" one for the guests VLans
 * Disk space: 
   * there *must* be at least as much free disk space as RAM in / on the controller host ( see https://labs.enovance.com/issues/374 for the rationale ).
   * there must be enough space in /tmp to accomodate for a copy of the largest primary disk for an instance snapshot ( [[https://github.com/openstack/nova/commit/70129ed19db187cc90f74abd5c93c86098d29c27|see the patch that fixes this in Folsom]] ). Alternatively, it is possible to set the temporary directory using the TMP variable. This should be done after nova-compute is installed:
     {{{
echo export TMP=/var/lib/nova/tmp >> /etc/default/nova-common
}}}
   * there must be enough space in /var/lib/glance to store all glance images
   * there must be enough space in /var/lib/nova to store all instance disk images ( not volumes )
 * Network :
  * public network
  * private network. If the machines are not on a LAN, [[https://labs.enovance.com/projects/openstack/wiki/L2-openvpn|create one with OpenVPN]].
  * fixed ip range for guests
  * number of networks for guests
  * network size for guests
  * public "floating" IPs (optional)
 * Base distribution :
  * Debian GNU/Linux squeeze (will be upgraded to wheezy)
  * As of June 8th, 2012 you must do the following because the most recent python-prettytable is partly broken
   {{{
apt-get install python-prettytable=0.5-1
echo python-prettytable hold |  dpkg --set-selections
}}} 

===  IMPORTANT ===

This HOWTO is valid for the OpenStack Nova, Glance and Keystone packages labelled 2012.1, currently available in Debian unstable (sid) and might need some adjustments with later versions.

=== Upgrade to Wheezy ===

Edit /etc/apt/sources.list to read :
{{{
deb http://ftp.fr.debian.org/debian/ wheezy main
deb-src http://ftp.fr.debian.org/debian/ wheezy main

deb http://security.debian.org/ wheezy/updates main
deb-src http://security.debian.org/ wheezy/updates main

# squeeze-updates, previously known as 'volatile'
deb http://ftp.fr.debian.org/debian/ squeeze-updates main
deb-src http://ftp.fr.debian.org/debian/ squeeze-updates main
}}}
Then :
{{{
apt-get update
apt-get dist-upgrade -y
reboot
}}}

=== Puppet ===

Install puppet agent (on the three nodes):
{{{
apt-get install -y puppet augeas-tools
}}}
Install puppetmaster (only on the controller node)
{{{
apt-get install -y puppetmaster sqlite3 libsqlite3-ruby libactiverecord-ruby git
}}}

Ensure ruby 1.9 is *not* installed
{{{
dpkg -l | grep ' ruby1.9'
}}}

==== Configure the puppet agents ====

On all the nodes

 * Enable pluginsync & configure the hostname of the puppetmaster
{{{
augtool << EOT
set /files/etc/puppet/puppet.conf/agent/pluginsync true
set /files/etc/puppet/puppet.conf/agent/server <hostname of the puppet master>
save
EOT
}}}


==== Configure the Puppet Master ====

On the controller node only.

 * Enable storedconfig and configure database
{{{
augtool << EOT
set /files/etc/puppet/puppet.conf/master/storeconfigs true
set /files/etc/puppet/puppet.conf/master/dbadapter sqlite3
set /files/etc/puppet/puppet.conf/master/dblocation /var/lib/puppet/server_data/storeconfigs.sqlite
save
EOT
}}}

 * Create a dummy site manifest
{{{
cat > /etc/puppet/manifests/site.pp << EOT
node default {
  notify { "Hey ! It works !": }
}
EOT
}}}

 * Restart puppetmaster
{{{
    service puppetmaster restart
}}}

==== Test the puppet agents ====

⚠ Warning ⚠:
  With sqlite3 as database backend, only one puppet agent can run at once.


Make sure '''hostname(1)''' and '''/etc/hostname''' agree with each other because it is the name under which it will be known to the puppet master. Send a request to the puppet master, asking it to accept a certificate for the machine with:
{{{
puppet agent -vt --waitforcert 60
}}}

And while the puppet agent is waiting, on the master/controller run:
{{{
puppetca sign -a
}}}

There should be no error and you should see a message saying "Hey ! It works !"

==== On the puppet master, install the openstack modules and the adapt the sample manifest ====

Get the modules
{{{
cd /etc/puppet/modules
git clone git://git.labs.enovance.com/puppet.git .
git checkout openstack
git submodule init
git submodule update
}}}

Copy the example manifest for use by the puppetmaster
{{{
cp /etc/puppet/modules/examples/openstack.pp /etc/puppet/manifests/site.pp
}}}

Edit '''/etc/puppet/manifests/site.pp''' to change the following lines:

The actual private IPs of the controller and compute hosts (see at the beginning of this HOWTO):
{{{
$db_host = '192.168.66.100' # IP address of the host on which the database will be installed (the controller for instance)
$db_allowed_hosts = ['192.168.66.%'] # IP addresses for all compute hosts : they need access to the database
}}}

The FQDN of the host providing the API server which must be the same as the '''<controller.hostname>''' used above. 
{{{
# The public fqdn of the controller host
$public_server = '<controller.hostname>'

# The internal fqdn of the controller host
$api_server = '<controller.hostname>'
}}}

If the interface used for the private network is not '''eth1''', replace '''eth1''' with the actual interface on which the IPs '''192.168.66.0/24''' are found (for instance '''br0''').

Append at the end of the file:
{{{
node /<controller.hostname>/ inherits controller {}
node /compute/ inherits compute {}
}}}

=== Installation ===

Run puppet agent on each node for which a '''node /fqdn/ {}''' stanza has been written to match.
{{{
puppet agent -vt
}}}

There should be no errors and after it is complete the services should be running.

=== Checking if it really works ===

The required services are advertised in the database
{{{
root@controller:~# nova-manage service list
Binary           Host                                 Zone             Status     State Updated_At
nova-consoleauth controller                           nova             enabled    :-)   2012-05-03 08:56:29
nova-scheduler   controller                           nova             enabled    :-)   2012-05-03 08:56:31
nova-cert        controller                           nova             enabled    :-)   2012-05-03 08:56:32
nova-compute     compute1                             nova             enabled    :-)   2012-05-03 08:56:50
nova-network     compute1                             nova             enabled    :-)   2012-05-03 08:56:49
nova-compute     compute2                             nova             enabled    :-)   2012-05-03 08:56:47
nova-network     compute2                             nova             enabled    :-)   2012-05-03 08:56:48
}}}

A file named 'openrc.sh' has been created in /root on the controller node. Source it & check the nova api works
{{{
root@controller:~# source openrc.sh
root@controller:~# nova list
+----+------+--------+----------+
| ID | Name | Status | Networks |
+----+------+--------+----------+
+----+------+--------+----------+
root@controller:~# nova flavor-list
+----+-----------+-----------+------+-----------+------+-------+-------------+
| ID |    Name   | Memory_MB | Disk | Ephemeral | Swap | VCPUs | RXTX_Factor |
+----+-----------+-----------+------+-----------+------+-------+-------------+
| 1  | m1.tiny   | 512       | 0    | 0         |      | 1     | 1.0         |
| 2  | m1.small  | 2048      | 20   | 0         |      | 1     | 1.0         |
| 3  | m1.medium | 4096      | 40   | 0         |      | 2     | 1.0         |
| 4  | m1.large  | 8192      | 80   | 0         |      | 4     | 1.0         |
| 5  | m1.xlarge | 16384     | 160  | 0         |      | 8     | 1.0         |
+----+-----------+-----------+------+-----------+------+-------+-------------+
root@controller:~# nova image-list
+----+------+--------+--------+
| ID | Name | Status | Server |
+----+------+--------+--------+
+----+------+--------+--------+
}}}

The openstack cluster is quite empty and useless like this, let's upload an image in glance::
{{{
root@controller:~# wget https://launchpad.net/cirros/trunk/0.3.0/+download/cirros-0.3.0-x86_64-disk.img
…
root@controller:~# glance add name="CirrOS 0.3" disk_format=qcow2 container_format=ovf < cirros-0.3.0-x86_64-disk.img
Uploading image 'CirrOS 0.3'
================================================================[100%] 7.73M/s, ETA  0h  0m  0s
Added new image with ID: 949bbc5c-e6fa-4ec3-91cb-65cbb6123c54
root@controller:~# glance index
ID                                   Name         Disk Format   Container Format  Size
------------------------------------ ------------ ------------- ----------------- ----------
949bbc5c-e6fa-4ec3-91cb-65cbb6123c54 CirrOS 0.3   qcow2         ovf                  9761280
}}}

Does it show up in nova ?
{{{
root@controller:~# nova image-list
+--------------------------------------+-----------------+--------+--------+
|                  ID                  |       Name      | Status | Server |
+--------------------------------------+-----------------+--------+--------+
| 949bbc5c-e6fa-4ec3-91cb-65cbb6123c54 | CirrOS 0.3      | ACTIVE |        |
+--------------------------------------+-----------------+--------+--------+
}}}

The nova network puppet module create a private network for the VM to use. Check that it has been created.
{{{
root@controller:~# nova-manage network list
id    IPv4                IPv6            start address   DNS1            DNS2            VlanID          project         uuid
1     169.254.200.0/24    None            169.254.200.3   None            None            2000            None            71681e09-c072-4281-b5b4-37f26ddc97bf
}}}

And some floating (public) ips (choose an IP range addressable on your network)::
{{{
root@controller:~# nova-manage floating create --ip_range 10.142.6.224/27
root@controller:~# nova-manage floating list
None  10.142.6.225  None  nova  eth0
None  10.142.6.226  None  nova  eth0
…
}}}

Now create a keypair (for ssh access) and save the output in a file
{{{
root@controller:~# nova keypair-add test_keypair > test_keypair.pem
root@controller:~# chmod 600 test_keypair.pem
}}}

Boot an instance and get the console log
{{{
root@controller:~# nova boot --image 949bbc5c-e6fa-4ec3-91cb-65cbb6123c54 --flavor 1 --key_name test_keypair FirstTest --poll
+-------------------------------------+--------------------------------------+
|               Property              |                Value                 |
+-------------------------------------+--------------------------------------+
| OS-DCF:diskConfig                   | MANUAL                               |
| OS-EXT-SRV-ATTR:host                | None                                 |
| OS-EXT-SRV-ATTR:hypervisor_hostname | None                                 |
| OS-EXT-SRV-ATTR:instance_name       | instance-00000001                    |
| OS-EXT-STS:power_state              | 0                                    |
| OS-EXT-STS:task_state               | scheduling                           |
| OS-EXT-STS:vm_state                 | building                             |
| accessIPv4                          |                                      |
| accessIPv6                          |                                      |
| adminPass                           | yab49fMqVHJf                         |
| config_drive                        |                                      |
| created                             | 2012-05-03T10:09:00Z                 |
| flavor                              | m1.tiny                              |
| hostId                              |                                      |
| id                                  | 06dd6129-f94a-488d-9670-7171491899e5 |
| image                               | CirrOS 0.3                           |
| key_name                            | meh                                  |
| metadata                            | {}                                   |
| name                                | FirstTest                            |
| progress                            | 0                                    |
| status                              | BUILD                                |
| tenant_id                           | d1c9085272d542eda98f7e08a1a779d6     |
| updated                             | 2012-05-03T10:09:00Z                 |
| user_id                             | cd04222b81004af5b0ff20c840fb629e     |
+-------------------------------------+--------------------------------------+
root@controller:~# nova console-log FirstTest
…
}}}

Allocate a floating ip and associate it to the instance::
{{{
root@controller:~# nova floating-ip-create
+--------------+-------------+----------+------+
|      Ip      | Instance Id | Fixed Ip | Pool |
+--------------+-------------+----------+------+
| 10.142.6.225 | None        | None     | nova |
+--------------+-------------+----------+------+
root@controller:~# nova add-floating-ip FirstTest 10.142.6.225
…
}}}

Update the rules for the default security group (allow icmp & ssh)::
{{{
root@controller:~# nova secgroup-add-rule default icmp -1 -1 0.0.0.0/0
+-------------+-----------+---------+-----------+--------------+
| IP Protocol | From Port | To Port |  IP Range | Source Group |
+-------------+-----------+---------+-----------+--------------+
| icmp        | -1        | -1      | 0.0.0.0/0 |              |
+-------------+-----------+---------+-----------+--------------+
root@controller:~# nova secgroup-add-rule default tcp 22 22 0.0.0.0/0
+-------------+-----------+---------+-----------+--------------+
| IP Protocol | From Port | To Port |  IP Range | Source Group |
+-------------+-----------+---------+-----------+--------------+
| tcp         | 22        | 22      | 0.0.0.0/0 |              |
+-------------+-----------+---------+-----------+--------------+
}}}

We now should be able to ping the instance::
{{{
root@controller:~# ping -c 1 10.142.6.225
PING 10.142.6.225 (10.142.6.225) 56(84) bytes of data.
64 bytes from 10.142.6.225: icmp_req=1 ttl=63 time=0.626 ms
 --- 10.142.6.225 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.626/0.626/0.626/0.000 ms
}}}

And ssh into it with the identity we created before::
{{{
root@controller:~# ssh -i test_key cirros@10.142.6.225
$ uname -a
Linux cirros 3.0.0-12-virtual #20-Ubuntu SMP Fri Oct 7 18:19:02 UTC 2011 x86_64 GNU/Linix
$ exit
Connection to 10.142.6.114 closed.
}}}














== Swift ==

Assuming three machines installed with squeeze, the primary node being the openstack mgmt.host node and no puppet or puppetmaster installed.

==== swift primary node ====

{{{
apt-get install libmysql-ruby ruby-activerecord-2.3 sqlite3 puppetmaster puppet ruby-sqlite3
}}}

Puppet configuration:
{{{
diff --git a/puppet/puppet.conf b/puppet/puppet.conf
index b18fae3..ce4ed22 100644
--- a/puppet/puppet.conf
+++ b/puppet/puppet.conf
@@ -7,6 +7,8 @@ factpath=$vardir/lib/facter
 templatedir=$confdir/templates
 prerun_command=/etc/puppet/etckeeper-commit-pre
 postrun_command=/etc/puppet/etckeeper-commit-post
+pluginsync=true
+storeconfigs=true

 [master]
 # These are needed when the puppetmaster is run by passenger

commit 507105065306433eec8f03dd72ab52ccaf268ccc
Author: root <root@sd-16961.dedibox.fr>
Date:   Mon Apr 2 15:04:53 2012 +0200

    configure database storage

diff --git a/puppet/puppet.conf b/puppet/puppet.conf
index ce4ed22..af220e9 100644
--- a/puppet/puppet.conf
+++ b/puppet/puppet.conf
@@ -9,10 +9,19 @@ prerun_command=/etc/puppet/etckeeper-commit-pre
 postrun_command=/etc/puppet/etckeeper-commit-post
 pluginsync=true
 storeconfigs=true
+server=mgmt.host

 [master]
 # These are needed when the puppetmaster is run by passenger
 # and can safely be removed if webrick is used.
 ssl_client_header = SSL_CLIENT_S_DN
 ssl_client_verify_header = SSL_CLIENT_VERIFY
+storeconfigs=true

+# Needed for storeconfigs=true
+dbadapter=mysql
+dbname=puppet
+dbuser=puppet
+dbpassword=password
+dbserver=localhost
+dbsocket=/var/run/mysqld/mysqld.sock
}}}

Setup mysql for puppet:

{{{
mysqladmin create puppet
mysql -e "grant all on puppet.* to 'puppet'@'localhost' identified by 'password';"
}}}

Install openstack modules for puppet:

{{{
cd /etc/puppet
git clone git://git.labs.enovance.com/openstack-puppet-modules.git modules && cd modules && git submodule init && git submodule update
cp /etc/puppet/modules/swift/examples/multi.pp /etc/puppet/manifests/site.pp
}}}


{{{
commit 8eb77223e25bfff1284612417efedd228e0c6696
Author: root <root@sd-16961.dedibox.fr>
Date:   Mon Apr 2 15:37:19 2012 +0200

    use tap0 for lan

diff --git a/puppet/manifests/site.pp b/puppet/manifests/site.pp
index a915aea..9b890b0 100644
--- a/puppet/manifests/site.pp
+++ b/puppet/manifests/site.pp
@@ -28,7 +28,7 @@
 $swift_shared_secret='changeme'
 # assumes that the ip address where all of the storage nodes
 # will communicate is on eth1
-$swift_local_net_ip = $ipaddress_eth0
+$swift_local_net_ip = $ipaddress_tap0

 Exec { logoutput => true }
}}}

Enable puppet autosign for all hosts:

{{{
echo '*' > /etc/puppet/autosign.conf
}}}

Deploy swift configuration on the proxy:

{{{
chown -R puppet:puppet /var/lib/puppet/
puppet agent --certname=swift_storage_1 --server=mgmt.host --verbose --debug --test
/etc/init.d/xinetd reload
}}}

==== swift secondary nodes ====

{{{
deb http://ftp.fr.debian.org/debian/ wheezy main
deb http://ftp.fr.debian.org/debian/ sid main

apt-get install  python2.7=2.7.2-8  python2.7-minimal=2.7.2-8 libpython2.7=2.7.2-8 python-prettytable=0.5-1
echo libpython2.7 hold |  dpkg --set-selections
echo python2.7 hold |  dpkg --set-selections
echo python2.7-minimal hold |  dpkg --set-selections
echo python-prettytable hold |  dpkg --set-selections

apt-get install puppet ruby-sqlite3

puppet agent --certname=swift_storage_3 --server=openstack-online-0001.dedibox.fr --verbose --debug --test
}}}

==== create swift ring ====

{{{
puppet agent --certname=swift_proxy --server=openstack-online-0001.dedibox.fr --verbose --debug --test
}}}

==== propagate the swift configuration ====

{{{
puppet agent --certname=swift_storage_1 --server=openstack-online-0001.dedibox.fr --verbose --debug --test

puppet agent --certname=swift_storage_2 --server=openstack-online-0001.dedibox.fr --verbose --debug --test

puppet agent --certname=swift_storage_3 --server=openstack-online-0001.dedibox.fr --verbose --debug --test
}}}

==== check that it works ====

On proxy / mgmt.host :

{{{
# cd /etc/puppet/modules/swift/ext
# ruby swift.rb
getting credentials: curl -k -v -H "X-Storage-User: test:tester" -H 'X-Storage-Pass: testing' http://127.0.0.1:8080/auth/v1.0
verifying connection auth:  curl -k -v -H "X-Auth-Token: AUTH_tk5d5a63abdf90414eafd890ed710d357b" http://127.0.0.1:8080/v1/AUTH_test
Testing swift: swift -A http://127.0.0.1:8080/auth/v1.0 -U test:tester -K testing stat
found containers/objects: 0/0
Uploading file to swift with command: swift -A http://127.0.0.1:8080/auth/v1.0 -U test:tester -K testing upload my_container /tmp/foo1
tmp/foo1
Downloading file with command: swift -A http://127.0.0.1:8080/auth/v1.0 -U test:tester -K testing download my_container
tmp/foo1

Dude!!!! It actually seems to work, we can upload and download files!!!!
}}}

==== Swift/Horizon ====

Edit '''/etc/keystone/default_catalog.templates''' like this:

{{{
catalog.RegionOne.object-store.publicURL = http://mgmt.host:8080/v1/AUTH_$(tenant_id)s
catalog.RegionOne.object-store.adminURL = http://mgmt.host:8080/
catalog.RegionOne.object-store.internalURL = http://mgmt.host:8080/v1/AUTH_$(tenant_id)s
catalog.RegionOne.object-store.name = 'Object Store Service'
}}}

{{{
diff --git a/swift/proxy-server.conf b/swift/proxy-server.conf
index 83dda1e..8364fe7 100644
--- a/swift/proxy-server.conf
+++ b/swift/proxy-server.conf
@@ -7,7 +7,8 @@ user = swift

 [pipeline:main]
 # ratelimit?
-pipeline = healthcheck cache tempauth proxy-server
+#pipeline = healthcheck cache tempauth proxy-server
+pipeline = healthcheck cache  tokenauth keystone  proxy-server

 [app:proxy-server]
 use = egg:swift#proxy
@@ -28,3 +29,17 @@ use = egg:swift#healthcheck
 use = egg:swift#memcache
 # multi-proxy config not supported
 memcache_servers = 127.0.0.1:11211
+
+[filter:tokenauth]
+paste.filter_factory = keystone.middleware.auth_token:filter_factory
+service_port = 5000
+service_protocol = http
+service_host = 127.0.0.1
+auth_port = 35357
+auth_protocol = http
+auth_host = 127.0.0.1
+admin_token = ADMIN
+
+[filter:keystone]
+paste.filter_factory = keystone.middleware.swift_auth:filter_factory
+operator_roles = admin, swiftoperator, projectmanager

}}}

{{{
/etc/init.d/swift-proxy restart
}}}

==== swift command line ====

{{{
apt-get install swift
swift -U $OS_TENANT_NAME:$OS_USERNAME list
}}}

==== Running the puppet agents daemons ====

Edit {{{/etc/default/puppet}}} and change to 

{{{
START=yes
}}}

and run the daemon:

{{{
/etc/init.d/puppet start
}}}