Contents
- Network topology used in this howto
- Setting-up quantum-plugin-openvswitch-agent
- Creating the tenant LAN
- Creating the tenant router
- Mapping your public IP address network in Quantum
- Defining the default gateway for the external network
- Booting your first VM inside the tenant LAN
- Accessing the sshd of your virtual machine from within your quantum-network machine
- Assigning a public IP address to a virtual machine
Network topology used in this howto
There are many ways of using Quantum. In fact, as many as there are network topologies. But we can't cover all of them. It is therefore left as an exercise to the reader to adapt this howto to his own need.
In this howto, we will assume that you have to your disposal, a bunch of IP addresses available on the 1.2.3.0/24 network. The IP address 1.2.3.1 will be the default gateway provided by your network provider, who granted you the use of a block of IP starting at 1.2.3.5 and ending at 1.2.3.9. Of course, in real life, you may need more IP addresses, but this is enough for this howto. The address 1.2.3.9 will be the "main" IP address of your server: it will be assumed that your server will be installed with a fresh Debian installation using this IP address.
Also, in this howto, we will setup a single router per tenant (if you don't know already, a "tenant" in OpenStack terms, can be thought as a customer). The customer router will be setup with the public IP address 1.2.3.5 and will route traffic for a LAN on 10.0.0.0/24, with 10.0.0.1 as default gateway for this LAN.
This small below drawing will help understanding better. The green network is the public network, while the blue one is the virtual network which Quantum creates.
Setting-up quantum-plugin-openvswitch-agent
On your compute node, make sure that quantum-plugin-openvswitch-agent (this howto covers only this case, which doesn't need specific hardware). Make sure that Open vSwitch is installed and running, and that the local_ip directive in /etc/quantum/plugins/openvswitch/ovs_quantum_plugin.ini is set to the management IP address of your compute node (it is normally set automatically by the debconf and postinst scripts). Then make sure that the daemon is up and running (ps axuf, then check the logs in /var/log/quantum). Once this is done, create the br-int bridge using Open vSwitch:
# ovs-vsctl add-br br-int
Creating the tenant LAN
Using python-quantumclient, simply issue the following commands on the shell, which will create a network called "testnetwork" with 10.0.0.0/24 attached to it as subnet:
# quantum net-create testnetwork Created a new network: +---------------------------+--------------------------------------+ | Field | Value | +---------------------------+--------------------------------------+ | admin_state_up | True | | id | 710620b7-63be-4a04-9168-35e19d7ca995 | | name | testnetwork | | provider:network_type | gre | | provider:physical_network | | | provider:segmentation_id | 1 | | router:external | False | | shared | False | | status | ACTIVE | | subnets | | | tenant_id | 5f3fe6621a884758b37ca30506c161ff | +---------------------------+--------------------------------------+ # quantum subnet-create -- testnetwork 10.0.0.0/24 Created a new subnet: +------------------+--------------------------------------------+ | Field | Value | +------------------+--------------------------------------------+ | allocation_pools | {"start": "10.0.0.2", "end": "10.0.0.254"} | | cidr | 10.0.0.0/24 | | dns_nameservers | | | enable_dhcp | True | | gateway_ip | 10.0.0.1 | | host_routes | | | id | b5cd1984-4a45-4233-90b1-19eb4f7a6e32 | | ip_version | 4 | | name | | | network_id | 710620b7-63be-4a04-9168-35e19d7ca995 | | tenant_id | 5f3fe6621a884758b37ca30506c161ff | +------------------+--------------------------------------------+
Take a note of the ID of the net and subnet which you just created (eg: 710620b7-63be-4a04-9168-35e19d7ca995 for the network, and b5cd1984-4a45-4233-90b1-19eb4f7a6e32 for the subnet in this example).
If you wish, you can also set the DNS server for that subnet (in this example, sets the famous 8.8.8.8 Google public DNS resolver):
quantum subnet-update b5cd1984-4a45-4233-90b1-19eb4f7a6e32 --dns_nameservers list=true 8.8.8.8
Creating the tenant router
Now, we create a tenant router called "testrouter":
# quantum router-create testrouter Created a new router: +-----------------------+--------------------------------------+ | Field | Value | +-----------------------+--------------------------------------+ | admin_state_up | True | | external_gateway_info | | | id | 0d259ad9-7727-44f2-9d86-a8d682073d87 | | name | testrouter | | status | ACTIVE | | tenant_id | 5f3fe6621a884758b37ca30506c161ff | +-----------------------+--------------------------------------+
Note the ID of the router (0d259ad9-7727-44f2-9d86-a8d682073d87 in this example). Then we attach to it a network interface which will be on the tenant LAN. This network interface will be assigned automatically the IP address of the default gateway for the tenant LAN (eg: 10.0.0.1/24). The first parameter is the ID of the router we just created, and the 2nd one is the ID of tenant LAN subnet:
# quantum router-interface-add 0d259ad9-7727-44f2-9d86-a8d682073d87 b5cd1984-4a45-4233-90b1-19eb4f7a6e32 Added interface to router 0d259ad9-7727-44f2-9d86-a8d682073d87
Mapping your public IP address network in Quantum
Low level setup of the node running quantum-server
Now is the time to man 1.2.3.0/24 in Quantum. Before doing it, we need to configure the machine running openstack-proxy-node. What is needed to be done, is to create a "br-ex" bridge, attached to your eth0 interface, which will connect your quantum-server and all of your virtual machines to the public IP network.
WARNING: this operation is potentially dangerous! If you do a mistake with the bridging, it is possible that you will loose access to your server. So make sure you can recover from mistakes, either by having physical access to your server, or by having a KVM over IP access. When you will run "ovs-vsctl add-port br-ex eth0" (see below), instantly, you will loose any kind of connectivity until the rest of the commands are run.
So, we prepare the below script:
set -x ovs-vsctl add-br br-ex ovs-vsctl add-port br-ex eth0 ifconfig br-ex 1.2.3.9 netmask 255.255.255.0 up ifconfig eth0 0.0.0.0 up route add default gw 1.2.3.1
Then run GNU screen (or tmux, or anything like that...) to make sure that this script will run until it is finished, even if you loose connectivity with ssh.
Creating the public IP network in Quantum
If everything works as expected, then you can create the ext_net network as per below:
# quantum net-create ext_net -- --router:external=True Created a new network: +---------------------------+--------------------------------------+ | Field | Value | +---------------------------+--------------------------------------+ | admin_state_up | True | | id | cd5dea18-1d25-492d-88b1-c439a0a93dad | | name | ext_net | | provider:network_type | gre | | provider:physical_network | | | provider:segmentation_id | 2 | | router:external | True | | shared | False | | status | ACTIVE | | subnets | | | tenant_id | 5f3fe6621a884758b37ca30506c161ff | +---------------------------+--------------------------------------+
and attach a subnet to it:
# quantum subnet-create --allocation-pool start=1.2.3.5,end=1.2.3.8 ext_net 1.2.3.0/24 -- --enable_dhcp=False Created a new subnet: +------------------+----------------------------------------+ | Field | Value | +------------------+----------------------------------------+ | allocation_pools | {"start": "1.2.3.5", "end": "1.2.3.8"} | | cidr | 1.2.3.0/24 | | dns_nameservers | | | enable_dhcp | False | | gateway_ip | 1.2.3.1 | | host_routes | | | id | 5fd4033b-3609-4abc-bc17-2da0c1b1ccf9 | | ip_version | 4 | | name | | | network_id | cd5dea18-1d25-492d-88b1-c439a0a93dad | | tenant_id | 5f3fe6621a884758b37ca30506c161ff | +------------------+----------------------------------------+
Note that these IP addresses, in the range from 1.2.3.5 to 1.2.3.8 are called "floating IPs" in OpenStack. Later on, you can assign some floating IPs to virtual machines (see below).
Defining the default gateway for the external network
First, get the IDs of your ext_net and router:
# quantum net-list +--------------------------------------+-------------+------------------------------------------------------+ | id | name | subnets | +--------------------------------------+-------------+------------------------------------------------------+ | 710620b7-63be-4a04-9168-35e19d7ca995 | testnetwork | b5cd1984-4a45-4233-90b1-19eb4f7a6e32 10.0.0.0/24 | | cd5dea18-1d25-492d-88b1-c439a0a93dad | ext_net | 5fd4033b-3609-4abc-bc17-2da0c1b1ccf9 1.2.3.0/24 | +--------------------------------------+-------------+------------------------------------------------------+ # quantum router-list +--------------------------------------+------------+-----------------------+ | id | name | external_gateway_info | +--------------------------------------+------------+-----------------------+ | 0d259ad9-7727-44f2-9d86-a8d682073d87 | testrouter | null | +--------------------------------------+------------+-----------------------+
Then simply use the router-gateway-set command using the router ID as first parameter, and the ext_net ID as second parameter:
# quantum router-gateway-set 0d259ad9-7727-44f2-9d86-a8d682073d87 cd5dea18-1d25-492d-88b1-c439a0a93dad Set gateway for router 0d259ad9-7727-44f2-9d86-a8d682073d87
All this should be enough configuration to have a working networking setup.
Booting your first VM inside the tenant LAN
Before starting the virtual machine, check that you really do have 2 networks, as expected:
# quantum net-list +--------------------------------------+-------------+------------------------------------------------------+ | id | name | subnets | +--------------------------------------+-------------+------------------------------------------------------+ | 710620b7-63be-4a04-9168-35e19d7ca995 | testnetwork | b5cd1984-4a45-4233-90b1-19eb4f7a6e32 10.0.0.0/24 | | cd5dea18-1d25-492d-88b1-c439a0a93dad | ext_net | 5fd4033b-3609-4abc-bc17-2da0c1b1ccf9 1.2.3.0/24 | +--------------------------------------+-------------+------------------------------------------------------+
Then you can boot the VM (note: you can use "nova image-list" to list AMI images):
# nova boot --flavor 1 --image 7f2cae48-a017-4950-9f2c-d45e618367ca --nic net-id=710620b7-63be-4a04-9168-35e19d7ca995 firstinst +-------------------------------------+--------------------------------------+ | Property | Value | +-------------------------------------+--------------------------------------+ | OS-EXT-STS:task_state | scheduling | | image | tty-linux | | OS-EXT-STS:vm_state | building | | OS-EXT-SRV-ATTR:instance_name | instance-00000001 | | flavor | m1.tiny | | id | d0c9b41f-56e4-499e-9d00-0414a8aab922 | | security_groups | [{u'name': u'default'}] | | user_id | 87d27a6fcac94ab7b5c2e3da89339bc1 | | OS-DCF:diskConfig | MANUAL | | accessIPv4 | | | accessIPv6 | | | progress | 0 | | OS-EXT-STS:power_state | 0 | | OS-EXT-AZ:availability_zone | None | | config_drive | | | status | BUILD | | updated | 2013-04-09T07:49:56Z | | hostId | | | OS-EXT-SRV-ATTR:host | None | | key_name | None | | OS-EXT-SRV-ATTR:hypervisor_hostname | None | | name | firstinst | | adminPass | vsJ7iPvBXxAL | | tenant_id | 5f3fe6621a884758b37ca30506c161ff | | created | 2013-04-09T07:49:55Z | | metadata | {} | +-------------------------------------+--------------------------------------+ # nova list +--------------------------------------+-----------+--------+----------------------+ | ID | Name | Status | Networks | +--------------------------------------+-----------+--------+----------------------+ | d0c9b41f-56e4-499e-9d00-0414a8aab922 | firstinst | ACTIVE | testnetwork=10.0.0.2 | +--------------------------------------+-----------+--------+----------------------+
Accessing the sshd of your virtual machine from within your quantum-network machine
Since Quantum is using IP namespaces, you have to use namespaces to access your VM. To do that, we first list all IP namespaces:
# ip netns qdhcp-710620b7-63be-4a04-9168-35e19d7ca995 qrouter-0d259ad9-7727-44f2-9d86-a8d682073d87
We can then check the IP addresses of our testrouter router this way:
# ip netns exec qrouter-0d259ad9-7727-44f2-9d86-a8d682073d87 ifconfig lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:0 (0.0 B) TX bytes:0 (0.0 B) qg-5cb2227b-60 Link encap:Ethernet HWaddr fa:16:3e:ba:61:29 inet addr:1.2.3.5 Bcast:1.2.3.255 Mask:255.255.255.0 inet6 addr: fe80::f816:3eff:feba:6129/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:813 errors:0 dropped:0 overruns:0 frame:0 TX packets:20 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:53506 (52.2 KiB) TX bytes:1462 (1.4 KiB) qr-f2ed7a54-66 Link encap:Ethernet HWaddr fa:16:3e:e2:9b:61 inet addr:10.0.0.1 Bcast:10.0.0.255 Mask:255.255.255.0 inet6 addr: fe80::f816:3eff:fee2:9b61/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:228 errors:0 dropped:0 overruns:0 frame:0 TX packets:149 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:19189 (18.7 KiB) TX bytes:18120 (17.6 KiB)
As you can see, this shows the 1.2.3.5 and 10.0.0.1 IP addresses assigned to the router. If everything works, then you should be able to ping your virtual machine:
# ip netns exec qrouter-0d259ad9-7727-44f2-9d86-a8d682073d87 ping 10.0.0.2 PING 10.0.0.2 (10.0.0.2) 56(84) bytes of data. 64 bytes from 10.0.0.2: icmp_req=1 ttl=64 time=16.9 ms 64 bytes from 10.0.0.2: icmp_req=2 ttl=64 time=0.282 ms 64 bytes from 10.0.0.2: icmp_req=3 ttl=64 time=0.234 ms 64 bytes from 10.0.0.2: icmp_req=4 ttl=64 time=0.229 ms ^C --- 10.0.0.2 ping statistics --- 4 packets transmitted, 4 received, 0% packet loss, time 3003ms rtt min/avg/max/mdev = 0.229/4.422/16.943/7.229 ms
And also ssh into it, with access to the public internet from within this virtual machine:
# ip netns exec qrouter-0d259ad9-7727-44f2-9d86-a8d682073d87 ssh root@10.0.0.2 The authenticity of host '10.0.0.2 (10.0.0.2)' can't be established. RSA key fingerprint is 60:d3:89:19:28:f3:f7:c9:08:9c:bd:3e:23:86:c6:d9. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added '10.0.0.2' (RSA) to the list of known hosts. root@10.0.0.2's password: <password is password...> Chop wood, carry water. # ifconfig eth0 Link encap:Ethernet HWaddr FA:16:3E:F7:12:76 inet addr:10.0.0.2 Bcast:10.0.0.255 Mask:255.255.255.0 UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:208 errors:0 dropped:0 overruns:0 frame:0 TX packets:253 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:26560 (25.9 KiB) TX bytes:23116 (22.5 KiB) lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:0 errors:0 dropped:0 overruns:0 frame:0 TX packets:0 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:0 (0.0 B) TX bytes:0 (0.0 B) # ping 8.8.8.8 PING 8.8.8.8 (8.8.8.8): 56 data bytes 64 bytes from 8.8.8.8: seq=0 ttl=49 time=26.985 ms 64 bytes from 8.8.8.8: seq=1 ttl=49 time=11.038 ms 64 bytes from 8.8.8.8: seq=2 ttl=49 time=10.944 ms ^C --- 8.8.8.8 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max = 10.944/16.322/26.985 ms
Assigning a public IP address to a virtual machine
Why?
Above, we could access the virtual machine using the IP namespace. But that is kind of cheating, since normally a customer will not be able to login in your management network. It is therefore possible to assign floating IP addresses to virtual machines. You can see these as a kind of NAT thing: the public IP address will redirect to the LAN IP.
Creating floating IPs
This is super easy. No parameter, since everything is already configured. Just do this:
quantum floatingip-create ext_net Created a new floatingip: +---------------------+--------------------------------------+ | Field | Value | +---------------------+--------------------------------------+ | fixed_ip_address | | | floating_ip_address | 1.2.3.6 | | floating_network_id | cd5dea18-1d25-492d-88b1-c439a0a93dad | | id | 27f9989b-3128-440c-9d42-21706c560bc4 | | port_id | | | router_id | | | tenant_id | 5f3fe6621a884758b37ca30506c161ff | +---------------------+--------------------------------------+
The IP address will be taken from the pool of IPs you created when defining the ext_net subnet.
Assigning a floating IP address to a VM
First, list your VM ID and its port:
# nova list +--------------------------------------+-----------+--------+----------------------+ | ID | Name | Status | Networks | +--------------------------------------+-----------+--------+----------------------+ | d0c9b41f-56e4-499e-9d00-0414a8aab922 | firstinst | ACTIVE | testnetwork=10.0.0.2 | +--------------------------------------+-----------+--------+----------------------+ # quantum port-list -- --device_id d0c9b41f-56e4-499e-9d00-0414a8aab922 +--------------------------------------+------+-------------------+---------------------------------------------------------------------------------+ | id | name | mac_address | fixed_ips | +--------------------------------------+------+-------------------+---------------------------------------------------------------------------------+ | 1a1a6984-344a-419c-ab5f-bb26280a17e2 | | fa:16:3e:f7:12:76 | {"subnet_id": "b5cd1984-4a45-4233-90b1-19eb4f7a6e32", "ip_address": "10.0.0.2"} | +--------------------------------------+------+-------------------+---------------------------------------------------------------------------------+
Then assign the floating IP address to it:
# quantum floatingip-associate 27f9989b-3128-440c-9d42-21706c560bc4 1a1a6984-344a-419c-ab5f-bb26280a17e2 Associated floatingip 27f9989b-3128-440c-9d42-21706c560bc4
Now, you can check that the IP address 1.2.4.6 is really redirecting to 10.0.0.2:
quantum floatingip-show 27f9989b-3128-440c-9d42-21706c560bc4 +---------------------+--------------------------------------+ | Field | Value | +---------------------+--------------------------------------+ | fixed_ip_address | 10.0.0.2 | | floating_ip_address | 1.2.4.6 | | floating_network_id | cd5dea18-1d25-492d-88b1-c439a0a93dad | | id | 27f9989b-3128-440c-9d42-21706c560bc4 | | port_id | 1a1a6984-344a-419c-ab5f-bb26280a17e2 | | router_id | 0d259ad9-7727-44f2-9d86-a8d682073d87 | | tenant_id | 5f3fe6621a884758b37ca30506c161ff | +---------------------+--------------------------------------+
Normally, doing a ping of 1.2.4.6 should now work from anywhere on the internet.
Security groups
Security groups with Nova
Before you can ssh into your virtual machine, you need to allow the ssh port. By default, everything is blocked on the floating IP address. This is done this way:
nova secgroup-add-rule default tcp 22 22 0.0.0.0/0
Then normally, you should be able to ssh into your VM:
# ssh root@1.2.4.6 The authenticity of host '1.2.4.6 (182.54.237.6)' can't be established. RSA key fingerprint is 60:d3:89:19:28:f3:f7:c9:08:9c:bd:3e:23:86:c6:d9. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added '182.54.237.6' (RSA) to the list of known hosts. root@182.54.237.6's password: Chop wood, carry water. #
Listing security groups rules is done this way:
# nova secgroup-list-rules default +-------------+-----------+---------+-----------+--------------+ | IP Protocol | From Port | To Port | IP Range | Source Group | +-------------+-----------+---------+-----------+--------------+ | tcp | 22 | 22 | 0.0.0.0/0 | | +-------------+-----------+---------+-----------+--------------+
ICMP echo requests can also be added:
# nova secgroup-add-rule default icmp -1 -1 0.0.0.0/0
but I saw it working by default anyway.
Security groups with Quantum
The new way to configure security groups is to do it through Quantum. This is now the default with the latest version of the Debian packages for Grizzly (as of 24th june 2013). A "default" security group should exist. If it's not there, then you can create it this way:
# quantum security-group-create default
Then allow ssh and ICMP:
#quantum security-group-rule-create --direction ingress --protocol tcp --port_range_min 22 --port_range_max 22 default #quantum security-group-rule-create --protocol icmp --direction ingress default