Cloud Computing with Debian and its descendants
This page summarises on what to expect from Debian when it comes to compute clouds. Please refer to Wikipedia or other sources for details about what clouds are about. The idea is that you get virtual instances of some self-assembled or off-shelf booting disk image. You start them, pay for their lifetime (likely), pay for the data that goes in or out, and stop them again.
Our fine distribution can play any role in here. It can
- run as a cloud OS
- run the infrastructure that runs the clouds
- run as the joe the plumber's regular desktop OS that contacts the cloud infrastructure with requests to control the clouds
What cloud software are already available in Debian
There's 2 big cloud computing suites that are available in Debian: OpenStack and XCP (Xen Cloud Platform). None of them are available in Debian Stable (Squeeze, at the time of writing), but they work pretty well in SID/Wheezy.
XCP is the open source version of ?XenServer, which is a Citrix commercial product. It's of course based on the Xen hypervisor. XCP is written mainly in the OCaml language, with some parts also written in Python and C. It is a very mature product, which works easily, and is fairly easy to install. If you have Recommends: activated, then simply do "apt-get install xcp-xapi" to get it installed, then read the README.Debian to know how to use it. If you don't have the requirement to have EC2 compatibility, and that you are searching for something easy, then it's the option of choice.
The other suite of packages is huge, and it's of course OpenStack. OpenStack is a much less mature project, but that has really a lot of contributors. Its goal is to be the Apache of the cloud. In other words, it is supposed to become the open source standard that everybody will use. It has a lot of interesting features, but it's a lot harder to setup and understand than XCP. Don't expect to be ready within an hour, you'll need few days, if not weeks, to just understand the basic concepts behind it, and how to set it up. But if you are a service provider, and wish to setup a cloud computing service, then it will be your choice. It has a partial implementation of EC2 and S3 APIs (I wrote partial, but really, everything that you will need is there).
Then, what's also nice, is that if you wish to run Openstack with Xen, you can use OpenStack with XCP. OpenStack will talk directly to XenAPI (which is the API of XCP), instead of using libvirt. It has been designed this way, and not use libvirt, because XenAPI has many concepts that libvirt doesn't, and using libvirt would have been too much limiting features. If you wish to know how to do such a setup, there's a README in the source package of nova (otherwise known as openstack compute). I am not sure if we packaged it yet with the binaries of nova-compute-xen though.
As for an EC2 client, you can use euca2ools, also available in SID. These would be what to use if you wish to control OpenStack using the EC2 API (but of course, you can use "nova" on the shell, which uses the native OpenStack compute API).
I don't think there's anything else available in Debian to work with cloud computing, if I'm mistaking, feel free to correct me.
How to contribute
Join the mailing lists:
- pkg-xen-devel@lists.alioth.debian.org (which is also the one for XenAPI / XCP)
- openstack-devel@lists.alioth.debian.org
Grab the sources in the /git/openstack or /git/pkg-xen on Alioth.
Note that, especially OpenStack, is BIG. We already have more than 50 binary packages of it in SID. So of course, we'd be more than happy to have help for its packaging/testing.
Where to get more information
Eucalyptus - a free cloud infrastructure with Debian packages pending
euca2ools - free clone of Amazon ec2 console tools
?Cloud/CreateImage - instruction on how to prepare a cloud-ready image
Cloud/TorqueCluster - instruction how to setup a torque cluster within a Eucalyptus cloud
?Cloud/Integration - changes or additions to Debian to prepare and run images using only packaged material.
OpenNebula - toolkit for IaaS cloud handling (with mixed private and public nodes)
opennebula - Debian package for OpenNebula toolkit
Amazon - the EC2 that got the craze going
Cloud/AmazonEC2Image - Registered Amazon Machine Image
- The Net At Large - various howtos that linking Debian with clouds
LinuxConfig.org - how to prepare an image for Amazon's EC2
Who in the Debian community is interested in clouds and why?
Firstly, the extension from providing packages together is to use them together. Thus, the clouds, just like computational grids, may become a way to extend our current way we think about our community. The following groups were observed to show some sincere interest in the cloud technology:
- sharing of administrative burden also for data that needs to be regularly updated
- its Bioinformatics wing sees lots of parallel computing to be performed on clouds
The RightScale OSS team builds and publishes images for cloud users. For a list of available images, see Cloud/AmazonEC2Image.
See also
HighPerformanceComputing - wiki page gathering tools for distributed computing which can/should partially be combined with cloud technology
VMBuilder - tool for the creation of virtual machines for Xen, KVM, VirtualBox with the prospect to run them in a cloud
euca2ools - handling of virtual machines for the Eucalyptus and Amazon elastic compute cloud.