Differences between revisions 1 and 36 (spanning 35 versions)
Revision 1 as of 2012-07-17 15:41:55
Size: 6
Editor: ?Alexander Pashaliyski
Comment:
Revision 36 as of 2012-07-21 10:01:59
Size: 5565
Editor: ?Alexander Pashaliyski
Comment:
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
test #language en
This is a step by step tutorial about Build services setup for Wanna-build.
||<tablestyle="width: 100%;" style="border: 0px hidden">~-[[DebianWiki/EditorGuide#translation|Translation(s)]]: none-~||<style="text-align: right;border: 0px hidden"> (!) [[/DebianWannaBuildInfrastructureDiscussion|Discussion]]||
----

## If your page gets really long, uncomment this Table of Contents
<<TableOfContents(2)>>

This tutorial follows the installation of [[DebianWannaBuildInfrastructureOnOneServer|Wanna Build]].

== Setup Buildd and Sbuild ==

=== Install sbuild and buildd ===

Currently sbuild and buildd are located in a single package named sbuild. We're going to install sbuild and its dependencies.
{{{
apt-get install sbuild sudo schroot debootstrap
}}}

=== Configure sbuild ===

sbuild uses schroot to build packages. We'll create a directory for it and configure sbuild to use it.
{{{
distribution=sid
architecture=amd64

mkdir -p /srv/chroot/$distribution-$architecture-sbuild
debootstrap $distribution /srv/chroot/$distribution-$architecture-sbuild
chown -R buildd. /srv/chroot/$distribution-$architecture-sbuild
}}}
Lets configure the chroot:
{{{
vim /etc/schroot/schroot.conf
}}}
{{{
echo "
[sid]
description=Debian sid (unstable)
directory=/srv/chroot/$distribution-$architecture-sbuild
users=buildd
groups=buildd
root-groups=buildd
aliases=unstable,default
" >> /etc/schroot/schroot.conf
}}}
Configuring sbuild is explained here: http://wiki.debian.org/sbuild
{{{
vim /etc/sbuild/sbuild.conf
}}}
{{{
# Need at least sbuild v0.60.6
sbuild-update --keygen
mirror=http://ftp.us.debian.org/debian/
sbuild-createchroot --keyring=/etc/apt/trusted.gpg --arch=$architecture $distribution /srv/chroot/$distribution-$architecture-sbuild $mirror
}}}

To configure auto mount proc and devpts in chroot:
{{{
echo "
proc /srv/chroot/$distribution-$architecture-sbuild/proc proc defaults 0 0
devpts /srv/chroot/$distribution-$architecture-sbuild/dev/pts devpts rw,noexec,nosuid,gid=5,mode=620 0 0
" >> /etc/fstab

mount -a
}}}
We'll now configure the schroot environment. We'll create two users and copy the sources.list file to chroot.
{{{
cp /etc/apt/sources.list /srv/chroot/$distribution-$architecture-sbuild/etc/apt/
usersfile=/srv/chroot/$distribution-$architecture-sbuild/root/users
echo "#\!/bin/sh" > "$usersfile"
echo "groupadd -g $(id -g buildd) buildd" >> "$usersfile"
echo "useradd -u $(id -u buildd) -g $(id -g buildd) buildd" >> "$usersfile"
echo "groupadd -g $(id -g sbuild) sbuild" >> "$usersfile"
echo "useradd -u $(id -u sbuild) -g $(id -g sbuild) sbuild" >> "$usersfile"
chmod a+x "$usersfile"

chroot /srv/chroot/$distribution-$architecture-sbuild/
# Run the script
/root/users
}}}
=== Disable sbuild environment filters ===
The latest version of sbuild has a feature to clean up the environment before starting the building process. However there are some bugs associated with it and the simplest way to go around them is to disable the filters from the following script (you may want to create a backup copy of the file just in case):
{{{
vim /usr/share/perl5/Sbuild/Chroot.pm
}}}
Comment the lines about deleting environment variables (300 and 301 in the current version):
{{{
#delete $ENV{$var} if
# $match == 0;
}}}
Save the changes

=== Patch buildd ===
#TODO: add a link to the patch
# patch buildd.patch

'''Tip'''
This file:
{{{
/var/lib/buildd/build/SBUILD-GIVEN-BACK
}}}
contains packages that will be skipped because they've already failed. There are cases in which you may want to edit or delete it.

'''Tip'''
If while generating gpg-key there's not enough entropy, here's a command that can be useful for generating entropy:

{{{
find /usr /var /tmp /opt -type f -print0 | xargs -0 cat > /dev/null
}}}

--------


#TODO: What is next ? How to test ?

== Setup reprepro for local repository ==

0. Installing reprepro
{{{
apt-get install reprepro
}}}
1. Creating directories for the repository
{{{
mkdir -p /org/ftp.debian.org/ftp/apt/
cd /org/ftp.debian.org/ftp/apt/
mkdir conf dists incoming indices logs pool project
}}}
2. Configuring reprepro
{{{
cd /org/ftp.debian.org/ftp/apt/conf/
vim distributions
}}}
{{{
Origin: Alexander Pashaliyski
Label: Alexander Pashaliyski
Codename: sid
Architectures: i386 amd64 source
Components: main
Description: Alexander Pashaliyski APT Repository
SignWith: yes
}}}

Create and edit incoming conf file:
{{{
vim incoming
}}}
{{{
Name: default
IncomingDir: incoming
TempDir: /tmp
Allow: sid
Cleanup: on_deny on_error
}}}

Create and edit options conf file:
{{{
vim options
}}}
{{{
verbose
ask-passphrase
basedir .
}}}

Create and edit uploaders conf file:
{{{
vim uploaders
}}}
{{{
allow * by unsigned
}}}

Then create some other files
{{{
cd ../indices
touch override.sid.main
touch override.sid.main.debian-installer
touch override.sid.main.src
cd ..
}}}

3. Generating gpg key
{{{
gpg --gen-key
}}}

4. Export the repository
{{{
reprepro -vb /org/ftp.debian.org/ftp/apt export
}}}

5. Adding packages to the repository
To add a package to the repository, use one of these commands (replace <...> with the name of your package):
{{{
reprepro includedeb sid <deb filename>
reprepro includedsc sid <dsc filename>
}}}

Example:
{{{
reprepro includedeb sid openssh-client_5.9p1-5_amd64.deb
reprepro includedsc sid eglibc_2.13-32.dsc
}}}

This is a step by step tutorial about Build services setup for Wanna-build.

Translation(s): none

(!) ?Discussion


This tutorial follows the installation of Wanna Build.

Setup Buildd and Sbuild

Install sbuild and buildd

Currently sbuild and buildd are located in a single package named sbuild. We're going to install sbuild and its dependencies.

apt-get install sbuild sudo schroot debootstrap

Configure sbuild

sbuild uses schroot to build packages. We'll create a directory for it and configure sbuild to use it.

distribution=sid
architecture=amd64

mkdir -p /srv/chroot/$distribution-$architecture-sbuild
debootstrap $distribution /srv/chroot/$distribution-$architecture-sbuild
chown -R buildd. /srv/chroot/$distribution-$architecture-sbuild

Lets configure the chroot:

vim /etc/schroot/schroot.conf

echo "
[sid]
description=Debian sid (unstable)
directory=/srv/chroot/$distribution-$architecture-sbuild
users=buildd
groups=buildd
root-groups=buildd
aliases=unstable,default
" >> /etc/schroot/schroot.conf

Configuring sbuild is explained here: http://wiki.debian.org/sbuild

vim /etc/sbuild/sbuild.conf

# Need at least sbuild v0.60.6
sbuild-update --keygen
mirror=http://ftp.us.debian.org/debian/
sbuild-createchroot --keyring=/etc/apt/trusted.gpg --arch=$architecture $distribution /srv/chroot/$distribution-$architecture-sbuild $mirror

To configure auto mount proc and devpts in chroot:

echo "
proc            /srv/chroot/$distribution-$architecture-sbuild/proc   proc    defaults        0       0
devpts          /srv/chroot/$distribution-$architecture-sbuild/dev/pts devpts rw,noexec,nosuid,gid=5,mode=620  0  0
" >> /etc/fstab

mount -a

We'll now configure the schroot environment. We'll create two users and copy the sources.list file to chroot.

cp /etc/apt/sources.list /srv/chroot/$distribution-$architecture-sbuild/etc/apt/
usersfile=/srv/chroot/$distribution-$architecture-sbuild/root/users
echo "#\!/bin/sh" > "$usersfile"
echo "groupadd -g $(id -g buildd) buildd" >> "$usersfile"
echo "useradd -u $(id -u buildd) -g $(id -g buildd) buildd" >> "$usersfile"
echo "groupadd -g $(id -g sbuild) sbuild" >> "$usersfile"
echo "useradd -u $(id -u sbuild) -g $(id -g sbuild) sbuild" >> "$usersfile"
chmod a+x "$usersfile"

chroot /srv/chroot/$distribution-$architecture-sbuild/
# Run the script
/root/users

Disable sbuild environment filters

The latest version of sbuild has a feature to clean up the environment before starting the building process. However there are some bugs associated with it and the simplest way to go around them is to disable the filters from the following script (you may want to create a backup copy of the file just in case):

vim /usr/share/perl5/Sbuild/Chroot.pm 

Comment the lines about deleting environment variables (300 and 301 in the current version):

#delete $ENV{$var} if
#    $match == 0;

Save the changes

Patch buildd

#TODO: add a link to the patch # patch buildd.patch

Tip This file:

/var/lib/buildd/build/SBUILD-GIVEN-BACK

contains packages that will be skipped because they've already failed. There are cases in which you may want to edit or delete it.

Tip If while generating gpg-key there's not enough entropy, here's a command that can be useful for generating entropy:

find /usr /var /tmp /opt -type f -print0 | xargs -0 cat > /dev/null


#TODO: What is next ? How to test ?

Setup reprepro for local repository

0. Installing reprepro

apt-get install reprepro

1. Creating directories for the repository

mkdir -p /org/ftp.debian.org/ftp/apt/
cd /org/ftp.debian.org/ftp/apt/
mkdir conf dists incoming indices logs pool project

2. Configuring reprepro

cd /org/ftp.debian.org/ftp/apt/conf/
vim distributions

Origin: Alexander Pashaliyski
Label: Alexander Pashaliyski
Codename: sid
Architectures: i386 amd64 source
Components: main
Description: Alexander Pashaliyski APT Repository
SignWith: yes

Create and edit incoming conf file:

vim incoming

Name: default
IncomingDir: incoming
TempDir: /tmp
Allow: sid
Cleanup: on_deny on_error

Create and edit options conf file:

vim options

verbose
ask-passphrase
basedir .

Create and edit uploaders conf file:

vim uploaders

allow * by unsigned

Then create some other files

cd ../indices
touch override.sid.main
touch override.sid.main.debian-installer
touch override.sid.main.src
cd ..

3. Generating gpg key

gpg --gen-key

4. Export the repository

reprepro -vb /org/ftp.debian.org/ftp/apt export

5. Adding packages to the repository To add a package to the repository, use one of these commands (replace <...> with the name of your package):

reprepro includedeb sid <deb filename>
reprepro includedsc sid <dsc filename>

Example:

reprepro includedeb sid openssh-client_5.9p1-5_amd64.deb
reprepro includedsc sid eglibc_2.13-32.dsc