This page describes how to build proper udebs. Since I have only done this once, and I haven't even tested the result, there may be many errors here. Please correct them!

debian/control

You need to define that you're building the udeb using debian/control. It looks very much like any other binary package. Special things are:

the rest

The rest is the same as a normal package. Except that for udebs, you will probably want to strip a lot of stuff (documentation, themes, that sort of things). Udebs don't have to follow policy in many respects, but it is important that they are small. Most of this is taken care of automatically by debhelper if you have set XC-Package-Type correctly.