You can put Debian on a memory stick (keydrive) and boot it directly from the memory stick.

The DebianInstaller can both boot from USB keydrives, and install to them.

See also: Keydrive