What you need

To establish an IPSec tunnel to a netscreen device you need at least openswan, strongswan or racoon on your debian box. The example describes the configuration of openswan but I tested the connections with strongswan and racoon too. The scenario is to establish a tunnel as a roadwarrior with a dynamic IP address and authenticate via a x509 based certificate. I built a self signed CA and use it to sign the client requests.
Generate your certificate request on your firewall itself and sign it with your CA. Netscreen uses X509v3 Subject Alternative Name to identify the certificates, so don't remove these extension when you're signing the Certificate. You also need to set an extension to the client certificate, otherwise you'll have trouble to establish a tunnel if you don't have a RSA signature in your DNS.

Create a self signed Certificate

generate your CA RSA key

openssl genrsa -aes256  -out CA.key 4096

Generating RSA private key, 4096 bit long modulus
e is 65537 (0x10001)
Enter pass phrase for CA.key:
Verifying - Enter pass phrase for CA.key:

The command above creates an aes encrypted RSA key with a lengh of 4096 bit and saves it in CA.key.This key is the heart of your PKI, so you need to keep it private.

generate your self signed CA

openssl req -new -x509  -sha1 -days 365 -key ca.key -out ca.pem

Enter pass phrase for ca.key:
You are about to be asked to enter information that will be incorporated
into your certificate request.

The command above generates a self signed certificate which is valid for one year, this certificate is public and it's used to check other certificates which has been signed with your key. Please refer http://en.wikipedia.org/wiki/Public_key_infrastructure for more information.

Now switch to your netscreen and generate a certificate request, you can do that via browser or cli. I prefer the cli since I'm much faster with that instead using a browser and searching for everything.

set pki x509 dn country-name <>
set pki x509 dn email <>
set pki x509 dn ip <>
set pki x509 dn local-name <>
set pki x509 dn name <>
set pki x509 default send-to <>
exec pki rsa new-key 2048

Now upload the signed certificate and the CA to your netscreen by using the browser, tft or scp. The netscreen can now authenticate itself and can check a certificate which has been signed by your CA.
We will now generate a RSA key for the client, a client certificate request and we will sign it with the CA key.

Generate the client key:
openssl genrsa -aes256 -out client.key 4096
Set an extension:
echo "subjectAltName=email:email@example.com">extension

Generate the certificate request:
openssl req -new -key client.key -out client.csr

Sign the client csr and add the extension:
openssl x509 -req -days 365 -CA ca.pem -CAkey ca.key -CAcreateserial -CAserial serials -in client.csr -out client.pem -extfile extension

Upload the client.pem certificate to your netscreen, this is the cert you are using to authenticate against the firewall. It's also possible to verify the client cert against the CA without uploading the client Cert into the firewall, but I'll upload the cert usually to the firewall itself
You should generate a CRL too, to handle certificates they are not allowed to connect anymore etc.

Configuration Debian

Now we have everything to authenticate and we'll setup the Roadwarrior, I prefer a static setup but you can set client parameters via dhcp too.
The client get a virtual 32 bit IP address, only this IP can connect to the remote network, there is no need to setup iptables NAT rules, we route the packets to this IP.
Your configuration files are /etc/ipsec.conf, ipsec.secrets and /etc/ipsec.d/*, you need to copy your client.pem to /etc/ipsec.d/certs and your client.key to /etc/ipsec.d/private, place your cacert.pem to /etc/ipsec.d/cacerts and your CRL to /etc/ipsec.d/crls.
Open /etc/ipsec.secrets with your favorite editor and set your private RSA key.
Set the following line in your ipsec.secrets, you need to change $password to the password you may have use while you've generated your private key. If you created the key without using any encryption parameters the RSA key will not be encrypted and you don't need to set the password parameter.

RSA : client.key "$password"

Edit /etc/ipsec.conf and make sure you have at least the following in your config.

version 2.0     # conforms to second version of ipsec.conf specification

# basic configuration
config setup
        # if the roadwarrior is behind a router/firewall or whatever translate his address
        # useful for debugging
        #plutodebug="control parsing"
        klipsdebug = none

        # enable this if you see "failed to find any available worker"
        # nhelpers=0

# define the std connection parameter

conn %default

conn your_connection_name
        leftsubnet= # this will be your virtual address
        rightsubnet= # your remote network behind the netscreen
        rightid=@netscreen.example.com # the subject alt name used in the netscreen cert

Please refer to man 5 ipsec.conf for more details. Now get your virtual ip running on your external interface.

ifconfig $EXT_INTERFACE:0 netmask up

Restart or start your ipsec daemon via /etc/init.d/ipsec (start|restart) and check that you see in /var/log/auth.log the following line:
adding interface $EXT_INTERFACE:1/wlan0:1
adding interface $EXT_INTERFACE:1/wlan0:1