Translation(s): none

(!) ?Discussion

DDE Tutorial: exploring DDE

Introduction

DDE Is a tool to make it easy to publish Debian information. In this tutorial we have a little tour to see how DDE works.

If you just want to know a quick&dirty way to get at the data, just go to a DDE site (DDE has a list) and follow the instructions.

Prerequisites

You should install some Python packages : 

# Note: you can use python-syck if you do not have python-yaml
apt-get install python-json python-yaml python-paste python-debianbts python-werkzeug python-cherrypy

... this should help get rid of messages like : 

"WARNING:static:Cannot find a way to decode the data in ..."

Downloading the code

git clone git://git.debian.org/debtags/dde.git
cd dde

Explore the information tree

./testdde --help /
./testdde ls -l /apt
./testdde get /apt/packages/apt
./testdde get -t json /apt/packages/apt  (requires python-json)
./testdde get -t yaml /apt/packages/apt  (requires python-yaml or python-syck)

DDE exports data as a read-only tree that can be queried using paths. Given a tree path, you can get its value, list its child nodes or get some documentation about what it contains.

Tree paths are provided by plugins loaded at program startup. It is quite easy to write new plugins that add new branches to the tree, exporting new information.

When you get the value of a path, you can choose the format of the result. Currently supported are json, yaml, csv and pickle, but more can be added.

Publish the information on the web

Start DDE as a HTTP server (requires python-paste or python-cherrypy): 

./testdde --server

You can now point your browser at http://localhost:8080

The DDE server shows an introduction about DDE, and allows you to navigate the tree.

Once you find the information that you want, you can download it in a format of your choice:

From the DDE main page you also have links to further information about all the supported data formats.

The internal web server is not the only way to publish information on the web: DDE is implemented as a WSGI application, and can be deployed as CGI, FastCGI, SCGI, mod_wsgi and pretty much any other way you can think of.

Explore JavaScript integration

Load the file ./complete.html into a browser while the DDE server is running and type the first few letters of a package name: after a short moment a popup should appear suggesting possible completions.

Have a look at the source code to see how simple it can be to use DDE to add autocompletion to web forms.

Write software that gets data from DDE

This is how you get some data from DDE in python: 

import json, urllib2

def dde_get(url):
  return json.read(urllib2.urlopen(url+"?t=json").read())

print dde_get("http://dde.debian.net/dde/q/aptfile/byfile/lenny-amd64/bin/sh")

The same can be done with Yaml, or Pickle, or any other format. Other programming languages have it just as easy: you are welcome to add here examples in other languages.

If you decode Pickle data, make the decoder safe by using cPickle and deactivating find_global. This is an example code that decodes a long stream of data from DDE: 

import urllib2
import cPickle as pickle

def dde_get_stream(url):
    unpickler = pickle.Unpickler(urllib2.urlopen(url+"?t=pickle"))
    # Disallow arbitrary code execution
    unpickler.find_global = None
    while True:
        try:
            yield unpickler.load()
        except EOFError:
            break

for package in dde_get_stream("http://dde.debian.net/dde/q/udd/dist/d:debian/c:main/r:lenny/a:armel"):
    print repr(package)

Example in Perl: 

use strict;
use warnings;

use JSON;
use LWP::Simple;

use Data::Dumper;

sub dde_get ($) {
    my ($url) = @_;
    my $json = get( $url . '?t=json' );
    my $perl = from_json( $json, { utf8 => 1 } );
    return $perl;
}

my $result = dde_get(
    'http://dde.debian.net/dde/q/aptfile/byfile/lenny-amd64/bin/sh');
print Dumper $result;

See also: