Differences between revisions 14 and 15
Revision 14 as of 2006-10-16 10:13:15
Size: 7972
Editor: ?SimonJosefsson
Comment:
Revision 15 as of 2006-10-16 11:46:13
Size: 10278
Editor: ?SimonJosefsson
Comment:
Deletions are marked like this. Additions are marked like this.
Line 182: Line 182:
== Theory of operation ==

1. Rsync the Debian repository.

{{{
MIRROR=ftp.se.debian.org::debian
OUT=/data/debian
TMPFILE=/tmp/jas-junk-$$

rsync -av --exclude Sources.diff $MIRROR/dists/testing/main/source $OUT/dists/testing/main/
rsync -av --exclude Packages.diff $MIRROR/dists/testing/main/binary-i386 $OUT/dists/testing/main/

rsync ftp.se.debian.org::debian/pool/main/ | awk '{print $5}' | grep -v -e rsync -e \\. > $TMPFILE

LIST=`cat $TMPFILE`

for i in $LIST; do
    echo Working in $i...
    rsync -av --delete --delete-excluded \
 --exclude '*.udeb' \
 --exclude '*_alpha.deb' \
 --exclude '*_amd64.deb' \
 --exclude '*_arm.deb' \
 --exclude '*_hppa.deb' \
 --exclude '*_hurd-i386.deb' \
 --exclude '*_ia64.deb' \
 --exclude '*_m68k.deb' \
 --exclude '*_mips.deb' \
 --exclude '*_mipsel.deb' \
 --exclude '*_powerpc.deb' \
 --exclude '*_s390.deb' \
 --exclude '*_sparc.deb' \
 $MIRROR/pool/main/$i $OUT/pool/main/
done
}}}

2. Get list of package contents.

{{{
OUT=/data/debian

mkdir -p $OUT/contents/

find $OUT/pool/main -name \*.orig.tar.gz |
( while read file; do
  oldIFS=$IFS
  IFS=/
  set -- $file
  IFS=$oldIFS
  dir=$6
  pkg=$7
  basename=$8
# echo file $file dir $dir pkg $pkg basename $basename
  if ! test -f $OUT/contents/$basename; then
      echo unpacking $file
      tar tvfz $file > $OUT/contents/$basename
  fi
done
)
}}}

3. Search for suspicious filenames.

{{{
DIR=/data/debian
SRCLIST=$DIR/dists/testing/main/source/Sources

gzip -cd $SRCLIST.gz > $SRCLIST

cd $DIR/contents
find . -type f|sort|xargs egrep \
    -e rfc[0-9]+\.txt \
    -e draft-.*[0-9][0-9]\.txt \
| (
while read line; do
  oldIFS=$IFS
  IFS=:
  set -- $line
  IFS=$oldIFS
  file=$1
  file=`echo $file | sed 's,^./,,g'`
  pkg=`echo $file | sed 's,_.*,,g'`
  ver=`grep -A 3 "^Package: $pkg\$" $SRCLIST | grep ^Version: | sed 's/Version: //'`
# echo line $line
# echo file $file
  if test -n "$lastpkg" -a "$pkg" != "$lastpkg"; then
    if test -n "$files"; then
      echo pkg $lastpkg ver $lastver files $files
    fi
    files=""
  fi
  if grep -q $file $SRCLIST; then
     set -- $line
     files="$files $6"
  fi
  lastpkg=$pkg
  lastver=$ver
done
echo last pkg $pkg ver $ver files $files
)
}}}

This page provides information on IETF RFC/I-Ds within Debian.

Background

The majority of IETF documents, such as RFCs, are not licensed under DFSG-free terms, and should thus not be included in Debian's main. However, RFC documents are frequently included in many packages that would otherwise be free software. Some care is required to remove then when packaging software for Debian.

Some links to discussion of the license problems:

Some general background:

In particular, the etch_rc_policy.txt document above says:

  1. DFSG-freeness
    • All content in main and contrib must meet the DFSG, both in .debs and in the source (including the .orig.tar.gz)

Status

All non-free RFC files in non-source packages are believed to be reported, see:

Currently I'm investigating source packages too. A list of packages with non-free files in them are found at:

On 2006-10-16, I began sending bug reports for RFC/I-Ds in source packages too. They will eventually show up under the first link in this section too.

Known exceptions

These documents have been made available under a free license:

Template for RFC authors to release additional rights

If you as a RFC editor wish to grant additional rights within the document directly, to avoid having the Debian community ask you for additional rights later on, you may include the following text in the document. Similar text has been approved in RFCs before (e.g., RFC 3492, RFC 4501) and the IETF do not appear to object to this practice.

x. Copying conditions

        The author(s) agree to grant third parties the irrevocable
        right to copy, use and distribute the work, with
        or without modification, in any medium, without royalty,
        provided that, unless separate permission is granted,
        redistributed modified works do not contain misleading
        author, version, name of work, or endorsement information.

The text is derived from draft-josefsson-ipr-notice-update.

Template requesting additional rights from RFC authors

This is based on RFC 4663 and draft-josefsson-ipr-rules-update (see <http://josefsson.org/bcp78broken/>).

Subject: Requesting additional rights to RFC xxxx

Dear Author,

The Debian GNU/Linux distribution wishes to incorporate the
IETF RFC xxxx as part of its distribution, and to allow
users to develop, modify and evolve the document.

Because the authors of contributions to the IETF standards retain
most intellectual property rights with respect to such contributions
under IETF policies in effect during the development of RFC xxxx, and
because you are an author of said document, the Debian community hereby
requests that you kindly agree to release your contributions in
RFC xxxx under the license below, for inclusion in Debian.

        I agree to grant third parties the irrevocable
        right to copy, use and distribute the work, with
        or without modification, in any medium, without royalty,
        provided that, unless separate permission is granted,
        redistributed modified works:

             (a) do not contain misleading author, version, name
                 of work, or endorsement information, and

             (b) do not claim endorsement of the modified work by
                 the Contributor, or any organization the
                 Contributor belongs to, the Internet Engineering
                 Task Force (IETF), Internet Research Task Force
                 (IRTF), Internet Engineering Steering Group
                 (IESG), Internet Architecture Board (IAB),
                 Internet Assigned Numbers Authority (IANA),
                 Internet Society (ISOC), Request For Comments
                 (RFC) Editor, or any combination or variation of
                 such terms (including without limitation the
                 IETF "4 diamonds" logo), or any terms that are
                 confusingly similar thereto, and

             (c) remove any claims of status as an Internet
                 Standard, including without limitation removing
                 the RFC boilerplate.

        The IETF suggests that any citation or excerpt of
        unmodified text reference the RFC or other document from
        which the text is derived.

To indicate that you agree to these terms, please reply to this e-mail
and quote the license above and indicate that you agree to this.

If you prefer another widely recognized free license instead, the
following ones are also fine:
 * the 3-clause BSD license
   http://www.gnu.org/licenses/info/BSD_3Clause.html
 * the GNU GPL version 2
   http://www.fsf.org/licensing/licenses/gpl.txt
 * the Expat/MIT license
   http://www.opensource.org/licenses/mit-license.php

 Sincerely yours,
   Simon Josefsson

Bug report template

This will be used for the reports on non-free RFCs in source packages.

{{{ Subject: Source package contains non-free IETF RFC Severity: serious Package: [package] Version: [version] Hi!

This bug has been filed on multiple packages, and general discussions are kindly requested to take place on debian-legal or debian-devel in the thread with Subject: "Non-free IETF RFC/I-Ds in source packages".

It seems this source package contains the following files from the IETF under non-free license terms:

FILES The license on RFC/I-Ds is not DFSG-free, see:

The etch release policy says binary and source packages must each be free:

The severity is serious, because this violates the Debian policy:

There are (at least) three ways to fix this problem. In order of preference:

1. Ask the author of the RFC to re-license the RFC under a free

2. Remove the non-free material from the source, e.g., by re-packaging

  • the upstream archive and adding a 'dfsg' version name to it.

3. Move the package to non-free.

I went over many packages looking for names of likely non-free files, and there may be false positives. If this is the case for your package, I'm sorry for the noise. I'll modify the scripts to take into account false positives when I learn of them, and publish the list of exceptions under "Known exceptions" at <http://wiki.debian.org/NonFreeIETFDocuments>.

Thanks, Simon }}}

Theory of operation

1. Rsync the Debian repository.

MIRROR=ftp.se.debian.org::debian
OUT=/data/debian
TMPFILE=/tmp/jas-junk-$$

rsync -av --exclude Sources.diff $MIRROR/dists/testing/main/source $OUT/dists/testing/main/
rsync -av --exclude Packages.diff $MIRROR/dists/testing/main/binary-i386 $OUT/dists/testing/main/

rsync ftp.se.debian.org::debian/pool/main/ | awk '{print $5}' | grep -v -e rsync -e \\. > $TMPFILE

LIST=`cat $TMPFILE`

for i in $LIST; do
    echo Working in $i...
    rsync -av --delete --delete-excluded \
        --exclude '*.udeb' \
        --exclude '*_alpha.deb' \
        --exclude '*_amd64.deb' \
        --exclude '*_arm.deb' \
        --exclude '*_hppa.deb' \
        --exclude '*_hurd-i386.deb' \
        --exclude '*_ia64.deb' \
        --exclude '*_m68k.deb' \
        --exclude '*_mips.deb' \
        --exclude '*_mipsel.deb' \
        --exclude '*_powerpc.deb' \
        --exclude '*_s390.deb' \
        --exclude '*_sparc.deb' \
        $MIRROR/pool/main/$i $OUT/pool/main/
done

2. Get list of package contents.

OUT=/data/debian

mkdir -p $OUT/contents/

find $OUT/pool/main -name \*.orig.tar.gz | 
( while read file; do
  oldIFS=$IFS
  IFS=/
  set -- $file
  IFS=$oldIFS
  dir=$6
  pkg=$7
  basename=$8
#  echo file $file dir $dir pkg $pkg basename $basename
  if ! test -f $OUT/contents/$basename; then
      echo unpacking $file
      tar tvfz $file > $OUT/contents/$basename
  fi
done
)

3. Search for suspicious filenames.

DIR=/data/debian
SRCLIST=$DIR/dists/testing/main/source/Sources

gzip -cd $SRCLIST.gz > $SRCLIST

cd $DIR/contents
find . -type f|sort|xargs egrep \
    -e rfc[0-9]+\.txt \
    -e draft-.*[0-9][0-9]\.txt \
| (
while read line; do
  oldIFS=$IFS
  IFS=:
  set -- $line
  IFS=$oldIFS
  file=$1
  file=`echo $file | sed 's,^./,,g'`
  pkg=`echo $file | sed 's,_.*,,g'`
  ver=`grep -A 3 "^Package: $pkg\$" $SRCLIST | grep ^Version: | sed 's/Version: //'`
#  echo line $line
#  echo file $file
  if test -n "$lastpkg" -a "$pkg" != "$lastpkg"; then
    if test -n "$files"; then
      echo pkg $lastpkg ver $lastver files $files
    fi
    files=""
  fi
  if grep -q $file $SRCLIST; then
     set -- $line
     files="$files $6"
  fi
  lastpkg=$pkg
  lastver=$ver
done
echo last pkg $pkg ver $ver files $files
)