Differences between revisions 23 and 82 (spanning 59 versions)
Revision 23 as of 2006-10-26 12:51:39
Size: 15651
Editor: ?SimonJosefsson
Comment:
Revision 82 as of 2014-10-23 20:37:34
Size: 10980
Editor: ?MartinSteghoefer
Comment: Dual license RFC 5215
Deletions are marked like this. Additions are marked like this.
Line 5: Line 5:
The majority of IETF documents, such as RFCs, are not licensed under DFSG-free terms, and should thus not be included in Debian's main. However, RFC documents are frequently included in many packages that would otherwise be free software. Some care is required to remove then when packaging software for Debian. The majority of IETF documents, such as RFCs, are not licensed under DFSG-free terms, and should thus not be included in Debian's main. However, RFC documents are frequently included in many packages that would otherwise be free software. Some care is required to remove the documents when packaging software for Debian.
Line 9: Line 9:
 * http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=199810  * DebianBug:92810
 * DebianBug:199810
Line 15: Line 16:
 * http://release.debian.org/removing-non-free-documentation
 * http://release.debian.org/etch_rc_policy.txt

In particular, the etch_rc_policy.txt document above says:
 * http://release.debian.org/squeeze/rc_policy.txt
 * http://release.debian.org/wheezy/rc_policy.txt

In particular, the latter document above says:
Line 26: Line 27:
All non-free RFC files in non-source packages are believed to be reported, see: All non-free RFC files in source packages are believed to be reported, see:
Line 30: Line 31:
Currently I'm investigating source packages too. A list of packages with non-free files in them are found at:

 * http://josefsson.org/bcp78broken/ietf-in-src.txt

On 2006-10-16, I sent bug reports for RFC/I-Ds in source packages too. They will eventually show up under the first link in this section too.

Some raised a concern that there may be false positives in the list. I modified the script (see last on this page) to compute MD5's on the RFC in the source packages, and run diff+cmp on the files if the MD5 doesn't match. The output from the script is published at:

 * http://josefsson.org/bcp78broken/debian-ietf-documents-diff.txt

There doesn't seem to be any false positives, but I'm going through the MISMATCHES one by one.
After the initial report against a lot of packages without manual checking, there were claims of false positives in the list. To this date, the only at least likely false positive was DebianBug:393411 and DebianBug:674940.

On 2006-10-24 I modified the script to recursively look into archives-within-archives in the source packages, to find deeply nested documents. Since that date, I run the debian-run script (see below) from time to time and report any changes. Since the amount of changes is now small, I try to check each package manually before reporting. As of 2009-04-21, this is still the current status.

On 2009-09-11 the last bug related to a package in testing was fixed (DebianBug:459705) and on 2009-09-22 the package entered testing. In order words, by that date, testing no longer contain any non-free RFCs! Since then, the status has regressed and some packages with RFCs in them has entered testing, but as of 2010-05-17 testing do not contain any RFCs.

As of 2010-05-17 the list of remaining packages (in unstable!) that contains IETF documents is 'samba4'.

On 2011-10-19 another check was made, and bugs were found (and reported) in lusca, open-iscsi, libpgm, isc-dhcp, libreoffice, and libmath-base85-perl.

Other checks can be made without referring to this page, like the http://bugs.debian.org/cgi-bin/pkgreport.cgi?tag=gnewsense;tag=libreplanet;users=gnewsense-dev@nongnu.org;, which spotted bacula (DebianBug:658326).
Line 47: Line 48:
   See http://bugs.debian.org/390658    See DebianBug:390658
Line 50: Line 51:
   See http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=393408    See DebianBug:393408
Line 52: Line 53:

 * The I-D draft-ietf-cat-kerb-chg-password-02.txt
   See DebianBug:393380
   and http://packages.debian.org/changelogs/pool/main/k/krb5/current/copyright

 * RFC 2629.
   See the copyright of the source file in xml2rfc.

 * RFC 5864.
   See the copyright of the source file in openafs.

 * RFC 5215.
   See section 11 "Copying Conditions" of the RFC: http://tools.ietf.org/html/rfc5215#section-11
Line 60: Line 74:
   The RFC editor has something else to say on this {{{    See bug DebianBug:365201 for some discussion.

The RFC editor has something else to say on this
{{{
Line 89: Line 106:
}}}

== Bug report template ==

This will be used for the reports on non-free RFCs in source packages.

{{{
Subject: Source package contains non-free IETF RFC/I-D
Severity: serious
Package: [package]
Version: [version]
User: debian-release@lists.debian.org
Usertags: nonfree-doc rfc

Hi!

This source package contains the following files from the
IETF under non-free license terms:

FILES

The license on RFC/I-Ds is not DFSG-free, see:

 * http://wiki.debian.org/NonFreeIETFDocuments
 * http://bugs.debian.org/199810

According to the squeeze/wheezy release policy, source packages must be
DFSG-free, see:

 * http://release.debian.org/squeeze/rc_policy.txt
 * http://release.debian.org/wheezy/rc_policy.txt

The severity is serious, because this violates the Debian policy:

 * http://www.debian.org/doc/debian-policy/ch-archive.html#s-dfsg

There are (at least) three ways to fix this problem. In order of
preference:

1. Ask the author of the RFC to re-license the RFC under a free
   license. A template for this e-mail request can be found at
   http://wiki.debian.org/NonFreeIETFDocuments

2. Remove the non-free material from the source, e.g., by re-packaging
   the upstream archive and adding 'dfsg' to the Debian package
   version name.

3. Move the package to non-free.

General discussions are kindly requested to take place on debian-legal
or debian-devel in the thread with Subject: "Non-free IETF RFC/I-Ds in
source packages".

Thanks,
Simon
Line 179: Line 251:
== Bug report template ==

This will be used for the reports on non-free RFCs in source packages.

{{{ Subject: Source package contains non-free IETF RFC
Severity: serious
Package: [package]
Version: [version]
Hi!

This bug has been filed on multiple packages, and general discussions
are kindly requested to take place on debian-legal or debian-devel in
the thread with Subject: "Non-free IETF RFC/I-Ds in source packages".

It seems this source package contains the following files from the
IETF under non-free license terms:

FILES
The license on RFC/I-Ds is not DFSG-free, see:
 * http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=199810
 * http://release.debian.org/removing-non-free-documentation
 * http://wiki.debian.org/NonFreeIETFDocuments

The etch release policy says binary and source packages must each be free:
 * http://release.debian.org/etch_rc_policy.txt

The severity is serious, because this violates the Debian policy:
 * http://www.debian.org/doc/debian-policy/ch-archive.html#s-dfsg

There are (at least) three ways to fix this problem. In order of
preference:

1. Ask the author of the RFC to re-license the RFC under a free
   license. A template for this e-mail request can be found at
   http://wiki.debian.org/NonFreeIETFDocuments

2. Remove the non-free material from the source, e.g., by re-packaging
   the upstream archive and adding a 'dfsg' version name to it.

3. Move the package to non-free.

I went over many packages looking for names of likely non-free files,
and there may be false positives. If this is the case for your
package, I'm sorry for the noise. I'll modify the scripts to take
into account false positives when I learn of them, and publish the
list of exceptions under "Known exceptions" at
<http://wiki.debian.org/NonFreeIETFDocuments>.

Thanks,
Simon
}}}
Line 232: Line 253:
1. Rsync the Debian repository.

The file is called debian-rsync.

{{{
#!/bin/sh

# debian-rsync: Mirror a Debian repository.

# Written by Simon Josefsson <simon@josefsson.org> during September
# 2006. Released under GPLv2 or later. See
# <http://wiki.debian.org/NonFreeIETFDocuments>.

MIRROR=ftp.se.debian.org::debian
OUT=/data/debian

rsync -av --exclude Sources.diff $MIRROR/dists/testing/main/source \
    $OUT/dists/testing/main/
rsync -av --exclude Packages.diff $MIRROR/dists/testing/main/binary-i386 \
    $OUT/dists/testing/main/
rsync -av --exclude Sources.diff $MIRROR/dists/unstable/main/source \
    $OUT/dists/unstable/main/
rsync -av --exclude Packages.diff $MIRROR/dists/unstable/main/binary-i386 \
    $OUT/dists/unstable/main/

LIST=$(rsync $MIRROR/pool/main/ \
    | grep '^drwx' \
    | awk '{print $5}' \
    | grep -v -e '^.$')

for i in $LIST; do
    echo Working in $i...
    rsync -av --delete --delete-excluded \
 --exclude '*.udeb' \
 --exclude '*_alpha.deb' \
 --exclude '*_amd64.deb' \
 --exclude '*_arm.deb' \
 --exclude '*_hppa.deb' \
 --exclude '*_hurd-i386.deb' \
 --exclude '*_ia64.deb' \
 --exclude '*_m68k.deb' \
 --exclude '*_mips.deb' \
 --exclude '*_mipsel.deb' \
 --exclude '*_powerpc.deb' \
 --exclude '*_s390.deb' \
 --exclude '*_sparc.deb' \
 $MIRROR/pool/main/$i $OUT/pool/main/
done
}}}

2. Get list of package contents.

There are two files here, first tar-recursive and then debian-unpack.

{{{
#!/bin/sh

# tar-recursive: Recursively list members of an archive.

# Inspired by find-in-tars by Timo Juhani Lindfors
# <timo.lindfors@iki.fi>, but re-written by Simon Josefsson
# <simon@josefsson.org> during September 2006. Released under GPLv2
# or later. See <http://wiki.debian.org/NonFreeIETFDocuments>.

for archive in "$@"; do
    case "$archive" in
 *.tar.gz) MEMBERS=`tar tvfz $archive | awk '{print $6}'`;;
 *.tar.bz2) MEMBERS=`tar tvfj $archive | awk '{print $6}'`;;
 *.jar) MEMBERS=`unzip -vqq $archive | awk '{print $8}'`;;
 *.zip) MEMBERS=`unzip -vqq $archive | awk '{print $8}'`;;
    esac

    for member in $MEMBERS; do
 echo $member
 case "$member" in
     *.tar.gz | *.tar.bz2 | *.jar | *.zip)
  echo recursing into $archive $member 1>&2
  tempdir=`mktemp -d -t tar-recursive.XXXXXXXXXX` || exit 1
  case "$archive" in
      *.tar.gz) tar xfz $archive -C $tempdir $member
   err=$? ;;
      *.tar.bz2) tar xfj $archive -C $tempdir $member
   err=$? ;;
      *.jar) unzip -q $archive $member -d $tempdir
   err=$? ;;
      *.zip) unzip -q $archive $member -d $tempdir
   err=$? ;;
  esac
  if test "$err" != "0"; then
      echo "tar failed on $member with error $?" 1>&2
      exit 1
  fi
  echo $member
  tar-recursive $tempdir/$member | sed "s@^@$member:@"
  rm -fr $tempdir
  ;;
 esac
    done
done
}}}

{{{
#!/bin/sh

# debian-unpack: Extract a list of members in *.orig.tar.gz in a
# Debian repository.

# Written by Simon Josefsson <simon@josefsson.org> during September
# 2006. Released under GPLv2 or later. See
# <http://wiki.debian.org/NonFreeIETFDocuments>.

OUT=/data/debian

mkdir -p $OUT/contents/

find $OUT/pool/main -name \*.orig.tar.gz | (
    while read file; do
 oldIFS=$IFS
 IFS=/
 set -- $file
 IFS=$oldIFS
 dir=$6
 pkg=$7
 basename=$8
# echo file $file dir $dir pkg $pkg basename $basename
 if ! test -f $OUT/contents/$basename; then
     echo unpacking $file
     tar-recursive $file > $OUT/contents/$basename
     if [ "x$?" != "x0" ]; then
  echo "tar failed on $filename with error $?" 1>&2
  exit 1
     fi
 fi
    done)
}}}

3. Search for RFC matches.

(note: this one isn't updated to use the same contents file generated by more modern scripts above. It is preserved here until there is a better script available.)

{{{
DIR=/data/debian
SRCLIST=$DIR/dists/testing/main/source/Sources
RFCDIR=/home/jas/rfc
TMP=/tmp
IDURL=http://bgp.potaroo.net/ietf/all-ids/

gzip -cd $SRCLIST.gz > $SRCLIST

cd $DIR/contents
find . -type f|sort|xargs egrep \
    -e rfc[0-9]+\.txt \
    -e draft-.*[0-9][0-9]\.txt \
| (
while read line; do
  oldIFS=$IFS
  IFS=:
  set -- $line
  IFS=$oldIFS
  file=$1
  file=`echo $file | sed 's,^./,,g'`
  pkg=`echo $file | sed 's,_.*,,g'`
  ver=`grep -A 3 "^Package: $pkg\$" $SRCLIST | grep ^Version: | sed 's/Version: //'`
# echo line $line
# echo file $file
  if test -n "$lastfile" -a "$file" != "$lastfile"; then
    if test -n "$files"; then
      echo pkg $lastpkg ver $lastver lastfile $lastfile files $files

      X=$DIR/bad/$lastpkg-$lastver/
      mkdir -p $X
      cd $X

      f=`find $DIR/pool/main -name $lastfile`
      for i in $files; do
   echo tar xfz $f $i
   tar xfz $f $i

   if echo $i | grep -q '.bz2$'; then
       echo bzip2 -d $i
       bzip2 -d $i
       i=`echo $i | sed 's/.bz2$//'`
   fi

   md5in=`cat $i | md5sum`
   echo $md5in $i

   base=`basename $i`
   rfc=`echo $base | sed 's/rfc0*/rfc/'`
   if ! test -f $RFCDIR/$rfc; then
       # Probably some unknown draft...
       pushd $RFCDIR
       if ! wget $IDURL/$rfc 2>&1; then
    echo FETCH-FAIL $rfc
       fi
       popd
   fi
   md5real=`cat $RFCDIR/$rfc | md5sum`
   echo $md5real $RFCDIR/$rfc

   if test "$md5in" != "$md5real"; then
       echo MISMATCH $rfc
       diff -urw $i $RFCDIR/$rfc 2>&1
       cmp $i $RFCDIR/$rfc 2>&1
   else
       echo MATCH $rfc
   fi
      done
    fi
    files=""
  fi
  if grep -q $file $SRCLIST; then
     set -- $line
     files="$files $6"
  fi
  lastfile=$file
  lastpkg=$pkg
  lastver=$ver
done
echo last pkg $pkg ver $ver files $files
)
}}}
The source code to search for RFCs inside Debian can be found at http://git.josefsson.org/cgi-bin/gitweb.cgi?p=tools.git;a=tree;f=rfc-search;hb=HEAD
----
## This page is widely referenced from hundreds bug reports
## (search NonFreeIETFDocuments in the BTS. example: http://bugs.debian.org/393400 )
CategoryPermalink

This page provides information on IETF RFC/I-Ds within Debian.

Background

The majority of IETF documents, such as RFCs, are not licensed under DFSG-free terms, and should thus not be included in Debian's main. However, RFC documents are frequently included in many packages that would otherwise be free software. Some care is required to remove the documents when packaging software for Debian.

Some links to discussion of the license problems:

Some general background:

In particular, the latter document above says:

  1. DFSG-freeness
    • All content in main and contrib must meet the DFSG, both in .debs and in the source (including the .orig.tar.gz)

Status

All non-free RFC files in source packages are believed to be reported, see:

After the initial report against a lot of packages without manual checking, there were claims of false positives in the list. To this date, the only at least likely false positive was 393411 and 674940.

On 2006-10-24 I modified the script to recursively look into archives-within-archives in the source packages, to find deeply nested documents. Since that date, I run the debian-run script (see below) from time to time and report any changes. Since the amount of changes is now small, I try to check each package manually before reporting. As of 2009-04-21, this is still the current status.

On 2009-09-11 the last bug related to a package in testing was fixed (459705) and on 2009-09-22 the package entered testing. In order words, by that date, testing no longer contain any non-free RFCs! Since then, the status has regressed and some packages with RFCs in them has entered testing, but as of 2010-05-17 testing do not contain any RFCs.

As of 2010-05-17 the list of remaining packages (in unstable!) that contains IETF documents is 'samba4'.

On 2011-10-19 another check was made, and bugs were found (and reported) in lusca, open-iscsi, libpgm, isc-dhcp, libreoffice, and libmath-base85-perl.

Other checks can be made without referring to this page, like the http://bugs.debian.org/cgi-bin/pkgreport.cgi?tag=gnewsense;tag=libreplanet;users=gnewsense-dev@nongnu.org;, which spotted bacula (658326).

Known exceptions

These documents have been made available under a free license:

The following may hold but is questionable:

  • US-authored RFCs earlier than RFC around RFC 1000-1100.
    • These did not carry a copyright notice, and since the US did not sign the Berne convention until 1989, they are in the public domain.

      See bug 365201 for some discussion. The RFC editor has something else to say on this

From: RFC Editor <rfc-editor@rfc-editor.org>
Subject: Re: Copyright and copying conditions for RFC 1510?
To: Simon Josefsson <jas@extundo.com>
Cc: RFC Editor <rfc-editor@rfc-editor.org>
Date: Mon, 16 Dec 2002 11:07:28 -0800

Simon,

The copyright statement applies retroactively.  Please follow the
instructions as stated at:

   ftp://ftp.rfc-editor.org/in-notes/rfc-editor/rfc-copyright-story

Thank you.

RFC Editor


On Sun, Dec 15, 2002 at 10:38:30AM +0100, Simon Josefsson wrote:
> rfc1510.txt does not mention copyright or copying condition. Does the
> copyright notice in
> 
> ftp://ftp.rfc-editor.org/in-notes/rfc-editor/rfc-copyright-story
> 
> apply retroactively?  If not, do you know who owns the copyright of
> the document and what the copying conditions are?
> 
> Thanks.

Bug report template

This will be used for the reports on non-free RFCs in source packages.

Subject: Source package contains non-free IETF RFC/I-D
Severity: serious
Package: [package]
Version: [version]
User: debian-release@lists.debian.org
Usertags: nonfree-doc rfc

Hi!

This source package contains the following files from the
IETF under non-free license terms:

FILES

The license on RFC/I-Ds is not DFSG-free, see:

 * http://wiki.debian.org/NonFreeIETFDocuments
 * http://bugs.debian.org/199810

According to the squeeze/wheezy release policy, source packages must be
DFSG-free, see:

 * http://release.debian.org/squeeze/rc_policy.txt
 * http://release.debian.org/wheezy/rc_policy.txt

The severity is serious, because this violates the Debian policy:

 * http://www.debian.org/doc/debian-policy/ch-archive.html#s-dfsg

There are (at least) three ways to fix this problem.  In order of
preference:

1. Ask the author of the RFC to re-license the RFC under a free
   license.  A template for this e-mail request can be found at
   http://wiki.debian.org/NonFreeIETFDocuments

2. Remove the non-free material from the source, e.g., by re-packaging
   the upstream archive and adding 'dfsg' to the Debian package
   version name.

3. Move the package to non-free.

General discussions are kindly requested to take place on debian-legal
or debian-devel in the thread with Subject: "Non-free IETF RFC/I-Ds in
source packages".

Thanks,
Simon

Template for RFC authors to release additional rights

If you as a RFC editor wish to grant additional rights within the document directly, to avoid having the Debian community ask you for additional rights later on, you may include the following text in the document. Similar text has been approved in RFCs before (e.g., RFC 3492, RFC 4501) and the IETF do not appear to object to this practice.

x. Copying conditions

        The author(s) agree to grant third parties the irrevocable
        right to copy, use and distribute the work, with
        or without modification, in any medium, without royalty,
        provided that, unless separate permission is granted,
        redistributed modified works do not contain misleading
        author, version, name of work, or endorsement information.

The text is derived from draft-josefsson-ipr-notice-update.

Template requesting additional rights from RFC authors

This is based on RFC 4663 and draft-josefsson-ipr-rules-update (see <http://josefsson.org/bcp78broken/>).

Subject: Requesting additional rights to RFC xxxx

Dear Author,

The Debian GNU/Linux distribution wishes to incorporate the
IETF RFC xxxx as part of its distribution, and to allow
users to develop, modify and evolve the document.

Under IETF policies that were in effect during the development of
RFC xxxx, the authors of contributions to the IETF standards retain
copyright with respect to such contributions. Because you are an
author of said document, the Debian community hereby requests that
you kindly agree to release your contributions in RFC xxxx under
the license below, for inclusion in Debian.

        I agree to grant third parties the irrevocable
        right to copy, use and distribute the work, with
        or without modification, in any medium, without royalty,
        provided that, unless separate permission is granted,
        redistributed modified works:

             (a) do not contain misleading author, version, name
                 of work, or endorsement information, and

             (b) do not claim endorsement of the modified work by
                 the Contributor, or any organization the
                 Contributor belongs to, the Internet Engineering
                 Task Force (IETF), Internet Research Task Force
                 (IRTF), Internet Engineering Steering Group
                 (IESG), Internet Architecture Board (IAB),
                 Internet Assigned Numbers Authority (IANA),
                 Internet Society (ISOC), Request For Comments
                 (RFC) Editor, or any combination or variation of
                 such terms (including without limitation the
                 IETF "4 diamonds" logo), or any terms that are
                 confusingly similar thereto, and

             (c) remove any claims of status as an Internet
                 Standard, including without limitation removing
                 the RFC boilerplate.

        The IETF suggests that any citation or excerpt of
        unmodified text reference the RFC or other document from
        which the text is derived.

To indicate that you agree to these terms, please reply to this e-mail
and quote the license above and indicate that you agree to this.

If you prefer another widely recognized free license instead, the
following ones are also fine:
 * the 3-clause BSD license
   http://www.gnu.org/licenses/info/BSD_3Clause.html
 * the GNU GPL version 2
   http://www.fsf.org/licensing/licenses/gpl.txt
 * the Expat license
   http://www.jclark.com/xml/copying.txt

 Sincerely yours,
   Simon Josefsson

Theory of operation

The source code to search for RFCs inside Debian can be found at http://git.josefsson.org/cgi-bin/gitweb.cgi?p=tools.git;a=tree;f=rfc-search;hb=HEAD


CategoryPermalink