Wiki

• FrontPage
• RecentChanges
• FindPage
• HelpContents
• DebianPAN/OpenCL

================================================================================================================================================================ =

Modif : 

$gitk --all

On est sur la branche master actuellement 
Passage sur la branche patch queue 

$gbp pq import 

(fait les modifs) 
$git commit 

$gbp pq export
$git gui

Commit les fichiers

$sbuild (pour relancer la construction)

I-SETUP

How to add autopkgtest for OpenCL software , using the DebianPackage:rocm-opencl-icd

Take the exemple of a package :

1. Place into /slix and /debian/tests

cd silx 

cd debian/tests

2. Open the control file in order to write into :

nano control

3. Copy and paste the part pocl in the same file to implement ROCm tests:

In the copied part of ontrol dedicated to ROCm test, replace all Pocl by rocm. Add in the section Depends of the new ROCm test the command :

pkg-rocm-tools

Add into the section Restrictions :

skippable

And then modify the following line :

Tests: opencl

Into : 

Test-Command:rocm-test-launcher opencl

And it may works !

Before implementation :

       Tests: no-opencl
        Depends:
         @,
         python3-all,
         python3-pytest,
         python3-pytest-mock,
         python3-pytest-xvfb,
         xauth,
         xvfb,
        Restrictions: allow-stderr

        Tests: opencl
        Depends:
        @,
        clinfo,
        pocl-opencl-icd,
        python3-all,
        python3-pytest,
        python3-pytest-mock,
        python3-pytest-xvfb,
        xauth,
        xvfb,
        Architecture: !i386
        Restrictions: allow-stderr
        
        Test-Command: xvfb-run -s "-screen 0 1024x768x24 -ac +extension GLX +render -noreset" sh debian/tests/gui
        Depends:
         mesa-utils,
         silx,
         xauth,
         xvfb,
        Restrictions: allow-stderr

Implemented ROCm test ->

       Tests: no-opencl
        Depends:
         @,
         python3-all,
         python3-pytest,
         python3-pytest-mock,
         python3-pytest-xvfb,
         xauth,
         xvfb,
        Restrictions: allow-stderr

        Tests: opencl
        Depends:
        @,
        clinfo,
        pocl-opencl-icd,
        python3-all,
        python3-pytest,
        python3-pytest-mock,
        python3-pytest-xvfb,
        xauth,
        xvfb,
        Architecture: !i386
        Restrictions: allow-stderr
        
        Test-Command:rocm-test-launcher opencl
        Depends:
         @,
         clinfo,
         rocm-opencl-icd,
         pkg-rocm-tools,
         python3-all,
         python3-pytest,
         python3-pytest-mock,
         python3-pytest-xvfb,
         xauth,
         xvfb,
        Architecture: !i386
        Features: test-name=opencl-ROCm
        Restrictions: allow-stderr , skippable
        
        Test-Command: xvfb-run -s "-screen 0 1024x768x24 -ac +extension GLX +render -noreset" sh debian/tests/gui
        Depends:
         mesa-utils,
         silx,
         xauth,
         xvfb,
        Restrictions: allow-stderr

So, run the test in /silx by using :

sbuild

Then, tests are truly passed : Add the repository to git

git add .
git commit -m "the name of the version"
git push

## to check the repository 
git log 

And so on for others

* setup Sbuild to run rocm tests

* Add a rocm autopkgtest to your package

For a package

How to launch autopkgtest with podman+rocm :

1.Create a container , an image by using :

$ sudo rocm-podman-create rocm/debian:unstable.img

##To check the image use : 
$ podman images 

##To check the container : 
$ podman ps

##Configure the system for GPU-in-container use
$ rocm-podman-setup -u <user>

##To run autopkgtest : 

# Run rocrand's autopkgtests (from the official archive) in that image
$ autopkgtest -B <pkg> -- podman+rocm rocm/debian:unstable.img

Commands for dgit -gbp sbuild :

1. To see if everything is clean:

git diff in longeac@grades-01:~/ufo-filters$

2. Clean up and start again:

 git clean -xdf
 git reset --hard
 dgit --gbp sbuild

3. Manual directory creation:

.config/sbuild/config.pl et déplacement .sbuildrc 

Writing the config.pl file :

$chroot_mode = "unshare";
$unshare_mmdebstrap_keep_tarball = 1;
$run_autopkgtest = 1;
$external_commands = {
  "chroot-update-failed-commands" => [ [ '%SBUILD_SHELL' ] ],
  "build-deps-failed-commands" => [ [ '%SBUILD_SHELL' ] ],
  "build-failed-commands" => [ [ '%SBUILD_SHELL' ] ],
};
$unshare_mmdebstrap_extra_args = [
  "*" => [ '--aptopt=Acquire::http { Proxy "http://195.221.0.35:8080"; }',
           '--aptopt=Acquire::https { Proxy "http://195.221.0.35:8080"; }',
           '--customize-hook=echo "export http_proxy=http://195.221.0.35:8080" > "$1"/etc/environment',
           '--customize-hook=echo "export https_proxy=http://195.221.0.35:8080" > "$1"/etc/environment',
]
];
$autopkgtest_opts=['--shell-fail', '--apt-upgrade', '--env=https_proxy=http://195.221.0.35:8080', '--', 'podman+rocm', 'localhost/rocm/debian:%r']

4. schroot Upgrade :

schroot -l sbuild-update -ud testing

5. New procedure for modifying sbuild

**SETUP

Install necessary packages:

sudo apt install sbuild mmdebstrap uidmap

Create a directory for the chroot tarball :

mkdir -p ~/.cache/sbuild

Create/Update the tarball :

mmdebstrap --include=ca-certificates --skip=output/dev --variant=buildd unstable ~/.cache/sbuild/unstable-amd64.tar.zst https://deb.debian.org/debian

Error results :

If you are using apt-cacher-ng, specify http:// instead of https:// and enable the apt-cacher-ng proxy using --aptopt='Acquire::http { Proxy "http://195.221.0.35:8080"; }'. You can use any tarball compression extension you prefer :

        mmdebstrap --skip=output/dev --variant=buildd unstable ~/.cache/sbuild/unstable-amd64.tar.gz http://deb.debian.org/debian --aptopt='Acquire::http { Proxy "http://195.221.0.35:8080"; }'

*** Validation check :

perl -c ~/.config/sbuild/config.pl

Result:

> longeac@grades-01:~$ perl -c ~/.config/sbuild/config.pl
/home/experiences/grades/longeac/.config/sbuild/config.pl syntax OK

6. Test and launch the command:

In the directory : longeac@grades-01: ~/ufo-filters

sbuild

And it works !!!!!

Commands for /usr/bin/autopkgtest -B ufo-filters --test-name=reconstruct -- unshare --release unstable --arch amd64 :

sudo apt install  mmdebstrap 

sudo nano /etc/subuid

longeac:100000:65536

sudo nano /etc/subuid

longeac:100000:65536

/usr/bin/autopkgtest -B ufo-filters --test-name=reconstruct -- unshare --release unstable --arch amd64

For very specific case

For the package Nabu, after the following implementation, it might be possible that Opencl's tests would be marked as SKIPPED for an unkown reason even if the test ran truly include Opencl. First, the SKIPPED mark , appears if we have no tests data or not enough. So that Pierre Paleo, maintainers of Nabu , suggest to proceed like this to takle the issue :

  apt update
  apt install python3 python3-venv
  apt install git g++
  apt install opencl-headers opencl-c-headers opencl-clhpp-headers ocl-icd-opencl-dev pocl-opencl-icd clinfo

  python3 -m venv /tmp/paleo/pydeb
  source /tmp/paleo/pydeb/bin/activate
  pip install --upgrade pip setuptools wheel
  pip install pyopencl
  pip install pyvkfft
  pip install nabu

 # check that (py)OpenCL works
  python -c "import pyopencl as cl; import pyopencl.array as parray; ctx = cl.create_some_context(); queue = cl.CommandQueue(ctx); d = parray.zeros(queue, 500, 'f'); d+= 1; print(d)"
 
# run nabu tests
  nabu-test -rs -v

II-List of packages implemented

https://codesearch.debian.net/search?q=pocl-opencl-icd+path%3Adebian%2Ftests%2Fcontrol&literal=1&perpkg=1

https://codesearch.debian.net/search?q=path%3Adebian%2Ftests%2Fcontrol+pyopencl&literal=1

https://codesearch.debian.net/search?q=path%3Adebian%2Fcontrol+pocl-opencl-icd&literal=1&perpkg=1

https://codesearch.debian.net/search?q=path%3Adebian%2Fcontrol+pyopencl&literal=1

https://codesearch.debian.net/search?q=path%3Adebian%2Fcontrol+opencl&literal=1 -> TODO find interesting packages.

The table was created before implementation to have an overview of the work

After implémentation of tests :

||<#FFFF00> O PASS|| The autopkgtest was present, at the origin we have errors and after implementation it pass

||<#FFFF00> PASS|| The autopkgtest was absent but it pass

For all that had been done , please check the report

III-Other tests

Inventory of all other pkg tests

ufo-filters

Test files : test_swap_quadrants.py / test_memin.py / test_fft.py / check-gradient / check-multipage-readers / test-142.sh / test-149.sh / test-153.sh / test-161.sh / test-177.sh /test-core-149.sh /test-file-write-regression.sh /test-gradient.sh /test-multipage-readers.sh /test-nlm.sh /

nabu

Test files : testutils.py utils.py tests.py nabu/preproc/tests nabu/io/tests nabu/estimation/tests nabu/misc/tests nabu/pipeline/tests nabu/processing/tests nabu/reconstruction/tests nabu/resources/tests nabu/stitching/tests

pynx

Test files : master/debian/tests/nabu-pocl pynx/-/blob/master/.gitlab-ci.yml pynx/cdi/test pynx/holotomo/test pynx/processing_unit/test pynx/ptycho/test pynx/scattering/test pynx/scripts/pynx_test.py pynx/-/tree/master/pynx/test pynx/-/tree/master/pynx/utils/test pynx/-/tree/master/pynx/wavefront/test

pocl

Test files : pocl/-/tree/main/cmake pocl/-/tree/main/examples/PyOpenCL pocl/-/tree/main/examples/ViennaCL pocl/-/tree/main/examples/chipStar pocl/-/tree/main/examples/shoc pocl/-/tree/main/tests pocl/-/tree/main/tools/scripts pocl/-/tree/main/tools/docker pocl/-/tree/main/tools/data

examl

Test files : examl/-/tree/master/testData

clblast

Test files :clblast/-/tree/debian/sid/.github/workflows clblast/-/tree/debian/sid/test clblast/-/blob/debian/sid/doc/testing.md clblast/-/blob/debian/sid/scripts/generator/generator/routine.py clblast/-/blob/debian/sid/doc/testing.md

clpeak

Test files : clpeak/-/tree/main/android/app/src/androidTest

genx

Test files : genx/-/blob/debian/master/doc_source/source/_templates/test genx/-/tree/debian/master/genx/tests genx/genx/models/lib/testing.py genx/tests/data/test.conf genx/genx/plugins/add_ons/Test.py genx/genx/examples/widget_test.ipynb

haskell-futhark

sasmodels

Test files : .github/workflows/test.yml TESTS.txt explore/beta/testPolyDiseperseSphricalBeta.m sasmodels/model_test.py data_files/testPolydisperseGaussianSphere.dat explore/beta/data_files/testPolydisperseGaussianSphere2.dat pytest.ini conftest.py debian/patches/0005-Clean-up-model-files-from-tests.patch

hashcat

Test files : tools/test.pl tools/test.sh tools/test_rules.pl tools/rules-test-cases.yaml src/selftest.c include/selftest.h tools/test_modules tools/cl_tests tools/pdf_tests tools/tc_tests tools/vc_tests?ref_type=heads

libhmsbeagle

Test files : tests examples/swig_python/test.py examples/hmctest examples/matrixtest examples/complextest examples/tinytest examples/oddstatetest examples/synthetictest

pyopencl

Test files : test .test-conda-env-py3.yml

compyle

Test files : compyle/tests/test_jit.py .github/workflows/tests.yml

gpyfft

Test files : gpyfft/test debian/patches/0001-add-also-the-CPU-devices-for-tests.patch debian/tests/control

pyvkfft

Test files : /doc/source/scripts pyvkfft/test doc/source/api/testing.rst pyvkfft/scripts/pyvkfft_test_suite.py examples/pyvkfft-tests-CUDA.ipynb doc/source/examples/pyvkfft-tests-OpenCL.nblink

pytools

/

python-dtcwt

Test files : tests

xpra

/

pysph

Test files : pysph/base/tests pysph/sph/tests .github/workflows/tests.yml pysph/solver/tests pysph/parallel/tests pysph/tools/tests requirements-test.txt pysph/examples/tests/test_examples.py

pyfai

Test files : src/pyFAI/test

silx

Test files : src/silx/app/test_.py src/silx/test src/silx/gui/test src/silx/image/test/test_bb.py src/silx/io/test src/silx/utils/test src/silx/gui/plot3d/test/test_gl.py src/silx/gui/_glutils/test/test_gl.py src/silx/math/fft/test src/silx/math/test/test_combo.py src/silx/opencl/test/test_image.py

python-xrt

Test files : tests/raycing examples/withRaycing/_QookBeamlines/testGrating.xml examples/withRaycing/_QookBeamlines/testAlignment.xml

• FIN

pytools, compyle, gpyfft, pyvkfft, python-dtcwt, xpra, pysph, pynx, pyfai, sasmodels, silx, python-xrt

* running test via command line

AUTOPKGTEST_TMP=/tmp OCL_ICD_VENDORS=/etc/OpenCL/vendors/amdocl64.icd rocm-test-launcher debian/tests/script 

* autopkgtest setup

try this https://salsa.debian.org/rocm-team/pkg-rocm-tools