Differences between revisions 1 and 10 (spanning 9 versions)
Revision 1 as of 2018-07-23 15:23:57
Size: 5258
Editor: ?Celejar
Comment:
Revision 10 as of 2021-02-26 04:12:03
Size: 8886
Editor: PaulWise
Comment: betterise apt links
Deletions are marked like this. Additions are marked like this.
Line 1: Line 1:
= GPU Accelerated Video Decoding = #language en
~-[[DebianWiki/EditorGuide#translation|Translation(s)]]: none-~
----
With modern graphics cards, it's often possible to [[https://en.wikipedia.org/wiki/Video_processing_unit#GPU_accelerated_video_decoding|offload]] the jobs of video encoding and decoding to them from the CPU in order to reduce power usage and make more resources available to the rest of the system. Compared to CPUs, GPUs are much more efficient at the job. However, both hardware as well as software support are required for this offloading, and the latter in particular (at least in the Linux world) has undergone much evolution in recent years. Online documentation is therefore sparse and incomplete, inconsistent, and often outdated. Furthermore, support for hardware video acceleration under Linux world is unfortunately split across different APIs with different levels of support.
Line 3: Line 6:
Modern GPUs often offer [[https://en.wikipedia.org/wiki/Video_processing_unit#GPU_accelerated_video_decoding|hardware accelerated video decoding]], which can potentially yield a dramatic reduction in CPU usage during video playback, depending on the codec used and other properties of the video. Both hardware as well as software support are required for this functionality, and the latter in particular (at least in the Linux world) has undergone much evolution in recent years. Online documentation is therefore sparse and incomplete, inconsistent, and often outdated. The following are some notes regarding the use of hardware accelerated video decoding on Debian Stretch. <<TableOfContents(4)>>
Line 5: Line 8:
=== Benefits === == Benefits ==
Historically, the benefits of hardware acceleration under Linux have been [[https://github.com/mpv-player/mpv/commit/dbef5b737e2f994f02923c8214cba368b663a655|uncertain]], but it seems likely that support today has improved drastically. In at least some relatively typical scenarios, the performance gains of using hardware decoding can be huge, [[https://lists.debian.org/debian-user/2018/06/msg00306.html|with reductions in CPU usage of around 90%]].
Line 7: Line 11:
Whether hardware acceleration is a good idea in general is actually debatable. One of the MPV developers has claimed, as recently as July 2017, that “[[https://github.com/mpv-player/mpv/commit/dbef5b737e2f994f02923c8214cba368b663a655|hwdec is usually a bad idea unless absolutely needed]]”, and [[https://wiki.archlinux.org/index.php?title=Mpv&oldid=527776#Hardware_decoding|ArchWiki still claims]] (as of July 20, 2018), that:
 Warning: Hardware decoding is known to sometimes cause problems and as such it is considered "usually a bad idea unless absolutely needed" by the developers. For that reason and because it typically offers very similar performance to software decoding it is disabled by default. Moreover, depending on the video card, drivers installed, and file being decoded hardware decoding may perform differently or may not even be used at all leading to inconsistent behavior that can be difficult to debug.
== APIs and Hardware / Software Support ==
Line 10: Line 13:
On the other hand, in at least some relatively typical scenarios, the performance gains of using hardware decoding can be huge, [[https://lists.debian.org/debian-user/2018/06/msg00306.html|with reductions in CPU usage of around 90%]]. The three main APIs that are in use are VA-API, VDPAU, and NVENC/NVDEC.
Line 12: Line 15:
=== Library and API Support ===  * VA-API - Supported on Intel, AMD, and NVIDIA (only via the open-source Nouveau drivers). Widely supported by software, including Kodi, VLC, MPV, Chromium, and Firefox. Main limitation is lacking any support in the [[NvidiaGraphicsDrivers|proprietary NVIDIA drivers.]]
Line 14: Line 17:
There are two main software frameworks for hardware acceleration currently in use on Linux: WikiPedia:Video_Acceleration_API [VA-API] and WikiPedia:VDPAU. The former should generally be used with Intel Graphics hardware, and the latter with Nvidia hardware. For more information, see [[https://wiki.archlinux.org/index.php/Hardware_video_acceleration|the ArchWiki page on Hardware video acceleration]].  * VDPAU - Supported fully on AMD and NVIDIA (both proprietary and Nouveau). Supported by most desktop applications like Kodi, VLC, and MPV, but has no support at all in Chromium or Firefox. Main limitations are poor and incomplete Intel support and not working with browsers for web video acceleration.
Line 16: Line 19:
==== VA-API ====  * NVENC/NVDEC - A proprietary API supported exclusively by NVIDIA. Only supported in a few major applications (FFmpeg and OBS Studio for encoding, FFmpeg and MPV for decoding). Main limitation is limited software and hardware support across the board because of its proprietary nature.
----
== Installation ==
=== VA-API ===
Line 18: Line 24:
VA-API software support is provided by various Debian packages, including the various driver packages (e.g., `va-driver-all`, `i965-va-driver`, etc.) and the underlying library package (`libva1`). A useful utility is `vainfo` (provided by the `vainfo` package), which provides information about available VA-API capabilities, including the list of codecs for which hardware acceleration is available. VA-API sees broad software support and is even used by default in applications like MPV when it's available.

For Nouveau and the various AMD drivers, support can be added simply by installing the DebianPkg:mesa-va-drivers package.

For Intel, it's split generationally and into free and non-free drivers. The non-free drivers '''are necessary to encode media''' while the free drivers can only decode.

For Gen 8+ Intel hardware, the free driver can be installed with the DebianPkg:intel-media-va-driver package. The non-free driver can be installed with the DebianPkg:intel-media-va-driver-non-free after adding a non-free component to your [[SourcesList|apt sources]].

For older Intel hardware, the free driver can be installed with the DebianPkg:i965-va-driver package. This driver supports of to Gen 9 GPUs. If both drivers are installed, the driver from DebianPkg:intel-media-driver is preferred over DebianPkg:i965-va-driver (since bullseye). The non-free driver can be installed with the DebianPkg:i965-va-driver-shaders package after adding a non-free component to your [[SourcesList|apt sources]].

Driver selection can be overridden by setting the environment variable `LIBVA_DRIVER_NAME` to a specific driver, e.g., `LIBVA_DRIVER_NAME=i965` (to use the driver from DebianPkg:i965-va-driver on bullseye) or `LIBVA_DRIVER_NAME=iHD` (to use the driver from DebianPkg:intel-media-va-driver on buster). See EnvironmentVariables for more details on how to set this environment variable system-wide or per user.

=== VDPAU ===

VDPAU faces considerably more limitations compared to VA-API, but nonetheless, it's the only option for some users. Particularly, anyone using the NVIDIA proprietary drivers. It's not supported in any major browser except for GNOME Web but is useful for local playback. MPV is recommended for this.

To enable VDPAU support for the AMD drivers ({{{radeon}}} and {{{amdgpu}}}), along with the open-source Nouveau driver for NVIDIA cards, install the DebianPkg:vdpau-driver-all package.

This will also enable VDPAU support over the OpenGL/VA-API backend for Intel GPUs. However, this has severe stability issues and may not work at all on some Intel devices. If possible, you are heavily recommended to use VA-API instead with Intel.

To enable VDPAU support for the proprietary NVIDIA drivers, you must choose the relevant package for your driver version. If you installed the latest drivers via the DebianPkg:nvidia-driver package, then you can simply install the DebianPkg:nvidia-vdpau-driver package.

Otherwise, if you're using a legacy driver, choose the appropriate package from the following: DebianPkg:nvidia-legacy-304xx-vdpau-driver, DebianPkg:nvidia-legacy-340xx-vdpau-driver, DebianPkg:nvidia-legacy-390xx-vdpau-driver

=== NVENC/NVDEC ===
NVDEC is supported by the DebianPkg:libnvcuvid1 package.

For the legacy drivers, instead choose the DebianPkg:libnvidia-legacy-304xx-nvcuvid1, DebianPkg:libnvidia-legacy-340xx-nvcuvid1, or DebianPkg:libnvidia-legacy-390xx-nvcuvid1 package.

NVENC is supported by the DebianPkg:libnvidia-encode1 package.

For the legacy drivers, instead choose either the DebianPkg:libnvidia-legacy-340xx-encode1 or DebianPkg:libnvidia-legacy-390xx-encode1 package.

These are only non-free runtime libraries however. Applications in Debian's main archive, including FFmpeg and OBS Studio, are not compiled with support for NVENC or NVDEC and so the usefulness is limited to third-party binaries not supplied by Debian.
----
== Checking support ==
You can find a full report of whether or not VA-API or VDPAU are functional, and what codecs they support, by installing the DebianPkg:vainfo and DebianPkg:vdpauinfo packages and running their commands. {{{vainfo}}} and {{{vdpauinfo}}}
Line 24: Line 67:
==== MPV ==== ==== mpv ====
Line 26: Line 69:
Mpv has good hardware acceleration support, although it is not enabled by default (see above). To enable it, use the `–hwdec` command line switch. It can also be made the default by adding a line like “`hwdec`” to the mpv configuration file (e.g., `$HOME/.config/mpv.conf`). [`hwdec` can also take various values, although ideally supplying the switch with no value should be sufficient. See the mpv manpage for more details.] mpv has good hardware acceleration support, although it is not enabled by default. To enable it, use the `–hwdec` command line switch. It can also be made the default by adding a line like “`hwdec`” to the mpv configuration file (e.g., `$HOME/.config/mpv.conf`). [`hwdec` can also take various values, although ideally supplying the switch with no value should be sufficient. See the mpv manpage for more details.]
Line 56: Line 99:
===== Multithreading Conflict ===== ==== Browser support ====
Web video is one of the most important use cases for hardware video acceleration, as without it sites like YouTube will cause heavy CPU usage (and therefore power consumption), a particular concern on mobile devices such as laptops and tablets.
Line 58: Line 102:
An unfortunate limitation of VLC (actually of FFmpeg, on which VLC is based) is that [[https://forum.videolan.org/viewtopic.php?t=134770|hardware acceleration cannot be used if multithreading is enabled]] (Threading is controlled via [All preferences] “Input / Codecs → FFmpeg → Threads”: a value of 0 enables multithreading, and a value of 1 disables it). If multithreading is enabled, the request for hardware acceleration will be silently ignored, although the output of VLC invoked with the `-v` flag will include a line like: As of [[DebianBullseye|Debian 11/Bullseye]], VA-API support is enabled by default in Chromium and Firefox. However, in the case of Firefox, it currently only works on [[Wayland]] desktops. [[WikiPedia:GNOME Web]] has VA-API support through the [[DebianPkg:gstreamer1.0-vaapi]] package if installed.
Line 60: Line 104:
{{{
[00007fdf8ce97280] avcodec decoder warning: thread type 1: disabling hardware acceleration
}}}
VDPAU isn't supported at all in Chromium or Firefox. The only browser that supports it is [[WikiPedia:GNOME Web]], available in the DebianPkg:epiphany-browser package. GNOME Web leverages GStreamer for this support, also requiring the [[DebianPkg:gstreamer1.0-plugins-bad]] package for VDPAU support to work properly.

Translation(s): none


With modern graphics cards, it's often possible to offload the jobs of video encoding and decoding to them from the CPU in order to reduce power usage and make more resources available to the rest of the system. Compared to CPUs, GPUs are much more efficient at the job. However, both hardware as well as software support are required for this offloading, and the latter in particular (at least in the Linux world) has undergone much evolution in recent years. Online documentation is therefore sparse and incomplete, inconsistent, and often outdated. Furthermore, support for hardware video acceleration under Linux world is unfortunately split across different APIs with different levels of support.

Benefits

Historically, the benefits of hardware acceleration under Linux have been uncertain, but it seems likely that support today has improved drastically. In at least some relatively typical scenarios, the performance gains of using hardware decoding can be huge, with reductions in CPU usage of around 90%.

APIs and Hardware / Software Support

The three main APIs that are in use are VA-API, VDPAU, and NVENC/NVDEC.

  • VA-API - Supported on Intel, AMD, and NVIDIA (only via the open-source Nouveau drivers). Widely supported by software, including Kodi, VLC, MPV, Chromium, and Firefox. Main limitation is lacking any support in the proprietary NVIDIA drivers.

  • VDPAU - Supported fully on AMD and NVIDIA (both proprietary and Nouveau). Supported by most desktop applications like Kodi, VLC, and MPV, but has no support at all in Chromium or Firefox. Main limitations are poor and incomplete Intel support and not working with browsers for web video acceleration.
  • NVENC/NVDEC - A proprietary API supported exclusively by NVIDIA. Only supported in a few major applications (FFmpeg and OBS Studio for encoding, FFmpeg and MPV for decoding). Main limitation is limited software and hardware support across the board because of its proprietary nature.


Installation

VA-API

VA-API sees broad software support and is even used by default in applications like MPV when it's available.

For Nouveau and the various AMD drivers, support can be added simply by installing the mesa-va-drivers package.

For Intel, it's split generationally and into free and non-free drivers. The non-free drivers are necessary to encode media while the free drivers can only decode.

For Gen 8+ Intel hardware, the free driver can be installed with the intel-media-va-driver package. The non-free driver can be installed with the intel-media-va-driver-non-free after adding a non-free component to your apt sources.

For older Intel hardware, the free driver can be installed with the i965-va-driver package. This driver supports of to Gen 9 GPUs. If both drivers are installed, the driver from intel-media-driver is preferred over i965-va-driver (since bullseye). The non-free driver can be installed with the i965-va-driver-shaders package after adding a non-free component to your apt sources.

Driver selection can be overridden by setting the environment variable LIBVA_DRIVER_NAME to a specific driver, e.g., LIBVA_DRIVER_NAME=i965 (to use the driver from i965-va-driver on bullseye) or LIBVA_DRIVER_NAME=iHD (to use the driver from intel-media-va-driver on buster). See EnvironmentVariables for more details on how to set this environment variable system-wide or per user.

VDPAU

VDPAU faces considerably more limitations compared to VA-API, but nonetheless, it's the only option for some users. Particularly, anyone using the NVIDIA proprietary drivers. It's not supported in any major browser except for GNOME Web but is useful for local playback. MPV is recommended for this.

To enable VDPAU support for the AMD drivers (radeon and amdgpu), along with the open-source Nouveau driver for NVIDIA cards, install the vdpau-driver-all package.

This will also enable VDPAU support over the OpenGL/VA-API backend for Intel GPUs. However, this has severe stability issues and may not work at all on some Intel devices. If possible, you are heavily recommended to use VA-API instead with Intel.

To enable VDPAU support for the proprietary NVIDIA drivers, you must choose the relevant package for your driver version. If you installed the latest drivers via the nvidia-driver package, then you can simply install the nvidia-vdpau-driver package.

Otherwise, if you're using a legacy driver, choose the appropriate package from the following: nvidia-legacy-304xx-vdpau-driver, nvidia-legacy-340xx-vdpau-driver, nvidia-legacy-390xx-vdpau-driver

NVENC/NVDEC

NVDEC is supported by the libnvcuvid1 package.

For the legacy drivers, instead choose the libnvidia-legacy-304xx-nvcuvid1, libnvidia-legacy-340xx-nvcuvid1, or libnvidia-legacy-390xx-nvcuvid1 package.

NVENC is supported by the libnvidia-encode1 package.

For the legacy drivers, instead choose either the libnvidia-legacy-340xx-encode1 or libnvidia-legacy-390xx-encode1 package.

These are only non-free runtime libraries however. Applications in Debian's main archive, including FFmpeg and OBS Studio, are not compiled with support for NVENC or NVDEC and so the usefulness is limited to third-party binaries not supplied by Debian.


Checking support

You can find a full report of whether or not VA-API or VDPAU are functional, and what codecs they support, by installing the vainfo and vdpauinfo packages and running their commands. vainfo and vdpauinfo

Application Support

Application support for hardware acceleration varies, and each application requires individual configuration. Following are details for various applications.

mpv

mpv has good hardware acceleration support, although it is not enabled by default. To enable it, use the –hwdec command line switch. It can also be made the default by adding a line like “hwdec” to the mpv configuration file (e.g., $HOME/.config/mpv.conf). [hwdec can also take various values, although ideally supplying the switch with no value should be sufficient. See the mpv manpage for more details.]

If hardware acceleration is being used, mpv’s output will contain lines like the following:

libva info: VA-API version 0.39.4
libva info: va_getDriverName() returns 0
libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/i965_drv_video.so
libva info: Found init function __vaDriverInit_0_39
libva info: va_openDriver() returns 0
AO: [alsa] 48000Hz stereo 2ch float
Using hardware decoding (vaapi).
VO: [opengl] 1920x816 vaapi

VLC

Hardware acceleration in VLC is controlled in the GUI via “Tools → Preferences → Input / Codecs → Hardware-accelerated decoding”, or via the CLI option –avcodec-hw value [‘value’ is mandatory].

If hardware acceleration is being used, VLC’s output will contain lines like the following:

libva info: VA-API version 0.39.4
libva info: va_getDriverName() returns 0
libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/i965_drv_video.so
libva info: Found init function __vaDriverInit_0_39
libva info: va_openDriver() returns 0
[00007f082ce97280] avcodec decoder: Using Intel i965 driver for Intel(R) Broadwell - 1.7.3 for hardware decoding

Browser support

Web video is one of the most important use cases for hardware video acceleration, as without it sites like ?YouTube will cause heavy CPU usage (and therefore power consumption), a particular concern on mobile devices such as laptops and tablets.

As of Debian 11/Bullseye, VA-API support is enabled by default in Chromium and Firefox. However, in the case of Firefox, it currently only works on Wayland desktops. GNOME Web has VA-API support through the gstreamer1.0-vaapi package if installed.

VDPAU isn't supported at all in Chromium or Firefox. The only browser that supports it is GNOME Web, available in the epiphany-browser package. GNOME Web leverages GStreamer for this support, also requiring the gstreamer1.0-plugins-bad package for VDPAU support to work properly.