[libav-devel] qsv: How about replace software parsers with MFXVideoDECODE_DecodeHeader

Mark Thompson sw at jkqxz.net
Mon Jan 22 14:19:22 CET 2018

On 22/01/18 11:20, Li, Zhong wrote:
>   MSDK provides an API (MFXVideoDECODE_DecodeHeader) to parse video parameters. Currently it hasn't been used.
> Instead, software parsers are used. It works well for h264 decoder, and basically works well for hevc decoder (some issues found by Mayxm due to width/height are unaligned, I also found some hevc clips without setting profile can be decoded by software but qsv failed) .
> More issues found on other decoders such as VP8. The decoding conformance pass rate is low and looks like it is due to some missing/incompatible header information is passed to qsv decoder (though Mark provides a patch 182cf17 but still something is missing).
>   Similar issues happens on MJPEG decoding which I am going to add.

Have you looked into what is missing for VP8?  There shouldn't be much magic there, so I doubt that particular case is difficult to fix.

I imagine MJPEG will be nastier, though, because there is a lot more variation in layout.

>   Maybe we can continue to work on software parsers for qsv, but I believe replace software parser with MFXVideoDECODE_DecodeHeader is a better choice:
> 1.      It can remove the dependence on various software parsers, and just need a unified interface for all codes.
> 2.      It will be very easy to add new decoder such as MJPEG decoding support without any software parser patches.
> 3.      MFXVideoDECODE_DecodeHeader is used by MSDK sample decoder (i.e: sample_decode), so it is reliable for MSDK decoder. (As my test, it can effectively improve decoding conformance pass rate, especially for vp8 decoding.)
> 4.      CUVID decoder is using CUVID parser instead of software parser, maybe qsv can align with it.

The CUVID decoder only exists in ffmpeg and is considered deprecated in favour of the hwaccel (which is the only thing in libav).

>    Negative effect:
> 1.      May cause some regression since it will take effect to all codecs.
> 2.      Others?

I believe a problem with this method is that you don't have any libmfx session at the point where you do the initial parsing (since you only get it after the get_format() callback, which needs information from that parsing).  How would you intend to get the session to use for this purpose?

- Mark

PS:  Feel free to ignore anything I say about qsvdec - I regard qsvdec as deprecated, because nowadays I find that using the platform API hwaccels (DXVA2/VAAPI) and mapping back if necessary is just more flexible and better supported.  (Though it would be nice if libmfx encode supported D3D11 array textures too...)

More information about the libav-devel mailing list