It’s disappointing that v1.43.0 got released despite the various regression I found. It’s also disappointing that the original thread got closed, which would’ve been a great way to collect those issues. But I guess most Plex users don’t make heavy use of the Transcoder like I do? Many probably just download rip to MP4.
Anyway, to sum up:
Window Size
Not fixed (but can be fixed locally so don’t care)
Green burnt-in VOBSUB/PGS
Run test.mkv through this: LIBVA_DRIVERS_PATH="/var/lib/plexmediaserver/Library/Application Support/Plex Media Server/Cache/va-dri-linux-x86_64" FFMPEG_EXTERNAL_LIBS="/var/lib/plexmediaserver/Library/Application Support/Plex Media Server/Codecs/74455c4-a119aa85fdef32b7dbfbf8a2-linux-x86_64/" /usr/lib/plexmediaserver/Plex\ Transcoder -hwaccel vaapi -hwaccel_output_format vaapi -analyzeduration 20M -probesize 20M -i test.mkv -init_hw_device vaapi=vaapi:/dev/dri/renderD128,driver=iHD -filter_complex "[0:s]scale=1920:1080,hwupload[s];[0:v]hwupload,scale_vaapi=w=1920:h=-1:format=p010[v];[v][s]overlay_vaapi[o]" -map "[o]" -c:v hevc_vaapi -b:v 2000K -f dash dash
External PGS wrong timestamp
External PGS (.sup) always start at 00:00:00.000 instead of the using the actual start time of the subtitle. Although this is also a bug in current ffmpeg (v8.0.1). I don’t use external PGS and this can be fixed locally by adding a setpts filter (e.g. python -c 'print(f"PTS+{int.from_bytes(open(pgs_file,"rb").read(6)[2:])/90000}/TB")').
Regarding #2:
This seems to be hardware bug indeed. Running the command through linuxserver/ffmpeg:amd64-latest results in the same green subtitles, but on my desktop results in white subtitles.
I don’t know if it’s useful but comparing both vainfos shows a few missing formats:
I tried playing around with alpha, converting to YUV, shuffling channels, shifting colors, etc. but it’s always green. The only thing that works is doing the overlay in SW, which decreases performance from 1.61x to 0.74x, thus making transcoding unusable.
Update: Through testing I found out that it’s just the BT.2020 primaries that are bugged. If I instead use another wide-color gamut like DCI-P3, I get white subtitles, e.g. tonemap_vaapi=format=p010:p=smpte431:t=smpte2084:m=bt2020nc. The subtitle and the overlay chain can be left untouched.
Unfortunately this is only HDR-ish, because you do lose the Mastering display information (and since this is technically tonemapping, you’re at the mercy of Intel doing it correctly) but it does look like HDR. If this bug can’t be fixed through other means, then this would be a decent workaround. And if Plex really wanted to they could merge some hevc_metadata patches early, which would allow them to reinsert the metadata after VAAPI is done, e.g. -bsf:v "hevc_metadata=max_cll='1000,400':master_display='G(13250,34500)B(7500,3000)R(34000,16000)WP(15635,16450)L(10000000,1)'".
Edit: Like I predicted, the VAAPI tonemapping is not great; it makes the image too contrasty. But luckily enough, I can just lie to VAAPI. So what I did is simply tag the video as Display P3 (setparams=color_primaries=smpte432), which, let’s be real here, is probably the truth (even if it says Mastering display color primaries: BT.2020) and then retag it BT.2020 after the overlay. The image looks the same, the HDR metadata is there and the subtitles are white. Might even make sense to always do that (unless you wanna maintain a database of broken GPUs) even if you’re potentially losing color information because P3 is smaller.
TLDR: All three issues can be fixed locally so consider this whole thread low priority.
issue #1 is a known issue that has been marked low priority due to the effort it would take to fix compared to how incredibly rare the issue is. For issue #2 i’ve only heard of it occuring for 1 other user and it went away when he upgraded his system. this leads me to beleive it is a device/driver issue. have you tried this build with updated drivers? This is the first I am hearing about issue #3 can you provide a sample file?