Server Version#: 1.43.2.10687
Player Version#: Various (tested Web (chrome), Android TV, iOS)
I found a PMS hardware transcoding bug affecting valid video files that contain sections encoded at different frame rates, and the fault is in the FFmpeg 6.1 code Plex transcoder is currently based on. Weird coincidence, it’s just been fixed upstream this April, but the fix has not even made it into the latest stable FFmpeg release yet… But, good news, the Plex fix would be either a really small four-line backport matching the FFmpeg commit, or just adding -noautoscale when PMS is already outputting hardware frames through an explicit scaling/filter chain.
I originally noticed this the past week because an entire show of episodes would play normally for about halfway through and then suddenly hitch or stop.
What was actually happening was that PMS had already transcoded ahead into the final minute of the episode, and when it reached roughly same area each time when the credits section started, the transcoder crashed immediately, even though playback itself was still much earlier in the episode.
Depending on the client, that would either cause a long black screen hitch while Plex restarted the transcode and fell back to software (with a misleading warning that the connection was not fast enough) or exit playback completely with an error popup.
Direct Play was fine. Hardware decoding with software encoding was also fine. It only broke when PMS was doing both hardware decoding and hardware encoding - in this case, burning subs with a hevc-hevc hardware transcode.
After digging into the file I found that the episodes switch between 23.976 and 29.97 fps because different parts of the programme were encoded separately and joined together (turns out this is fairly common with some TV and a lot of anime encodes, particularly around credits, previews and other separately mastered sections - apparently that doesn’t break the codec spec, they’re valid hevc encodes and all the timestamps are correctly calculated).
When it hits the FPS change, Plex transcoder triggers a rebuild of the encode chain, but accidentally mixes a software only scaling step into the hardware pipeline when it does this. The two do not work together , so the transcode fails with
Impossible to convert between the formats supported by 'Parsed_hwupload_5' and 'auto_scale_0'
Error reinitializing filters!
Failed to inject frame into filter network: Function not implemented
I reproduced the same failure outside Plex using stock FFmpeg 6.1.1. Identical issue, when the hardware decoder reconfigures at the frame-rate change, FFmpeg incorrectly inserts a software auto_scale filter after a hw filter, vomits, exits the transcode with an error. The same test completes with no issue if you add -noautoscale, which is what should be done when outputting hardware frames through an explicit scaling/filter chain.
I wanted to check if it had been fixed already in newer version of FFmpeg, so I built current FFmpeg master and reran the test and it played just fine without the -noautoscale… by chance, a fix was made literally just this April after being present for like a decade, the commit is b796d72 (fftools/ffmpeg_filter: skip autoscale for hardware format).
The patch is only a few lines - it just stops FFmpeg adding the normal software scaler when the output format is hardware.
I tested the workaround inside PMS directly by injecting -noautoscale to Plex’s normal transcoder command without changing anything else, episode plays fine with hardware encoding and decoding all the way through the frame-rate change and to the end of the file.
So! Should be a real easy fix for PMS. Can either backport the upstream FFmpeg commit, or if you don’t want to patch your custom ffmpeg too much, could add your own basic check to add -noautoscale to the flags when Plex is using an explicit hardware output chain that sets the final size and pixel format, something like this should probably work
const bool explicitHardwareOutput =
videoFilterGraphSetsOutputSize &&
videoFilterGraphSetsHardwarePixelFormat &&
videoEncoderUsesHardwareFrames;
if (explicitHardwareOutput)
transcoderArgs.emplace_back("-noautoscale");
transcoderArgs.emplace_back(outputTarget);
The bug does also highlight that there’s some room to make the plex playback a bit more robust when hitting this sort of server side transcoding issue… AFAICT, DASH is built around separately generated media segments referenced by a manifest, and FFmpeg’s DASH muxer can retain a moving window of completed segment.
So PMS successfully transcoding almost the entire episode, hitting an error right at the end, then successfully falling back to software transcode, doesn’t need to kill the client playback while there’s still loads of good segments left to keep client playback going while the transcoder restarts.
Obviously I assume this is easy for me to say and a lot harder to get working right, but ideally PMS could keep serving the completed segments → restart a fallback software transcode from the last known current playback timecode → wait until the replacement transcode has caught up and hit a valid aligned segment → then update the existing manifest to continue from that segment - and only interrupt playback if it cannot catch up or pull off a smooth swap, where you would then just see it as a regular old buffering pause for a moment… or, after exhausting fallback options with transcoder erroring out, only then give an error about the media file.
I don’t know if it’s better UX or not, but in that completely failed scenario, could also just let the available segments play out before hitting the client with a playback stopping error, I suppose in some cases that would let them possibly finish the main content and just miss credits or end card etc.
