Intro detection crashing when scanning truehd_eae media

Server Version#: 1.29.2.6334
Player Version#: 4.92.0

Hi, it has been few months now my intro detection keep crashing on media with TrueHD eae, example sample of the logs, keep in mind this is just sample of few thousands possibly more of the logs.

Oct 28, 2022 16:59:03.610 [0x7f7b0ccc2b38] ERROR - [Req#2ea2a1/Transcode/6b16fdd6-74a5-4000-aa2c-bce32eee7e14/fbacc32a-068f-4f66-8916-9b19e40dc405] [truehd_eae @ 0x7f09f715c040] error reading output
Oct 28, 2022 16:59:03.610 [0x7f7b155ecb38] ERROR - [Req#2ea2a2/Transcode/6b16fdd6-74a5-4000-aa2c-bce32eee7e14/fbacc32a-068f-4f66-8916-9b19e40dc405] Error while decoding stream #0:1: Unknown error occurred
Oct 28, 2022 16:59:03.611 [0x7f7b0c6b9b38] ERROR - [Req#2ea2a3/Transcode/6b16fdd6-74a5-4000-aa2c-bce32eee7e14/fbacc32a-068f-4f66-8916-9b19e40dc405] [truehd_eae @ 0x7f09f715c040] error reading output
Oct 28, 2022 16:59:03.611 [0x7f7b124e2b38] ERROR - [Req#2ea2a4/Transcode/6b16fdd6-74a5-4000-aa2c-bce32eee7e14/fbacc32a-068f-4f66-8916-9b19e40dc405] Error while decoding stream #0:1: Unknown error occurred
Oct 28, 2022 16:59:03.611 [0x7f7b093fcb38] ERROR - [Req#2ea2a5/Transcode/6b16fdd6-74a5-4000-aa2c-bce32eee7e14/fbacc32a-068f-4f66-8916-9b19e40dc405] [truehd_eae @ 0x7f09f715c040] error reading output
Oct 28, 2022 16:59:03.611 [0x7f7b15df8b38] ERROR - [Req#2ea2a6/Transcode/6b16fdd6-74a5-4000-aa2c-bce32eee7e14/fbacc32a-068f-4f66-8916-9b19e40dc405] Error while decoding stream #0:1: Unknown error occurred
Oct 28, 2022 16:59:03.612 [0x7f7b0cabfb38] ERROR - [Req#2ea2a7/Transcode/6b16fdd6-74a5-4000-aa2c-bce32eee7e14/fbacc32a-068f-4f66-8916-9b19e40dc405] [truehd_eae @ 0x7f09f715c040] error reading output
Oct 28, 2022 16:59:03.612 [0x7f7b153d9b38] ERROR - [Req#2ea2a8/Transcode/6b16fdd6-74a5-4000-aa2c-bce32eee7e14/fbacc32a-068f-4f66-8916-9b19e40dc405] Error while decoding stream #0:1: Unknown error occurred
Oct 28, 2022 16:59:03.612 [0x7f7b0c8bcb38] ERROR - [Req#2ea2a9/Transcode/6b16fdd6-74a5-4000-aa2c-bce32eee7e14/fbacc32a-068f-4f66-8916-9b19e40dc405] [truehd_eae @ 0x7f09f715c040] error reading output
Oct 28, 2022 16:59:03.613 [0x7f7b15ffbb38] ERROR - [Req#2ea2aa/Transcode/6b16fdd6-74a5-4000-aa2c-bce32eee7e14/fbacc32a-068f-4f66-8916-9b19e40dc405] Error while decoding stream #0:1: Unknown error occurred
Oct 28, 2022 16:59:03.613 [0x7f7b126e5b38] ERROR - [Req#2ea2ab/Transcode/6b16fdd6-74a5-4000-aa2c-bce32eee7e14/fbacc32a-068f-4f66-8916-9b19e40dc405] [truehd_eae @ 0x7f09f715c040] error reading output
Oct 28, 2022 16:59:03.613 [0x7f7b159f2b38] ERROR - [Req#2ea2ac/Transcode/6b16fdd6-74a5-4000-aa2c-bce32eee7e14/fbacc32a-068f-4f66-8916-9b19e40dc405] Error while decoding stream #0:1: Unknown error occurred
Oct 28, 2022 16:59:03.613 [0x7f7b0f310b38] ERROR - [Req#2ea2ad/Transcode/6b16fdd6-74a5-4000-aa2c-bce32eee7e14/fbacc32a-068f-4f66-8916-9b19e40dc405] [truehd_eae @ 0x7f09f715c040] error reading output
Oct 28, 2022 16:59:03.614 [0x7f7b0f513b38] ERROR - [Req#2ea2ae/Transcode/6b16fdd6-74a5-4000-aa2c-bce32eee7e14/fbacc32a-068f-4f66-8916-9b19e40dc405] Error while decoding stream #0:1: Unknown error occurred
Oct 28, 2022 16:59:03.923 [0x7f7b1b2e5b38] ERROR - IntroDetector: Failed to transcode file (69): /storage/media/tv/ShowTitle (2004)/Season 03/ShowTitle (2004) - S03E23 [Bluray-1080p] [h265 TrueHD 10bit].mkv

media info for the related file

Video
ID                                       : 1
Format                                   : HEVC
Format/Info                              : High Efficiency Video Coding
Format profile                           : Main 10@L4@Main
Codec ID                                 : V_MPEGH/ISO/HEVC
Duration                                 : 43 min 13 s
Bit rate                                 : 5 079 kb/s
Width                                    : 1 920 pixels
Height                                   : 1 080 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Constant
Frame rate                               : 23.976 (24000/1001) FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 10 bits
Bits/(Pixel*Frame)                       : 0.102
Stream size                              : 1.53 GiB (75%)
Default                                  : Yes
Forced                                   : No
Color range                              : Limited
Color primaries                          : BT.709
Transfer characteristics                 : BT.709
Matrix coefficients                      : BT.709

Audio
ID                                       : 3
Format                                   : MLP FBA
Format/Info                              : Meridian Lossless Packing FBA
Commercial name                          : Dolby TrueHD
Codec ID                                 : A_TRUEHD
Duration                                 : 43 min 14 s
Bit rate mode                            : Variable
Bit rate                                 : 1 570 kb/s
Maximum bit rate                         : 9 597 kb/s
Channel(s)                               : 6 channels
Channel layout                           : L R C LFE Ls Rs
Sampling rate                            : 48.0 kHz
Frame rate                               : 1 200.000 FPS (40 SPF)
Compression mode                         : Lossless
Stream size                              : 486 MiB (23%)
Language                                 : English
Default                                  : Yes
Forced                                   : No

Text
ID                                       : 2
Format                                   : PGS
Codec ID                                 : S_HDMV/PGS
Codec ID/Info                            : Picture based subtitle format used on BDs/HD-DVDs
Duration                                 : 42 min 11 s
Bit rate                                 : 21.3 kb/s
Count of elements                        : 818
Stream size                              : 6.43 MiB (0%)
Language                                 : English
Default                                  : Yes
Forced                                   : No

Menu
00:00:00.000                             : en:Chapter 1
00:04:46.495                             : en:Chapter 2
00:13:28.891                             : en:Chapter 3
00:20:36.277                             : en:Chapter 4
00:25:59.349                             : en:Chapter 5
00:31:04.196                             : en:Chapter 6
00:42:32.007                             : en:Chapter 7

i did the usual, remove the codec folder to reinstall, remove cache etc, and the problem still persist, it’s not on one server either. I have two plex servers and both of them experiencing the same problem.

I am happy to provide any needed information.

No Linux user here, but I seem to remember that you must not use the noexec mount option for the volume which holds the plex data folder.
Otherwise it would prevent the special decoder binary from running which is used for all Dolby audio in Plex.

Thanks for the tip, but sadly this is at least not the case in my setup as plex running inside of docker

doing mount -l for the config endpoint shows the following

/dev/nvme0n1p2 on /config type ext4 (rw,relatime,errors=remount-ro)

for the media files endpoint is mergerfs endpoint with the following mount options

1:8:2:3:4:5:6 /storage/media type fuse.mergerfs (ro,relatime,user_id=1000,group_id=1000,default_permissions,allow_other)

other mount points

/dev/nvme0n1p2 on /tmp type ext4 (rw,relatime,errors=remount-ro)
tmpfs on /transcode type tmpfs (rw,nosuid,nodev,inode64)

May I see the full DEBUG Plex Logs ZIP file which captures this?

Sure, give me few as i cleaned the logs turned on the logs and manually started intro detection, once it’s finishes i’ll upload it you just want Plex Media Server.log correct? and is there an email or something that i can share them with or just attach them here

Just attach your server logs here.

May I ask if the truehd track is an official encode (=found on Blu-ray) or is it an encode made by someone else? I guess it’s a user encode. (The avg/max bitrate difference seems off for an official encode) The problem with self-encoded truehd streams made by ffmpeg is that they are not 100% dolby truehd conform. This can be verified by trying to pass through the audio on an AVR setup that supports truehd. The official stream will play no problem, a self-encoded one will not play anything (or audio is distorted). Why does this matter? Plex uses official and licensed Dolby software and therefore it may not surprise if it fails to decode ffmpeg truehd streams. (guess)

Plex Media Server.zip (224.4 KB)

I didn’t attach the other plex media server.N.log files as they are filled with just the ERROR: logs i linked in the previous post

It is indeed rip from bluray, you might be right however it seems to only effect the truehd files not AC3 EAC or anything else.

Has the stream been altered after ripping it from the disc? Reencode with handbrake?

I wasn’t the person who ripped it as such i do not have this information, however looking at the metadata it was ripped using Writing application : DVDFab 12.0.4.2 as such i don’t think it was altered.

Ah okay, so you don’t have access to the disc for a rerip and retry?

That’s a great hint. DVDFab has the capability to recode a rip to a format wanted by the user… That would also explain the low bitrate 10bit HEVC 1080p video. A bluray stream would be high bitrate (20mbit+) 8bit H264 1080p. So I guess the audio stream was changed alongside that rip. What you can try is recode the audio to AAC or AC3 (with ffmpeg - those encodes are Dolby compliant and should work with the Plex Dolby decoder). The tool xmedia recode can help with that if you are not familiar with FFmpeg cli commands. You also don’t need to reencode the video… When you have done this Intro detection should work.

Yeah, if turns out the media is the problem I’ll go ahead and reencode the audio using ffmpeg, however i want to know what’s the error, and why it’s spamming the logs for so many lines it’s thrashing my SSD, i would assume if the encoder fails it should log log one error and move on, however this one spam around few thousands error per media item :frowning:

Just FYI, the media works in VLC, mpv, and basiclly anything including other media servers.

Maybe start with one file and look at what happens. :wink:

I guess the Dolby decoder logs one line per truehd audio package/snipped it tries to decode and there can be many on one file. This is probably nothing plex can change as the decoder is closed-sourced and built by Dolby and then shipped to plex. (Again just a guess, but the file has dolby copyright)

This isn´t surprising as all the other players/systems rely on ffmpeg for all de- and encoding. FFmpeg has no problem with tracks that are not 100% dolby compliant. Just plex uses Dolby licensed software for Dolby stream decoding.

Yeah, I am holding off and waiting for @ChuckPa response if he agrees with the current assessment that the media is broken, I’ll re-rip or just re-encode. gonna take few weeks to do so hope it’s software related :frowning:

How much media is indexed by Plex?

Specifically, I’m looking for the total number of directories which are being monitored for changes (inotify service)

The container relies on the host capabilities here.

If the inotify table is full, ALL EAE services will fail. (A limitation of the Dolby EAE)

Older kernels have a default limit of 8192 maximum directory entires. Newer kernels support 65536.

Large movie/tv libraries or substantive Music or Photo libraries easily can fill (overflow) the table.

Table overflow will only be detected in the first few minutes after PMS start (while the table is loaded)

I am going to need to see the start of the failure.

I suggest

  1. Start playback
  2. Stop after 10 seconds
  3. Download Logs
  4. Attach everything

@ChuckPa i have Scan my library automatically disabled i notify plex via api when media changes i only have Run a partial scan when changes are detected enabled.

(23:21:50) user@server ~
$ inotify_list -l

Current limits
-------------
fs.inotify.max_user_instances = 400
fs.inotify.max_user_watches = 524288

(23:21:50) user@server ~
$ inotify_list

   INOTIFY   INSTANCES
   WATCHES      PER
    COUNT     PROCESS   PID USER         COMMAND
------------------------------------------------------------
      37         4       39492 user      /lib/systemd/systemd --user
       7         1       39578 user      /usr/libexec/tracker-miner-fs-3
       4         1       39525 user      /usr/bin/dbus-daemon --session --address=systemd: --nofork --nopidfile --systemd-activation --syslog-only
       2         2       39501 user      /usr/bin/pulseaudio --daemonize=no --log-target=journal
       1         1       39627 user      /usr/libexec/goa-identity-service
       1         1       39609 user      /usr/libexec/goa-daemon
       1         1       39595 user      /usr/libexec/gvfs-afc-volume-monitor
       1         1       39584 user      /usr/libexec/gvfs-udisks2-volume-monitor
       1         1       39500 user      /usr/bin/pipewire-media-session

  55  WATCHES TOTAL COUNT

INotify instances per user (e.g. limits specified by fs.inotify.max_user_instances):

INSTANCES    USER
-----------  ------------------
13           user

if automatic detection is disabled then I really do need to see the logs.

There are two possibilities:

  1. Codec issue (damaged or “operation not permitted”)
  2. Media issue

I’ll debug this further tomorrow as you suggested I’ll do

Start playback
Stop after 10 seconds
Download Logs
Attach everything

It’s really late here 23:30 right now.

I’ll ping you once i have the logs ready tomorrow, Thanks for your time.

@ChuckPa

Hi sorry took to long, i had setup new plex server just to make sure the test case is clean, what i did

  1. created new plex server
  2. added the problematic media
  3. scanned and attempted to play. video started but no sound.

see attached files

Thank you.

Thank you for the logs.

Oct 29, 2022 19:38:59.980 [0x7f420f3ecb38] ERROR - CryptoHash: Failed to open file: "/config/Library/Application Support/Plex Media Server/Codecs/EasyAudioEncoder-1785-linux-x86_64.zip.tmp-d054-31bb-045a-fbbf"
Oct 29, 2022 19:38:59.981 [0x7f420f3ecb38] ERROR - Codecs: Download failed: Failed to verify SHA1sum of codec!
Oct 29, 2022 19:38:59.981 [0x7f420e517b38] ERROR - Error configuring transcoder: Decoder install failed: truehd_eae
Oct 29, 2022 19:38:59.981 [0x7f420e517b38] DEBUG - Streaming Resource: Terminated session 0x7f4212765308:93352234-ba7d-4a84-a17f-b31c92617421 with reason Conversion failed. A required codec could not be found or failed to install.
Oct 29, 2022 19:38:59.981 [0x7f420e517b38] ERROR - IntroDetector: Failed to start a transcode session: /media/tv/Lost (2004)/Lost (2004) - S03E23 - Through The Looking Glass (2) [Bluray-1080p] [h265 TrueHD 10bit].mkv
Oct 29, 2022 19:38:59.981 [0x7f4213cedb38] DEBUG - Killing job.

Can you jump into the shell and see what’s going on in the Codecs directory?
It makes no sense for you, individually, being unable to download that single codec when other codecs download correctly.

Verification of other codecs downloading is the next step :slight_smile: