Transcode Folders Getting Deleted

Plex Media Server NAS & Devices
1

I have a strange issue that just started occurring tonight. During streaming, either locally or remotely, shows will randomly buffer about every 5-10 seconds.

Upon looking at my Plex, I noticed that, in the transcode Sessions directory, folders were being created. The folder then would be deleted, the show would buffer, and a brand new folder would be created and start to get filled with .ts files.

This certainly seems like a problem. Each time the folder gets recreated (with a different UUID), the CPU spikes to 100% and the TV show starts playing for its next 5-10 segment stint.

Can anyone point me to where I might start looking for a root cause as to why this might be?

I’m using Plex via Docker version 1.11.0.4666 on Ubuntu 17.04. I’ve attempted to watch shows locally on an AppleTV (Direct Play) and remotely on a Roku (Transcoded) and both have the same issues.

Edit: I should add - I’ve attempted to completely delete my Docker images and rebuild from the plexpass Docker tag and I am still experiencing the same issues. The SSD for transcoding that I have mapped to the Docker volume has 863GB free, so space is not an issue.

2

I have a dump from the log, if it might help:

Jan 13, 2018 01:23:22.047 [0x7f7674ff0700] WARN - Got a transcode session ping without a session GUID (or with an invalid one).
Jan 13, 2018 01:23:22.368 [0x7f7674ff0700] ERROR - [FFMPEG] - No VA display found for device: .
Jan 13, 2018 01:23:22.368 [0x7f7674ff0700] ERROR - [FFMPEG] - No VA display found for device: /dev/dri/renderD128.
Jan 13, 2018 01:23:22.369 [0x7f7674ff0700] ERROR - [FFMPEG] - Cannot load libcuda.so.1
Jan 13, 2018 01:23:22.369 [0x7f7674ff0700] WARN - avcodec_open2 returned -1313558101 for encoder 'h264_nvenc'
Jan 13, 2018 01:23:22.369 [0x7f7674ff0700] ERROR - [FFMPEG] - No VA display found for device: /dev/dri/renderD128.
Jan 13, 2018 01:23:22.369 [0x7f7674ff0700] ERROR - [FFMPEG] - No VA display found for device: .
Jan 13, 2018 01:23:22.370 [0x7f7674ff0700] ERROR - [FFMPEG] - No VA display found for device: /dev/dri/renderD128.
Jan 13, 2018 01:23:22.370 [0x7f7674ff0700] ERROR - [FFMPEG] - No VA display found for device: /dev/dri/renderD128.
Jan 13, 2018 01:23:22.370 [0x7f7674ff0700] ERROR - [FFMPEG] - Cannot load libcuda.so.1
Jan 13, 2018 01:23:22.370 [0x7f7674ff0700] WARN - avcodec_open2 returned -1313558101 for encoder 'h264_nvenc'
Jan 13, 2018 01:23:22.371 [0x7f7674ff0700] ERROR - [FFMPEG] - No VA display found for device: /dev/dri/renderD128.
Jan 13, 2018 01:23:22.591 [0x7f76833ff700] WARN - Got a request to stop a transcode session without a session GUID (or with an invalid one).
Jan 13, 2018 01:23:22.769 [0x7f76737fe700] WARN - PubSub: Received notifyConnectivity event with incorrect async identifier (a51ba616-482c-42e1-bb32-9d3ae507b159, expected a3a1f6a6-d4ec-4d71-87e1-63228091a00c)
Jan 13, 2018 01:23:23.621 [0x7f7676ff4700] ERROR - [Transcoder] [Parsed_inlineass_2 @ 0x339ab60] fontselect: Using default font family: (Arial, 400, 0) -> /usr/lib/plexmediaserver/Resources/Fonts/DejaVuSans-Regular.ttf, 0, DejaVuSans[Parsed_inlineass_2 @ 0x339ab60] 
Jan 13, 2018 01:23:25.582 [0x7f76737fe700] WARN - PubSub: Received notifyConnectivity event with incorrect async identifier (55799f2d-26b2-4cef-9560-7df1a4abf0d7, expected f3d7d905-8ac3-4990-8d18-be73cb8b53fd)
Jan 13, 2018 01:23:25.583 [0x7f76737fe700] WARN - PubSub: Received notifyConnectivity event with incorrect async identifier (e37b2134-50b3-4d9e-902f-63b0899b8b31, expected f3d7d905-8ac3-4990-8d18-be73cb8b53fd)
Jan 13, 2018 01:23:25.583 [0x7f76737fe700] WARN - PubSub: Received notifyConnectivity event with incorrect async identifier (a3a1f6a6-d4ec-4d71-87e1-63228091a00c, expected f3d7d905-8ac3-4990-8d18-be73cb8b53fd)
Jan 13, 2018 01:23:25.583 [0x7f76737fe700] WARN - PubSub: Received notifyConnectivity event with incorrect async identifier (10c088df-f56b-4770-8ad7-c687a490eef9, expected f3d7d905-8ac3-4990-8d18-be73cb8b53fd)
Jan 13, 2018 01:23:25.583 [0x7f76737fe700] WARN - PubSub: Received notifyConnectivity event with incorrect async identifier (a78e08e3-0c8f-4fcd-b43e-d6d2d07c27c0, expected f3d7d905-8ac3-4990-8d18-be73cb8b53fd)
Jan 13, 2018 01:23:25.583 [0x7f76737fe700] WARN - PubSub: Received notifyConnectivity event with incorrect async identifier (97a9c3f0-4de7-4baa-a9e3-5ff7e00aa03c, expected f3d7d905-8ac3-4990-8d18-be73cb8b53fd)
Jan 13, 2018 01:23:25.899 [0x7f76737fe700] WARN - PubSub: Received notifyConnectivity event with incorrect async identifier (0a774049-e237-4491-a04d-b3ee440d4fd2, expected f3d7d905-8ac3-4990-8d18-be73cb8b53fd)
Jan 13, 2018 01:23:30.314 [0x7f76883fd700] ERROR - [FFMPEG] - Cannot load libcuda.so.1
Jan 13, 2018 01:23:30.314 [0x7f76883fd700] WARN - avcodec_open2 returned -1313558101 for encoder 'h264_nvenc'
Jan 13, 2018 01:23:30.314 [0x7f76883fd700] ERROR - [FFMPEG] - No VA display found for device: /dev/dri/renderD128.
Jan 13, 2018 01:23:30.324 [0x7f76883fd700] WARN - Crash: Crash reporting disabled

Edit - it seems that Plex is trying to pick up on hardware that does not exit. I turned off Hardware Acceleration in the options, but the stream is still experiencing the issues in the original post. Now the logs are empty…but the buffering/file deletions are continuing.

3

Using hardware transcoding in Docker requires manual intervention. It is best to ask that question in the Docker forum as they have the procedures already defined.

Respectfully, since PMS is native on Ubuntu, and the Docker container is formatted as Ubuntu, with portability as simple as a tar ball, why not run natively on Ubuntu?

I take care of the /dev/dri entries (udev) as part of PMS installation. You never have to deal with it.

Before digging further into that, I will need to know more.

  1. Your full log files (Settings - Server - Help - Download Logs) to see all the errors presented
  2. Can you tell me what CPU you’re using?
  3. Are you trying to use an nVidia GPU for hardware transcoding?
4

Thanks, ChuckPA!

I use Docker for several reasons - single-command Plex updates, the ability to run other containers on my machine with relative ease, it’s what I used to learn the basics of Docker Compose, etc… I also prefer to use minimalist installs on my machines to prevent overhead - so I keep small images with just Core+Docker around to run projects, including Plex. Happy to answer any questions you have on that matter, if you’re interested.

Regarding the topic, these issues existed before I attempted to try Hardware Acceleration. The errors mentioned above were obviously from when I turned that on, so I immediately turned them back off. Those specific errors went away; however, the issues still persist, but only with certain media files.

I do want to add that this issue seemed to happen after I added 10-bit HEVC media to the Plex. The Transcoder seemed to fail, hard, when trying to transcode these files for AppleTV. It would show a black screen and scroll the video time (~30 minutes) in about 15 seconds before moving on to the next episode. In an attempt to get direct stream working on my AppleTV, I modified the tvOS.xml profile. When that failed, I reverted the file, and for good measure, completely rebuilt the Plex Docker image from scratch (PlexPass tag). After that, as far as I can tell, these buffering issues with h.264 files started occurring.

I’m not attempting, to my knowledge, to use nVidia Transcoding as I do not have a GPU installed. Do you have a recommendation on what I might look at to ensure I haven’t misconfigured that?

I’ve tailed all of the Plex logs with debugging (I’ve checked verbose, too) when the media pauses, buffers, or outright fails. I’m hoping you might see something in there that I’m not picking up on. To make the search easier, I ensured Debug (not Verbose) logging was on and attempted to play a couple different problem media sources (which I am certain was working a few days ago). It paused 5 seconds into playing and never recovered. I ensured that my AppleTV was logging to the Plex beforehand. You can find my logs here:

http://cloud.andrewvaughan.io/2x3D3u0U1z0I

I’m not sure if this is a signal, but even tailing the log file seems to fail when the video starts buffering. It locks up at something that looks like this:

Jan 13, 2018 12:40:11.835 [0x7f1aea3ff700] DEBUG - Streaming Resource: Last segment has been delayed for 1.04 and last request was 0.00s ago, so changing to client buffered on 6DED3FA1-C823-4B4B-921D-0A2CA742C32D with an estimate of 22839kbps
Jan 13, 2018 12:40:11.835 [0x7f1aea3ff700] DEBUG - Streaming Resource: Changing client bandwidth 6DED3FA1-C823-4B4B-921D-0A2CA742C32D to 20000kbps from 7977kbps
Jan 13, 2018 12:40:11.837 [0x7f1aea3ff700] DEBUG - TranscodeUniversalRequest: using augmented profile tvOS
Jan 13, 2018 12:40:11.837 [0x7f1aea3ff700] DEBUG - Downloading document http://127.0.0.1:32400/library/metadata/1261?includeBandwidths=1&offset=-1&X-Plex-Client-Capabilities=audioDecoders%3Dmp3%2Caac%2Cac3{bitrate%3A800000%26channels%3A8}&X-Plex-Session-Identifier=6DED3FA1-C823-4B4B-921D-0A2CA742C32D
Jan 13, 2018 12:40:11.838 [0x7f1aea3ff700] DEBUG - Auth: authenticated user 1 as wspnut
Jan 13, 2018 12:40:11.843 [0x7f1aea3ff700] DEBUG - We're going to try to auto-select an audio stream for account 1.
Jan 13, 2018 12:40:11.843 [0x7f1aea3ff700] DEBUG - Selecting best audio stream for part ID 2107 (autoselect: 1 language: en)
Jan 13, 2018 12:40:11.843 [0x7f1aea3ff700] DEBUG - We're going to try to auto-select a subtitle.
Jan 13, 2018 12:40:11.843 [0x7f1aea3ff700] DEBUG - Found a suitable native subtitle language [eng] for a foreign film
Jan 13, 2018 12:40:11.843 [0x7f1aea3ff700] DEBUG - Audio Stream: 5091, Subtitle Stream: 5094
Jan 13, 2018 12:40:11.844 [0x7f1aea3ff700] DEBUG - Streaming Resource: Applying codec restriction restrict-transcode-target-codec(audioCodec=ac3&context=streaming&protocol=hls&type=videoProfile&videoCodec=h264) on 6DED3FA1-C823-4B4B-921D-0A2CA742C32D
Jan 13, 2018 12:40:11.845 [0x7f1aea3ff700] DEBUG - Scaled up video bitrate to 6957Kbps based on 4.500000x fudge factor.

Re-tailing the log brings up the latest again. It’s very odd.

Before I start my dump of data, thanks in advance for your help with this!

Here are the specs of my CPU and RAM; please let me know if I can provide you more details. I’m aware that the i5-4590 model does not support QSV - thus the errors above:

andrew@plex:~$ cat /proc/cpuinfo 
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 60
model name	: Intel(R) Core(TM) i5-4590 CPU @ 3.30GHz
stepping	: 3
microcode	: 0x1c
cpu MHz		: 800.024
cache size	: 6144 KB
physical id	: 0
siblings	: 4
core id		: 0
cpu cores	: 4
apicid		: 0
initial apicid	: 0
fpu		: yes
fpu_exception	: yes
cpuid level	: 13
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid xsaveopt dtherm ida arat pln pts
bugs		:
bogomips	: 6599.89
clflush size	: 64
cache_alignment	: 64
address sizes	: 39 bits physical, 48 bits virtual
power management:

...etc x4 cores...

andrew@plex:~$ cat /proc/meminfo
MemTotal:        8053988 kB
MemFree:          156348 kB
MemAvailable:    7226828 kB
Buffers:          666244 kB
Cached:          6473572 kB
SwapCached:            8 kB
Active:          2221300 kB
Inactive:        5311372 kB
Active(anon):     358780 kB
Inactive(anon):    44020 kB
Active(file):    1862520 kB
Inactive(file):  5267352 kB
Unevictable:           0 kB
Mlocked:               0 kB
SwapTotal:       2097148 kB
SwapFree:        2097136 kB
Dirty:               900 kB
Writeback:             0 kB
AnonPages:        301516 kB
Mapped:           151940 kB
Shmem:              9944 kB
Slab:             302756 kB
SReclaimable:     244496 kB
SUnreclaim:        58260 kB
KernelStack:        4192 kB
PageTables:         8876 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:     6124140 kB
Committed_AS:    1497060 kB
VmallocTotal:   34359738367 kB
VmallocUsed:           0 kB
VmallocChunk:          0 kB
HardwareCorrupted:     0 kB
AnonHugePages:    116736 kB
ShmemHugePages:        0 kB
ShmemPmdMapped:        0 kB
CmaTotal:              0 kB
CmaFree:               0 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:      171544 kB
DirectMap2M:     6000640 kB
DirectMap1G:     3145728 kB

With the rebuild, the only thing consistent has been my included /config directory. If I build /config again from scratch, it seems to resolve my issue. When I reinstate my backup, the issue returns. I’d really rather not have to reconfigure my server, as I have made extensive and significant configuration changes to it over the years.

PS - there may be some verbosity in the logs from my own testing. Apologies in advance; I can likely repeat the issue with a fresh set of logs, if it would make things easier.

5

I appreciate your desire for minimal footprint but you’re adding docker on top of Linux already. Native is the minimal footprint . The Ubuntu DEB file doesn’t install anymore or less than the Docker image does . :slight_smile: Enabling the Plex repo in apt’s sources gives you that single-command update. My point here is I assert zero net gain.

Regarding your issue: The transcoder does create temporary session directories and then deletes them through normal processing.

Your use of docker here is what complicates debugging. I have no idea how file locking (which IS required) will translate through the docker layer.
Further, Is everything local to the host (with exception of the actual media of course)? I ask because the transcoder does require file locking support. Network mounts do not enable file locking by default. You must expressly activate it in the mount options.

6

I can work within the Docker container if that would simplify your help. You’re basically just dealing with a standard Ubuntu environment with that. I can also pretty easily remove Docker from this equation by installing and configuring the folders I’m currently passing into the Docker container, if that’s a concern. I anticipate that this issue will continue, but will let you know if it is otherwise.

The config, transcode, and media directories are mounted as volumes, so they are the only things that persist beyond the Plex Docker configuration. File locking should be created and controlled no different in Docker than in Ubuntu standard. The platform has had no issue locking for quite some time, including this current version. Most media still plays fine on the Plex - I would imagine if there were issues on the file system, I’d be experiencing more wide-spread issues. Please correct me if I’m wrong.

Regarding what you said:

The transcoder does create temporary session directories and then deletes them through normal processing.

Is it normal, however, for the transcoder to re-create the folder every 5-10 seconds? I noticed that most transcodes will build a folder and stay within it. However, on these problematic pieces of media, the playback pauses, the transcode folder is deleted, a new one is created (with a new UUID) and it repeats itself.

Is everything local to the host (with exception of the actual media of course)? I ask because the transcoder does require file locking support. Network mounts do not enable file locking by default. You must expressly activate it in the mount options.

All media, folders, transcode, etc are local HDDs an SDDs.

Given the fact that a clean “config” seems to repair the issue - and the fact that I’ve not had any issues with this media on this same hardware and setup for about ~5 months - makes me think the issue is focused in the Plex config directory itself.

Again, if needed I could blank the config and redo all my Plex configuration work (anxiety intensifies), but I’d really like to determine what might be the root cause.

7

Regarding any kind of mount points. I don’t know what mechanism Docker users for mounts. If it uses a network mount then it’s clear something changed in how locks are supported. Diagnosis of that is far beyond my skill with Docker.

If you are able, and willing, to pull PMS out of the Docker config (tar ball), relocate into the /var/lib/plexmediaserver/Library environment, and create the few symlinks needed to replicate your real media with how mounts appear in the container, it would be a valid test of whether a Docker / Ubuntu issue or a PMS issue.

If you would prefer, I can relocate this thread to the Docker forum and you can continue working with the staff there.

8

If you feel that would be best, that’s fine by me. Given the functionality I described above, with this issue being intermittent depending on selected media and target client, I highly doubt this is a Docker issue.

Edit: That said, I am setting up the tar and links now, so I’ll get back to you.

9

Okay, I’ve moved everything out of Docker, installed the latest PlexPass version via .deb, symlinked all of my folders, and I’m still dealing with the same problem.

10

May I see those log files? They will make a lot more sense to me.

Settings - Server - Help - Download Logs

Attach the ZIP file

11

Here’s the latest:

Let me know if you need me to go more verbose.

12

For future reference, you can attach the ZIP file here and after I open it, either you or I can delete it.

13

You forgot one link.

/config/Library/ -> /var/lib/plexmediaserver/Library

Also, while you’re there, if you don’t need IPv6, Please turn it off in Settings - Server - Network and on the host. IPv6 is known to cause issues with metadata retrieval (none of the servers use IPv6 and not all ISPs perform V6 <-> V4 translation correctly)

Then restart PMS and recreate the issue.

14

I did make that symlink. Was there something in the log that made it seem I didn’t?

andrew@plex:/var/lib/plexmediaserver$ ll
total 12
drwxr-xr-x  3 plex plex 4096 Jan 13 16:30 ./
drwxr-xr-x 40 root root 4096 Jan 13 16:29 ../
lrwxrwxrwx  1 plex plex   15 Jan 13 16:30 Library -> /config/Library/
drwxr-xr-x  3 plex plex 4096 Jan 13 16:29 Library-bk/

I’ll turn off IPv6 and repost here in a bit. If it’s not an issue, I’ll repost the zip on my cloud (just because I own the service and can ensure removal afterward). The wife is watching a movie right now - so I have to be good and wait :slight_smile:

Thanks again for all the help.

15

Your logs showed the error.

That symlink is incorrect. It should not exist in /var/lib/plexmediaserver. It should exist in root (/)

sudo ln -s /var/lib/plexmediaserver /config is the correct one.

That will provide resolution for /config/Library which I found in your logs

This is one of them.

com.plexapp.agents.opensubtitles.log.2:2018-01-11 08:27:25,742 (7fe848ff9700) :  ERROR (model:205) - Cannot read model from /config/Library/Application Support/Plex Media Server/Metadata/Movies/b/756d9e2fc0f98b4ed94075e9d37fda74d3d95ef.bundle/Contents/com.plexapp.agents.opensubtitles
16

That doesn’t make any sense to me - I’m just basically reinstalling Plex if I do that because I’ve replaced my config folder with the one recently installed by Plex.

The /config folder contains the /var/lib/plexmediaserver folder from my previous install (this is the Docker standard). By simlinking this way, I retain my previous config. So in fact, I want the opposite of what you listed:

sudo ln -s /config /var/lib/plexmediaserver

Now, in this case, I simply simlinked the Library since there was nothing else in the directory, but you tell me if that’s incorrect:

sudo ln -s /config/Library /var/lib/plexmediaserver/Library

Is it possible that the error you stated could be pointing to a source of my problems, however. Could that Cannot read model error be caused by anything else? Again, keep in mind that the rest of my Plex and it’s configuration is working very well - something that wouldn’t be the case if I had the linking issues you mentioned.

17

Which way did you do the relo?

Did you take the native desktop system and make it point to a docker tarball output or did you drop the docker tarball in /var/lib/plexmediaserver/Library and then provide the /config link ?

I do think we’re at the limit of what I can help with.

At this point, I would like to ask @gbooker02 to take a look and assist. He is far more skilled with docker issues than I and can probably address the main issue and avoid what we’re trying to do.

18

Here’s a summary of my system’s state if it can help move things in the right direction:

Docker is not being used - I’ve removed all Docker images and containers from the system.

  1. I installed plexmediaserver_1.11.0.4666-fc63598ba_amd64.deb
  2. /var/lib/plexmediaserver was backed up to /var/lib/plexmediaserver-bk
  3. I symlinked the configuration directory to the configuration directory I was using for Docker’s config volume (/media/speedy/config)
  4. I symlinked the transcode and data locations the Docker configuration expects locally to best replicate the Docker directory configuration

I cleaned up my symlinks to avoid confusion, these are the only ones that exist currently:

  • /media/speedy/config => /var/lib/plexmediaserver (this was setup by the Docker install)
  • /media/speedy/transcode => /transcode
  • /media/media01 => /data/media01
  • /media/media02 => /data/media02

Regarding the posts above, /config was symlinked to /media/speedy/config - they’re one in the same.

Just so people don’t have to read through everything, too - I’m experiencing significant delays and errors with only certain types of media. Watching the /transcode folder, I noticed that each time the media freezes, the ongoing transcode directory is deleted, created under a new name, the CPU spikes for a second; wash-rinse-repeat. Having the transcode directory be re-created every 5 seconds or so definitely seems abnormal.

The media, hardware, and Plex versions haven’t changed recently. This media has been playable for well over a year. So far, we have yet to find anything in the Movies libraries that have these issues - they only have occurred on select TV shows as of yet.

If I completely trash my config and rebuild it from scratch, the issue goes away, but I very-much want to find the root cause so I don’t have to completely reconfigure my Plex.

Please let me know what else I can provide, or if this doesn’t explain everything I can provide. If you think this would be best in Docker, I’ll definitely be obliged if you could move this to the correct place, and thank you for your help thus far!

Edit: Also adding - we’ve only been having this problem specifically with AppleTV. I haven’t been able to replicate it on any device (that said, we almost universally watch Plex via AppleTV).

19

Hey all -

Just bumping this. Any ideas?

20

Sorry for being away.

The only thing I can think of is somehow the docker container is losing connectivity (drive spinning down and not spinning up in time ?) and PMS seeing things as “Unavailable” . It shouldn’t delete them and should recover normally when they are available again.

As a test of this, turn off “Empty trash automatically after every scan”. See what impact this has.