Transcoder failure docker plex folder full

server-docker

#1

Looking for some help. I am using plex via docker (ubuntu server host via VM vSphere). My movies are all stored on a NAS drive via NFS. The docker image is only maintaining its own DB on a HD local to the host machine.

Here is how i have it setup
Esxi Host
--- Ubuntu headless server install VM
--- VM image stored on esxi HD
--- Movie / TV Show directories stored on NFS (v3) NAS

The Host is solely setup for plex and is providing 12 cores and 26gb of ram.

What is happening is when watching movies periodically they will fail with a message saying that transcoding failed. Everything on the host itself isn't taxed from a resource perspective at all.

Looking in the logs i see messages such as this

Aug 21, 2017 23:03:56.395 [0x7f9779fff700] ERROR - [Transcoder] av_interleaved_write_frame(): No space left on device
Aug 21, 2017 23:03:56.520 [0x7f97aebfd700] ERROR - [Transcoder] [segment @ 0x1e7c380] Failure occurred when ending segment 'chunk-00480'

This lead me down the road of checking my VM host to see if there were space issues which there shouldn't be considering it is allocated for 150GB.

When I inspect the space on the host here is what i notice. If i look at the docker aufs readout below it sits at 76% with nothing running. As movies are running and transcoding that number will increase until it finally hits 100% and when that happens the transcoder crashes.

root@DockerHost:/var/lib/docker# df -h
Filesystem Size Used Avail Use% Mounted on
udev 12G 0 12G 0% /dev
tmpfs 2.4G 18M 2.4G 1% /run
/dev/mapper/DockerHost--vg-root 14G 9.8G 3.3G 76% /
tmpfs 12G 244K 12G 1% /dev/shm
tmpfs 5.0M 0 5.0M 0% /run/lock
tmpfs 12G 0 12G 0% /sys/fs/cgroup
tmpfs 2.4G 0 2.4G 0% /run/user/1000
none 14G 9.8G 3.3G 76% /var/lib/docker/aufs/mnt/722af28d4904149ec4228a79068dfab6268c7247dae076478c36002fab86db4a
shm 64M 8.0K 64M 1% /var/lib/docker/containers/38e4b17d4f9f0f33c45f11640bba12626192ce4f7645306e7fd82481fd90ad77/shm

If i use other file services it doesn't show that folder being full. It appears as if the docker folder has a max size of 14GB and many of my movies are larger than that. Is there any way to increase that size from the 14GB displayed to something larger such as 40GB?

Any suggestions on what to do to resolve this issue?


#2

To add on this. All of the plex cache and media files are all stored back out on the NAS. Transcoding is set to a temp directory on the docker host and is confirmed to be increasing space on the host

“Mounts”: [
{
“Source”: “/nfs/nas/TVShows”,
“Destination”: “/TVShows”,
“Mode”: “”,
“RW”: true,
“Propagation”: “rprivate”
},
{
“Source”: “/nfs/nas/Videos”,
“Destination”: “/Videos”,
“Mode”: “”,
“RW”: true,
“Propagation”: “rprivate”
},
{
“Source”: “/nfs/nas/Plex/Config”,
“Destination”: “/config”,
“Mode”: “”,
“RW”: true,
“Propagation”: “rprivate”
},
{
“Source”: “/tmp”,
“Destination”: “/transcode”,
“Mode”: “”,
“RW”: true,
“Propagation”: “rprivate”
}
],


#3

Your /transcode is a volume mount on /tmp which appears to be on the / filesystem. That has 3.3G free (which is the same for the aufs mounts because it is ultimately stored on /). This is rather small for a transcode temp. Given what I can glean from your intended setup, you likely need to add an NFS mount for the transcode temp and put the /transcode docker mount somewhere on that NFS mount.


#4

Thank you gbooker02. My goal was to get the transcoding to process in ram. Based on one of the guides i viewed it says to do it that way, and it doesn’t seem to be working. I’m unsure why the tmp volume is 14 gb as i didn’t set it. And honestly unsure why it is 76% full as it isn’t storing anything that i can tell since all of the videos and config files are all on the NFS. What would be ideal would be to get that partition to ~ 40gb or so if possible, and figure out why it says that 9.8gb of the 14 is full.

when i run ncdu on the /var/lib/docker/aufs/mnt/ directory the max size returned is only about 500 meg


#5

I think i may have resolved this issue by extending my base partition (DockerHost–vg-root). I provisioned the VM to have more space but needed to configure the file system itself to use the additional size. I followed the guide at the following page

http://www.geoffstratton.com/expand-hard-disk-ubuntu-lvm

Now my free space is as follows

root@DockerHost:~# df -h
Filesystem Size Used Avail Use% Mounted on
udev 12G 0 12G 0% /dev
tmpfs 2.4G 18M 2.4G 1% /run
/dev/mapper/DockerHost–vg-root 146G 7.8G 132G 6% /
tmpfs 12G 0 12G 0% /dev/shm
tmpfs 5.0M 0 5.0M 0% /run/lock
tmpfs 12G 0 12G 0% /sys/fs/cgroup
tmpfs 2.4G 0 2.4G 0% /run/user/1000


#6

The key is the location of /transcode in the container which is a mounted volume of /tmp. The instructions you were following were likely assuming that /tmp was tmpfs which will be in RAM, but in your case it was not.

Also, there’s no real significant advantage to putting the transcode temp in RAM. With modern OSs, the disk cache gives all the advantage you’d get out of a ram disk for this content and is capable of purging the cache on content that’s not been used recently. I don’t understand why this recommendation keeps showing up because it seems to stem from a fundamental misunderstanding on the kind of workload the transcode temp sees and how the OS works.

Expanding / is one fix as that expands the size of /tmp. You could have also put the transcode temp on an NFS export. It’s up to you which best fits your use case.


#7

Hi. I seem to have the same issue. I’ve already tried to move the transcode volume to NFS, and a local system to no avail.

Should I start another thread?