If you have not already, we suggest setting your Plex username to something else rather than email which is displayed on your posts in forum. You can change the username at https://app.plex.tv/desktop#!/account
Welcome to our forums! Please take a few moments to read through our Community Guidelines (also conveniently linked in the header at the top of each page). There, you'll find guidelines on conduct, tips on getting the help you may be searching for, and more!

Transcoder failure docker plex folder full

liquidfrolloliquidfrollo Posts: 11Members, Plex Pass Plex Pass

Looking for some help. I am using plex via docker (ubuntu server host via VM vSphere). My movies are all stored on a NAS drive via NFS. The docker image is only maintaining its own DB on a HD local to the host machine.

Here is how i have it setup
Esxi Host
--- Ubuntu headless server install VM
--- VM image stored on esxi HD
--- Movie / TV Show directories stored on NFS (v3) NAS

The Host is solely setup for plex and is providing 12 cores and 26gb of ram.

What is happening is when watching movies periodically they will fail with a message saying that transcoding failed. Everything on the host itself isn't taxed from a resource perspective at all.

Looking in the logs i see messages such as this

Aug 21, 2017 23:03:56.395 [0x7f9779fff700] ERROR - [Transcoder] av_interleaved_write_frame(): No space left on device
Aug 21, 2017 23:03:56.520 [0x7f97aebfd700] ERROR - [Transcoder] [segment @ 0x1e7c380] Failure occurred when ending segment 'chunk-00480'

This lead me down the road of checking my VM host to see if there were space issues which there shouldn't be considering it is allocated for 150GB.

When I inspect the space on the host here is what i notice. If i look at the docker aufs readout below it sits at 76% with nothing running. As movies are running and transcoding that number will increase until it finally hits 100% and when that happens the transcoder crashes.

root@DockerHost:/var/lib/docker# df -h
Filesystem Size Used Avail Use% Mounted on
udev 12G 0 12G 0% /dev
tmpfs 2.4G 18M 2.4G 1% /run
/dev/mapper/DockerHost--vg-root 14G 9.8G 3.3G 76% /
tmpfs 12G 244K 12G 1% /dev/shm
tmpfs 5.0M 0 5.0M 0% /run/lock
tmpfs 12G 0 12G 0% /sys/fs/cgroup
tmpfs 2.4G 0 2.4G 0% /run/user/1000
none 14G 9.8G 3.3G 76% /var/lib/docker/aufs/mnt/722af28d4904149ec4228a79068dfab6268c7247dae076478c36002fab86db4a
shm 64M 8.0K 64M 1% /var/lib/docker/containers/38e4b17d4f9f0f33c45f11640bba12626192ce4f7645306e7fd82481fd90ad77/shm

If i use other file services it doesn't show that folder being full. It appears as if the docker folder has a max size of 14GB and many of my movies are larger than that. Is there any way to increase that size from the 14GB displayed to something larger such as 40GB?

Any suggestions on what to do to resolve this issue?

Comments

  • liquidfrolloliquidfrollo Posts: 11Members, Plex Pass Plex Pass

    To add on this. All of the plex cache and media files are all stored back out on the NAS. Transcoding is set to a temp directory on the docker host and is confirmed to be increasing space on the host

    "Mounts": [
    {
    "Source": "/nfs/nas/TVShows",
    "Destination": "/TVShows",
    "Mode": "",
    "RW": true,
    "Propagation": "rprivate"
    },
    {
    "Source": "/nfs/nas/Videos",
    "Destination": "/Videos",
    "Mode": "",
    "RW": true,
    "Propagation": "rprivate"
    },
    {
    "Source": "/nfs/nas/Plex/Config",
    "Destination": "/config",
    "Mode": "",
    "RW": true,
    "Propagation": "rprivate"
    },
    {
    "Source": "/tmp",
    "Destination": "/transcode",
    "Mode": "",
    "RW": true,
    "Propagation": "rprivate"
    }
    ],

  • gbooker02gbooker02 Posts: 636Members, Plex Employee, Plex Pass, Plex Ninja Plex Employee

    Your /transcode is a volume mount on /tmp which appears to be on the / filesystem. That has 3.3G free (which is the same for the aufs mounts because it is ultimately stored on /). This is rather small for a transcode temp. Given what I can glean from your intended setup, you likely need to add an NFS mount for the transcode temp and put the /transcode docker mount somewhere on that NFS mount.

  • liquidfrolloliquidfrollo Posts: 11Members, Plex Pass Plex Pass
    edited August 23

    Thank you gbooker02. My goal was to get the transcoding to process in ram. Based on one of the guides i viewed it says to do it that way, and it doesn't seem to be working. I'm unsure why the tmp volume is 14 gb as i didn't set it. And honestly unsure why it is 76% full as it isn't storing anything that i can tell since all of the videos and config files are all on the NFS. What would be ideal would be to get that partition to ~ 40gb or so if possible, and figure out why it says that 9.8gb of the 14 is full.

    when i run ncdu on the /var/lib/docker/aufs/mnt/ directory the max size returned is only about 500 meg

  • liquidfrolloliquidfrollo Posts: 11Members, Plex Pass Plex Pass

    I think i may have resolved this issue by extending my base partition (DockerHost--vg-root). I provisioned the VM to have more space but needed to configure the file system itself to use the additional size. I followed the guide at the following page

    http://www.geoffstratton.com/expand-hard-disk-ubuntu-lvm

    Now my free space is as follows

    root@DockerHost:~# df -h
    Filesystem Size Used Avail Use% Mounted on
    udev 12G 0 12G 0% /dev
    tmpfs 2.4G 18M 2.4G 1% /run
    /dev/mapper/DockerHost--vg-root 146G 7.8G 132G 6% /
    tmpfs 12G 0 12G 0% /dev/shm
    tmpfs 5.0M 0 5.0M 0% /run/lock
    tmpfs 12G 0 12G 0% /sys/fs/cgroup
    tmpfs 2.4G 0 2.4G 0% /run/user/1000

  • gbooker02gbooker02 Posts: 636Members, Plex Employee, Plex Pass, Plex Ninja Plex Employee

    The key is the location of /transcode in the container which is a mounted volume of /tmp. The instructions you were following were likely assuming that /tmp was tmpfs which will be in RAM, but in your case it was not.

    Also, there's no real significant advantage to putting the transcode temp in RAM. With modern OSs, the disk cache gives all the advantage you'd get out of a ram disk for this content and is capable of purging the cache on content that's not been used recently. I don't understand why this recommendation keeps showing up because it seems to stem from a fundamental misunderstanding on the kind of workload the transcode temp sees and how the OS works.

    Expanding / is one fix as that expands the size of /tmp. You could have also put the transcode temp on an NFS export. It's up to you which best fits your use case.

  • micheajpmicheajp Posts: 5Members, Plex Pass Plex Pass

    Hi. I seem to have the same issue. I've already tried to move the transcode volume to NFS, and a local system to no avail.

    Should I start another thread?

Sign In or Register to comment.