I know they (transcoder team) are working on the next update.
There is a big upgrade in it.
Of what I do know, the core (ffmpeg) is full functional replacement. I don’t know details but what I remember was (??) ffmpeg once had this same problem. It was fixed upstream.
Fingers crossed, That same correction, and all it entailed, comes downstream into Plex (which I believe is their goal).
My role with the company is that of a part-part-timer. (I am retired from my career after 35 years in the industry). I do this to help and keep my mind occupied.
AFAIK, code goes from development <-> regression testing right now.
It sounds to me like the work is done and they’re ironing out kinks.
Engineering is being very tight lipped on it but I hear they are making a lot of progress.
revr3nd’s hardware decode script facilitated by my install script for unraid.
I’m not sure if other users are running Docker in this thread, but I have a feeling that it may be some sort of strange inheritance issue. To test my theory, I modified the wrapper script to detach the transcode process from the PMS process entirely.
#!/bin/bash
PLEX_PATH="/usr/lib/plexmediaserver"
ALLOWED_CODECS=("h264" "hevc" "vp8" "vp9")
contains() {
typeset _x;
typeset -n _A="$1"
for _x in "${_A[@]}" ; do
[ "$_x" = "$2" ] && return 0
done
return 1
}
allowed_codec() {
for i in "$@"; do
if [ "-i" == "$i" ]; then
return 1
elif contains ALLOWED_CODECS "$i"; then
return 0
fi
done
return 1
}
if allowed_codec $*; then
"$PLEX_PATH/Plex Transcoder2" -hwaccel nvdec "$@" &
else
"$PLEX_PATH/Plex Transcoder2" "$@" &
fi
transcodepid=$!
disown
trap "pkill -P $transcodepid" INT TERM EXIT
while kill -0 $transcodepid 2> /dev/null; do
sleep 1
done
trap - EXIT
##patched
To explain, we first call the original Plex Transcoder process, and fork it into the background. Then we capture the PID of the Plex Transcoder process so that we can monitor it, and then we finally disown that process, while keeping our script running.
Plex thinks the transcode process is running normally, and the user gets the desired content. When the transcode process exits, the script ends but there is no chain of inheritance between the transcode process and the Plex process.
End result:
Not only am I now able to have the GPU idle, it only enters the P0 state if the driver decides it needs to enter a higher P-State. For some reason, the GPU driver was latching into a P0 state without even needing to. I’m not sure what exactly was causing that weird interaction, but now I can transcode 1080p video without even leaving the P8 state. Fantastic.
Also, I hope this information is useful in the debugging process for the PMS team.
EDIT:
It’s not quite perfect - I need to debug killing the transcode process a bit, is anybody with Plex able to comment on what signal is sent to the transcode process when a stream is ended?
For those experiencing this issue, does it only happen when running the unofficial script to enable NVDEC support on Linux? if it does then it is due to the fact that NVDEC is not fully implemented in the current PMS releases even when enabled by the unofficial script.
In an upcoming PMS release we will have offical support and should resolve this issue.
i just tried it. with and without patch. it makes no difference. in both cases the main process (in my case pid 12362) touches the nvidia device after transcoding is ended and hold the gpu in p0 state. if i restart the plexmediaserver process the gpu swaps instantly from P0 to P8 and stay in P8 until the next transcode is started.
while transcoding
root@mediaserver:~# fuser -v /dev/nvidia*
BEN. PID ZUGR. BEFEHL
/dev/nvidia0: root kernel mount /dev/nvidia0
plex 12362 F.... Plex Media Serv
plex 12567 F...m Plex Transcoder
/dev/nvidiactl: root kernel mount /dev/nvidiactl
plex 12362 F.... Plex Media Serv
plex 12567 F...m Plex Transcoder
/dev/nvidia-uvm: root kernel mount /dev/nvidia-uvm
plex 12362 F.... Plex Media Serv
plex 12567 F...m Plex Transcoder
after transcoding is stopped
root@mediaserver:~# fuser -v /dev/nvidia*
BEN. PID ZUGR. BEFEHL
/dev/nvidia0: root kernel mount /dev/nvidia0
plex 12362 F.... Plex Media Serv
/dev/nvidiactl: root kernel mount /dev/nvidiactl
plex 12362 F.... Plex Media Serv
/dev/nvidia-uvm: root kernel mount /dev/nvidia-uvm
plex 12362 F.... Plex Media Serv
fresh started plexmediaserver with gpu in p8 state
root@mediaserver:~# fuser -v /dev/nvidia*
BEN. PID ZUGR. BEFEHL
/dev/nvidia0: root kernel mount /dev/nvidia0
/dev/nvidiactl: root kernel mount /dev/nvidiactl
/dev/nvidia-uvm: root kernel mount /dev/nvidia-uvm
@chrisallen
This issue affects all Plex installations on Linux, regardless of the presence of the wrapper script.
Without the wrapper script as soon as a transcode process is started the GPU latches to a P0 state. Once the transcode process itself has ended, control of the open handles to the GPU is passed to the parent “PlexMediaServer” process rather than the handles being closed. Since the PMS process is also being executed with a high “nice” value the GPU continues to be stuck in the P0 state. I suspect the handles are being left open in ffmpeg, and then ffmpeg’s process is ending, causing the parent process to inherit these open handles. I believe this has been solved upstream in ffmpeg, but could be mistaken - I haven’t taken the time to strace the issue as frankly it’s not my job as a paying customer to do so.
By decoupling the process as I have above in the script, this solves the problem. The script above does not work properly either, however - as it doesn’t actually kill the transcode process as intended, rather it dies gracefully after finishing the encode for the current chunk.
This issue should be easy to replicate in a testing environment.
Also important to note that it happens on both bare metal and docker containers. So it’s not a docker issue, either.
P0 is the boost mode at Nvidia, P0 in IDLE is therefore a bad thing, the consumption is the most important, the normal IDLE state is P8 (for P2000, ~ 5/6W, not 18w).
So in your case, your gpu is never in idle state.
Your GPU does not go into P8 because without a graphics server, the GPU does not have a self-management policy, so you have to set up and launch an Xorg to apply the nvidia-settings.
Do you have nvidia persistence daemon running? It should take care of proper performance states without X server.
I think I have properly described the test case in march btw…
ComasSky1 via Plex Forum plex@discoursemail.com ezt írta (időpont: 2019. aug. 6., Ke 9:56):
Echoing the above. I actually have a Quadro P2000 myself. I was able to get the transcode process to not only end properly and discard it’s open handles to the GPU by decoupling it, but also using default nice levels allowed the GPU driver to decide when to use higher power states (for example I was able to transcode multiple 1080p streams without leaving the P8 state)
Persistenced is what I use since I’m running unraid with a docker environment, though I also dabbled with a fake X server so I could run nvidia-settings in gui mode and play around in there to see if anything had an impact.
Note that an X server will use a minimum of 8mb of vram and will keep a process on the card at all times, even though the power state functions properly. On the other hand, persistenced will not use any vram and will not have an open process on the card. Persistenced does not allow certain configuration to be done that requires an X server (coolbits, et al) but does make power management at least attempt to function.
Note in my above screenshot:
I am not using Persistenced (it shows “Off” rather than “On”) and am in a P8 state with a P2000 encoding a 1080p stream. P8 utilizing just 5w of power, in other words a 75% power reduction. At this time I was running my fake X server:
#!/bin/bash
##This will only work on single GPU systems:
GPUID=$(nvidia-xconfig --query-gpu-info | grep BusID | sed 's/^[^:]*: //')
#Now that we know the PCI BusID of the card we can create the X server with a fake display:
nvidia-xconfig -s -a --allow-empty-initial-configuration --use-display-device=None --virtual=640x480 --busid "$GPUID" -o /dev/stdout | X :99 -config /dev/stdin&
This script will spawn an empty virtual display with a 640x480 resolution on the next available TTY. Note that if you are using a physical display attached to the machine that the TTY will switch from the TTY you are currently using to this TTY, which will result in “no video” You can use Ctrl+Alt+F1-7 to switch to other active VTs. Only the TTY with this X server spawned will result in no video. I could have added a clause to switch VT’s automatically but I didn’t feel like adding those tools to unraid as it’s pretty manual.
You can also launch processes running on this display, referencing it as “:99”
For example, using ssh -X user@hostname you could then run:
sudo -E nvidia-settings -c :99
and it would start nvidia-settings on the local X server of another machine. This also works from Windows boxes running xming. Just starting the display that way should be sufficient to get the driver to enter a lower power state.