Transcode

Server Version#: 1.18.2.2058

Hey all -

Before y’all start… yes I know its dumb to transcode 4k… please just help a guy in need. :slight_smile:

Just built a new server… hopefully one that is capable of transcoding for subtitles. I am a bit hard of hearing. Its an i7-9700K running on ubuntu 18.04. Should be plenty of horsepower from my research. Here is the media info:

  • Bitrate 119209 kbps

  • Width 3840

  • Height 2160

  • Container MKV

  • Display Title 4K (HEVC Main 10 HDR)

  • Codec TRUEHD

  • Channels 6

  • Bitrate 3766 kbps

  • Language English

  • Audio Channel Layout 5.1(side)

  • Bit Depth 24

  • Sampling Rate 48000 Hz

  • Title TrueHD 5.1

  • Display Title English (TRUEHD 5.1)

I’ve done some testing and checked my logs. I average anywhere from .8 - 1.1 transcode speed. It is right on the edge of being to do it… however… My CPU according to plex is around 20% under the status window. Running HTOP from terminal it shows anywhere from 15-30% per core. The CPU at no point that I can see gets pegged or is even breaking a sweat. It almost seems like its being held back somehow. Any ideas? When I decommission my old desktop I will add a 1070 into the mix and the problem may become moot… but I want to figure out why this isn’t fully utilizing my compute power.

Thanks.

Bump…

DEBUG, not VERBOSE, log file ZIP please capturing the problem?

Stop playback after the problem shows,
wait 30 seconds,
download logs and attach the ZIP

I’m having a similar issue using a 1660 to Transcode. The GPU decode shows 8% and never goes above .9 speed. I wish plex would just peg my hardware at 100% and call it a day. Same thing when I was running software Transcode on my dual xeons. 30% usage .9 speed. I’ve given up

1 Like

DEBUG (not VERBOSE) Logs ZIP , capturing the problem, and XML are needed…

… or it didn’t happen :slight_smile: (just like college)

Plex Media Server Logs_2019-12-07_14-17-09.zip (4.9 MB)

Thanks Chuck. See attached.

@Clmcm400

Thanks for the logs.

  1. Subtitles are selected. English for foreign language film.
  2. Audio conversion to AAC enabled. TV not capable of the source audio ?
  3. VAAPI (CPU’s internal ASIC) selected.
Dec 07, 2019 14:12:55.626 [0x7ff9feffd700] DEBUG - TPU: hardware transcoding: using hardware decode accelerator vaapi
Dec 07, 2019 14:12:55.626 [0x7ff9feffd700] DEBUG - [Universal] Using local file path instead of URL: /home/clmcm400/media/Movies/Blade Runner (1982)/Blade Runner (1982).mkv
Dec 07, 2019 14:12:55.626 [0x7ff9feffd700] DEBUG - TPU: hardware transcoding: zero-copy support present
Dec 07, 2019 14:12:55.626 [0x7ff9feffd700] DEBUG - TPU: hardware transcoding: not using zero-copy because subtitle burning is required
Dec 07, 2019 14:12:55.626 [0x7ff9feffd700] DEBUG - Codecs: hardware transcoding: testing API vaapi
Dec 07, 2019 14:12:55.628 [0x7ff9feffd700] DEBUG - [FFMPEG] - Format 0x41524742 -> bgra.
Dec 07, 2019 14:12:55.628 [0x7ff9feffd700] DEBUG - [FFMPEG] - Format 0x42475241 -> argb.
Dec 07, 2019 14:12:55.628 [0x7ff9feffd700] DEBUG - [FFMPEG] - Format 0x41424752 -> rgba.
Dec 07, 2019 14:12:55.628 [0x7ff9feffd700] DEBUG - [FFMPEG] - Format 0x52474241 -> abgr.
Dec 07, 2019 14:12:55.628 [0x7ff9feffd700] DEBUG - [FFMPEG] - Format 0x58524742 -> bgr0.
Dec 07, 2019 14:12:55.628 [0x7ff9feffd700] DEBUG - [FFMPEG] - Format 0x42475258 -> 0rgb.
Dec 07, 2019 14:12:55.628 [0x7ff9feffd700] DEBUG - [FFMPEG] - Format 0x58424752 -> rgb0.
Dec 07, 2019 14:12:55.628 [0x7ff9feffd700] DEBUG - [FFMPEG] - Format 0x52474258 -> 0bgr.
Dec 07, 2019 14:12:55.628 [0x7ff9feffd700] DEBUG - [FFMPEG] - Format 0x50424752 -> unknown.
Dec 07, 2019 14:12:55.628 [0x7ff9feffd700] DEBUG - [FFMPEG] - Format 0x50524742 -> unknown.
Dec 07, 2019 14:12:55.628 [0x7ff9feffd700] DEBUG - [FFMPEG] - Format 0x36314752 -> unknown.
Dec 07, 2019 14:12:55.628 [0x7ff9feffd700] DEBUG - [FFMPEG] - Format 0x3231564e -> nv12.
Dec 07, 2019 14:12:55.628 [0x7ff9feffd700] DEBUG - [FFMPEG] - Format 0x3132564e -> unknown.
Dec 07, 2019 14:12:55.628 [0x7ff9feffd700] DEBUG - [FFMPEG] - Format 0x32595559 -> yuyv422.
Dec 07, 2019 14:12:55.628 [0x7ff9feffd700] DEBUG - [FFMPEG] - Format 0x59565955 -> uyvy422.
Dec 07, 2019 14:12:55.628 [0x7ff9feffd700] DEBUG - [FFMPEG] - Format 0x32315659 -> yuv420p.
Dec 07, 2019 14:12:55.628 [0x7ff9feffd700] DEBUG - [FFMPEG] - Format 0x30323449 -> yuv420p.
Dec 07, 2019 14:12:55.628 [0x7ff9feffd700] DEBUG - [FFMPEG] - Format 0x48323234 -> yuv422p.
Dec 07, 2019 14:12:55.628 [0x7ff9feffd700] DEBUG - [FFMPEG] - Format 0x56323234 -> yuv440p.
Dec 07, 2019 14:12:55.628 [0x7ff9feffd700] DEBUG - [FFMPEG] - Format 0x50343434 -> yuv444p.
Dec 07, 2019 14:12:55.628 [0x7ff9feffd700] DEBUG - [FFMPEG] - Format 0x33434d49 -> unknown.
Dec 07, 2019 14:12:55.628 [0x7ff9feffd700] DEBUG - [FFMPEG] - Format 0x30313050 -> p010le.
Dec 07, 2019 14:12:55.629 [0x7ff9feffd700] DEBUG - [FFMPEG] - Created surface 0.
Dec 07, 2019 14:12:55.629 [0x7ff9feffd700] DEBUG - [FFMPEG] - Direct mapping possible.
Dec 07, 2019 14:12:55.629 [0x7ff9feffd700] DEBUG - TPU: hardware transcoding: final decoder: vaapi, final encoder: vaapi
Dec 07, 2019 14:12:55.630 [0x7ff9feffd700] DEBUG - Job running: EAE_ROOT='/tmp/pms-5cc7c45c-ae92-45a1-946a-65fdfa709a53/EasyAudioEncoder' FFMPEG_EXTERNAL_LIBS='/var/lib/plexmediaserver/Library/Application\ Support/Plex\ Media\ Server/Codecs/395e79c-2735-linux-x86_64/' XDG_CACHE_HOME='/var/lib/plexmediaserver/Library/Application Support/Plex Media Server/Cache' XDG_DATA_HOME='/usr/lib/plexmediaserver/Resources' X_PLEX_TOKEN='xxxxxxxxxxxxxxxxxxxx' '/usr/lib/plexmediaserver/Plex Transcoder' '-codec:0' 'hevc' '-hwaccel:0' 'vaapi' '-hwaccel_fallback_threshold:0' '10' '-codec:1' 'truehd_eae' '-eae_prefix:1' 'd6e40b3357eace54-com-plexapp-android_' '-analyzeduration' '20000000' '-probesize' '20000000' '-i' '/home/clmcm400/media/Movies/Blade Runner (1982)/Blade Runner (1982).mkv' '-filter_complex' '[0:6]scale=3840:2160[0];[0:0][0]overlay[1];[1]scale=w=3840:h=2160[2];[2]format=pix_fmts=nv12[3];[3]hwupload[4]' '-filter_complex' '[0:1] aresample=async=1:ocl='\''5.1'\'':osr=48000[5]' '-map' '[4]' '-codec:0' 'h264_vaapi' '-b:0' '62208k' '-maxrate:0' '82944k' '-bufsize:0' '165888k' '-r:0' '23.975999999999999' '-force_key_frames:0' 'expr:gte(t,0+n_forced*1)' '-map' '[5]' '-metadata:s:1' 'language=eng' '-codec:1' 'aac' '-strict:1' 'experimental' '-aac_coder:1' 'fast' '-q:1' '0' '-segment_format' 'mpegts' '-f' 'ssegment' '-individual_header_trailer' '0' '-segment_time' '1' '-segment_start_number' '0' '-segment_copyts' '1' '-segment_time_delta' '0.0625' '-segment_list' 'http://127.0.0.1:32400/video/:/transcode/session/d6e40b3357eace54-com-plexapp-android/0a05da6d-602f-435d-912c-bd6ec37ca596/seglist' '-segment_list_type' 'csv' '-segment_list_size' '5' '-segment_list_separate_stream_times' '1' '-segment_list_unfinished' '1' '-max_delay' '5000000' '-avoid_negative_ts' 'disabled' '-map_metadata' '-1' '-map_chapters' '-1' 'media-%05d.ts' '-start_at_zero' '-copyts' '-vsync' 'cfr' '-y' '-init_hw_device' 'vaapi=vaapi:,driver=iHD,kernel_driver=i915' '-hwaccel_device' 'vaapi' '-filter_hw_device' 'vaapi' '-nostats' '-loglevel' 'quiet' '-loglevel_plex' 'error' '-progressurl' 'http://127.0.0.1:32400/video/:/transcode/session/d6e40b3357eace54-com-plexapp-android/0a05da6d-602f-435d-912c-bd6ec37ca596/progress'
Dec 07, 2019 14:12:55.630 [0x7ff9feffd700] DEBUG - Jobs: Starting child process with pid 25045
Dec 07, 2019 14:12:55.631 [0x7ffa75de0700] DEBUG - Request: [127.0.0.1:48364 (Loopback)] PUT /video/:/transcode/session/d6e40b3357eace54-com-plexapp-android/0a05da6d-602f-435d-912c-bd6ec37ca596/progress?status=startup (17 live) Signed-in Token (Clmcm400) (range: bytes=0-) 
Dec 07, 2019 14:12:55.631 [0x7ffa86ffd700] DEBUG - Completed: [127.0.0.1:48364] 204 PUT /video/:/transcode/session/d6e40b3357eace54-com-plexapp-android/0a05da6d-602f-435d-912c-bd6ec37ca596/progress?status=startup (17 live) 0ms 203 bytes (pipelined: 1) (range: bytes=0-) 
Dec 07, 2019 14:12:55.640 [0x7ffa5b7fe700] DEBUG - Request: [127.0.0.1:48364 (Loopback)] PUT /video/:/transcode/session/d6e40b3357eace54-com-plexapp-android/0a05da6d-602f-435d-912c-bd6ec37ca596/progress?status=startup (17 live) Signed-in Token (Clmcm400) (range: bytes=0-) 
Dec 07, 2019 14:12:55.640 [0x7ffa877fe700] DEBUG - Completed: [127.0.0.1:48364] 204 PUT /video/:/transcode/session/d6e40b3357eace54-com-plexapp-android/0a05da6d-602f-435d-912c-bd6ec37ca596/progress?status=startup (17 live) 0ms 203 bytes (pipelined: 2) (range: bytes=0-) 
Dec 07, 2019 14:12:55.640 [0x7ff9fdffb700] DEBUG - Request: [127.0.0.1:48364 (Loopback)] PUT /video/:/transcode/session/d6e40b3357eace54-com-plexapp-android/0a05da6d-602f-435d-912c-bd6ec37ca596/progress?status=opening (17 live) Signed-in Token (Clmcm400) (range: bytes=0-) 
Dec 07, 2019 14:12:55.640 [0x7ffa877fe700] DEBUG - Completed: [127.0.0.1:48364] 204 PUT /video/:/transcode/session/d6e40b3357eace54-com-plexapp-android/0a05da6d-602f-435d-912c-bd6ec37ca596/progress?status=opening (17 live) 0ms 203 bytes (pipelined: 3) (range: bytes=0-) 

This CPU is coasting along. Subtitle burning on an 8 core CPU (4 phys, 4 logical) will show as 12.5% at maximum for the subtitle part + what little bit is needed for the particular audio codec; maybe 20-25% max. Subtitle burning is single threaded. IF subtitle burning is ever added to the VA-API library, it will be offloaded to the ASIC as it is capable of it.

I don’t know what the playback settings on the Shield are, or those of the AVR/TV but I see AAC often when transcoding to a shield. Does the TV support it?

Unfortunately I use a SONOS playbar for audio… and their codec support is garbage. It also doesn’t support connectivity to a receiver so I need to replace my whole audio setup to support the audio. You mention 12.5% maximum… I take it that is what plex will report from the status window. While running the playback I had the processor utilization up from the OS and I never saw a single core peak above 30%. I knew subtitles were single threaded from prior research (read a lot of threads where you provided feedback). Since the playback is fine without subtitles, I assume that is the bottleneck… but since I don’t see a single core beyond 30% from the HTOP command in the linux terminal I can only assume plex isn’t maxing the capabilities of my processor. Can you shed some light on that theory? Or am I wrong and its the audio that is bogging down my system? Every thread I read says my setup should be able to tear through this. I haven’t yet added a GPU and I have one and plan to do so once I offline my old plex box. Is this the only solution to my issue even though I have a processor that should be able to handle this?

Thanks.

Be careful of single core vs entire processor percentages.

When I state: 12.5%, I am referring to 100% (of the physical processor) / 8 threads.

If you are seeing 30% of a single core, that’s a very light load.

If Plex doesn’t need to max the CPU’s capabilities, why should it?

  1. Initial spike to fill the output buffers
  2. As video and audio are consumed (played),
  3. the transcoder wakes up for a moment to refill the buffers.

You don’t want the CPU fully maxed all the time. If it is, there is a much bigger problem to resolve.

GPU performance as measured for gaming isn’t the same as it is for video processing. In video, it’s a lot of small block conversions from one format to another. It’s a different type equation being processed through the pipelines.

Ahh thanks for explaining 12.5. Math, duh.

I am getting stuttering when playing that movie back - that is why I think it should use more compute power. If plex isn’t maxing my CPU in any way why would I get stuttering? Did you see anything in the logs to explain it? I feel like something is still amiss.

Also you mention why would it max the cores if it doesn’t need to… my transcode speed is .8-1.1ish or so. I have seen other transcodes go much higher than that speed. Are you saying that for some reason on my setup plex is buffering because it doesn’t need to go faster?

I only brought up the GPU as a work around for the current issue I am experiencing.

Thanks.

Why stuttering? Is it wireless? If so, is it 2.4 Ghz (the trash band) on an overloaded channel which makes throughput a challenge?

Well I was hoping the forums could explain the stuttering. :slight_smile:

It is wired. Both the server and the shield.

Any other thoughts?

Thanks.

Without a hands-on, forgive me, is like taking your car to the mechanic & saying it sputters without letting him test drive it. No way to know what the problem is.

Stuttering with internet is the act of:

  1. Data is sent to the playback buffer
  2. Player consumes the data
  3. Buffer is emptied to the point where it asks for more data, knowing it will run out soon.
  4. Stuttering occurs when that data does not arrive before the buffer has run out.
  5. Playback suspends until ample data to continue.

What’s involved here?

  1. Network
  2. Computational Resources.

How about installing iperf3 and running it to quantify network performance at the client

iperf3 runs on the server and the client.

Here is what it looks like. This shows the UP and DOWN sustainable throughput.

[chuck@lizum ~.372]$ iperf3 -c 192.168.0.21
Connecting to host 192.168.0.21, port 5201
[  5] local 192.168.0.13 port 50078 connected to 192.168.0.21 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  1.09 GBytes  9.40 Gbits/sec    0   2.56 MBytes       
[  5]   1.00-2.00   sec  1.10 GBytes  9.41 Gbits/sec    0   2.56 MBytes       
[  5]   2.00-3.00   sec  1.09 GBytes  9.41 Gbits/sec    0   3.12 MBytes       
[  5]   3.00-4.00   sec  1.09 GBytes  9.41 Gbits/sec    0   3.12 MBytes       
[  5]   4.00-5.00   sec  1.09 GBytes  9.41 Gbits/sec    0   3.12 MBytes       
[  5]   5.00-6.00   sec  1.10 GBytes  9.42 Gbits/sec    0   3.12 MBytes       
[  5]   6.00-7.00   sec  1.09 GBytes  9.41 Gbits/sec    0   3.12 MBytes       
[  5]   7.00-8.00   sec  1.10 GBytes  9.42 Gbits/sec    0   3.12 MBytes       
[  5]   8.00-9.00   sec  1.09 GBytes  9.41 Gbits/sec    0   3.12 MBytes       
[  5]   9.00-10.00  sec  1.09 GBytes  9.41 Gbits/sec    0   3.12 MBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  11.0 GBytes  9.41 Gbits/sec    0             sender
[  5]   0.00-10.00  sec  10.9 GBytes  9.41 Gbits/sec                  receiver

iperf Done.
[chuck@lizum ~.373]$ iperf3 -c 192.168.0.21 -R
Connecting to host 192.168.0.21, port 5201
Reverse mode, remote host 192.168.0.21 is sending
[  5] local 192.168.0.13 port 50094 connected to 192.168.0.21 port 5201
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec  1.08 GBytes  9.27 Gbits/sec                  
[  5]   1.00-2.00   sec  1.10 GBytes  9.41 Gbits/sec                  
[  5]   2.00-3.00   sec  1.10 GBytes  9.41 Gbits/sec                  
[  5]   3.00-4.00   sec  1.10 GBytes  9.41 Gbits/sec                  
[  5]   4.00-5.00   sec  1.09 GBytes  9.40 Gbits/sec                  
[  5]   5.00-6.00   sec  1.10 GBytes  9.41 Gbits/sec                  
[  5]   6.00-7.00   sec  1.10 GBytes  9.41 Gbits/sec                  
[  5]   7.00-8.00   sec  1.10 GBytes  9.41 Gbits/sec                  
[  5]   8.00-9.00   sec  1.10 GBytes  9.41 Gbits/sec                  
[  5]   9.00-10.00  sec  1.10 GBytes  9.41 Gbits/sec                  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  10.9 GBytes  9.40 Gbits/sec    0             sender
[  5]   0.00-10.00  sec  10.9 GBytes  9.40 Gbits/sec                  receiver

iperf Done.
[chuck@lizum ~.374]$

Perform this level testing on the LAN and, if there is a problem, it will show.

Forgive my candor, We’re here to help support Plex. We are not here to support or fix network problems. Guide? Ok. Solve? It’s your network. :smiley:

Yeah I hear you… but I doubt it is a network issue unless you saw something in the logs that demonstrate that. I am an IT systems engineer (not bragging just saying I have a bit of experience) - I have a 120TB freenas array on a 10G network. The plex server will have 10G I just haven’t yet installed that card. So it is only 1G for now - but I can transfer files back and forth around 105-110MB/sec (with TCP/IP overhead factored in that is pretty good on a 125MB/sec pipe). A saturated 1 gigabit per second link can transfer 450 gigabytes per hour… in order for the network to not be a factor it needs to be able to transfer in real time at a minimum. Back of napkin math says I can transfer that file at least 6 times during the playback time span. Getting an iperf up and running would require me to sideload it onto the shield… a bit of a pain. I have built my network numerous times and have multiple switch stacks of Cisco 3750’s and Ubiquiti 10G switch/WAPs on cat6e. I don’t run your standard home network - its enterprise level (home labs ftw). So it -shouldn’t- be a factor.

It feels like we are grasping at straws… I have already confirmed the server is transcoding at .8x-1.1x speed. You have also stated my processor is coasting along. I have more compute available and we haven’t explained why it isn’t using more available CPU cycles when the transcoding speed is too slow. Isn’t that the issue that is causing my stuttering? I just want to get at the root cause of that and fix it so this machine can perform as well as its specs say it should.

Thanks.

If you want to throw equipment lists (which i have at home here in my office) around, I can do that too, but don’t think it’s really called for here. Suffice it to say my backbone is 10 GbE as well. My education and career are in EE & CE (Computer Engineering) with all the necessary credentials. I’ve been doing this for 30+ years so I too have experience under my belt.

With that out of the way…

What I don’t understand is where you’re getting the .8-1.1x speed value from?

If this were a CPU-only problem, the speed value would be a direct reflection of the CPUs ability to deliver content as compared to real-time demand. A transcoding speed of 2.0 means that it is transcoding 2.0x faster than real time. A speed of 0.8 means it’s transcoding at 80% of real time playback.

How did you confirm the server is transcoding at that speed? By which test(s)?

How about this as a test?

  1. Select one of your rips which has less than 100 Mbps video demand. (80 Mbps?)
  2. Play it and observe result.

I suggest this because

  1. You have an i7-9xxx CPU.
  2. I do not know of any quantitative performance benchmark results for the iHD driver which that CPU requires.
  3. Knowing the Intel i965 driver would ‘crap out’ at 100 Mbps,
  4. Is it possible iHD, which the CoffeeLake requires, also ‘craps out’ at that bitrate?
  5. Transcode 80 Mbps and test. Transcode 90 Mbps and test.
  6. Assertion: There is a maximum transcode bandwidth through the iHD_drv_video driver which is less than the ASIC’s true limit.

Controlled test files which are excellent for this type testing: http://jell.yfish.us/
These are known good test files at a variety of bit rates in both H.264 and H.265.

If there is a performance knee / fallover point, they will expose it.

Run playback to your browser . Nothing special is required.

I only threw out equipment lists to state what I was working with as networking was being discussed - not a measuring contest. :slight_smile: If I were running some junk Linksys cable/dsl router from back in the day I would definitely blame the network, lol.

I confirmed that speed by verbose logging and checking Plex Media Server.log (ala Why is my video stream buffering? | Plex Support). I don’t have those logs anymore otherwise I would upload those for you. I will have to regenerate them.

Fascinating theory on the driver being a limiting factor for the ASIC. I will definitely love to test that. Give me some time to work on the scenarios you described (work week and all). We had similar(ish?) testing at my job - line ASICs on a fiber SAN director could pump more data than the back-plane could handle… fun to figure out but what a pain.

Do you know #3 as fact? Wondering if I am using crappy drivers on Ubuntu. Do you know if there are certain drivers I should be running? Maybe something you guys recommend from in house testing?

I could also swap drives and run this on a Windows OS… wouldn’t be apples to apples but could point at an OS/driver issue instead of a Plex one.

Hmmm gotta stir on this a bit… thanks.

I wasn’t ‘measuring’. I figured it would give you a comparable reference point from where I draw.

Point #3, regarding the i965, is fact. It was designed at a time of 720p and 1080p. Typical BluRay 1080p don’t sustain more than 60-70 Mpbs with the occasion spike above 100 for high-cgi bursts on reference frame loading.

The advent of 2160p changes everything. Rather than continue to nurse an aging technology, and in concert with Intel’s future plans (whatever they are), they decided to start over with the iHD family.

I ocncur, Windows uses a different rendering mechanism entirely. Running these tests on Windows and comparing against Linux is not valid because it’s not the same iHD driver code. I wouldn’t bother wasting time but if you want to run 80, 100, and 120 Mbps then ok… I wouldn’t go through any trouble reconfiguring just for the test though.

If you just play the JellyFish control videos, it will tell you quickly enough.
I’ve run them. I have references for an i7-7700, transcoding down to 10 Mbps output.
It will be interesting to see how your datapoints plot out. (The source files available exceed 200 Mbps)

me too

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.