Please do that one more time. You had confused what I requested somehow. This tells me the browser is doing the logging and not the transcoder.
Jun 08, 2018 19:45:04.296 [0x7f552a6db700] INFO - /usr/lib/plexmediaserver/Plex Media Server
Jun 08, 2018 19:45:04.128 [0x7f5522fff700] VERBOSE - * User-Agent => Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_5) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/11.1.1 Safari/605.1.15
Jun 08, 2018 19:45:04.128 [0x7f5522fff700] VERBOSE - * Accept-Language => en
Jun 08, 2018 19:45:04.128 [0x7f5521fff700] VERBOSE - * Connection => close
Jun 08, 2018 19:45:04.128 [0x7f5522fff700] VERBOSE - * Referer => https://app.plex.tv/desktop
Jun 08, 2018 19:45:04.128 [0x7f5522fff700] VERBOSE - * Accept-Encoding => br, gzip, deflate
The steps I need are:
Normal Plex logs set to DEBUG (not VERBOSE)
Set the transcoder level to verbose as I showed you above. (Your PlexOnlineToken is in your Preferences.xml file)
rerun the test until failure
Secondary items found
Jun 08, 2018 19:46:44.137 [0x7f550f7f4700] VERBOSE - * vdec_hw_status => 0
Jun 08, 2018 19:46:44.548 [0x7f551c3fe700] WARN - SLOW QUERY: It took 630.000000 ms to retrieve 1 items.
Jun 08, 2018 19:46:46.186 [0x7f550f7f4700] VERBOSE - It took 0.0 sec to serialize a list with 0 elements.
Jun 08, 2018 19:46:46.491 [0x7f5524ffe700] VERBOSE - WebSocket: processed 1 frame(s)
There is something seriously sideways happening as it’s intermittently happening
Which directory should I ben in when I run? cat bin/set-transcoder-error
curl 'http://192.168.0.21:32400/:/prefs/set?TranscoderLogLevel=error&X-Plex-Token=Your-Plex-Online-Token-Here'```
I tried running it via copy/paste and got:
```cat: bin/set-transcoder-error: No such file or directory```
I then got:
```curl: (7) Failed to connect to 192.168.0.21 port 32400: Connection timed out```
Apologies again for the confusion and thanks for bearing with me. I am comfortable with running Linux from the command line, but was just confused by your inclusion of: cat bin/set-transcoder-error #!/bin/sh
I just set the logging level for transcoding via curl, and then started the optimization for the same movie. I’ll monitor it closely and grab the logs as soon as I spot the error.
The first line you show (cat bin/set-transcoder-error) is from my command line #!/bin/sh is the standard method to invoke a specific shell (/bin/sh) in a shell script.
Most, if not all, of my plex tools are self-written shell scripts to automate tasks
The logs were incredibly helpful in showing the remote connection to the media drops. If that’s on your network share, then check the file… something is causing a read error. (The ‘transport endpoint not connected’ means the file block it’s trying to read is ‘gone’)
Plex Media Server.log:Jun 09, 2018 03:46:37.404 [0x7f5521fff700] ERROR - HTTP 408 downloading url https://plex.tv/updater/products/1/check.xml?build=linux-ubuntu-x86_64&channel=16&distribution=ubuntu&version=1.13.2.5142-3ba6662e9
Plex Media Server.log:Jun 09, 2018 03:46:51.408 [0x7f5524ffe700] ERROR - Exception getting remote address: remote_endpoint: Transport endpoint is not connected
Plex Media Server.log:Jun 09, 2018 03:46:52.170 [0x7f5524ffe700] ERROR - Exception getting remote address: remote_endpoint: Transport endpoint is not connected
Plex Media Server.log:Jun 09, 2018 03:47:04.724 [0x7f54feffc700] ERROR - [Transcoder] /content/Movies/Plex Versions/Optimized for TV/.inProgress/Blade Runner 2049 (2017).mp4.18: No such file or directory
(END)
I’m glad it was helpful. Can you clarify what you mean by, “where in the network is this?”
Since this keeps happening for 5 files, I don’t think they’re the issue. I can try optimizing the same files on a different machine, if that would help.
I was refreshing the Web UI to see when the error would occur, in case that matters. Also, I was running htop monitoring as well, and the RAM was close to maxed-out at the time.
I grabbed this snippet from the syslog at the point of failure, which has some interesting info that hopefully helps.
Thanks for that log snippet. That tells me everything I need to know.
Your VM is taking a kernel fault when it goes tries to do the disk I/O for the transcoder temp file and doesn’t have any memory left to allocate
Notice here.
16- Jun 9 03:47:02 cloudvault kernel: [1663271.504395] [<ffffffff813ffc13>] dump_stack+0x63/0x90
15- Jun 9 03:47:02 cloudvault kernel: [1663271.504411] [<ffffffff8121012e>] dump_header+0x5a/0x1c5
14- Jun 9 03:47:02 cloudvault kernel: [1663271.504424] [<ffffffff811968f2>] oom_kill_process+0x202/0x3c0
13- Jun 9 03:47:02 cloudvault kernel: [1663271.504427] [<ffffffff81196d19>] out_of_memory+0x219/0x460
12- Jun 9 03:47:02 cloudvault kernel: [1663271.504434] [<ffffffff8119cd45>] __alloc_pages_slowpath.constprop.88+0x965/0xb00
11- Jun 9 03:47:02 cloudvault kernel: [1663271.504437] [<ffffffff8119d168>] __alloc_pages_nodemask+0x288/0x2a0
10- Jun 9 03:47:02 cloudvault kernel: [1663271.504442] [<ffffffff811e6ccc>] alloc_pages_current+0x8c/0x110
09- Jun 9 03:47:02 cloudvault kernel: [1663271.504445] [<ffffffff81192e0b>] __page_cache_alloc+0xab/0xc0
08- Jun 9 03:47:02 cloudvault kernel: [1663271.504448] [<ffffffff81195390>] filemap_fault+0x150/0x400
07- Jun 9 03:47:02 cloudvault kernel: [1663271.504464] [<ffffffff812a81d6>] ext4_filemap_fault+0x36/0x50
06- Jun 9 03:47:02 cloudvault kernel: [1663271.504469] [<ffffffff811c2216>] __do_fault+0x56/0xf0
05- Jun 9 03:47:02 cloudvault kernel: [1663271.504472] [<ffffffff811c5d75>] handle_mm_fault+0xfa5/0x1820
04- Jun 9 03:47:02 cloudvault kernel: [1663271.504486] [<ffffffff8106c747>] __do_page_fault+0x197/0x400
03- Jun 9 03:47:02 cloudvault kernel: [1663271.504489] [<ffffffff8106ca17>] trace_do_page_fault+0x37/0xe0
02- Jun 9 03:47:02 cloudvault kernel: [1663271.504494] [<ffffffff81064fb9>] do_async_page_fault+0x19/0x70
01- Jun 9 03:47:02 cloudvault kernel: [1663271.504513] [<ffffffff81851a08>] async_page_fault+0x28/0x30
We read this from bottom -> top. I’ve added index numbers on the front to help you understand.
“memory pages” and “disk blocks” are the same size (4K bytes) for a really slick reason… they’re treated equally… watch
Normal virtual memory page fault to read in the next file block (files are memory mapped in Linux)
Begin async I/O reading of the block
Set a trace point in case we fail (device not present and have to fall back)
Actually ‘page fault’ the file block into memory (the top most level of the steps to get it)
Handle the fault and dispatch into the file system
Call the fault handler (whichever disk read routine we need)
Arrive in the EXT4 filesystem handler and start looking to see if this block is already in memory somewhere
It’s not, so we generate a ‘hard fault’ and go get it
Allocate a page (block) of cache memory for the next block of the file to be loaded into.
We don’t have the block in cache, go get a fresh block of memory
Make sure we can actually have another block of memory
We’ve run into a problem allocating memory.
We determined we are out of memory (The transcoder can’t have more… Not enough Virtual memory + swap for the VM to give it)
Kill the process asking for memory with Out_Of_memory (OOM) error
Dump the header to the .core file for crash analysis
Dump the process stack to the .core file for crash analysis
Thanks so much for the detailed response. After reading that, I figured it was a memory issue. I’m trying an optimization now with the transcode setting set for optimal quality, so hopefully it consumes less RAM as it goes more slowly through the file.
You’re running in a VM. The VM’s memory is swappable to the real HD. Increase the max memory the VM can have a bit.If your machine runs out of RAM, it will swap your main memory to disk.
yes, the best solution is to add more RAM (16 GB is a nice round number to run Linux + PMS + transcoding)
The VM has the physical disk size you allocated to it when the VM was created. When you installed the OS, it partitioned that space into normal filesystem and swap for use within the VM. This is independent of your main system’s memory.
If you look in the VM’s settings, there is a “Memory” allocation setting. When that is too small, the Guest OS (Linux in the VM) will start swapping sooner than normal. This is the value you want to increase. If you have a 2GB memory allocation, increase to 3GB. You don’t need much but HW transcoding is all about having enough ‘memory’ available to the Guest OS.
What’s happened to you is:
You didn’t allocate enough Memory limit for the VM so the Linux kernel in the VM started to Swap.
The size of the VM’s HD wasnt’ big enough to allow anything else to get swapped out and afford the needed space for transcoding.
That’s where the Linux kernel in the VM said “STOP” and killed the transcoder.
Winding back, Increase VM main memory. If the VM memory fills up, it will swap to your host’s Swap space if it needs. (Remember, Host and Guest are completely independent ).
If your main host (computer) itself starts to swap, THEN you need more main memory
What are the specs you have for everything? Installed system memory right now? VM memory allocation?
Thanks, since my VM is on a VPS that was preconfigured by the hosting company, it seems I don’t have any swap space: total used free shared buff/cache available Mem: 9.8G 2.2G 2.9G 105M 4.6G 7.1G Swap: 0B 0B 0B
I’ve contacted them to see my options to add swap space and/or expand my RAM.
I have set all files and folder to writable and readable for anybody in OS X
then I open the file (internet connection available) few times with manually selected audio tracks (4k HEVC Main 10 hdr file with 3 track DTS-HD MA5.1, DTS 5.1, TRUEHD 7.1). After that optimisation for TV is start working.
Not sure why optimization were failed before since were no error it is just stack on start
Hope this helps someone
I’ll report back with the logs.