Recording failures after update to 1.19.1 beta

I updated to 1.19.1.2621 and every time a DVR recording starts, it fails to complete.

I temporarily rolled back to version 1.18.8.2527 which was more stable for me.

These are the logs for the failed recordings before I rolled back.

Plex Media Server Logs_2020-04-03_20-13-26.zip (5.8 MB)

You mentioned earlier here that you encountered the same commercial skipper issue on 1.19.1.2589 as stated in this thread.

So did recordings work ok in that version but now fail on 1.19.1.2621 ?

The type of errors I am seeing in the logs were seen in the past when “Malwarebytes Premium” was in use - Do you use this ?

See https://support.plex.tv/articles/windows-transcoder-failures-during-playback-live-tv-or-dvr-recording/

I think we should move your posts and my replies to a separate new forum topic as this has nothing to do with the Commercial Skipper

@jcal65 I moved your posts from Commercial skipper putting commercial_markers.mkr file in the show directory with 1.19.1.2589 beta to here as the problem is not related to the commercial skipper

Please see my replies / questions above

I looked back at previous recordings and noticed that they have not been fully completing since March 31st. On that date I updated to 1.19.1.2589. The problem for me has persisted up to and including the current release 1.19.1.2621

I am currently using 1.18.8.2527 and have gotten good recordings ever since. repost

Not using Malwarebytes. I do use Trend Micro Maximum Security.

I noticed that the CC Cleaner program I have used for years stopped loading recently. I temporarily disabled Trend and CC Cleaner started working properly again. I am guessing a recent Trend update has caused this and still trying to find a solution. Searching the web, it seems others have had a similar problem.

Could this be impacting my PMS based on the logs?

I am currently running 1.18.8.2527 and getting good recordings with Trend enabled

Correction on my post: I temporarily disabled Trend and CC Cleaner began working properly.

These errors show the transcoder not being able to get to the Plex Media Server process via port 32400 on localhost.

Apr 03, 2020 20:02:53.601 [10424] ERROR - [Transcoder] [tcp @ 04e2a880] Connection to tcp://127.0.0.1:32400 failed: Error number -138 occurred
Apr 03, 2020 20:02:53.602 [10424] ERROR - [Transcoder] [stream_segment,ssegment @ 05061a40] Failed to open segment list 'http://127.0.0.1:32400/video/:/transcode/session/c2ff9583-19e0-4b60-96df-e0952df4340b/685129a3-2d7e-4477-b413-9c54baaae45b/seglist?X-Plex-Http-Pipeline=infinite'
Apr 03, 2020 20:02:53.602 [10424] ERROR - [Transcoder] av_interleaved_write_frame(): Unknown error

These are similar errors to what was seen when “Malwarebytes Premium” was found to be blocking the transcoder requests

So I suggest trying the following:

Upgrade again to 1.19.1.2621. Do a recording and getting it to fail.

Then disable / stop or uninstall “Trend Micro Maximum Security”
Restart Plex Media Server to get fresh logs created
Repeat the recording test and see what happens
If it still fails - then get the logs zip and I can see if it is still the same error

If recordings now succeed then look into all advance settings for Trend Micro Maximum Security and see if there are options for whitelisting the Plex .exe files - all the .exe files within C:\Program Files (x86)\Plex\Plex Media Server directory

If there is such an option, try to do that and again repeat the tests - this time after rebooting and double checking that the whitelisting is still in place before repeating the test

I will try this and let you know the results. Thanks!

I installed 1.19.1.2621 and ran three tests.

  1. Trend Micro running normally
  2. Trend Micro turned off by system tray
  3. Trend Micro completely uninstalled, and computer rebooted

All three times the DVR recordings failed, about 13 minutes after they started, with this version PMS.

I am open to any suggestions or further tests you would like me try.

Plex Media Server Logs_2020-04-04_12-19-30 version 1.19.1.2621 Trend Micro running normal.zip (5.3 MB)

Plex Media Server Logs_2020-04-04_12-47-02 version 1.19.1.2621 Trend Micro turned off system tray.zip (6.0 MB)

Plex Media Server Logs_2020-04-04_13-14-16 version 1.19.1.2621 Trend Micro uninstalled.zip (7.1 MB)

Thanks for the tests

You are sure there is no other 3rd party security / malware related software running?

The failure is still the same

Apr 04, 2020 13:11:45.687 [4592] ERROR - [Transcoder] [tcp @ 02668880] Connection to tcp://127.0.0.1:32400 failed: Error number -138 occurred
Apr 04, 2020 13:11:45.687 [4592] ERROR - [Transcoder] [stream_segment,ssegment @ 040ed640] Failed to open segment list 'http://127.0.0.1:32400/video/:/transcode/session/98e62e91-594d-4f5f-9c5f-4e0c7c140944/093d5e88-8646-4d89-a3f1-d03be3bb1281/seglist?X-Plex-Http-Pipeline=infinite'
Apr 04, 2020 13:11:45.692 [4592] ERROR - [Transcoder] av_interleaved_write_frame(): Unknown error
Apr 04, 2020 13:11:45.692 [4592] ERROR - [Transcoder] Error writing trailer of media-%05d.ts: Invalid argument

Will need to see if packet capture and procmon capture helps show what it might be

Suggest concentrating on test 3 without the Trend Micro and having two specific tests - one with 1.19.1.2621 and one with 1.18.8.2527 and for each get logs as well as packet capture and for the 1.19.1.2621 test to also have procmon capture - started perhaps 5 minutes into the recording so we do not use too much resources.

For packet capture, would need to capture both network adaptor traffic as well loopback packets and wireshark does have ncap loopback capture capability - selecting both that as well as the network adaptor
An alternative product is rawcap

Links:
Wireshark: Wireshark · Go Deep
or
Rawcap: RawCap - A raw socket sniffer for Windows

So would like to see capture of a test using 1.19.1.2621 and 1.18.8 - with capture stopped when it fails on 1.19.1 and for the 1.18.8 which does not fail - capture stopped after say 15 minutes.
You can start the captures 5 minutes or so into the recordings and save as pcap files

and zip and provide with the zipped logs together

For the 1.19.1 test I am going to complicate it a bit more and add a procmon capture as well
Download - Process Monitor - Sysinternals | Microsoft Learn
You can also start the procmon capture 5 minutes or so in - assuming the failure happens about 13-15 minutes into the recording. Do capture all evenets - nothing filtered. When the failure occurs save procmon capture as PML file - again not filtered and zip

These will be big files and may need to upload to google drive or dropbox etc and send me link privately using forum Message

This is a list of all the programs I have on this computer.

2020-04-04_15-54-32

I will work on the tests you requested and report back to you. Thanks!

Update: have been working with @jcal65 to look into this and with extensive diagnostics and tests from @jcal65 (thank you very much) - this has been tracked down to tcp port exhaustion - which can be seem in the Windows System Event log entry at the time - Event ID 4227 tcpip

TCP/IP failed to establish an outgoing connection because the selected local endpoint was recently used to connect to the same remote endpoint. This error typically occurs when outgoing connections are opened and closed at a high rate, causing all available local ports to be used and forcing TCP/IP to reuse a local port for an outgoing connection. To minimize the risk of data corruption, the TCP/IP standard requires a minimum time period to elapse between successive connections from a given local endpoint to a given remote endpoint.

@jcal65 is helping with testing a development build for this

2 Likes

Mysteriously the problem went away !

There was a non standard dynamic port rule for tcp
netsh int ipv4 show dynamicport tcp
showed port range to be

Protocol tcp Dynamic Port Range

Start Port : 1024
Number of Ports : 64511

The default for windows is:

Protocol tcp Dynamic Port Range


Start Port : 49152
Number of Ports : 16384

This was corrected and the default range set for ipv6 as well and also for udp - but at t the same time, also applied an optional windows update

This brought windows 10 Pro version 1909 from OS build level 18363.752 to 18363.753

After making those 2 changes and a reboot the problem went away

What is baffling is that uninstalling that update and also re-instating the non standard port range made no difference - the tcp port exhaustion was no longer happening

Microsoft did have a bug in this area that was resolved

Addresses an issue that causes all Transmission Control Protocol (TCP) dynamic ports to be consumed. As a result, network communications will fail for any protocol or operation using dynamic port

But this was already applied before and was subsequently superseded when the windows 10 version went up from 1903 to 1909

My only guess at this point is that the windows update from OS build 18363.752 to 18363.753 corrected something that the uninstall of the update left in place

1 Like

That’s really great data. And weird. Thanks for sharing.

Of all the plex employees I’ve encountered on the forums, you really seems to be customer-service oriented, so thank you. We appreciate your hard work on the DVR/live-TV.

I agree, Plex team member sa2000 has been very responsive, great to work with while testing my issue and determined to find a solution.

Although we are not 100% sure what fix actually solved my problem, it has been corrected and it is the result of excellent Plex support!

1 Like

I have been having what looks like a similar problem, where eventually, whatever it is I’m watching, we come across a spurious ‘weak signal’ message. The host machine is still on Win7 x64 (for now) and dynamic tcp port range for ipv4 was set to start at 1025 and number of ports to 13976 ports, strangely enough. FWIW, UDP and IPv6 ranges were nominal. I’ve since changed it to the defaults you’ve mentioned. Not sure if this has made a difference yet, but just thought I’d mention it.

(Oh, and I don’t know if there is any equivalent to the KB you mentioned above for Windows 7, or if they’ve made changes there recently to the stack at all.)

I have responded to your linked forum thread. Yes there was evidence of running out of tcp ports. The latest investigation that I did is here Permanent Transcoder error since update - #14 by sa2000

We are expecting a new version of curl to be released soon with the fix for this and once it is available then we will incorporate this version of curl within Plex Media Server

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.