I regularly update all system drivers and both GPU and NIC drivers were updated within the last two weeks (well after this problem started about a month ago) so I don’t think it’s that. I will check for and update all drivers just in case.
My ‘Insecure Connections’ setting was at default - I’ve changed that to Never and if that doesn’t do the trick will try with Always.
I also backed up and deleted my Plug-Ins folder entirely. A PMS restart recreated the Plug-Ins folder with Services.bundle and that’s it. I don’t actually need trakt.tv or Subzero and I’m not sure what Webtools even does, so I’ll leave it at default for the time being.
It’s now been about 4 hours and it seems like everything is fine so far, so I’m going to tentatively call this resolved. Thanks for your help, Jon_C! This is a huge relief! If this had gone on any longer I might’ve had to stop using Plex.
For what it’s worth, deleting the Plex Plug-Ins folder seems to have been the fix. I don’t think I need any of the plug-ins that were previously installed so I’m not going to bother re-adding them one by one, but if anyone else has this problem, that would be a good place to start.
Thanks very much for the update and news, my fingers are crossed for you as I really do understand the frustration 100%, I was in the same spot you are you (were in, hopefully if all goes to plan)…
Look forward to an update over the next hours/day and really hope it’s all sorted for you… Also thanks for following up with your steps as I’m sure others will run into this as well and it was very frustrating as you know when no one seems to know why…
Well, I am sorry to report that deleting my Plug-Ins folder and letting Plex recreate it did not resolve the problem.
While my CPU is no longer pegging at 100%, I am still seeing the behavior where after ~two hours, PlexScriptHost starts running (presumably a scheduled task of some kind) and the process consumes 25% of CPU resources persistently until I shut down PMS or reboot the computer. Previously, I had four PlexScriptHost processes running, each at 25% - so while killing three of those processes has reduced the cost, the underlying cause still remains.
In Task Manager the command line is:
"C:\Program Files (x86)\Plex\Plex Media Server\PlexScriptHost.exe" “C:\Program Files (x86)\Plex\Plex Media Server\Resources\Plug-ins-9d51fdfaa\Framework.bundle\Content\Resources\Versions\2\Python\bootstrap.py” “C:\Program Files (x86)\Plex\Plex Media Server\R” - And it cuts off there as it is presumably too many characters.
Unless anyone has any suggestions, my next step is going to be backing up my DBs, uninstalling and reinstalling PMS and then re-importing my library.
So if this happened - the next stage of investigation would be capturing diagnostics that would cover the time the two back-to-back connections are removed.
This was achieved before through SysInternals Process Monitor which showed what closed the connections
Process Explorer is showing the second situation - one listening connection and no established connections - so it definitely looks like TCP connections are being closed.
What could cause that to happen? I had a bunch of 127.0.0.1 entries in my hosts file, which I have now reset back to default (no entries). I then restarted PMS and checked PlexScriptHost in ProcEx again and there are now three entries - although who knows what it will look like in two hours from how.
I’m still very unclear about what on my system would be terminating TCP sockets (and what that means exactly) and how I would use ProcMon to determine when and how the connections are removed.
The last two cases investigated were 3rd party products closing down the plex connections. It is only through Process Monitor capturing it when they are closed down that one can identify what is doing it. But leaving procmon running for long time would use up all system resources - so would need to check periodically and clear all captured events (clear display) if the connections are still active - until it happens
why did you have these there ? it is normally 3rd party products such as a proxy / security and parental control type of products that would add such entries to the hosts file
I have a few different content servers running (like Calibre) and have subdomains pointed at them - at one time in the process of troubleshooting something else, I added lines to my host file (ie. 127.0.0.1 books.XXXXXXXXXXX.net). I don’t remember why I did that but removing them does not seem to have hurt (or helped) anything.
I will run a Process Monitor capture over the next few hours to see if I can identify what is killing TCP connections - Is there anything in particular I should be looking for?
An update on this - I’m still unable to determine exactly what is causing TCP socket termination on my system. This is what I tried or discovered today:
Looking in my Windows System event logs, I noticed a number of Tcpip warnings (Event ID 4227) indicating that outgoing connections are opened and closed at a high rate. This is consistent with the fact that the PlexScriptHost connections are being terminated somehow. There was nothing corresponding in the Applications event logs.
I saw that there were a number of SkypeApp errors. I don’t use Skype and have never installed it. I downloaded, installed and uninstalled it to clean up the problem files. This resolved the Skype errors but not the Plex error.
I ran a Process Monitor capture just before the PlexScriptHost loop occurred, but I wasn’t able to determine which application caused the socket termination. I also ran TCPView and saw the PlexScriptHost connections being terminated, but I’m not familiar enough with either of these programs to identify what could be causing it to happen.
My next step will be disabling everything that runs automatically at boot except PMS, rebooting, and waiting for two hours to see if the problem occurs again or not.
Save unfiltered PML file
Zip and upload to Dropbox or similar service and send me link by private message
Give me screenshot of process explorer showing when there were 3tcp entries and when they dropped to just 1 - if not - the times when you checked would be enough plus the PID number
Zipped server logs as normal including all the plugin logs
Started having this issue recently, and ‘just’ did your step 1, and that solved the problem for me. My CPU usage would come in at 8-9% immediately upon starting the computer, and remain there.
The screenshot showing 3 tcp ports listening on 10555 and back to back connections on 10556 & 10557 you say was taken at the beginning of the ProcMon capture - around 10:26
and the screenshot showing 1 tcp port - the one the com.plexapp.system process (PID 10420) is kistening on - 10555 and this screenshot was before you stopped the procmon capture - before 10:36:29
The Procmon capture shows no references to tcp ports 10555, 10556, 10557 in the Path column for the captured TCP operations. That is strange.
Procmon was definitely not filtered ? We have 1.6 million events in 10 minutes - that is lower than what i expect. May be it was filtered or procmon.exe needs to have been started with run as administrator? (just thinking of what could be reason for the PML not having the tcp disconnects for 10556 and 10557
I am going to leave PML running on my laptop for 10 minutes and see how many events get captured - i am sure it would be at least an order of magnitude more - we shall see
It is not an order of magnitude different - 1.6 million events on my laptop in 4 minutes. So may be it was captured correctly and not filtered.
Anyway - did some more testing - simulating what happened by using currports utility CurrPorts: Monitoring TCP/IP network connections on Windows and then disconnecting the port myself and the disconnect did not show up in procmon capture - will have to look at other clues. rawcap.exe captured the RST sent to the remote port (the other port in the back-to-back connection)
may be you do the same experiment - killing the port yourself and trying to establish what gets logged where and then we can look for that in procmon
If we assume for a moment, that it was an application that was running at the time that closed the PlexScriptHost.exe tcp connection, then could come to it from this angle:
i looked at the Process Profiling events in your PML and excluded processes that started after 10:30 (i excluded 10:32 onwards) and also excluded processes that show up in a processes profiles list from my laptop - then this brings us down to a short list of candidates to eliminate - and this is the list, Of course it is unlikely to be Photoshop.exe but it could be EventGhost.exe - so tackle the list in the order you think that would exclude the unlikely candidates - initially
After your second-to-last response where you mentioned that there was nothing obviously causing this behavior in the capture, it got me thinking: what if this was being caused by something that was deliberately hiding itself? In other words, malware of some kind?
I run Windows Defender scheduled scans regularly and I expected that would catch anything malicious, but out of curiosity I ran a manual scan to see if anything came up. Nothing did. Still not convinced, I ran a Malwarebytes scan, which I haven’t done since the summer - and sure enough, it brought up results for something called PCCleaner, which I definitely did NOT install, as a rogue application (along with a rootkit which may or may not have been related to PCCleaner). I allowed MWB to quarantine and remove it, rebooted, ran another scan to ensure things were clean, ran a Sophos scan to be doubly sure, and then let Plex run.
It’s been about 24 hours now and PlexScriptHost is still behaving! I think I can say with confidence that this PCCleaner malware was the cause of the problem. Upon further research, it turns out that it was one of those add-on “do you also want to install” packages included with various freeware app installers that someone in my household failed to uncheck while installing something else.
If anyone else is running into this issue and can’t identify the cause, try running a scan with a reputable third party AV\malware program like MWB! PCCleaner isn’t caught by Windows Defender and although it includes an uninstaller executable, it doesn’t show up in the list of installed programs.
Thank you sa2000 for your help on this! It’s very much appreciated!
