100% disk utilisation

Plex Media Server
, ,
23

Your transcode location absolutely can have an impact.

If your transcode location is on your DATA array (where your media / videos are stored) then that can REALLY slow things down, because it will go like this…

  1. Plex starts a transcode.
  2. The movie is read from your RAID array → DISK READS
  3. The transcoded chunks are written back to the RAID array → DISK WRITES
  4. The transcoded chunks that have just been written to the RAID array are now read into memory whilst they are streamed out via Plex → DISK READS

So as you can see, doing it this way can make your RAID array far busier that would be preferred.

Now if your transcode area is on the OS drive, which looks like it could be on an SSD, then that could actually wear out your SSD much faster.

The ideal situation, would be to have enough RAM in order to create a RAM disk, and move the Transcode area to the RAM disk.

24

@axemanuk666 I agree this did and still does puzzle me and the RAID controller was the first thing I checked to see what was going on. After further testing it became apparent that when I run the file outside of Plex, either remotely or locally using another Media Player, the issue does not occur. This problem only occurs when transcoding movies. I can replicate this every time using the Transformers movie, because it happens in the first 5 minutes. Other movies it takes a lot longer.
@anon18523487 by the way, the Beta version tries to transcode VC1 using FFMPEG and that was what was crashing out Plex Server. I’ve reverted back to previous version and The Matrix runs fine.

25

I would be curious to know what speeds you are getting on your RAID array. You can test this using crystal disk mark…

As a RAID5 array is essentially “stripe with parity”, depending on the controller / disks / number of disks, you should get fairly good read speeds, however RAID5 is notoriously bad for writes, as the controller takes some extra time to calculate and write out the parity bits.

26

Does it not try to transcode on the prior version?

27

@axemanuk666 That was one of the first things I ran. Here is the results.
image
RAID 5 is using RocketRAID 640L controller with 4x 4TB Western Digital disks. I think there Green edition.
@anon18523487 In the console, it just comes up as [Transcode] [Now]. but when on the latest beta, if came up with Transcode FFMPEG, which I know is used by many other media players for video encoding. This crashed the Plex Server each time I tried to play The Matrix [Video VC1/Audio TRUEHD 5.1].

28

@axemanuk666 for comparison the c:\ SSD Samsung:

image

and the other RAID5 running from motherboard Intel chipset x4 2TB Western Digital Green.
image
The speeds do vary but between the two RAID arrays, other then the Read speed, not much of a difference and still plenty of performance to run these films.

29

Huh. Can you post a screen shot? Plex does use FFMPEG but it’s always done that, just a newer version in 1.25.0. However, It should show “(hw)” for hardware transcoding, or nothing for software transcoding using FFMPEG. I’m just confused by what you are seeing.

Let me try reproducing the VC1 issue. I’m sure I tested this before and it was fine.

30

Transcoding VC1 seems to work fine for me.

image

I’ll need a sample of your file to test. Generating Sample Files from Media | Plex Support

31

I assume the log files replicate what I saw on the console when I launched The Matrix, which is 1080p VC1 / Audio TRUEHD 5.1. Your sample above is DTS, and I have no issues running DTS audio. I’ve downloaded logs and can see what I saw. on the console. You will see the FFMPEG errors that crashes Plex.
Plex Media Server Logs_2021-11-12_21-11-37.zip (2.9 MB)

32

@Guru_Panda - You’re getting some pretty good read speeds from those arrays, and as expected, the write speeds are affected by the need to calculate the parity bits, however your sequential writes are not too terrible.

But that SSD though!! Damn that’s fast!!

In regards to your problem, is this only happening with content on the array with the controller, or does it also happen with content on array connected to the motherboard?

Curious to know if this is a “global” problem on your machine, or more specific to one array.

And also, where do you actually have your transcode location pointing to?

For reference, here are the speeds I am getting on my 9 year old HP ProLiant ML350 Gen6 server…

My data storage is a dedicated single DAS unit (Direct Attached Storage), that has a built in RAID controller, housing 8 drives, which is divided into 2 RAID10 arrays, and connected via USB3…

D-Drive → 4x8TB Seagate Ironwolf NAS Drives → RAID10

image

E-Drive → 4x6TB Seagate Ironwolf NAS Drives → RAID10

image

C-Drive → 2x 120GB SSD’s RAID1

image

R-Drive → 16GB RAM Drive ( RAM is DDR3 ECC, its an old box! )

image

I chose to go with RAID10 for 3 reasons… Extra speed, extra redundancy, and to avoid the very long rebuild times you tend to get with RAID5.

33

I’m able to reproduce the issue with TrueHD and PMS 1.25.0. I’ll get this looked at.

1 Like
34

@anon18523487 I’m pleased I was of help to find a bug in beta :slight_smile: Here is a sample of my movie. I used the Plex transcoder to split 10 minutes from the beginning of the film. It took ages and my disk was at 100% whilst it was processing it. It could be the movie, but if it is, I’d like to know why it worked fine before.Processing: output - Transformers Dark side of the moon.mkv… not sure if this work as I got a message saying this file cant be uploaded. If not, please let me know how I can get this sample over to you.

@axemanuk666 I haven’t tried running the film from another drive yet. I’m really not convince it is the RAID array. Its either the transcoder, the film or both.

35

Here’s an experiment:

Take an affected movie file (or three),
throw it into mkvtoolnix and thus remux it without changing anything else.
Put it into your Plex (save the original file, to be able to go back)
Let pass one server maintenance period (usually performed during the night).
Try to play the remuxed file in Plex.

Do you see a difference?

36

@OttoKerner Thank you for your suggestion. The more I’ve been trying to play this file through Plex, the worse this file has become. Not sure how this is possible, but I now can’t play the file after 5 minutes in, from any location or media player. I’ve tried remuxing the file and I now see the same behaviours on the hard disk when using the MVKToolNix GUI. I’m now in the process of repairing the movie using Wondershare Repairit. Hard disk is running at 100% with hardly any read/writes. So far its got to 98.99% and sitting at this point. I’m going to give it a few hours to see if this works and will test again through Plex. Thank you for everyone’s input on this one. If this works, maybe the mvk was corrupt, which the Plex transcoder started to pick up on. If that was the case, hopefully this post wasn’t a complete waste of time, as we have identified a bug in the latest Beta of Plex. I’ll keep everyone posted on my results, assume the mkv file can be repaired.

37

Please try the file on your other array.

What you are now describing, where the behaviour gets worse and worse, very much sounds to me like a deteriorating array.

For the sake of half an hour’s work, try the file on the other array just to rule things out.

The one thing I’ve learnt in the world of IT, is never taking anything for granted, never make any assumptions, and always prove everything.

Good luck.

38

@axemanuk666 I tried to repair the mvk using Wondershare Repairit, but after waiting for it to go through the whole file, it then lets you know that it repaired successful (i doubt it), but you have to pay to save the repaired file. What a con… I don’t think it has done anything, because after that, I tried to copy the file off onto the SSD, and it hung at the beginning of the transfer and the disk was at 100% again. Same behaviour… So you may be correct and the Array is doing a ‘creeping death’ on me :-(. During my initial tests of elimination, the file ran perfectly fine locally and remote using another media player, hence why it seemed like the issue was with Plex. I’ve initialised a Check Disk and Fix bad sectors on all drives in the array. Its going to take some hours before this is finished. I will post back again once this is finished.

1 Like
39

Yup, that’s a deteriorating array for sure.

See if you can find some tools from the controller’s manufacturer, which should then allow you to see the true status of the array and the controller.

Hopefully, you’ve only got 1 failing disk :+1:

40

I’m running the Highpoint RAID management GUI. I can’t see any other tools available from their site. When I first came across this issue and the HDD at 100%, the first things I checked was the RID array and if there were any errors etc. Everything looks in the green and normal and all the SMART results are also coming back okay on each drive. Now that I’ve run a manual disk check, I guess I’ll just have to wait to see if any bad sectors are found and hopefully repaired. This is a really strange one that definitely has me scratching my head. All the normal signs at the start didn’t indicate an array issue, but has time has gone on, it seems more likely.

41

Yes, that’s most likely what’s happening here.

1 Like
42

Now isn’t that just frustrating!

OK, so based on the above, it sounds like you do have the right tools available, and it’s just a shame that it’s not reporting anything glaringly obvious.

But now looking back in hindsight…

  1. Windows shows 100% disk usage but with no read / writes happening.
  2. Cannot copy a file off of the array.

This is definitely all relative.

I would suggest that whilst you are waiting for the checkdisk to finish, you start looking into how you can backup as much of the content from that array as possible.