So my NAS is in the living room for a few months since I am moving houses. It’s running TrueNAS with a 8 disk RAIDZ2 using Exos X16 drives (they are noisy).
Whenever I play a file through Plex (not transcoding) it’s making the array noisy. It sounds like the heads constantly have to seek for the right location to read from. The weird thing is, when I do: cat file.mkv > /dev/null it’s almost totally quiet.
ZFS being a CoW filesystem the file is supposed to be written totally sequentially so whenever the head it in place it should be fairly quiet reading the file. Which is is when using cat or cp but not when playing in Plex or Infuse. If I skip a few mins forward in the video then obviously the drives have to find the correct position and it makes a bit more noise for a second or two, it just sounds like it’s doing that all the time when playing.
I monitored the array with fatrace and the only thing it’s doing it reading that MKV file. Nothing else is going on.
How can Plex make such a big difference? It’s almost like its constantly skipping back and forth through the file when playing.
[EDIT] Playing the file in Infuse directly (over SMB) makes almost no noise. fatrace output seems like Infuse reads the file in chunks every few seconds. Compared to a continous read/stream when doing it over Plex. Although that shouldn’t really matter, once the HDD head is in the correct place it should be smooth sailing usually.
Is the Plex data folder also on the array? If so, you want to redirect at least the transcoder temp folder to a single drive.
It doesn’t make much sense to write these temporary files to an array with parity etc.
When you played the file directly in infuse, if this was after playing it multiple times during testing, could it now be pulling that from the ZFS cache hence no noise?
My NAS was in the same room as where we watch most of our stuff. I have all 4U servers so the large fans can be quiet, and the drive noise from the array was bothering me enough that I built a new array with all SSD’s and use the old array as a backup target.
That’s not how that works. In fact CoW file systems tend to have greater fragmentation than those that are not. That aside, ZFS’s metaslab allocation system tends to result in low fragmentation.
The frequency of these “seek sounds” is likely relevant. My guess what you are really hearing is the sound of an idle drive (with its head parked) seeking to the right cylinder, reading a short run of data, and then parking the head again. This is exactly what you would expect from a bursty short read work load which is what you get when a client is playing content over an HTTP connection.
If this is really bothersome to you, you should consider investing in more RAM which ZFS will allocate to ARC or adding an SSD as a L2ARC device. This will increase the read-ahead performed by ZFS.
ZFS has fragmentation of free space, that’s different. I have head parking turned off for these drives but even if that was it, why would it only make so much noise when running through Plex? Unless Plex uses some weird algo to go back and forth in the file.
I have enough RAM for ARC, which also won’t help if you open a file for the first time.
CoW file systems result in higher fragmentation of files. If a file is linearly written, and then subsequently a part of the file is rewritten, a non-CoW FS will typically not result in fragmentation of the file but a CoW FS will. These types of operations occur with media files in nearly every single non-file-copy case such as ripping a disk, compressing a file, etc. This is because there is often a length indicator near the beginning of the file and exact final size is not known when this length is first written.
There is a couple of seeks which occur and your cat and cp examples are not representative of that (nor are they representative of other aspects)
When playing an MKV, a client will first read the begging of the segment until it sees the seek information. Then it will typically seek toward the cues element (which is typically at the end of the file). Then a linear playback of the file will result in a linear read from near the beginning of the file.
After the client fills its media buffer, then it will end up reading the rest of the file at the rate the data in the buffer is consumed (essentially the media bitrate). The media server reads the file at the rate the client requests (basically the media bitrate after the client buffer files). ZFS reads from the disk at the rate its consumer (the Plex Media Server) reads after the ARC is full. Your cat/cp examples do not capture this aspect because they can only read the file as fast as they can.
If your ARC has enough space to keep the entire file, then it will fill with this file and ZFS will not need to go back to the disk after it finishes its read. That is why I suggested this as a potential solution.
