As the title suggests. I want to build a dedicated PMS box. I already have 2x Intel Xeon 2670’s and a 10Gb/s switch as well as a dual port SFP+ NIC. I was thinking of getting a NVMe SSD to run the PMS installtion, metadata, and transcoding on…but then I though, since I have over 200GB of Ram…that I could setup a RamDisk that’s 102 GB and use that for transcoding, then have a regular SATA ssd for the metadata.
My main question is, would there be any speed benefit with regards to the database and pictures/gui loading for the end user if I have PMS and it’s data installed on a NVMe drive vs a standard SATA SSD?
I switched my system drive from a 7200 rpm to a SSD and I did notice a difference in load times for metadata. From what others have said about this subject for the most part this is only applicable for people with large archives.
So if you are in a small eg sub 10TB, there wont be much of a difference.
Hey,
I also switched from HDD to SSD and I do notice a huge diffence. I mean metadata are like pictures and they are kb/s.
I think the higher the I/O (my Samsung has up to 100.000) the better is speed and loadings time for metadata.
I would give it a try!
PS: If I recall some or all clients do some picture cache.
I switched my system drive from a 7200 rpm to a SSD and I did notice a difference in load times for metadata. From what others have said about this subject for the most part this is only applicable for people with large archives.
So if you are in a small eg sub 10TB, there wont be much of a difference.
I currently have my metadata on a Crucial SSD. Right now I have 52 TB of data with an end goal of 1PB. Which is why I’m posing the question. Even locally, I notice a slight delay in loading images/data, while not bad…I was thinking NVMe drives like the Plextor M8Pe which has a Random Read IOPs of 280k while my current SSD is at 92k IOPs.
I switched my system drive from a 7200 rpm to a SSD and I did notice a difference in load times for metadata. From what others have said about this subject for the most part this is only applicable for people with large archives.
So if you are in a small eg sub 10TB, there wont be much of a difference.
I currently have my metadata on a Crucial SSD. Right now I have 52 TB of data with an end goal of 1PB. Which is why I’m posing the question. Even locally, I notice a slight delay in loading images/data, while not bad…I was thinking NVMe drives like the Plextor M8Pe which has a Random Read IOPs of 280k while my current SSD is at 92k IOPs.
If you’ve got the cash go for it, just make sure its a PCIE v3 x4 device and slot to get the full benefit from it. The SQLite DB will likely crap out before you get to 1PB of data though, not so much a perf issue, more it’s just not designed to handled huge numbers of records efficiently. You’ll likely also see gains from optimising the filesystem you are running on. I shall assume you’re running linux, cause, well, you want it fast so maybe run it on xfs or ext4 with journaling disabled and some mounts options like data=writeback,noatime,nodiratime
Fred, since you are talking about PBs being your goal then set your self up now for pcie based NVMe/AHCI dives to handle all the smaller meta data. I would guess and think you have 52 TB of media which i would assume you have about 2 to 5 TB of metadata. For now you won’t notice the speed difference but in the future you will have the same speed you have now with very slight degradation. If your board supports NVMe then go for it. You would also have to address the limits of SQLite as trudge mentioned. I would look into rqLite which clusters SQLite DB. That might be a solution until Plex supports mySQL or Oracle. Another possible problem is on the client side with storing the metadata from Plex Home Theater. Probably would be best to view only though Plex Web.
I switched my system drive from a 7200 rpm to a SSD and I did notice a difference in load times for metadata. From what others have said about this subject for the most part this is only applicable for people with large archives.
So if you are in a small eg sub 10TB, there wont be much of a difference.
I currently have my metadata on a Crucial SSD. Right now I have 52 TB of data with an end goal of 1PB. Which is why I’m posing the question. Even locally, I notice a slight delay in loading images/data, while not bad…I was thinking NVMe drives like the Plextor M8Pe which has a Random Read IOPs of 280k while my current SSD is at 92k IOPs.
If you’ve got the cash go for it, just make sure its a PCIE v3 x4 device and slot to get the full benefit from it. The SQLite DB will likely crap out before you get to 1PB of data though, not so much a perf issue, more it’s just not designed to handled huge numbers of records efficiently. You’ll likely also see gains from optimising the filesystem you are running on. I shall assume you’re running linux, cause, well, you want it fast so maybe run it on xfs or ext4 with journaling disabled and some mounts options like data=writeback,noatime,nodiratime
So as of right now. I don’t have a PB of stored data. But it is the end goal, obviously years from now. My main concern was as stated by lanboy, my concern is with the small files right now…That’s why i figured that the NVMe drives would be faster at running through those small files since it does have 280k IOPs. I don’t think I would run into Queue Depth issues or issues with transcoding as I would be offloading that to RAM.
I switched my system drive from a 7200 rpm to a SSD and I did notice a difference in load times for metadata. From what others have said about this subject for the most part this is only applicable for people with large archives.
So if you are in a small eg sub 10TB, there wont be much of a difference.
I currently have my metadata on a Crucial SSD. Right now I have 52 TB of data with an end goal of 1PB. Which is why I’m posing the question. Even locally, I notice a slight delay in loading images/data, while not bad…I was thinking NVMe drives like the Plextor M8Pe which has a Random Read IOPs of 280k while my current SSD is at 92k IOPs.
If you’ve got the cash go for it, just make sure its a PCIE v3 x4 device and slot to get the full benefit from it. The SQLite DB will likely crap out before you get to 1PB of data though, not so much a perf issue, more it’s just not designed to handled huge numbers of records efficiently. You’ll likely also see gains from optimising the filesystem you are running on. I shall assume you’re running linux, cause, well, you want it fast so maybe run it on xfs or ext4 with journaling disabled and some mounts options like data=writeback,noatime,nodiratime
So as of right now. I don’t have a PB of stored data. But it is the end goal, obviously years from now. My main concern was as stated by lanboy, my concern is with the small files right now…That’s why i figured that the NVMe drives would be faster at running through those small files since it does have 280k IOPs. I don’t think I would run into Queue Depth issues or issues with transcoding as I would be offloading that to RAM.
Side note: Set that cluster size to the smaller average on the NVMe drives to make the indexing a bit faster on Windows.
While the database/metadata will see benefit from fast access devices, I doubt you’ll see any benefit from putting the transcode temp there.
In terms of bandwidth, assume that you are transcoding to 40Mbps (which is quite high), and you are doing 10 simultaneous streams. That works out to 50MBps read and write (100MBps total). That’s within the performance of some spinning drives. Raid them together and you are well below what you can expect in terms of performance. Additionally if you have RAM, the writes will be committed to disk but the reads will just hit the disk cache in memory and not go to the disk, thus halving the disk access usage.
In terms of random access, unlike the metadata which are many small files, the transcode temp files are fewer and larger files. Additionally the database can be randomly read/written but the transcode temp files are linearly read and written.
So, do you really need to put the transcode temp on some media that has fast access?
@gbooker02 - You have to be careful with the idea that a single sequential read speed is not equivalent loadwise as 10 transcode sessions. Say a hard drive can maintain 100MBps direct transfer. If you start a second transfer, they will not each be 50MBps then, due to head travel. If you start 10 transfers, they will not each get 10MBps. It gets worse to more transfer you throw at it. This is where a good SSD will far outperform a spinning disk. I am not sure how many transcode sessions it would actually take to saturate a spinning disk, but 10 would be taxing I think. With a single head, you have to account for physical movement and latency.
@drinehart That would depend on the OSs caching/read-ahead/write-back policies. It’s quite conceivable that the OS and/or the drive can linearize many of the reads/writes. With the aforementioned disk cache, the fact that 40Mbps is a very very high bitrate transcode and most won’t be that high, and that there are consumer drives that can sustain over 200MBps (linear), its possible a spinning disk can handle 10 transcodes (depending on many factors). I’ve not benchmarked this but I wouldn’t be surprised if a single spinning disk could pull it off. Though this is moot since one wouldn’t use a single spinning disk for the transcode temp if they already have a raid array of them. In that setup, this performance requirement is well below what the array can deliver.
There is no doubt that an SSD will outperform a spinning disk. A good SATA SSD can saturate the 6Gbps of the SATA bus and there are PCI-e SSDs that can go well in excess of that, even at the consumer level. My point in the above approximations is that the SSD/Ram Disk performance may not be necessary for the transcode temp files. I don’t know how many simultaneous transcodes the OP needs nor the bitrate, but simply using the spinning storage in an existing raid array is certainly worth consideration.
@gbooker02 - This has gotten off track from the original request, huh? Sorry for that. In comparing an SSD to a RAM disk, it would be silly to implement a RAM disk for this purpose.
I mostly agree with your statements. I was simply pointing out that there is a limit on spinning disks. I would not assume that a RAID array will perform a lot different. RAID 1, 10 - yes. RAID 5/6, not sure that would be a good use case. RAID 0 - same thing applies, you are just busying two disks with the same issue as the first example I provided. RAID 0 will not provide more IOPs, generally, than non-raid, for transactional loads. I would consider treating the transcode temp location as a database type load. Thus, RAID 10 is recommended.
Ok…maybe I am not reading correctly or this may be above my understanding. So, I really posed a couple of questions in my original post. I also left out some information.
The PMS will have 2x Intel Xeon 2670’s connected through 2x 10Gb SFP+ NIC’s…connected to currently 2 servers that have 1x 10Gb SFP+ NIC’s. The end goal is to connect the PMS up to 5 servers.
These 5 servers will be running JBOD of 22 drives with 2 drives of parity. Hence why I figured if you have 5 different drives, in one server running at 100MB/s each…then 2 servers will max out a 10Gb link.
I was going to set the transcode directly to RAM as RAM has a read/write speed around 6GB/s +/- a few.
I wanted to know, if I had a large collection which I enabled the preview thumbnails, etc. Would the PMS metadata be best ran on NVMe SSD as we’re trying to access a ton of small files…or would a regular have about the same performance in this aspect. I know if we’re writing large amounts of data, since SSD’s or Half-Duplex, then the speed is effectively cut in half when doing read and writes at the same time…which is also why I started leaning towards NVMe SSD’s.
IDK if I missing more important information but this is kinda how my brain functions. Hard for me to get across what I’m trying to do.
Based on the post I will assume you mean 5 storage servers.
JBOD does not have parity. RAID 0,1,10, also have no parity. RAID5 or 6 does have parity. You need to be more clear here.
But why is that even needed? You can get the media over the wire anywhere near that…
PMS metadata, yes, SSD or NVMe (really doubt you will ever see a difference here). Transcoding - NVMe is a complete waste, and I agree with @gbooker02 that for the most part even using SSD for transcode temp is silly.
I’m using UnRaid which you can assign 2 parity drives and up to 28 other drives in the “array”
As I will eventually have 2x10Gb links connected to 5 storage servers with 1x10Gb link each. The idea is that if I have 10+ clients all transcoding at the same time, I’ll be able to get close to saturation of those links between the PMS and storage servers. I’ll also be allowing direct play if the end user can support the speed. With a 1Gb connection, I know that I will not be able to push out 10Gb to the end user. The point is being able to transcode a lot of streams and also have a mix of direct streams as well.
I’m not transcoding to NVMe, orSSD…it will be going to ram. I’m concerned about the speed of the metadata and other small files that are not related to transcoding. I want the system to be extremely responsive. And I can state from my own experience, when you’re transcoding multiple streams to the same SSD while it is also performing other operations, the drive slows down. Look at your SSD and copy a file to it while also reading from it and let me know if you see any decrease in performance.
Don’t come at this with the mindset of a normal person. Most normal people don’t have gig connections, work in an enterprise environment, have 10Gb in the home, have as much storage, etc.
I have the same question. Do you mean 5 storage arrays, 5 instances of PMS running, or something else?
Are doing 22+2 parity or 20+2 parity? Either way, 2 parity drives for this many data drives is a bit small and that’s a very wide stripe. You could end up with 3 drive failures before you can correct the parity lost from the first drive.
I’ve talked about this above. Not necessary unless you are going to be doing a very large number of transcodes. Given the passmark scores, you could be doing a lot but I imagine your spinning array is fast enough for this (depending on how it’s setup)
Putting the database and all the metadata on an SSD will have a notable speed gain. The database has frequent reads/writes. The metadata has frequent reads with very infrequent writes.
This is why i wish video preview thumbnails could be stored in a separate directory. I have my library on a ssd but the thumbnails really don’t need that speedy storage
5 physical storage arrays connected to 1 physical server running PMS
22+2 parity. I’ve used FreeNAS, OpenMediaVault, UnRaid…and UnRaid is what I’m going to use. I understand the risks and this is what I have chosen to run with. Now I will have another server running FreeNAS and I will have the array’s broken down into smaller arrays but I didn’t mention that as it won’t be the bulk of my storage.
Ok
I understand that…my setup is currently on an SSD…but I wanted to know if it would be better on a NVMe drive. Is NVMe a necessity, no, I know it’s not…but I just needed to ask.
Maybe and I think you do get a speed boost from SSD to NVMe, because of the 2 times IOPS. I say just do it! Why not? Such a big Array and Server. Why safe money than in NVMe.
Please give it try and compare it then.
And tell us your experience please.
@Streamerx said:
Maybe and I think you do get a speed boost from SSD to NVMe, because of the 2 times IOPS. I say just do it! Why not? Such a big Array and Server. Why safe money than in NVMe.
Please give it try and compare it then.
And tell us your experience please.
Yeah, I think I’m just going to try both ways and see what happens. Then I can create a thread about it. Not really great at documentation, but I’ll give it a try.
