Migrating PMS 1.21.0 to internal m.2 on qnap

[rich.jacot_gmail.com]

Server Version#:1.21.0
Player Version#:

I created a new datavol on two 500GB internal m.2s in a raid 1 with 5% over-provisioning and it’s a thick volume. The intent was to migrate Plex qpkg over to this new volume.

Once it was created. I stopped Plex in the AppCenter. Then I clicked the down arrow and Migrate to. Went through the step and selected the new volume. Well it’s been sitting saying “Migrating…” under Plex in the AppCenter for over an hour now. Is this normal? How long should it take? I’m not sure what I expected but I was thinking a minute or two.

BTW this is on a TVS-672N with 6 - 4TB WD Reds in a RAID5. It’s primarily used as a Plex Media Server but does of course have photos and a few other things on it. (QSonarr, Radarr, Jackett, Lidarr)

[rich.jacot_gmail.com]

It finally finished. Wow that took awhile!

[stpbase]

There are many small files in the server folder, so the transfer takes a while. Mine has over 400k files

[ChuckPa]

My TVS-1282 (i7), with 8x 8TB 7200 RPM drives needs 3-4 minutes to Migrate my data from SSD to SSD.

There is a lot of stuff in there.

[rich.jacot_gmail.com]

I just checked mine and found 411516 files. wow!

I noticed I had a full backup running at the time too. Poor planning on my part I guess.

Thanks!

[ChuckPa]

want to cringe?

[/share/CACHEDEV3_DATA/.qpkg/PlexMediaServer/Library] # find . -print | wc -l
2522920
[/share/CACHEDEV3_DATA/.qpkg/PlexMediaServer/Library] #

[rich.jacot_gmail.com]

Holy cow Batman!!!

How many byte per inode did you create your volume with? I’d be worried about running out of inodes/files.

I chose 4k cause I figured there were at bunch of small files and didn’t see any point in going larger.

[ChuckPa]

QNAP doesn’t give the option.

The Linux default is 256 (since 2.6-ish?)

Here is the tune2fs output from the SSD

[~] # tune2fs -l /dev/mapper/cachedev3
tune2fs 1.43.9 (8-Feb-2018)
Filesystem volume name:   DataVol3
Last mounted on:          /share/CACHEDEV3_DATA
Filesystem UUID:          87107fe0-7fc5-4fc6-ae07-7aaaa5393221
Filesystem magic number:  0xEF53
Filesystem revision #:    1 (dynamic)
Filesystem features:      has_journal ext_attr resize_inode filetype needs_recovery extent 64bit flex_bg sparse_super large_file huge_file uninit_bg dir_nlink extra_isize
Filesystem flags:         signed_directory_hash rehashed 
Default mount options:    (none)
Filesystem state:         clean
Errors behavior:          Continue
Filesystem OS type:       Linux
Inode count:              26918912
Block count:              215351296
Reserved block count:     131072
Free blocks:              188271438
Free inodes:              24363411
First block:              0
Block size:               4096
Fragment size:            4096
Group descriptor size:    64
Reserved GDT blocks:      1024
Blocks per group:         32768
Fragments per group:      32768
Inodes per group:         4096
Inode blocks per group:   256
RAID stride:              128
RAID stripe width:        128
Flex block group size:    16
Filesystem created:       Sun Nov  3 08:27:38 2019
Last mount time:          Sat Nov 21 12:52:21 2020
Last write time:          Sat Nov 21 12:52:21 2020
Mount count:              5
Maximum mount count:      -1
Last checked:             Sat Sep 26 17:56:18 2020
Check interval:           0 (<none>)
Lifetime writes:          40 TB
Reserved blocks uid:      0 (user admin)
Reserved blocks gid:      0 (group administrators)
First inode:              11
Inode size:	              256
Required extra isize:     28
Desired extra isize:      28
Journal inode:            8
Default directory hash:   half_md4
Directory Hash Seed:      21d5a429-9443-464c-afbf-a65db2320a61
Directory Hash Rev:       1
Directory Magic Number:   0x514E4150
Journal backup:           inode blocks
[~] # 

[rich.jacot_gmail.com]

@ChuckPa

918

Interesting. I got the option when I created it.

Like this: https://forum.qnap.com/download/file.php?id=25856&mode=view

Part of my tune2fs is:
cachedev1 cachedev2 control
[~] # tune2fs -l /dev/mapper/cachedev2
tune2fs 1.43.9 (8-Feb-2018)
Filesystem volume name: Volume3
Last mounted on: /share/CACHEDEV2_DATA
Filesystem UUID: 6cd98298-52b2-4610-8127-539af4c89ed3
Filesystem magic number: 0xEF53
Filesystem revision #: 1 (dynamic)
Filesystem features: has_journal ext_attr resize_inode filetype needs_recovery extent 64bit flex_bg sparse_super large_file huge_file uninit_bg dir_nlink extra_isize
Filesystem flags: signed_directory_hash rehashed
Default mount options: (none)
Filesystem state: clean
Errors behavior: Continue
Filesystem OS type: Linux
Inode count: 95027200
Block count: 95027200
Reserved block count: 131072
Free blocks: 88801841
Free inodes: 95027188
First block: 0
Block size: 4096
Fragment size: 4096
Group descriptor size: 64

[ChuckPa]

my advice? Don’t micro manage these things. There are times when it is important to and I won’t dispute that. Here, let QNAP deal with it because , in Plex’s case, the default is better than trying to estimate how much you’ll actually need.

My structure here:

1. QTS - cachedev1 (system default creation)
2. Main Array - cachedev2 (adjusted for 64K byte file blocking)
3. PMS - cachedev3
4. VMs - cachedev(4-6) raid

[ozbob65]

Hi guys, first ever comment on here. I see you all over the place, ChuckPa, so I’m guessing you’re the man to hit up for newbie help!!! I’m at the point where I’m wanting to be in control of my NAS (and Plex), and tell it where to put things and how instead of the other way around. I’ve been reading a lot about people using the RAM as cache, and having all their Plex info on SSD and so on; and want to go the same way… hell, I’ve only just worked out in the last week or so that the OS is Linux (I think?!)!!!

I have a TVS-873e running QTS 4.5.1.1540 with 32gb RAM (4x 8gb Crucial DIMM’s), a storage pool comprising 8x Seagate Barracuda HDD’s in RAID5 with 3x volumes: 1) OS, 2) Media, and 3) various files and data, and just installed 2x 860 EVO 500gb M.2 SATAIII in RAID1 as a second storage pool with one volume.

What I want to achieve is Plex (not the media) on the SSD’s. I tried the migrate thing yesterday, and while I could Plex was now on the SSD’s, all the data in the Plex Media Server folder was still in the original volume on the RAID5 Seagates…

I’ve never used anything other the QNAP web interface and am at a loss when you all start talking doing stuff at command lines and so on… HELP???

[ChuckPa]

@ozbob65

Plex’s data is actually stored wherever the application is installed.

I created a little trick (the PlexData shared folder) to be both an illusion and a diagnostic aid.

Here’s the nuts and bolts.

1. PlexData is a shared folder, wherever you want it.
2. It never actually holds any of your metadata. It contains one thing - a symbolic link.
3. The symbolic link is enough to fool File Station into thinking it really exists.
4. The actual data is stored under /share/CACHEDEVx_DATA/.qpkg/PlexMediaServer/Library/Plex Media Server.

Here’s the magic:

1. You use QTS to 'Migrate to" and move the Plex app to the SSD.
2. QTS moves PMS, and all its metadata, to that SSD.
3. When App Center restarts Plex, I update the symbolic link to point to this new location.
4. I do this because I wanted to guarantee you always have access to PMS’s data (logs, etc) even if it fails to start at some point.

To make this all happen the way you want;

1. Decide which SSD-based “CACHEDEV_DATA” volume you want Plex to run on.
2. Use App Center and 'Migrate To" that volume.
3. After a few minutes, App Center will finish and restart Plex.
4. I’ll update the PlexData share.
5. Your media never moved, only the application
6. All will be well.
   

Let me show you on mine.

1. The view from App Center showing PMS and metadata are on CACHEDEV3_DATA (one of my SSD volumes)
   

   

2. The view from FileStation of the PlexData share. PlexData (the shared folder) lives on CACHEDEV1_DATA (my M.2 SSD)
   

   

3. I click that link and FileStation transparently jumps to CACHEDEV3_DATA where you see Plex’s internal app data storage.
   

   

4. If you look at this from the Linux level it becomes instantly obvious.

[/share/PlexData] # cd /share/CACHEDEV1_DATA/PlexData
[/share/CACHEDEV1_DATA/PlexData] # ls -la
total 12
drwxrwxrwx  2 admin administrators 4096 2021-01-16 11:12 ./
drwxrwxrwx 47 admin administrators 4096 2021-01-19 03:16 ../
lrwxrwxrwx  1 admin administrators   69 2021-01-16 11:12 Plex Media Server -> /share/CACHEDEV3_DATA/.qpkg/PlexMediaServer/Library/Plex Media Server/
[/share/CACHEDEV1_DATA/PlexData] # 

5. My HDD array, CACHEDEV2_DATA, holds all my media.

6. I have installed the PCIe SSD card in my TVS-1282 with a 1TB SSD
   

   

7. I then told QTS to enable the cache and have it cache only CACHEDEV2_DATA (the HDD array). There’s no need to cache another SSD (Plex itself).
   

   

Did that answer your question?

[ozbob65]

Thanks for answering back!

Righto, give me half an hour to absorb what you’ve just said and try migrating again.

[ozbob65]

Okay… migration done.

PlexData files are still showing in Vo.2… not Vol.4?!

@ChuckPa

If you look at this from the Linux level it becomes instantly obvious.

How?!!!???!

[ozbob65]

@ChuckPa

Plex’s data is actually stored wherever the application is installed. I created a little trick (the PlexData shared folder) to be both an illusion and a diagnostic aid. Here’s the nuts and bolts.

Soooo… this “technically” isn’t “real”, and can be moved anywhere using the “Transfer to Dedicated Volume”??

[ChuckPa]

Look at the storage usage.

147 Bytes?

Looks fake to me.

It’s nothing more than a Symbolic link to where the real data is stored.
QNAP’s “QPKG” mechanism puts the data underneath the App.
Getting access to that storage normally requires you to use the command line.

My trick simply made that location accessible via FileStation.

I suggest:

1. SSH into the box

2. cd /share/PlexData

3. ls -la

4. du -ms .

5. cd “Plex Media Server” (this follows the link)

6. pwd

7. ls -la

8. du -ms .

If you:

cd /share/CA*/.qpkg/Plex*

you’ll be in the directory where PMS is installed.

Now, by examining plex.sh you can see where plex.sh is installation directory independent in everything it does. It asks QTS for the QPKG_DIR (directory where this package is installed). It does all remaining work using that base variable.

[ozbob65]

Man, you’re super helpful, but… SSH? This is where I am completely LOST!!

I’ve Googled SSH, Telnet, Putty, etc; but am lost!

I’ve inputted those commands… properly, I think, and this is the output (but I’ve no idea what I’m looking at so you’ll be able to tell me what’s what and if we’re right. I’m in the process of Googling what they mean.

Now, when I go back into File Station, I see the following with heaps of data, as current as of today, in Vol.2 (original location)…

But nothing showing in Vol.4 (new location on M.2) except a folder I created to store art I’ve made…

Storage Pool 1 is the 8x 8tb drives comprising 3x volumes, and Storage Pool 2 is the 2x 500mb M.2’s that are in Raid 1.

So, I guess what I’m now wondering is… why is everything still going to Vol.2, or can I just delete that shared folder?!

[ChuckPa]

To verify the Migrate To worked.

1. SSH into the QNAP

2. getcfg -f /etc/config/qpkg.conf PlexMediaServer Install_Path

It will tell you where PMS is installed.

Consider the following. It demonstrates the illusion the PlexData share creates.

[/share/PlexData/Plex Media Server] # cd /share
[/share] # ls -la | grep PlexData
lrwxrwxrwx  1 admin administrators   23 2021-01-27 01:09 PlexData -> CACHEDEV1_DATA/PlexData/
[/share] # getcfg -f /etc/config/qpkg.conf PlexMediaServer Install_Path
/share/CACHEDEV3_DATA/.qpkg/PlexMediaServer
[/share] # 

PlexData is on CACHEDEV1
PlexMediaServer is on CACHEDEV3.

All of the metadata is stored in /share/CACHEDEV3_DATA/.qpkg/PlexMediaServer/Library/Plex Media Server

Verification of disk usage:

[/share] # du -ms /share/PlexData/
1	/share/PlexData/
[/share/PlexData] # ls -la /share/PlexData/
total 12
drwxrwxrwx  2 admin administrators 4096 2021-01-27 01:14 ./
drwxrwxrwx 47 admin administrators 4096 2021-01-27 02:36 ../
lrwxrwxrwx  1 admin administrators   69 2021-01-27 01:14 Plex Media Server -> /share/CACHEDEV3_DATA/.qpkg/PlexMediaServer/Library/Plex Media Server/
[/share] # du -ms /share/CACHEDEV3_DATA/.qpkg/PlexMediaServer/
62733	/share/CACHEDEV3_DATA/.qpkg/PlexMediaServer/
[/share] #