Welcome to our forums! Please take a few moments to read through our Community Guidelines (also conveniently linked in the header at the top of each page). There, you'll find guidelines on conduct, tips on getting the help you may be searching for, and more!

Migrating from FreeNAS 9 plugin to FreeNAS Corral Docker

gbooker02gbooker02 Posts: 509 Plex Employee
edited March 18 in FreeBSD

Since I've seen a request for how to transition to the new FreeNAS docker containers with Plex and others are going to want to know the same, I figured I'd give a quick writeup on how to do this.

As a first note, I would highly discourage anyone from upgrading yet especially those who uses encrypted volumes. I initially ran through these tests using encrypted volumes and after the upgrade to Corral, the migration stopped until I unlocked the volumes (which is understandable). After doing so, the middleware crashed and never recovered. The system at this point is next to useless so I had to force a reboot. Upon it coming up again, the encrypted volumes were migrated and I could unlock them in the UI, but the migration could not be resumed as advertised; it was stuck in a partially migrated state. So I tried again without encryption. This time Corral failed to migrate the users (while it did migrate the groups). Clearly Corral is not ready yet.

Furthermore, if you rely on ACLs for permission access, these ACLs will not be honored in the container (known issue). I tested it with the media being owned by a different user than the plex user, and the plex user was granted access solely through the ACLs (those outside the user/group did not have access through unix permissions). You'll have to correct permissions on your media to work without ACLs in order for them to work within docker.

On to what did work: First, I had FreeNAS 9.10 installed with the main dataset called tank. I created a standard plugin (not a normal jail) since those not using this method should be able to infer the steps they need to change. I have my media exposed to the jail at /mymedia.

I then upgrade to FreeNAS Corral. After it migrated, I opened a shell (not Corral's command line, a bash shell) since it makes some steps much easier. In the command line, we are going to get the user id that was used within the plugin/jail for Plex (only execute the part following the % on a line that starts with %):

% sudo ls -ldn "/mnt/tank/jails/plexmediaserver_1/var/db/plexdata/Plex Media Server"
drwxr-xr-x  9 972  972  10 Mar 17 11:15 /mnt/tank/jails/plexmediaserver_1/var/db/plexdata/Plex Media Server

In the above, you'll have to change tank to match the volume name you used and you may need to change plexmediaserver_1 to match your jail name. See this page for more details on where to find this data. Here the user id is 972 and the group id is 972 (the third and fourth columns); we'll need that later.

In the Web UI, create a new dataset to store the Plex metadata. I created mine at tank/PlexMetadata but the name scheme and location is up to you. Now we need to create the skeleton directory structure and correct permissions. Again, change tank/PlexMetadata below to the name you used. The 972:972 is the user id and group id obtained earlier

% sudo mkdir -p "/mnt/tank/PlexMetadata/Library/Application Support/"
% sudo chown -R 972:972 /mnt/tank/PlexMetadata

Now copy the data over:

% sudo rsync -aP "/mnt/tank/jails/plexmediaserver_1/var/db/plexdata/Plex Media Server" "/mnt/tank/PlexMetadata/Library/Application Support"

Note: ensure that you don't have any trailing /s in either argument to the command. If such /s are present, it changes the meaning of the command and will not result in the correct directory structure.
This process will take a bit, depending on the size of your database. The command above is resumable, so you can kill it and restart at any time. Now let's go create the container.

In the web UI, go to Docker -> Containers and create a container.

  • You may select either plex or plexpass depending on whether you want Plex Pass versions or not. If your account does not have Plex Pass, you will not get Plex Pass versions either way.
  • Give the container a name. This is up to you.
  • Leave the ports alone
  • In Volumes, browse to the dataset you created above for /config
  • If you want transcode temps to be stored outside the container, you can set the location for /transcode otherwise you can remove that volume.
  • You should delete/modify the /data mount to match the mounts you had for your jail in FN 9.10. In my case I had tank/media mounted at /mymedia
  • Set the timezone for TZ (See this page for complete list)
  • Since we are migrating an existing installation, you can remove ALLOWED_NETWORKS and PLEX_CLAIM
  • Set PLEX_UID and PLEX_GID to the user id and group id above.
  • Expand Network Settings
  • If you change the network mode to Bridged, you then setup the IP Address or DHCP and generate a MAC address. Additionally, you can remove the ADVERTISE_IP setting in this mode.
  • If you leave the network mode on NAT, then you should set the ADVERTISE_IP to http://IpOfYourFreeNASBox:32400
  • Save.

If everything went well, you should see your shinny server at https://plex.tv/web (assuming you had previously logged it in). If you already have this page open, you may have to reload it so that the web app finds the server's new IP address.

As a final note, the docker VM is by default only given 1 core. You may want to change this if you do any transcoding. It is in VMs -> docker_host_0 -> CPU Cores. You'll have to manually reboot the VM (button at the top of this column) for this to take effect.

«1

Comments

  • cdelashmuttcdelashmutt Posts: 2 Plex Pass

    You are a god-like being, gbooker02.

    This guide worked perfectly for me! THANK YOU!

  • mysticpetemysticpete Posts: 76 Plex Pass

    @gbooker02 Quick Q: is there a way to setup Plex when you have a clean install of Corral and have imported unencrypted Volume/s from a freenas 9.10.x?

    Thanks in advance

  • gbooker02gbooker02 Posts: 509 Plex Employee

    @mysticpete said:
    Quick Q: is there a way to setup Plex when you have a clean install of Corral and have imported unencrypted Volume/s from a freenas 9.10.x?

    What you are describing is very similar to the scenario I outlined above. Your case just didn't go through the migration step that FreeNAS performs. AFAIK, this migration does nothing as far as this procedure is concerned. You should be able to follow the above steps in the same manner.

  • mysticpetemysticpete Posts: 76 Plex Pass

    @gbooker02 Thanks, OK I'll give it a go...

  • mysticpetemysticpete Posts: 76 Plex Pass
    edited March 18

    @mysticpete said:
    @gbooker02 Thanks, OK I'll give it a go...

    @gbooker02 OK (very pleased) got it running apart from one thing, I don't see the libraries when I open the web gui, is it probably a path error on my part?

  • angelus249angelus249 Posts: 5 Plex Pass
    edited March 18

    Well, thanks for the tutorial. I did everything as explained, but I run into the same issues as mysticpete does. I can get the plex server running, but I can't seem to get my database imported. Not sure what to do from here on out...

    Anyone got an idea?

  • angelus249angelus249 Posts: 5 Plex Pass
    edited March 18

    Alright, problem solved. A little slash "/" at the end of the source/destination folder was the culprit. When rsyncing my db, I worked with tab to complete the path, which ended up in

    % rsync -aP "/mnt/tank/jails/plexmediaserver_1/var/db/plexdata/Plex Media Server/" "/mnt/tank/PlexMetadata/Library/Application Support/"
    

    Instead you're required to commit the following command - also as stated above :) Otherwise only the content of the "Plex Media Server" folder is copied into the new folder but not the folder itself.

    % rsync -aP "/mnt/tank/jails/plexmediaserver_1/var/db/plexdata/Plex Media Server" "/mnt/tank/PlexMetadata/Library/Application Support"
    

    Easy to miss.

    Thanks for the tutorial. Works like a charm now.

    @mysticpete fyi

  • gbooker02gbooker02 Posts: 509 Plex Employee

    I added a note to the tutorial to watch for trailing /s in the command.

    BTW, zsh is much better about not creating trailing /s in tab completion ;)

  • surajravisurajravi Posts: 2

    @gbooker02 said:

    As a first note, I would highly discourage anyone from upgrading yet especially those who uses encrypted volumes. I initially ran through these tests using encrypted volumes and after the upgrade to Corral, the migration stopped until I unlocked the volumes (which is understandable). After doing so, the middleware crashed and never recovered. The system at this point is next to useless so I had to force a reboot. Upon it coming up again, the encrypted volumes were migrated and I could unlock them in the UI, but the migration could not be resumed as advertised; it was stuck in a partially migrated state. So I tried again without encryption. This time Corral failed to migrate the users (while it did migrate the groups). Clearly Corral is not ready yet.

    Hi gbooker, this is Suraj here, the idiot who wrote that migration which crashed for you :wink:

    Firstly, I am really sorry that you faced this, I tried migration with encrypted volumes (both passphrase-less as well as passphrased ones) and everything went smoothly for me. I wrote a small howto and FAQ for migrating to freenas corral (which I presume you already have read but posting here for the everyone else https://wiki.freenas.org/index.php/Migrating_from_9.X_to_FreeNAS_Corral)

    Now from what I understand from your post encrypted volumes migrated just fine and it was the rest of the migration task that failed for you (correct me if I am wrong here). Migrating to a whole new infrastruce is inherintly problematic and since there are so many starting points (freenas 9.10.x, 9.3.1, 9.x) that I cannot single handedly gurantee every single scenario is taken care of from the very beginning. This is where the power of the community trying it out and filing tickets comes into play. It is in that same spirit that I urge you to file a ticket on bugs.freenas.org with your debug file (GUI --> System --> Debug --> Download Debug File) and assign it to me (Suraj Ravichandran).

    I thank you for your time and effort in trying this out so early and paving the way with such informative tutorials for the rest of the people.

  • gbooker02gbooker02 Posts: 509 Plex Employee

    @surajravi

    First of all, my issues were with test VMs, not production machines. My production FN is going to stay on 9.10 until at least Corral SU1 (a determination I made before my above experiences). These VMs were created solely for testing out the migration and writing of the above tutorial.

    I did follow that wiki page for migrating an encrypted volume which I found immediately after the migration task stopped awaiting the volume unlock. It was after the second volume unlock that the webUI as well as the CLI complained about timeouts from the middleware and I restarted the VM. After that, the migration could not continue. The migration not continuing I did not report because it is already filed: https://bugs.freenas.org/issues/22255 I did try another test later on and it did successfully migrate. Unfortunately I didn't save the first VM (nor the successful one), but there were likely very few differences if any between them. I may have not setup the replication task to localhost on the successful VM but I'm not certain if that's the difference. As per a third VM (the one without encryption) that failed to migrate the user, I have no idea what happened there as the successful encryption VM did successfully migrate the user. All three of these VMs started with a virgin install of 9.10.2-U2 where I did some initial setup, installed Plex, copied some media, and then update to Corral so as to generate the tutorial the above.

    If I should try move VMs, I'll be sure to snapshot them prior to Corral upgrade and if I can reproduce the issues, perhaps I can get these (ESXi) VMs to you.

  • surajravisurajravi Posts: 2

    @gbooker02 as for https://bugs.freenas.org/issues/22255, it is already resolved (it mostly was a user configuration issue) and I got him the fix that was needed to rerun the migration successfully.
    As far as this is concerned: "
    It was after the second volume unlock that the webUI as well as the CLI complained about timeouts from the middleware and I restarted the VM."

    Here is what was most likely happening, your second volume unlock triggered a snapshot cache generation sequence (which takes a decent amount of time depending on the number of snapshots you have on that volume, during which cpu usage also spikes (which is normal). Things may have appeared "hung" but were not so. We are working on some progress reporting to indicate the same and should come with SU1. When you restarted the system in this fragile state, the migration and someother things went with it! For next time, I would recommend that please wait till the system becomes responsinve (for example a 30-45 minute cpu spike and heavy load was experienced on my freenas mini when I imported my volume which had 20K snapshots)

    The moving the vm's part would be a little too excessive, but if you ever get to reproduce this then I would very much like a debug file, you can send it over to me directly (suraj__at_ixsystems.com replace at with @).

    Again, thanks for trying this out so early in the cycle. Appreciate your effort

  • gbooker02gbooker02 Posts: 509 Plex Employee

    @surajravi said:
    Here is what was most likely happening, your second volume unlock triggered a snapshot cache generation sequence (which takes a decent amount of time depending on the number of snapshots you have on that volume, during which cpu usage also spikes (which is normal). Things may have appeared "hung" but were not so. We are working on some progress reporting to indicate the same and should come with SU1. When you restarted the system in this fragile state, the migration and someother things went with it! For next time, I would recommend that please wait till the system becomes responsinve (for example a 30-45 minute cpu spike and heavy load was experienced on my freenas mini when I imported my volume which had 20K snapshots)

    If I can reproduce I'll send you the debug file, but I doubt snapshot enumeration is at fault. There were two volumes: backup had 2 datasets of which 1 had 1 snapshot, tank had ~6 (3 + what jails produces) datasets with a combined total of 2 snapshots. I left the system running for 5 minutes before giving up.

    It is worth noting the snapshot enumeration and it not being displayed as I have ~12k on my production machine. I'll keep that in mind whenever I migrate it.

  • qBazqBaz Posts: 6 Plex Pass

    Thanks for this, btw -- instructions worked beautifully.

  • voltevolte Posts: 1 Plex Pass

    Thank you so much. This is excellent. FreeNAS Corral (in conjunction with Docker), AppleTV 4, and Plex, makes for an AMAZING experience.

  • brsullivanbrsullivan Posts: 8 Plex Pass
    edited March 29

    Had it working for a while... I would change the instruction to say point config to /PlexMetadata, I had it pointed to Application Support and it created two more subfolders, no biggie.

    After a reboot I'm now running into another issue with the docker/container. "Cannot start VM: Subnet 192.168.10.0/24 on interface igb0 overlaps with VM NAT subnet". Figured that out too, you can't set the VM IP to the same subnet as your container. If someone could point me to a primer I'd love to read more about this. I'm confused as to why there is a VM at all. I thought docker did away with VMs entirely.

    So system is finally up and running... another MAJOR MAJOR point to note. When you create the map points my install was looking for everything under /mnt/movies. So you have to put the /mnt string in order for it to see it. I was just creating entries mapped to /movies. Also note these mappings are case sensitive. The easiest way to see if you have all your mount points is to just create the container and then launch plex. It will tell you if a file is unavailable and you can click "unavailable" and it will show you the expected mount point. On a side note the container has some weird behavior. If you create a mount point and then later delete it, it will just recreate it and list it as a "VM" in the container. Not sure what that's all about.

  • mysticpetemysticpete Posts: 76 Plex Pass

    Its probably worth noting that you have to create the Docker Before you create the Container.

  • 242spot242spot Posts: 1 Plex Pass

    Thanks so much for this, it worked perfect for me, even if it did take me two days to remember enough about shell commands to be able to put this into practice.

  • gamecat69gamecat69 Posts: 5 ✭✭

    Worked perfectly, thank you!

  • DbldDbld Posts: 1 Plex Pass

    Hi @gbooker02 , thx for the tut, worked great for the most part. One thing that i'm not getting right, in the docker config you say "leave the ports alone".
    I did, however after completing everything and container starts there are no open ports to connect with at least according to nmap.

    What could i be doing wrong ?

  • gbooker02gbooker02 Posts: 509 Plex Employee

    @Dbld said:
    I did, however after completing everything and container starts there are no open ports to connect with at least according to nmap.

    If you used Bridged networking, you are likely hitting the wrong IP address. The IP address of the container in this case is NOT the IP address of your FreeNAS box.

«1
Sign In or Register to comment.