Cayars - Setup walk through and some tips and tricks


#2

[Followed]

Much of it is over my head, but I love to schmooze with the leading edge folks.


#3

Thanks for taking the time to post this.


#4

I've decided it's time to add a 1 TB SSD drive to my system exclusively for the MetaData and Transcoding folders.

I'm close to 500GB in the MetaData folder so a 1TB drive should hold me over for a bit. This should help speed things up a bit as sometimes I feel like the system is getting a bit sluggish.

On another note. I've been playing with HandBrake encoding for creating the "TC" files mentioned above. I wanted to see how much difference there was in both speed and in quality of files when using QuickSync to do the conversions.  Man was I surprised.  I used a couple of sample files for testing.

In a nutshell when converting MP4 1080p files with a bitrate of 10 to 12Mbit down to 720p 3.8Mbit I always got files that were under the 4Mbit limit I'm shooting for to use with the TC cache.

I used both QuickSync and conventional CPU to create similar files for comparison.

I honestly couldn't tell the difference between the two just watching them.  I did not try and stop to compare exact frames between the files as I really don't care about "exact quality".  I can say however that these "cached TC" files were of better quality then having PLEX transcode the files on the fly.  So it's an improvement in quality while reducing CPU resources.

The only computer I had available that had an integrated Intel GPU was my Surface Pro i5 2nd generation so that is what I used.

Using HIGH PROFILE with Picture set to 1280 (720 height) and a bitrate of 3800 specified I got:

14.5 fps using the software encoder with 97 to 99% CPU use

84.4 fps using same settings above BUT with H.264 (Intel QSV) selected to use QuickSync with 33 to 37% CPU use

Almost 6 times faster with plenty of CPU left to use for other things.

Strangely however is that Plex with the latest release or two has started doing something strange. It would previously all on it's own "merge" multiple movies together.  By that I mean if I had "Dave (1993).mp4" and also "Dave (1993) [TC720-4].mp4" somewhere in the library sections it would know it's the same movie. HOWEVER, now it's showing up as two different movies and I have to manually "merge" them together.  They are both found as the correct movie but have different icons etc.

One step forward, one step backward. :)

Once I get this sorted out I'll probably whip up a small util that can check the SQL database and take the last X movies added and create "cached" copies of these transcoded files. Probably instead of number of movies I'll use X GB/TB of space instead so we can for example designate 250GB, 2TB, etc and let the util manage it.

Carlo


#5

Thanks for posting this. This has given me plenty of ideas for my setup. I will take a much closer look at Media Center Master. As for the issues with CPU power and friends and family, I tend to persuade them to use Plex locally and use my own FTP server, but for those where FTP its above their head, I like your TC idea. For Live OTA TV I've started with HDHR Fling and a single HDHR Extend device. The Fling software has its own Roku channel, it's limited and still has ways to go since it's in beta, but been smooth so far (remote users could use this on the ROKU if you open the ports - but your limited to tuners). The guide isnt even on the same planet as WMC or remote potatoe, anyway, thanks  again for your write up.


#6

Thought I'd share a couple more things about CAST LISTINGS and TRAILERS:

In this first picture you will notice there is no trailer and the cast list doesn't have any of the key actors in the movie.

before.png

With a few changes to our settings we can have this which has a trailer and also shows all the key actors!

after.png

To get these results simple setup your agents like the following two screen shots.  Notice the order of Freebase and The Movie Database.

Also worth noticing if you click on the setting icon to the right of each agent (those that have it) I have every option enabled.  Adjust to your own preferences.

settings1.png

For trailers to work you will need to have a Plex Pass account and will want to make sure you are logged in before doing any scanning.

You will also want to make sure you select Freebase as the agent for a movie library

library.png

settings2.png


#7

After struggling for some time to try and organize NON movies or TV Shows I finally found a way to pretty easily organize some "personal" media".
NFL LIBRARY
NFLpng
 
NFL2png
NFL3png
 
LEARNING LIBRARY
Learning1png
Learning2png
Learning3png
 
SUMMARY
My choice of icons on this last set of images is only temporary and I'll soon be replacing them with some icons I'll make specifically for Plex. I just added them to grab some screen shots.  This "Learning" library is still a work in progress and I only have about 25% of my DVDs ripped and uploaded.  Just started it yesterday.
 
This way of organizing a library has made it easy to find media.  This is especially helpful on the xBox One as there is no search feature.  Trying to find an NFL game from last week WAS BLOODY HELL but now it's a piece of cake.
 
I know many people struggle with trying to make sense of putting their personal media up in Plex in a way that makes sense to navigate and I think these two different libraries show that this can be accomplished.  I'll be using this technique with a couple of other libraries such as personal family videos, other sports libraries, etc...
 
HOW TO DO THIS
The "Secret Sauce" to this setup is three things:
1) Addition of a new Scanner
2) Addition of a new Meta Agent
3) Naming your files properly so they "fit" a category in your new library.
 
Extended Personal Media Scanner
Go to the following thread and download the Extended Personal Media Scanner here: https://forums.plex.tv/topic/88982-rel-extended-personal-media-scanner/
The installation directions may be a bit "off" so this is what you want to do. Follow the instructions give to navigate to the "Scanners" folder.  Only don't put the contents of the zip here yet.
Create a sub-folder under Scanners called "Series" without the quotes.  Now find the .PY file in the zip and put this file in the Series folder.  You should end up with something like the following:
C:\Users\cayars\AppData\Local\Plex Media Server\Scanners\Series\Extended Personal Media Scanner.py
 
Extended Personal Media Shows Agent
Go to the following thread and download the Extended Personal Media Shows Agent here: https://forums.plex.tv/discussion/83440/rel-extended-personal-media-shows-agent
Follow the instructions exactly as posted in the first message so you end up with something like the following:
C:\Users\cayars\AppData\Local\Plex Media Server\Plug-ins\ExtendedPersonalMedia-Agent.bundle  <-- this folder/directory will now have both "Contents" and "Resources" in it.
 
You may want to stop/start Plex server at this point if the next section doesn't work.
 
FILE NAMING
You will see the scanner/meta-agent can take various formats of naming and be able to work with them.  I've found the S2014E01 format to work the best for me especially with sports.  If I used date based formats (2014-01-20) then I get results I may not want.  For example while the Super Bowl this year will be played in 2015 I don't want this game or the playoff, etc listed under the year 2015.  I want them to be with the rest of the 2014 season.  This of course is a personal preference.  The date based naming style cold be super using with family movies.  Just use the style that works for the library at hand.
 
I'll share some file names:
2012 Week 1 NFL
Week 01 - S2012E01 - 49ers at Packers.mp4
Week 01 - S2012E02 - Bengals at Raven.mp4
Week 01 - S2012E03 - Bills at Jets.mp4
 
2013 Super Bowl
Super Bowl - S2013E01 - Broncos at Seahawks.mp4
 
2014 Week 15
Week 15 - S2014E01 - Bengals at Browns.mp4
Week 15 - S2014E01 - Broncos at Chargers.mp4
Week 15 - S2014E02 - Cardinals at Rams.mp4
 
So the format is "category" - "year/episode" - "file name". Make sure each file has a different episode number.
 
For my Learning library:
Educator Math - S04E01 - Expressions and Formulas.mp4
Here I'm not using date in the series but just a series number. Course one will be season 01, course two will be season 02, etc.
 
ADDING A NEW LIBRARY
You add a new library as normal EXCEPT you start by adding a "TV SHOWS" library.  Add your folder(s) as normal and then click on "Advanced"
Here you will choose both a different SCANNER and a different AGENT like so (The two we just installed):
Add_Librarypng
Have fun and let me know how this works out for you!
 
Carlo


#8

Excellent series of posts, solved a couple of little niggles for me.

Suggest that you pull all this into a PDF and ask the Mods to pin it somewhere, keep up the good work, looking forward to the next installments.

Regards


#9

following this as im basically trying to do same thing as op. would be nice if you could share plex channels easily to allow live tv for sports. Hdhomerun view channel works pretty well for that but limited to your own account.


#10

Excellent series of posts, solved a couple of little niggles for me.
 
Suggest that you pull all this into a PDF and ask the Mods to pin it somewhere, keep up the good work, looking forward to the next installments.
 
Regards


I can certainly do something like that and go into more details about certain things like how to use Handbrake, how to create "TC" files, etc. But I wouldn't mind trying to figure out a few more useful tips to add to the thread. I'm also still planning on creating a little util to automate a few "tasks".

Off the top of my head I can't think of anything else to add at the moment. If anyone has any questions about Plex that sort of fits in this thread then by all means feel free to ask and I'll see what I can do to answer/put together a "tip" for it. 
 

following this as im basically trying to do same thing as op. would be nice if you could share plex channels easily to allow live tv for sports. Hdhomerun view channel works pretty well for that but limited to your own account.


Agreed completely. I wasn't even aware that the channels weren't shared until a release or two ago when the new "Home" features were added. At that point I was able to create different family accounts and limit what a couple of people were able to see. Then I was able to add more media that I didn't have on the system previously.

Up until that point I just had all devices (in my family) login with my plex pass account. After the switch I quickly noticed several channels that weren't available to anyone but me. :)

That was pretty much the reason why I didn't go with any channel based live tv solution since it wouldn't be available to anyone but me. Hopefully the Plex team listens to us customers who want this feature and we get it in an upcoming release!


#11

So im currently using the cetontv/hdhomerun xbmc plugin for users needing live remote tv. im going to try to see if I can somehow integrate it with the hdhomerun view channel icons as a fork maybe. this would allow remote passworded access and better plex independent x264 encoding. plus the ceton/hdhomerun backend can handle multiple hdhomeruns and has on demand tuning of the available tuners. only changes to channel in plex would be where the channels connect to. in this case remote ip api call. Each client would need it installed as their own channel though but most of my family has their own local plex server.


only current option to integrate livetv for other users not on master account and plex is plexbmc with xbmc plugins


#12

Just in case you aren't aware of the great web browser extension known as Transmogrify for Plex.

It is a Chrome and Firefox extension that adds many cool features to the Plex web interface. (I prefer it on Chrome)

Transmogrify for Plex thread is located here: https://forums.plex.tv/topic/99209-transmogrify-for-plex-a-browser-extension-that-adds-features-to-plexweb/

I'll use a few pictures and descriptions directly from that thread to illustrate what this extension can do.

Features include:
Adding a link to view the trailer for movies within Plex/Web
Adding a link to view the letterboxd page for movies
Adding a link to view the themoviedb page for movies
Adding a link to view the tvdb page for tv shows
Adding a link to view the IMDB page for movies, and displays ratings
Adding a link to view the trakt page for movies and tv shows, and display user ratings
Adding a link to view the rotten tomatoes page for movies, and display critics and audience ratings
Adding a random tv show/movie picker
Adding a missing seasons and episodes view
Adding a Can I Stream it? widget to movie pages
Adding a movie/tv show filter for the main dashboard
Adding an actor profile for cast members on movie pages
Adding a server statistics page
 

61lCbn9.jpg

You have a watch trailer link under the movie.  This plug-in finds trailers when the "normal" one does not.  Also doesn't require a plex pass.

Note all the "Extras" at the bottom of the page.

Check out the right side of the page for additional ratings

zCmRb39.jpg

This shows the very cool feature where an actor's profile is shown just by hovering your mouse over the actor name.

6CKE3Bl.jpg

In this picture it's quite easy to see the browser extension has determined what TV shows are missing and fills in some information.  You can also click on them for additional info.

hM5amp2.png

Only a small showing of what the statistics page will show you about your collection of Movies and TV Shows.

So check out the thread and install this web browser plug-in/extension. You won't regret it.

Carlo


#13

I edited the first post and embedded a couple of videos on some of the applications used.


#14

WOW... Got some homework to do.

Very cool.


#15

I previously mentioned I was going to order a 1 TB SSD drive to install in my Plex server as I couldn't find one local.  I was then going to move the meta data directory over to this.

This has been bugging me because as you probably know my meta data directory is close to 500 GB in size.  90% of it is probably the index files. I personally think it's stupid to have these index files "parked" on a fast disk like an SSD but with the current setup/layout/architecture of Plex it's more or less an all or nothing proposition when it comes to the meta-data directory location.

Right?  Well not so fast, so says Carlo. :)

My Plex server runs on Windows 2012 R2 but what I'm about to share will work just as well with Windows 8 (preferably with 8.1 which is a free update).

What I'm getting at is something called "Storage Spaces".  If done/setup correctly it's the best of a RAID setup, freenas, drobo and unraid all in one and part of the operating system.

For those who might have heard of Storage Space and dismissed it due to stuff they have read on Linux biased threads...  It isn't so.  It's not EVERYTHING a freenas box can be, it's not everything a Drobo can be, it's not everything an unraid server is.  But is has the best qualities of each all wrapped up in one and is part of the operating system.

I work in a mixed environment of Linux and Windows on the server side so I try to follow all things (well what I can) related to both especially when it comes to storage because it a "personal" thing. :)
Storage spaces get knocked on many Linux threads by those who have never worked with it or really don't know how it works (plus many just don't like Windows or are biased).  I try and keep an open mind as I use both and test things out for myself.  All I'll say is that it's fast, reliable and pretty easy to expand on when you understand how it works.

Ironically, I'm not using storage spaces for Plex (yet), BUT I just ordered several WD Red 6 TB drives and a 1TB Samsung SSD drive for just this purpose.  I do however have a bunch of servers setup using this at work and it's faster then our SANs, DATs and RAIDed server machines.  Storage spaces has also passed every Disaster Recover test we threw at it when other technologies did not (kind of surprised me).

Storage Spaces can be setup using just a GUI or can be controlled via Powerscript.  Storage Spaces can have the best of unraind, freenas and drobo in one product more or less, especially for our purposes.  You can use thin or thick provisioning, data duplication (worthless for video files generally) and can choose to format as NTFS or the newer ReFS.

Where it gets interesting and back to the point I started with.  You can setup a tier of pure HDDs and another tier of pure SSD drives.  Then when you create a virtual disk (ie Drive F,G, whatever) you can pick and choose among the tiers you've setup (ie 30% SSD and 70% HDD). The operating system will then monitor and arrange your files so the most often used program reside on the SSD tier.  It will move them around as a maintenance process in the background. You can also manually "pin" files to one or the other tier.  In the world of Plex for example the database files would get put on the SSDs in this example and much of the meta items would also as they are constantly being viewed and used.  But things like the index files (part of the meta database) would almost surely still reside on the HDDs as they only get used when a person is actually viewing a file.  Many of these index files may never get used or only once or twice a year if someone watched the movie or TV show associated with it.

So using "tiers" when setting up the storage pools would intelligently "split" the meta database up on the proper tiers based on frequency of usage!

You can also set aside just pure SSD space as a drive letter which would be perfect for the transcode directory.

I'm attaching a YouTube video which kind of covers some of what I just talked about and does cover SSD and HDD tiers.  The guy in this video uses Power Script to setup everything but it can just as easily be done via the GUI.  I myself like to use Power Script for these types of tasks as I can save it on a master flash drive to serve as "documentation" of how the storage array was setup in case I need to refer back to it anytime down the road.

https://www.youtube.com/watch?v=https://www.youtube.com/watch?v=x8KlY-aP9oE

Here's another done in a virtual environemtn show how to use the GUI to do everything and also how to add a new disk to the storage pool after it's operational.

https://www.youtube.com/watch?v=https://www.youtube.com/watch?v=oF1VoIftDMs

Carlo

PS: Question for you guys: Are some of these posts too technical in this thread or is this type of info useful?


#16

I find the posts interesting, informative and not too technical but then again im not scared of a terminal window.


Why use indexing? seems like a waste of storage space. what does it give you for the space used? I have it turned off and I have barely any metadata on my ssd. As to storage spaces, what kind of parity options does it have? Im more of a linux guy but a good tool is a good tool


I read someone that ran tests and the testing showed it made no difference if the plex database was on an ssd vs spinning disk as far as measured client performance goes. its somewhere in the forum


#17

Indexes are great if you or any of your friends own a Roku box because any fast forwarding or reversing will have a small box on screen that shows you exactly where you are in the movie. This also works in Android but is less useful as you can easily just "jump" to any section of the movie unlike most hardware devices that require the use of FF/RW.  Although borked right now it's pretty clear that the devs have the intention of using the index files on the xBox platforms also.  Right now you get the big box that pops up when FF or RW but it's just all black.

Index files are also used from the web interface when you use the "Now Playing" feature.

I have read "conflicting" reports about the use of an SSD for Plex's meta-data directory in the forums.  However, from everything I know about software and hardware architecture and enterprise system storage provisioning and diagnostic/performance tuning combined with what I've analyzed and seen on my own system...  What I think this will achieve is faster load times for clients when they enter a library and move around.  Right now I see a slowly degrading pause which I think is due to the fact that I have around 70K meta objects which is primarily to all the music albums I've added.  6.5K albums x number of tracks = lots of meta records.

I haven't really looked into the music side of Plex much in regards to what is and what isn't stored and where so the last part of the above could be wrong but I doubt it.

So while "others" may not report any speed ups this goes in the "face" of what I predict will happen.

We will know in a week or two. :)

I'm going to play a bit with the system testing different "layout" strategies. Where appropriate I'll also test IOPS

Right off the bat the first thing I want to test is just adding the SSD as a normal drive and setting it up to use for plex meta-data and for transcoding

Next will be a storage spaces with no parity, mirror, parity with only HDD tested with both NTFS and ReFS to see what IOPS I can get

I'll play with both "thin" and "thick" provisioning of the above.

Then I'll play with storage spaces using "tiered" SSD and HDD combined for meta-data use using both NTFS and ReFS with no parity, mirrow, parity and I'll also check IOPS again in these configurations

For this set of tests only "thick" provisioning can be used of course. No "thin" provisioning when using "tiered" storage.

I'm not going to be in rush to start using the space right away but want to explore different methods of setup and use for a Plex setup. I'll probably just play for a week or two unless I strike gold right away.


#18

I use sickbeard and it has worked pretty flawlessly for me.  I had thought about trying sonarr but it looks to be very similar to sickbeard in what it can do.  

There is one feature I would love for one of these programs to do (not sure if mcm can do it) but that would be to download content from multiple sources (usenet and torrent).  Usenet has most things but there are a few that don't get tracked that I would love for it to "fail over" to a torrent.


#19

You can do this already with the programs you have. :)  When I was running SB I did exactly that.


#20

You can do this already with the programs you have. :)  When I was running SB I did exactly that.

Don't you need to run a branch/fork of Sickbeard other than the standard one that has torrent integration in it beyond just monitoring a black-hole folder to have it check both USENET and torrents for a particular media file?


#21

yes I think. get on the sickrage fork that is actively developed