Importing to PMS

General Discussions
1

I’m running PMS in MacOS Sierra on a dedicated Mac mini Server and have formerly been using iTunes to supply media to the whole house. I now think Plex can do a better job and am trying to effect the transition. But some issues.

First of all, iTunes has been (and will be for the foreseeable future) used to tag the media files. Everything I have in iTunes is correctly identified and iTunes of course ensures that the embedded tags are kept in sync with what data has been entered. I have it set to ‘Keep Media folder organised’ which means it names the files and places them in its predetermined structure which is in fact very sensible folder structure of varying depth, according to the media type being stored. But this cannot be changed. iTunes does what it does and the user just has to live with that. On the whole this is not a problem as it gets it right, but there are glaring holes in this eco system, most especially, no actual server or ability to be run as one. This where Plex comes in.

PMS is all set up and I created a Movies library and pointed it at iTunes’ Movies folder (most media type names are the same in both products). It was surprisingly slow, but it did eventually finish and has the exact same number of items as iTunes reports. So without checking further, it looks like everything is good.

I then set up another movies type Library and pointed at a folder of non-commercial videos. None of these would be found on IMDB etc. Not many in total, so didn’t take long and then I had a list of all the videos with nice poster artwork showing, presumably frames taken from the videos themselves and not the same as iTunes displays, but no problem, all is good.

After that I created a TV Shows type Library which basically worked. Some issues to sort, but at least it obviously found all the files, even at different levels within the folder to which the Library was pointed.

Then I wanted to add some other videos of TV Show type, but for which I don’t expect any metadata to be found on-line, so I created another TV-Show type Library and pointed it at the appropriate folder. One of the items is in a 3 level heirarchy of Show/Season/Episodes, the other 3 of them are simply Show/episode(s). As a matter of interest, iTunes will not bother with the Season level if there is no Season specified. Hence its TV Shows media folder can contain a mix of folder structures with different levels. But once scanning was complete, this library was empty and remains so.

At first, I suspected that varying levels of folder structure cannot exist in the same Library, but the main TV Shows Library does contain such, as created by iTunes due to the existence or not of any seasons and they have been imported ok by Plex. But this other TV Shows type Library fails to find ANYTHING.

So this is the main problem. I don’t care a rat’s ar** what the folder structure is or is not, nor what the file is called. It WILL be something sensible created by iTunes. The fact is that all the important and correct information is contained within the files’ embedded tags. So the first thing I want Plex to do is find ANY suitable files (video in this case and all either MP4 or M4V or MKV) at ANY level in the folder to which it has been pointed, read the embedded tags and ‘import’ that into its database and then add anything else that it finds on the 'Net and that doesn’t conflict with the embedded data (more on that later). Yes I know about Agents etc and ‘Personal Media Shows’ is the selected Agent and ‘Local Media Assets (TV)’ is selected and at the top of the list. I’ve deleted the Library (easy decision as there’s nothing in it) and created it again, but whereas when creating the main Movies and TV Shows Libraries, it reported each file as it checked. For this Library I see no such checking the screen just immediately changes to an empty Library display, i.e. largely blank.

So the big question here is why this Library is not being created correctly. As far as I can tell, it should find all the MP4 video files and add them to its database using the embedded metadata (all I actually need), whatever the folder structure and naming. I don’t want it to concern itself with folder and file names. All the metadata required is embedded in the actual files and should be used. Isn’t that what ’ ‘Local Media Assets (TV)’ is supposed to do. Why is Plex struggling with something so simple. iTunes may have its issues, but any fully tagged media file I throw at it gets perfectly imported with all that data then in its database. Can anyone please explain why Plex seems incapable of doing this.

2

The answer is very easy. Plex has a naming and directory structure convention. You don’t want to follow it, so you will have issues with your media being cataloged. It’s not Plex being incapable.

3

Well that’s not entirely true. Filenames can be anything, but the embedded metadata is the EXACT data you want Plex to use. So why have to abide by some arcane file naming convention when all the required data is already available. It’s not a question of me not wanting to follow it. I shouldn’t have to follow it. Not only that, but this media is being shared with iTunes that follows a more logical structure with the actual metadata all in the file itself and I have no control over the file naming. Wouldn’t normally need to since Apple actually got it basically right. So sorry, I have to completely disagree with you as I think it is most definitely a lack of ability on the part of Plex.

However, I’m not actually complaining about Plex, just trying to understand all the nuances of how media gets imported. In particular how some TV Shows with the same folder structure apparently import OK, yet in this other library of the same type, they do not. If you want to help:-

What does ‘Local Media Assets (TV)’ mean if it doesn’t use the file’s metadata? Using embedded metadata should be the FIRST step Plex takes and only if that’s not available resort to file and folder names and only then add on to anything found so far with information from the 'Net.

Why does it not even list the video files it can obviously find, just because it cannot match them up to some Internet data. If it finds a video file and cannot find any additional data on the 'Net, it should still use the embedded metadata. If it cannot even do that, then list the file with whatever name it does have and let the user then correct it as appropriate. Simply not listing it at all is dumb and I don’t believe Plex is that dumb. Hence I’m asking for some help in identifying why it is simply not finding anything in this library.

4

Also, I would think it advantageous for PMS to be able to import ANY media structure created by iTunes which I would suspect is probably the most widely used media controller/player. Love it or hate it (or just indifferent) there are a huge number of people using it.

Apple have benefited enormously by making it as easy as possible to switch from Windows to Mac and Plex would undoubtedly similarly benefit by making an easy import from iTunes. All the library types basically match up and ALL the required data will be embedded in the files. It should be a simple, one step process with NO holes. As long as the data has been filled in in iTunes, there should be NO gaps once PMS has ‘imported’ it. None at all, because every file will contain all the required metadata. If Plex can add to it, great, a good reason for the switch, but there is NO need for PMS to simply ignore any file.

In fact, it wouldn’t just be for iTunes. Making use of embedded tags means Plex would be able to make better use of media from any source.

So I think Plex is really missing a trick here, but if I can fully understand what Plex actually uses when importing and how, I can hopefully allow them to co-exist.

5

The more I think about this, the dumber and so more unlikely it seems that Plex cannot use embedded tag data. To ignore (and not even list) any video file for which it cannot find a match anywhere on the 'Net would be very stupid indeed. Any video/movie that has no on-line reference has to be usable in Plex and the folder and file naming is woefully inadequate. There is simply NO way to convey all the information that can be carried in tags, just using the file and folder names. So, is Plex then unable to display any such data? Force the user to enter it all again so it can be stored in the PMS database? Well stupid as that would be, it is impossible if Plex doesn’t even list the file.

So I can only come to the conclusion that I must be doing something wrong that is not allowing Plex to read the embedded tag data. Anyone able to offer any help here?

6

Here is the last few lines from the scanner log:-

Jan 03, 2017 18:35:00.534 [0x7fffa228e3c0] DEBUG - Directory had 40 files, database had 0 files, can’t skip.
Jan 03, 2017 18:35:00.851 [0x7fffa228e3c0] DEBUG - Removing 0 media items that were left.
Jan 03, 2017 18:35:00.851 [0x7fffa228e3c0] DEBUG - Removing 0 directories that were left.
Jan 03, 2017 18:35:00.855 [0x7fffa228e3c0] DEBUG - Refreshing section 7 of type: 2

For the other directories it scanned there was the same entry. It finds the files, says nothing in the database and can’t skip, then skips them and as can be seen, at the end does nothing. So why is it simply doing nothing with the files? It finds them but seems to make no attempt to do anything with them.

Anyone any ideas?

7

Well as has been pointed out, naming conventions are important to Plex. If you don’t follow them, you are going to get mixed results at best, absolutely nothing, such as you have at worst…

So, I suggest you start here: https://support.plex.tv/hc/en-us/articles/200220687-Naming-Series-Season-Based-TV-Shows and work through this an any other links regarding naming you might find in the support documents. (There are a bunch of them, so I just gave you the starting point.)

Personally, I have some issues with photography videos, but that’s mismatching most of the time. And this usually comes down to names as well… I generally strip out all of the metadata in my media as part of the process I use to get it Plex Ready.

8

The only suggestions are that naming is important, but I find it hard to believe that naming is the ONLY way Plex obtains the crucial information needed to identify the video. If the file is devoid of any tags, then I agree, naming is crucial, but the real, intended and ‘designed into the file format’ way to identify video (and music) files is to use the embedded tags as these have greater capacity and ability to store a breadth of information totally impossible just within folder and file names. It is THE correct way to do it. Only if tags are not available for whatever reason should naming be used as it’s just the fallback. It should NOT be used as the sole source of metadata. I’ve said it before, that is just so dumb I’m struggling to accept that Plex is so limited in this way.

It seems to me that the Plex developers have put in an inordinate amount of time to enable you to throw any old piece of movie ■■■■ at it and it will do its best to figure out what it is. This is laudable, but appears to be at the expense of being to provide it with fully identified and tagged media as it completely ignores all that valuable tag data and only wants to use folder and file names which if not completely how it wants can mean the file is totally ignored. This is monstrously stupid. I realise there are Plex devotees who will not want to hear that, but as an objective observer of this process, I cannot see it any other way.

I am still then confused by what ‘Local Media Assets’ is supposed to mean. I now see that for Music libraries, one can specify to use ‘embedded data’ as well. This would imply that ‘Local Media Assets’ therefore does not mean to use embedded data. This I did not realise, so apart from the stupidity of not being able to specify this for video libraries, what does ‘Local Media Assets’ actually mean. Nothing useful it would appear, but I am trying to get an overall grasp of exactly how Plex imports media so I can attempt to use it in an acceptable way.

Having to rename folders and files just to tell Plex what it can already extract from the files’ tags is not only immensely annoying and time-wasting, but presents a problem when trying to share media with any other media system. In my case this is iTunes, but whatever is being used, any file and folder naming scheme will be basically unique to that system. In my case, everything is logically organised and structured by iTunes. Set up the metadata as you want and iTunes can maintain this ordered structure for you and it does do it in a VERY logical way. I cannot fault it. But it is not the same as Plex wants, which makes it basically impossible to share media - unless the systems in question use the embedded tags as the real way to attach metadata to the media files. Whatever its other shortcomings, iTunes does this very well but it appears Plex does not. Eschewing this standard way of identifying media files with virtually as much metadata as you want and instead crippling itself by relying solely on folder and file names.

If this is true, I am utterly dumbfounded by the lack of sense of the developers. So much so I still cannot quite believe it. So 2 final questions:-

• Is it true, that PMS does not and CANNOT use embedded tag data when importing video files?

• In which case, what does ‘Local Media Assets’ mean?

9

read this -> https://support.plex.tv/hc/en-us/categories/200028098-Media-Preparation

You seem to have a very high percentage of whinging posts where you act all dumbfounded at how Plex works - there are a lot of guides on the support site, maybe read some, you might save yourself a heart attack and some sore finger tips.

Tags would be nice if there was a single standard for tagging file, and existing methods didn’t conflict with each other, this has been discussed at length multiple times before if you care to search.

I would suggest that maybe Plex just isn’t the system for you.

10

Well instead of posting useless remarks like that, perhaps you could answer the actual questions I asked. That would be helpful and assist me in understanding better how Plex works.

I already know about the naming schemes. but since most of my TV Shows imported OK even when apparently NOT conforming, they are obviously not absolute rules. I am trying to get to the bottom of exactly how it identifies files on import. I asked specific questions, you didn’t answer anything so why did you bother to post.

11

@UKenGB said:
Well instead of posting useless remarks like that, perhaps you could answer the actual questions I asked. That would be helpful and assist me in understanding better how Plex works.

I already know about the naming schemes. but since most of my TV Shows imported OK even when apparently NOT conforming, they are obviously not absolute rules. I am trying to get to the bottom of exactly how it identifies files on import. I asked specific questions, you didn’t answer anything so why did you bother to post.

The answer to both of your “final” questions are answered in the link, there are entire sections on the local media assets agents - maybe you should read though it again. I can’t see any other questions in your post.

12

And I quote:-

“Local Media Assets” is an Agent source that loads local media files or embedded metadata for a series."

This is how I understood it from first reading it, yet no-one on here has actually mentioned that and only stated it’s all about file naming. So which is it?

I can and have read all the docs, but they are woefully incomplete and ambiguous. I’ve come to expect this for software and then having to seek assistance from others on the appropriate forum with more experience. However, simply referring me back to the same docs is not helpful.

My problem is not an inability to read and assimilate such documentation, it is trying to disambiguate (is that a word :slight_smile: such information. This is what I have ascertained so far:-

The docs (and others here) state that it’s all about the naming, but elsewhere it also states the above. They cannot both be true and that is what I am, so far unsuccessfully, trying to clarify. No-one has yet been able to state categorically that PMS can or can not use embedded tags and if so under what circumstances and why would it then need any particular folder and file naming scheme.

The docs also specify that TV series have to be a 3 tier hierarchy and others here have said you HAVE to abide by those naming conventions. Yet I have several TV Shows with no Season level and they appear to have been correctly imported into my main TV Shows library. So more ambiguity.

I have also copied one of the troublesome files (a single ‘episode’) into a new folder with a hierarchy that conforms precisely to how The TVDB has it listed and set this up as a new TV Shows type library just to test. Still nothing. I have selected the scanner with Personal Media Assets at the top and have tried it with TheTVDB scanner selected (having deleted the Library and emptied the trash in between, so should be fresh each time), yet the result is the same. Nothing. Nada.

So whilst I may disagree with Plex’s policy on this, there seems to be some other actual problem that is preventing this import from working.

I am not about to give up on Plex since it should be able to do what I need, even if a have to do a lot more work setting up my ‘already setup in iTunes’ media files than I expected, but I am getting hacked off with this brick wall and re-reading again the same ambiguous docs will not prove fruitful.

So I ask again, under what circumstance will PMS read the embedded tags when importing video files into a TV Shows type library and if it does so when using ‘Personal Media Assets’, why is it apparently incapable of importing these media files. If it does not, then why is it failing to import a media file that IS correctly named as per The TVDB?

The more I know about the import process, the better I will be able to troubleshoot these things myself. But having come to Plex only a matter of days ago, I have no direct experience to rely on and that won’t change until I can get it working with my media. Chicken and egg really. So some real help would be much appreciated.

13

how are you at reading code?

14

Well a little more detail than I was expecting, but thanks. That apparently indicates that not only should it parse the video file for tags, but specifically with some iTunes specific code. So that’s all good. But if it’s doing that, what does it matter the folder/file structure and names? Also, why is it still not working even when I do set up a structure that conforms to the information from TheTVDB?

Seems to me that it should either be able to use the embedded tags, or just use the folder/file structure to ascertain the name etc. But it’s actually doing neither.

The file is MP4 with extension .m4v, but another one that fails has the .mp4 extension so I don’t think it’s that.

About 400 Movies and a similar number of TV Show episodes were all imported into their respective Libraries. Some errors I’ve noticed, but essentially PMS found them and tried to do something with them. With these files though, it always seems to simply ignore them. There has never been ANYTHING about these in the Plex Library. It has always simply shown a blank page saying nothing in the Library. Nothing I’ve done has changed that and I cannot understand how I can set up a new Library of the correct type and try to import just this one file which is correctly named etc, and it appears to do nothing. The screen doesn’t give any indication of anything happening at all. It’s an MP4 video file (that plays perfectly) with embedded tags and with the correct folder structure and names, yet it simply refuses to acknowledge the existence of this media.

Any ideas?

15

Well I spoke too soon it would appear. I returned to Plex to look into this further and found that the suitably named file had finally made an appearance and seems correct. So appropriate naming IS the solution, but where does that leave ‘Personal Media Assets’ which is supposed to extract the embedded metadata but appears to actually do no such thing.

My question remains, how can I make PMS import video files using their existing embedded tag data and not have to faff about trying to rename things before they can be imported?

16

You don’t. You either follow the naming convention and enjoy properly added media, or you continue trying to force Plex to follow your setup and you keep failing. You chose to use Plex, so choosing to use iTunes names and organization will prevent Plex from working correctly.

17

Plex’s metadata agents are based on Python code. It isn’t currently possible to read all tags from all file types. Right now, it will read some tags from MP4 (and mp4 related like m4v) only. So if you are using these and have the Title tag correct, PMS should be able to use this ti identify the movie. For TV shows, it’s a little more difficult since there aren’t standard tags for show and season. iTunes adds in their own tags, which can change with iTunes versions. This is why Plex relies more on the filename and for TV Shows, the folder structure.

The Local Media Assets (LMA) agent is what reads in these limited tags, so if you want PMS to try and gather this info, then you need to enable it and have it be at the top of list. If ti can’t find the right tags, then it fallsback to using the filename/folder. The LMA is also responsible for finding external support files life posters, subtitles, movie trailers and extras.

PMS does not skip showing files that it couldn’t match. They should still show up. The only reason files get skipped is if you put a movie into a TV Show library or vice versa.

If you want specific answers to why something doesn’t work, you’ll need to provide details like the actual names and tags.

18

Thanks MovieFan.Plex, very helpful.

I am committed to using Plex, but I have hundreds of videos (movies, TV etc) in iTunes and I need to find a way to ‘import’ them into Plex on a continuous basis, i.e. as new ones are added by iTunes, so they also drop into Plex, if not automatically, at least with the minimum of manual interaction. Can I say again, simply telling me to not use Plex is NOT helpful. What I need is real information as has just been provided.

I am making headway, but mostly it involves moving media out of the iTunes domain into Plex’s own unique one. For some smaller libraries that’s OK, but I’d like the main Movies and TV Shows libraries to still be able to share with iTunes. Movies is I think OK, but TV Shows is a problem.

It occurs to me that being able to import media from iTunes and maybe continue to share the media between both systems would be a good thing for Plex to be able to offer. There must be many users like me fed up with the limitations of iTunes and looking for something better. There certainly are many desperate for some sort of real server. So for Plex to be able to provide a simple import process for all media would attract a lot of users.

Plex can already import playlists from iTunes can it not? Would I be right in thinking that this is done by reading the library XML file? If so then it would be relatively easy to also parse this file to extract all the video tag data you require. It’s all in there and no need to worry about actual file tags etc. Even if the XML file is not currently used, parsing XML is not rocket science and could be incorporated quite easily.

I really have in mind a one-shot process that would read the entire iTunes library into Plex. Just grab ALL the data from the XML file and at the end of that perhaps then try and match on-line to find more data, or not, depending on the agent/scanner specified. At the very least it would guarantee everything was imported with sufficient metadata to be immediately usable as it would be the same as had been in iTunes, so anyone switching would feel happily at home. I can’t help but think this would attract a lot of iTunes users, especially if it provided on-going sync between the 2 systems.

Just a thought.

19

Plex does have a way of importing the playlist, but reading a list of files is pretty easy. This only imports the playlist, the music files themselves are scanned in the same way by PMS as it does for movies. You point PMS to your music folder and the tags are read from the file themselves (if you choose that option). I don’t know if a movie’s data is stored in the XML. If it is, then yes it might be easier to read it from the XML. You can make a feature request to include this option.

Importing the playlists is also just a one-time task. It doesn’t keep things synced between iTunes and PMS, so if you want to try and keep iTunes and PMS synced for movies, that is a totally different ballgame then a one-time import option. I don’t know of anyone who’s reporting doing this.

20

Hi UKenGB, I’d just like to say that I have total sympathy for you. I have a setup very similar to yours and I too have tried to go down the iTunes + Plex co-existence road multiple times over the past 5+ years and I am just trying it again at the moment. But every time I come up against the same issues as you :frowning: My situation is slightly different as I do not store my media files in the default iTunes location nor do I let iTunes ‘organise’ them (for various reasons) so it is less of an issue for me to adopt the Plex storage conventions (though I agree that ther should not be any need for that). Like you, my files (music, films, tv shows, music videos etc.) are all throroughly and rigorously tagged. All that would be needed is for Plex to extract all that info from the files and use that; no need to look anything up on the Internet. But no, Plex is extremely limited in this area. I have seen comments about the tagging format being non-standard and changing between iTunes releases but that really doesn’t wash. The tagging standards for MP4/M4V/M4A files have been pretty much stable for many years and while there are occasionally minor changes from the iTunes perspective it is simply not true that there are major changes between iTunes relases (I know because I have been doing this for many years). To me the sad fact seems to be that the Plex developers, like Apple to be honest, have their own mindset about how things should work and if someone wants things to be even a little different, even if the ‘want’ is a very reasonable and sensible one, then they are out of luck. This is a huge shame; I think the opportunities here for Plex as an iTunes ‘replacement’ for in-home media sharing is very large but until/unless Plex adopts a more iTunes/Apple friendly approach then they will continue to miss out on this potential major increment to their user base. I guess they feel that they are the only viable alterantive (largely true at the moment) and so people must just ‘suck it up’. Sigh.