Importing to PMS

General Discussions
21

Hey Chris, good to hear your thoughts. Yes it seems bizarre to me that there are so many iTunes users crying out for a server solution and yet Plex seem to simply not grasp the HUGE potential market that would be for them and die hard Plex users seem rather disdainful of iTunes with often unfounded ill-conceived notions about what it can or cannot do. The more I dig into Plex, the more respect I have for the brilliance of what iTunes does. Plex pretends it can do as well, but look at playlists. iTunes has that pretty much nailed and Plex’s offering is pitiful by comparison and I agree, blather about lack of standards in video file tagging is just an excuse to be lazy. EVERY other video media tagging app I have tried seems to be able to read the exact same tags that iTunes uses, so call it a de-facto standard if you like, but it’s usable. I often hear complaints about iTunes that are simply not true, like iTunes makes up and uses its own tags, but that’s not the case. There’s very little data in iTunes that is not held in a standard file tag.

But iTunes is far from perfect for me. As has been pointed out elsewhere, it is becoming more and more of a storefront for Apple’s media sales than its original purpose as a music manager, for which I have been using it since before it was bought by Apple and renamed iTunes. As a former database designer (and having worked with music industry data organisations) and being a music enthusiast since before I can remember, data structure (particularly music) is very important to me. I know how it should be done, but sadly most developers obviously do not. Actually all developers to date - including Apple. So iTunes is seriously lacking in some respects and for those reasons, Plex will be a step forward, even though far from perfect itself.

I do let iTunes keep its media folder organised as that means everything is kept in a regular and consistent structure. I don’t have to concern myself with folder and file naming. I just adjust the tags and I don’t believe there’s a better tagger than iTunes for dealing with multiple files. But that simplicity also implies a rigidity as you cannot point iTunes at different library folders for the different media categories to de-centralise data storage. I thought I had it cracked by automounting shares from different locations, but iTunes doesn’t play nice with that sort of thing. However, Plex is excellent at this.

Regarding sharing data between these 2 systems, I have come up with what I think is a workable solution. Music doesn’t seem to be a problem although to be honest I’ve not got into that yet, but since there is an option to ‘read embedded tags’, I think in this regard Plex have seen the light. The main area of concern is video based media for which Plex wants to rely on the file name and what a nonsense that is when as you point out, all the required data is readily available within the file itself. File names are not meant to convey vast amounts of data. They’re just a way to approximately identify the file. Like a book title which doesn’t try to always include the name of the author, the series it comes from, the ISBN, almost the entire book itself. It’s just a short reference. Not only that, but there are fundamental problems with trying to use the filename as different systems prohibit different special characters that may exist within the data being used, like Title or Series name. If those are simply dropped, or switched to something else, potentially you’ve lost any chance to match the file with some on-line database over which you have no control and may or may not have chosen to call it something entirely different. Matching on-line like that is a fantastic ability, but should be done ONLY when the file’s embedded tags are lacking. But I digress…

Video media needs to be named right to suit Plex and for this Filebot is a very useful tool. However the irony of using a tool to look up on-line databases (based on what Filebot can determine about the file itself) in order to rename the files, just so Plex can then use that name to match with the same on-line databases to obtain the metadata has not escaped me. But Filebot does have a trick up its sleeve. It CAN read the file’s tags. Ok 2 tricks…

I realised that I needed to keep the original filenames untouched to let iTunes continue to be my front end for organising and tagging (and other stuff Plex will never do like syncing apps to mobile devices), so I could create links back to them, but named as Plex wants. Currently I’m using symlinks as it’s easy to see what going on and this way iTunes gets to keep unfettered access to read and write to the actual files while Plex will happily read (all it ever does) via the symlinks. Obviously if iTunes changes the actual file path and/or name, the symlink will fail, but I’m in the process of creating the scripts that will ‘sync’ the links and then also sync the whole lot between my Mac and the server (so everything is stored twice for security). I could use hard links but I’m not sure it would be any better and introduces the potential to start chewing up storage if things go wrong and then much harder to see what’s what. So symlinks for now unless I find a reason to have to use hard links. Either way, Filebot is the business as it can not only rename the files if that’s what you want, but alternatively create hard or symlinks to them, with the names and destination folder at your control. Personally I find its interface to be somewhat lacking, but once you get the hang of it, it is very powerful and can do great things. Anyway…

Movies seem to work even as iTunes saves them, but it is easy to create the Filebot format to tweek it a bit, like adding the year to folder and file name and so keep Plex really happy. But it also means that based on embedded tags I can actually send the symlinks to different folders that form the basis of different Plex libraries. So I can have a different Library for TV documentaries and TV Sport etc. I’d like it if Plex could provide sub-Libraries, or Smart Collections for this, but it can’t, not yet anyway, so in the meantime I use Filebot to categorise my media in any way I want.

Once it is all set up (still creating the necessary scripts etc), anytime I’ve made what I consider to be significant changes to the media on my Mac, I run a script that syncs the links and then the whole lot to the server running PMS. In this way, iTunes gets to keep control over the files and provide the familiar (and on the whole excellent) front end for me to control the data and Plex gets to see everything with the folder and file naming it craves. Mainly using TVs for the front end (Plex Connect on several 3s) means I could also access iTunes as an alternative if it was ever needed although my intention is to no longer have to run it on the server and just have Plex as a server process in the background (but that’s another problem entirely).

Sorry, a bit long winded, but this seems to satisfy my desire to still be able to run iTunes as the front end to control the media and yet share that same media with Plex. What I’ve done so far seems to work, so I’m hopeful it will provide the overall solution I really want.

22

Have you guys tried creating a movie library using the Personal Media Agent. This will avoid all the filenaming and matching that Plex will normally do. Make sure the Local Media Agent is also listed at the top of your agent orders so it will read you file tags. If you can do this with say 10 movies, I’d be interested in seeing the results.

23

I tried that but there are drawbacks with some media. Mostly I would like it to be able to find additional metadata when available on-line, over and above what is already in the tags. Selected Personal Media Assets as far as I know should mean it then never looks anything up for that Library which is not what I usually want, although may be appropriate for other Libraries (Home Videos etc).

Which raises the subject of multiple Libraries. One big lack in iTunes is that you are totally restricted to the main Library types. E.g. there can only be 1 Movies library and only 1 TV Library etc. I want to be able to sub-divide these so I can have TV Documentaries, TV Sport etc. This is impossible in iTunes. But my method allows me to create the links in different folders, each of which is a separate Plex Library so this way I can use Plex for finer grained categorisation of my media collection, but still use iTunes when required, like for original sorting and tagging etc.

I think it’s going to work well. Still ironing out some of the issues involved in maintaining sync, but nothing major.

A question though. If Personal Media Assets only uses the embedded tags, why does a Music Library offer ‘Use embedded tags’ as an option you can select, even when PMA has already been chosen?

24

@UKenGB said:
A question though. If Personal Media Assets only uses the embedded tags, why does a Music Library offer ‘Use embedded tags’ as an option you can select, even when PMA has already been chosen?
Personal Media Assets does not read the embedded data, it just prevents Plex from pulling online data. It is the Local Media Agent (LMA) that reads the data. Typically, the LMA will read the tag data to help it match the artist, album, and track. Once matched, then it will use show the info from the tag. That option basically forces PMS to use the tag data even if it can’t find a match. For example if you have a typo in the Artist’s name, it will assume it’s not a typo and use that Artist, so instead of matching to the nearest artist, you will have an unmatched artist.

25

The terms ‘Local’ and ‘Personal’ I think have been swapped, but in any case, I’m still not entirely clear on this. When setting up Agents in Settings, all the medial types have 3 ‘groups’, one of which is ‘Personal Media…’ and in each case, there are 2 agents listed: Local Media Assets and Personal Media, but from what you say above, it is not clear to me which should be at the top. Or does it not matter as long as both are checked (the Personal Media agent seems to be checked and locked so cannot be unchecked).

For a Music Library, what does and ‘unmatched artist’ mean? What will be ‘missing’ compared to one that has been matched?

26

A quick answer is there are 2 type of agents. 1 - metadata agents, which provide the data and 2 - supporting add-on (sort of, there’s no actual name for these).

1 includes, Personal Media, Plex Movie, TheMovieDB, TheTVDB, Last.fm, Premium Music, etc. These are the headings when you go to Settings \ Server \ Agents. 2 are the ones in the list below the headings.

0ly54svzk5tb.png900×350 14 KB

1 is used to match and identify your media. 2 indicates the order in which the data is used.

So you pick an item from 1 when creating libraries and deciding how to match your files. It’s possible, Plex Movie for example, to have multiple sources of data. You can enable and disable where the data comes from and by movie items up and down the list, you prioritize which data is used, but the identification will come from the agent you picked. Picking Personal means not to obtain any online data.

The Local Media Assets (type 2) tells PMS to scan the file itself (some tags) and other local files (posters, subtitles, other artwork, etc.) for use as data. So if you picked Personal and have LMA turned off, you get 0 data. For Music, of you pick the Premium Music agent and have LMA on, PMS will use the tags as 1 criteria for matching the artist. Matching means to match artist/song/track with the online site so it can get the correct info.

Lets say you have a song from the Goo-Goo Dolls. Your mp3 has the right filename but a typo in the tag so it says Go Go Dolls. The filename would match correctly, but using the tag might match to the 80s girl band The Go Go’s. PMS gives scores to how well each name matches the online site and will pick which ever has a higher score. The option to “Use Embedded Tags” forces PMS to use the match based on the tag. If you have the LMA listed at the top, it will use the tag data first to populate the data fields in PMS and then fill in the blanks from the online site. If LMA is at the bottom, the online data is used first and the tag data will fill in the blanks.

If you don’t care for the online music data, there is little difference between using the Personal Music Agents versus either of the other music agents with the “Use Embedded Tags” option.

27

I really appreciate the explanation. Only recently coming to Plex means I don’t have those concepts fully straight and ‘embedded’ into my head, so this really helps.

Music is the most important and what I don’t want to have to mess with. My media is very carefully tagged how I want and probably better than any on-line database, so I want that data used in preference to anything else. However I like the idea of Plex being able to add more data to that by getting it on-line. In the same way that embedded tags provide are far greater capacity for metadata than a filename ever could, an on-line database has effectively unlimited capacity compared to tags. So I want to establish the best correlation between what I already have and want and what Plex can additionally provide.

One question has arisen about music import. It concerns Album Artist and the sort tags.

In both cases iTunes will not populate them when the basic version can be used. So Album Artist will be left blank when the Artist can be used for that purpose (i.e. nothing was specifically entered for the AA tag). Likewise with Sort tags that are left empty and the regular tag used for sorting that track. Unless the Sort tag contains data, in which case that is used in preference. This is consistent and it works perfectly and avoids duplication of data (always to be desired). But how does Plex handle these tags? I have 25,000 tracks and sorting is CRUCIAL. I can do it any which way I like in iTunes and I want to make sure it’s right in Plex. Do I have to fill in these tags that otherwise can be left empty in iTunes? So ALL Album Artist tags need to be filled in even if it’s identical to the Artist tag for that track. Likewise the Sort tags. Do I have to make sure these tags are ALL complete for Plex to use them or is it as smart as iTunes and will drop back to the other, normal tag if this special one (AA & Sort versions) is empty?

I’d like to fully clarify this before trying to import all the music and possibly getting it wrong.

28

If Album Artist is blank, the track will be tied to the Artist. This can cause issues if you have compilation albums as these will get split between the different Artists.

For the Sorting field, if you do have that tag, I don’t know if it gets read. If you don’t have it, PMS will automatically use the Artist or Album field. Keep in mind PMS will automatically remove articles (a, and, the, etc) from the name so something like “The Who” will show that way but be sorted as “Who”.

29

Ok, so I have to ensure all Album Artist tags are filled in. Annoying, but no big deal and an AppleScript will do that.

However, sorting is an absolute must. Eric Clapton HAS to sort under Clapton. That’s what the Sort tags are for, so everyone can arrange everything sorted exactly how they want.

Not only for the (Album) Artist, but also for Album. When I display a list of an artist’s albums, how are they sorted? By Album tag? That would be monumentally dumb. I require them sorted chronologically, which I CAN do by using the Sort Album tag.

Tracks of an Artist should be sorted by Sort Album and then by Track Sort Title (or just the non Sort version if the Sort version is empty, like [Sort Title] || [Title]). There can be Shuffle and or basic Alpha sort options too, but they make no sense to me. However I recognise that some may like them.

What is crucial is that the sorting can be controlled by the user. What I require may not suit others, but apart from a shuffle option (if you must), use the Sort version (if it exists, normal Tag otherwise, for each item, like [Sort Tag] || [Tag]) so EVERY user gets to have their lists sorted exactly as they need/want. The developer need do no more. The user can completely control how each list sorts just by creating suitable Sort tags. If they cannot be bothered to do that, then they’re not interested or they have a very small collection. Either way, it is transparent to them and they can ignore it all, but software MUST use the Sort tags in preference to the normal versions (use [Sort Tag] || [Tag]) when sorting any such lists whose prime column does have a Sort version (artists, albums, album artists composers and tracks).

Sorry if I’m repeating myself, but this is just so important. Can you please confirm that Plex does indeed make use of these essential (Sort) tags and whether they all need to be filled in like Album Artist, or does it prioritise like iTunes and use [Sort Tag] || [Tag]?

30

@UKenGB said:
However, sorting is an absolute must. Eric Clapton HAS to sort under Clapton. That’s what the Sort tags are for, so everyone can arrange everything sorted exactly how they want.
You can edit the sort title to whatever you like.
Not only for the (Album) Artist, but also for Album. When I display a list of an artist’s albums, how are they sorted? By Album tag? That would be monumentally dumb. I require them sorted chronologically, which I CAN do by using the Sort Album tag.
If you are listing all albums, you can set the sort order using the sort option. However, on the Artist page, where the albums are listed underneath the artist info, this is set by an Advanced PMS option. https://support.plex.tv/hc/en-us/articles/201105343-Advanced-Server-Settings The default is to sort in descending order based on the release year. If there is more than 1 album in a year, it will sort alphabetically based on the Sort Title. You can change the default to sort by Album Title if you like, see the article I linked.
Tracks of an Artist should be sorted by Sort Album and then by Track Sort Title (or just the non Sort version if the Sort version is empty, like [Sort Title] || [Title]). There can be Shuffle and or basic Alpha sort options too, but they make no sense to me. However I recognise that some may like them.
Within an album Tracks are sorted by track number. Plex doesn’t use a Track Sort Title. You can change the order by renumbering the tracks.

31

Yes, sorry got a bit carried away. Sort Track Title is entirely unnecessary as tracks within an album should of course sort by Disc Number, then Track Number, so I assume Plex has got that right. The rest sounds like it might be doing the right thing.

The problem usually arises when all tracks of an artist are listed. Even Apple tends to list them alphabetically which is of no value to anyone. Might as well just press shuffle. The logical sort order is to first sort by album in chronological order and then by Track No. (within each album). But as you point out, there could be more than 1 album released in the same year. For this reason it makes more sense to always sort albums by Sort Album as that can be created to set any sort order the user requires. I use YYYY.MM.Album Name as that allows to sort down the month of release and I don’t know of any artist releasing more than one album in a single month so correct chronological sorting can be preserved.

The reason I’m so determined about this is because by using the Sort tags, it then allows the user to create any order they want. If it’s left to the developer to always get it right for every user, that’ll never happen as it’s an impossibility. Give the user the ability to configure it how they want.

I’ guess I’ll get to see exactly how it does all this once it’s all up and running. One outstanding question though, do ALL the Sort tags need to be filled in, even if it’s the same actual value as the regular version of the tag? Of if left empty, will Plex use the regular version for that item, as does iTunes?

32

@UKenGB said:
I’ guess I’ll get to see exactly how it does all this once it’s all up and running. One outstanding question though, do ALL the Sort tags need to be filled in, even if it’s the same actual value as the regular version of the tag? Of if left empty, will Plex use the regular version for that item, as does iTunes?
If empty, Plex automatically creates a sort title from the title but removes articles as mentioned earlier. There isn’t a way to override this and if you want to sort using “The”, you will have to manually add it back in.

33

No, no, articles like ‘A’ and ‘The’ should of course be removed for Sort versions - as iTunes does, but it seems to do that internally and doesn’t actually populate the Sort Tag with that ‘reduced’ version of the text and simply shows it in grey text to indicate what is being used. However you can change it to whatever you want and that WILL be stored in the (Sort) Tag. From what you say, Plex will do something similar, but obviously never actually writes any tags to files.

So to be completely clear on all this:-

I need to ensure that ALL [Album Artist] tags are filled in, even when it is the exact same data as the [Artist] tag?

If any [Sort Tag] contains data, that will be used for sorting, otherwise the [Tag] will be stripped of clutter (‘A’, ‘The’ etc) and used for sort purposes. So I do NOT need to fill in any [Sort Tag] unless something else needs to be used (like wanting sorting by ‘Clapton, Eric’ etc.). This is how iTunes does it, so if Plex is the same, that is VERY good news.

34

That all sounds correct.

35

Thank you for all your help. Much appreciated.

36

I would like to return to the Album Artist issue. MovieFan.Plex states that if there is no [Album Artist] tag, the track will be tied to the [Artist]. Understandable, but is that global or on a track by track basis?

I have NO music track whose [Album Artist] is empty but whose [Artist] should not be used instead. IOW, the only tracks with empty [Album Artist] are those for which it would be the SAME as [Artist]. So my understanding of the above is that in fact, for that track, [Artist] would be used instead of [Album Artist], which is correct for my music media. But what about the next track if that DOES have an [Album Artist]?

I have no idea of the database structure used by Plex, but I think it creates an Album record as well as the Track record(s) and each have their own associated Artist, which may be the same or it may be different, typically for compilations, but can also occur when an artist’s album contains tracks with songs by, or including. other artists. So the question is, as Plex imports each track, does it associate the [Artist] tag with the Album record if there is no [Album Artist], but use that if it does exist? So each track may or may not have any [Album Artist], but any Album record will have as its associated Artist, the [Album Artist] if it was there or [Artist] otherwise?

This does make the most sense and if it is on a track by track basis, then iTunes method of prioritising (meaning [Album Artist] can be left blank if the same as [Artist]) will be perfectly usable in Plex. This causes NO confusion with any compilations as they would ALWAYS have a suitable [Album Artist] since it would be different from any [Artist] tag.

Does PMS consider this for every track as it imports, or apply the same method globally?

37

@UKenGB said:
I would like to return to the Album Artist issue. MovieFan.Plex states that if there is no [Album Artist] tag, the track will be tied to the [Artist]. Understandable, but is that global or on a track by track basis?
Sorry, I wasn’t very clear. PMS will look at both Artist and Album Artist fields, if one or the other is blank, it assumes the same for both fields. Album Artist is the field PMS wants to identify the top level in Plex. Artist is then used for individual tracks. So if you have a bunch of random files in a folder and they have Artist but no Album Artist, and they have an Album label, this will look like a compilation to PMS and they will show up under Various Artist. If you have Album Artist but not Artist, then you should get a bunch of individual Artists, all having the same album title with 1 track.

If you have either field but no Album field, well, things get confusing and the results are unpredictable.

38

Thanks for that. I’m pretty sure Plex will work then with my iTunes data as is. Fingers crossed.