Mark,
The “Plex Team Member” is a bit abstract. It allows me to be in the forums on a daily basis 
Thanks for your understanding about those other things. Linux & Linux NAS are my areas. I’ll do everything I can. As for that other issue you mentioned… It’s VERY well known and getting a lot of internal work.
Good to hear. 
Okay, so checked SMART. 5/196/197/198 all showing zero for all four disks, as of 18th March (so two weeks ago). I’ll kick off another check, but I doubt much has changed (the first issue I had with Plex failing was on a library which has been almost unchanged for over a month. I’ve kicked off an Extended SMART test on all drives though - should be done in about 12 hours.
Raid scrubbing was run around a week ago. I don’t think it’s possible to set up a scheduled file system scrubbing task, so I’ve kicked one off manually now.
But essentially, the volume looks (as expected) sound and without any errors or issues. What’s next?
Oh, and here’s a complete set of Plex Logs from the fresh install I did last night.
I got English this time… Nice
you logs show missing XML and all kinds of problems with metadata that PMS thinks it should have so you took a serious hit somewhere. Abrupt power off perhaps?
To get over this.
- Stop PMS
- Open the Plex share (add your username if needed) in FileStation
- Rename
LibrarytoLibrary.keep(We’ll use what we can later) - Start PMS
- Go through a full setup again… HOWEVER… do is this way:
a. Basic setup but skip adding library sections while in the wizard. wait until you’re at the dashboard.
b. Set your preferences the way you had them
c. Add a section.
d. Optimize the DB after it’s done with that section.
e. Back to step c and repeat for the next section
f. When done, see where you’re at overall.
No, there’s been no power off. And just to be absolutely clear, I basically did what you suggest last night. I did this:
- Deleted the entire Plex folder
- Installed a fresh copy of the latest version of PMS (v1.12.2.4929)
- Ran through the setup, skipping the Add Media section
- Adjusted my preferences, including disabling Local Media Assets
- Added my TV Shows library and let Plex scan.
I’ve literally done nothing else since then, other than send you the logs. The only difference to what you’ve suggested is that I didn’t do the Optimise DB step after setting up my preferences (but given it was a totally fresh install literally 15 minutes old, I can’t see why on earth that would be necessary - and if it is, Plex should auto-optimise).
I grepped the logs, and saw lots of ‘SLOW QUERY’ and Missing XML warnings. What XML is missing?
The only other thing I did was to install the iPlayer plug-in (which I doubt will have any impact) and the Trakt plugin (to sync my watched statuses). Are there known issues with Trakt interfering?
Okay, so here’s an interesting finding. I decided to go through and Plex Dance the individual series that are being misidentified and categorised as ‘McMafia’. So after optimising the DB, I moved all of the series out of the folder, ran a scan, emptied trash, cleaned bundles, optimised the DB, ran another scan, emptied the trash again, cleaned bundles, optimised the DB once more, and then moved one series (‘Winterwatch’) which has 3 seasons each of 3-4 shows, back under the TV shows folder.
Plex picked up the change and automatically started a new scan. Winterwatch appeared in the set of TV shows. Yay! Except that somewhere between that point (where it had identified Winterwatch as a separate show) and when it did the TVDB query to get the artwork and description metadata, the same thing went wrong, and it disappeared, moving back as a season under McMafia. When the shows moved, I could see the status ‘toast’ appearing saying ‘Deleted an Item’:
Then after that it started processing metadata for McMafia season 54 etc:
So I don’t think this is anything to do with load, or corruption or anything else like that. It looks like Plex is correctly picking up the show folder and seasons, but when it scans the metadata, it gets it wrong.
How can we debug exactly what caused this to happen?
I see “Adding get_iplayer” appearing in the Alerts screen. Could the problem be this? https://forums.plex.tv/discussion/298864/fix-incorrect-match-problems-with-get-iplayer
Is it possible that even though I have LMA disabled, Plex is picking up the get_iplayer metadata in the MP4 file and using that to identify the show (incorrectly)? It would make sense, as all the incorrect shows are from the BBC. Is there a way to explicitly disable metadata scanning and have Plex only rely on the file/folder structure?
Update: So, I used ffmpeg to strip the metadata from one of the MP4 files. Danced, and then dropped it into the TV folder. It’s been identified and metadata’d correctly. Going to batch strip all the MP4 data from the other broken files, and see if that solves it for all of them. If it does, it basically means that Plex either has a bug, isn’t honouring the ‘LMA disabled’ setting, or should have an additional setting to say “Never ever ever use MP4 metadata”.
If you use MP4/M4V files you’d also better know how to locate and move Local Media Assets. Sure, you can strip all the metadata from MP4/M4V files, but it’s easier to ‘demote’ Local Media Assets so it will stop giving Tip-Top Priority to embedded and (very likely) bogus Title Fields inside MP4/M4V files.
FileBot (link in my signature) or Sonaar (in your case) can handle that naming and structuring that’s so important for you automatically or manually in seconds.
What FileBot (or Sonaar?) can’t do is remove possible embedded metadata in the Title Field of MP4/M4V files. Plex will read this info and prefer it over a perfect file name/structure, but you can combat that situation by moving Local Media Assets to the bottom of every agent list you can find. All tabs in TV Shows and Movies here:
https://support.plex.tv/hc/en-us/articles/200241558-Agents
Just drag LMA to the bottom of the list and drop it. If you do have embedded metadata this will cure the issue, if you don’t it won’t matter. LMA will do what it has to from the bottom.
There’s the ‘work-around’ to Plex’s refusal to move the default location of LMA - that is the #1 thorn in the side of MP4/M4V users in the Plexiverse.
B)
You may not have read all of this thread, but I’ve moved LMA to the bottom across all agents, and disabled it by unchecking it - but this problem is still happening. This is how I set up my agent before I imported any TV shows:
So the LMA setting isn’t being honoured - and MP4 metadata is still being read - even though it shouldn’t be.
Well, that’s one for the ‘Top Men’. AFAIK if LMA is Lowered or Disabled reading embedded Title Fields is impossible.
BTW… you shouldn’t Disable LMA, but you should Demote it. Eventually you’re gonna need Local Artwork/Subtitles/etc and with LMA Disabled that can’t happen.
Why would I ever want to use LMA? I don’t use subtitles, and artwork all comes from the bundles (via the web/TVDB).
Eventually, you WILL need subtitles.
For instance - unless you’re fluent in Inuit, you’re going to miss some plot points in The Terror - now playing on AMC. I don’t know about you, but here in WV we don’t get a chance to use ‘The Old Language’ very often so I need a subtitle every now and then.
Sometimes the artwork that is drawn isn’t exactly up to my exacting standards and sometimes, for some items, artwork and metadata simply doesn’t exist, so sometimes that stuff has to be created/found and used and without LMA that can’t happen.
Subtitles, yes, I suppose.
Artwork, no. If I have a show that’s missing artwork, I’ll either ignore it and not bother, or I’ll make some and add it to TheTVDB. I haven’t got enough time* in my life to replace TVDB artwork with ‘better’ versions.
(*I’m too busy re-doing the Plex Dance pointlessly to try and fix obscure Plex metadata scanning bugs)
Okay, so looks like I’ve solved this.
I moved the affected shows into a new folder, and ran this command:
find /volume1/video/Dance -type f -iname '*.mp4' -exec ffmpeg -i "{}" -map_metadata -1 -c:v copy -c:a copy "{}.clean" \; -exec mv "{}.clean" "{}" \;
which iterates through all of the MP4 files, using ffmpeg to strip the metadata from the files. Once I’d done that, moving back into the collection results in Plex scanning and identifying the files and pulling in their metadata correctly.
My McMafia is as it should be - 8 shows in one season. So I think what was happening is this:
- Ripping shows via get_iplayer sometimes writes ‘get_iplayer’ into the Show metadata tag.
- Plex scans the files, correctly identifies the series/season from the folder, but then reads the metadata tag and pulls the wrong show information from TheTVDB.com
- Because McMafia was also ripped via get_iplayer, it was there first, so the other shows were grouped underneath it.
- Disabling LMA or moving LMA to the bottom had no effect and the local MP4 metadata tags were taken into consideration by Plex when identifying the shows/episodes.
- All the stuff about file system corruption, system load, SqlLite transactions, etc., etc., were nothing to do with this and a red herring.
- Doing the Plex Dance has no effect, because it’s the Plex scanning algorithm that’s fundamentally flawed in this case.
The solution for this, as surmised earlier, is for Plex to be fixed so that either:
a) Moving LMA to the bottom and/or disabling it really really causes it to ignore Local Media Assets - including MP4 tags or
b) Plex needs a new option to be set called “Never ever ever read MP4 tags - ignore them all”
Additionally, given how strict Plex is in its naming/directory structure requirements, it seems utterly absurd that it would group shows that fall under completely distinct series folders into a single series, purely on the basis of some MP4 tags. Either Plex requires the folder structure layout that’s recommended in all of the articles about media preparation, or it doesn’t. If it does, then it should have flagged up an error to the user: “Multiple shows in different series folders with the same series name” and asked the user to resolve it.
How do we log and track an issue with the Plex development team? Can you do that Chuck?
In the meantime, I’m going to put a little script together that runs to periodically clean up MP4 metadata tags (I might write a utility that checks for tags, and only deletes them if they exist, so it runs quickly, and have this run daily on my library).
@“Mark Otway” said:
No, there’s been no power off. And just to be absolutely clear, I basically did what you suggest last night. I did this:
- Deleted the entire Plex folder
- Installed a fresh copy of the latest version of PMS (v1.12.2.4929)
- Ran through the setup, skipping the Add Media section
- Adjusted my preferences, including disabling Local Media Assets
- Added my TV Shows library and let Plex scan.
I’ve literally done nothing else since then, other than send you the logs. The only difference to what you’ve suggested is that I didn’t do the Optimise DB step after setting up my preferences (but given it was a totally fresh install literally 15 minutes old, I can’t see why on earth that would be necessary - and if it is, Plex should auto-optimise).
I grepped the logs, and saw lots of ‘SLOW QUERY’ and Missing XML warnings. What XML is missing?
The only other thing I did was to install the iPlayer plug-in (which I doubt will have any impact) and the Trakt plugin (to sync my watched statuses). Are there known issues with Trakt interfering?
Mark,
The reason you saw the “SLOW QUERY” statements in your logs is because of how SQLITE3 works. It’s a flat-file database engine. As you add rows to tables, the tables naturally fragment. When performing a cold-load as you did, it does not take long before the elapsed time to complete a query exceeds the “SLOW QUERY” threshold. This is why I instructed you to perform the Optimize Database step. You skipping that step allowed DB fragmentation to increase, unchecked, until performance impact triggered the warning. During normal operation, the DB optimizes itself every 3 days. This is usually sufficient. When loading a lot of media, it’s clearly not enough. If you had optimized at each section, you would have seen a significant performance increase.
I was unaware of your use of MP4/M4V files until now. When Local Media Assets (LMA) is enabled, any ‘bad’ metadata in MP4/M4V usually does result in incorrect matches. The ideal placement for the LMA in the stacking order is at the bottom of the stack, not the top. When at the bottom, all other means of matching are used before resorting to file metadata (which is often wrong). Also, in this position, PMS ability to find your local media assets is not impacted. All your local extras are found. That’s all you really need it for.
Personally, I always strip all metadata from files and have the names “FileBot perfect”. This affords me of knowing exactly what PMS will be looking for. I also avoid the use of MP4 files because of what it takes to clean then. You had to perform a full remux to clean them. MKV files are a lot simpler.
My general cleanup is:
find . -name \*.mkv -exec mkvpropedit {} -s title=\"\" --edit track:a1 --set language=eng --edit --track:v1 --set language=eng \;
All this does is rewrite the properties page. Full remux and file swapping after the fact is avoided.
@“Mark Otway” said:
Update: So, I used ffmpeg to strip the metadata from one of the MP4 files. Danced, and then dropped it into the TV folder. It’s been identified and metadata’d correctly.Going to batch strip all the MP4 data from the other broken files, and see if that solves it for all of them. If it does, it basically means that Plex either has a bug, isn’t honouring the ‘LMA disabled’ setting, or should have an additional setting to say “Never ever ever use MP4 metadata”.
For another topic, lol
Chuck, don’t get me wrong, I’m not saying your comments on performance and optimisation weren’t helpful, but they weren’t relevant to this problem.
When Local Media Assets (LMA) is enabled, any ‘bad’ metadata in MP4/M4V usually does result in incorrect matches. The ideal placement for the LMA in the stacking order is at the bottom of the stack, not the top. When at the bottom, all other means of matching are used before resorting to file metadata (which is often wrong).
So this is not the behaviour I’m seeing at all. I have LMA at the bottom and disabled - so my understanding is that it should use file/folder naming only - and not use MP4 metadata at all if my files and folders are named correctly (which they are).
So how do we get this fixed? In my scenario, Plex should never have looked at the erroneous MP4 metadata.
Mark,
Can you obtain and provide me a link to one of the original files or something which replicates the behavior? If you can help me with that, and I can demonstrate / replicate here, I will write up all my findings and submit to engineering. This is the best next step to take.
IIRC, it’s been the long-standing behavior of PMS to read metadata from files. Only music provides the option to disable the tags. Do we need it, arguably so. Will there be pushback, yes and why, imho, the ability to turn off embedded metadata use needs the control. The pushback will be in the form of “Why? It’s always been this way.” That as expected, If LMA’s default position were at the bottom of the stack, which would otherwise have allowed the normal matching routines to take precedence, would that be sufficient? See my logic? Go for gold, settle for silver
Another point for us to deal with are those “Otherwise Questionable” media sources who seem to put superfluous self-promoting things in the Name and Title metadata fields. If those fields were correctly populated, you’d never have had this problem. (this might be another pushback point).
Luckily I backed up the files before I used ffmpeg to clean them. Let me see if I can put them somewhere that you can download - I’ll PM you a link.
Thanks Mark 
