Found the answer! Poking around the forum once I hit the issue of local extras not populating ran across this old post:
Local extras analysis and detection are handled as part of the metadata refresh not the rescan refresh. The reason every other movie is borked for them then is that I had rescanned the files multiple times, but not refreshed their metadata. Some of the items did which is why they partially worked, and anything added new more recently did as part of its being added to the library.
Given this is impacting nearly every entry I am going to bite the bullet and do a complete library metadata refresh. I have tweaked the title of the post slightly to hopefully be found more in the future for anyone having similar issues.
TL;DR Solution: Refresh metadata to refresh local extras (which also only show up if local media assets are enabled under the Agents as well)
