[BUG & FIX] Plex can't handle large EPG Mappings

Plex Players Desktop Players
,
1

Server Version#: 1.42.2.10102
Player Version#: N/A
Tuner Make/Model: HDHomeRun
Guide/Lineup name: N/A
Using XMLTV?: Yes
Channel number/Name: N/A
Browser: Chrome-ish
Environment: Unraid/Docker (plexinc/pms-docker)

The Problem

I had issues when adding a medium-large EPG for a given channel lineup (~350 Channels). The EPG was supplied via XMLTV, and every channel has a mapping configured.

When trying to save the EPG/mapping, an error would show up: “There was a problem saving your DVR. Please try again.”

Looking at the network requests, I was seeing a preflight/CORS error, but at a glance, the more likely issue seemed to be that the EPG Mapping request (PUT media/grabbers/devices/##/channelmap) was using too large a query string for either the browser or webserver.

The URI that Plex builds up looks something like this:

PUT https://<ip>.<hash>.plex.direct:<port>/media/grabbers/devices/35/channelmap?channelsEnabled=WGBH.us%2CWCVB.us%2CWBZ.us%2CWBTS.us…&channelMappingByKey%5BWGBH.us%5D=WGBH.us&channelMapping%5BWGBH.us%5D=123&channelMappingByKey%5BWCVB.us%5D=WCVB.us&channelMapping%5BWCVB.us%5D=234&channelMappingByKey%5BWBZ.us%5D=WBZ.us&channelMapping%5BWBZ.us%5D=456&channelMappingByKey%5BWBTS.us%5D=WBTS.us&channelMapping%5BWBTS.us%5D=567

The above example is for just four channels, and you can see how the URL is already quickly growing long. In total, the PUT request for my full ~350 channel playlist was 32839 characters long. At that point, either Plex or its webserver chokes on such a massive URL, and fails to add any mappings at all.

The Workaround

There’s a few ways to work around this issue until a fix is released.

1: Live with less channels
If you add less channels, the requests will go through. If you’re experiencing this issue, try adding one channel, and then adding more after that.

If you saw the “error saving your DVR” message and it was due to the EPG mapping being too large, this won’t solve you getting your full EPG-- the request to update EPG mappings will always include every channel mapped, not just the new ones. This means that you will eventually reach the ‘cap’ of how many channels you can add, and then you will need to choose which channels to ignore or keep. To ‘forget’ a channel and make room for another, just un-tick the box next to a channel, so that it is no longer mapped to the EPG.

This solution gets you somewhere, but it still isn’t great, since you’re settling with a sacrifice.

The ‘cap’ for how many channels you can enable isn’t a set number, since it depends on the length of each of your channel/EPG IDs, since those determine the length of your total EPG mapping request.

2: Make your Channel and EPG IDs shorter
Because the issue is the overall length of the EPG mapping request, if you can make that request shorter, then you can keep more channels overall!

I use open-source middleware (called m3u-editor) that allows me to re-map my EPG’s channel names and IDs. Whether you use this or any other tool that lets you change channel IDs in your EPG, you can make it work.

I changed all of my outputted channel IDs from their original values (WGBH.us, WBTS.us, etc) to shortened values; (AA, AB, etc). With the shorter values, the full mapping URL was 22104 characters. Still massive, but about a third smaller than our original EPG mapping request. It worked!

This solution doesn’t scale forever, and some quick math says it would stop working around 500 channels, but for my case, this was enough to get around the issue.

As a bonus, Plex seemed to do much better at matching the shortened channel IDs. My guess is that this is because Levenshtein works great on equal-length keys (AA/AB/AC/etc were all just 2 characters), whereas previously Plex did some gnarly similarity matching (I think it scores with Levenshtein, but it doesn’t consider length and also scores in sort-order of channels. I have a lot of concerns with the XMLTV scoring system, but that’s best for another post).

:information_source: Plex Devs: For the XMLTV similarity match, could you consider the following?

A similarity scoring system that values both length and edit distance

similarity_score = 1 - damerau(a, b) / max(lenA, lenB)

(where ‘damerau’ is the Damerau-Levenshtein distance.)

  1. Calculate the similarity score between each Channel and every [remaining] EPG candidate
  2. Map the channel with the highest similarity-score to the corresponding EPG candidate
  3. Remove that mapped EPG entry from the available set of candidates. (Since EPG entires can only be mapped 1:1 with Channels)
  4. Repeat step 1, until we run out of Channels or EPG options.

There are more efficient ways to handle that loop, but the logic should be roughly the same.

Also, check out Jaro-Winkler distances, those may work well for the typical Channel and EPG naming formats.

The Solution

To really fix this issue, the PUT request to media/grabbers/devices/##/channelmap should be using a request body rather than stuffing it all in the query string.

It’s simple, it’s clean, and it’s best practice.

1 Like
2

I think i have the same problem, can max. save 159 Channel-mappings.

3

In my case, the URL was breaking because of my reverse proxy setup.

Accessing plex locally and mapping 260 channels without modifications worked perfectly for me.

After knowing that worked, also increased a couple of settings for my Plex reverse proxy and that also worked with all channels.

large_client_header_buffers 4 32k;
client_header_buffer_size 32k;

The query params is not ideal but there’s workarounds now that I know the problem. Big +1 on this being a request body rather than query parameters.

1 Like
4

Thanks J,

I hadn’t actually checked the reverse proxy limits at the time, and that is a great first spot to investigate.

I can’t recall in my testing whether I was accessing the instance via the reverse proxy at the time, or if I was connecting directly to the instance.

Either way, query strings are always going to hit some ceiling, so fingers crossed that this goes into a request body.

5

I suspect your issue might be caused by that excessively long URL. I think you could approach it differently: you could have an AI help you write a Python middleware. This middleware would be specifically responsible for updating your lengthy URL and caching the latest results. Then, using Python’s Flask framework, you could provide an API interface; accessing this interface would return the processed and cached EPG information. This way, you would have a short domain name or IP address for the middleware to replace the long URL (although it’s a bit of a workaround, using middleware gives you a concise domain or IP, right?). While this does seem like a PLEX issue (a bug that can’t handle overly long URLs properly?), it’s uncertain when it will be fixed. So, we can first try using middleware to help convert and handle the problem of lengthy URLs.

6

Perhaps someone has an idea how to manually edit the database using the database browser.

I think the main problem is the long URL for saving the mapping.

The channel assignment is also poorly implemented. If you assign a channel twice, an error is reported, but the location of the error isn’t specified.

There’s still little to no differentiation between audio broadcast, SD, and HD.

7

@Olli – I had written a separate post about how to edit the DB in the appropriate spot for changing the URL host for an EPG.

I haven’t tried manipulating the EPG channel mappings, but they are present in the same fields. Worth a shot if someone wants to try that angle.

8

I think i have noticed it, but won’t change my EPG host.

I haven’t tried manually changing the EPG channel mapping yet; that would be too much work. The risk of completely messing it up is too high. Besides, several of the higher channels only play rubbish anyway, and I don’t really need radio channels in Plex.

Also in Public broadcasting there’s 3-5 times the same Channel, the only difference are local news.