How to use zap2xml for Plex EPG & combining multiple EPG sources

I’ve been using zap2xml as my EPG source in plex for some time, and have had a few questions on how to set it up, as well as how to combine multiple EPG sources, which currently plex is unable to do with the native EPG.

So what is zap2xml? it is a command line script that connects to zapit or tvguide to generate an xmltv file that we can use with plex to populate the EPG.

I’ve used it with zap2it (https://tvlistings.zap2it.com) so that’s what we’ll be covering here.

So first thing, you need to decide what system that you’ll be running this on. Ideally it’ll be your Plex media server. I have successfully used zap2xml, with windows, Linux (Ubuntu), as well as with my QNAP NAS.

Note: If you are using this with Linux…or a NAS which likely uses some form of Linux, you may need to install perl. For a NAS, you’ll need to see if there is a plugin/addon for perl on your NAS, or check your NAS forum. On my QNAP NAS I’ve successfully used the QPerl 5.26.0.2 addon, which i believe is a non supported community developed addon. If you’re using Ubuntu, I believe is it already installed my default, but you may need to install some additional modules. Using Ubuntu Server 18.04 & 18.04.1 LTS I needed to install the following perl modules.

sudo apt install libhttp-cookies-perl
sudo apt install liblwp-protocol-https-perl
sudo apt install libjson-perl

Note: I had an issue installing the libjson-perl module on Ubuntu 18.04.1 when using the iso with the new Subiquity Installer (ubuntu-18.04.1-live-server-amd64.iso). I found that if I installed it using the iso with the traditional installer (ubuntu-18.04.1-server-amd64.iso), that everything worked well.

After some investigation, I discovered that the package sources / repositories on these two build are different. You can read more about it, and check out some possible fixes here apt - Different sources.list entries between 18.04 server and 18.04.1 server ISOs - Ask Ubuntu

Next head over to http://zap2xml.awardspace.info/ and download the appropriate version of zap2xml. Use the perl download for linux based systems and the window one for Windows based systems

While you’re here check out the documentation for the command line options

You’ll also need to head over to https://tvlistings.zap2it.com and set up a free account.
Once your account is setup, go ahead and log in to zap2it.
Click on Change provider or Time Zone

Select your country (Canada or United States), and type in your Postal code or Zip Code, then click on find providers. You’ll notice that there are a number of cable, satellite or antenna providers. Since I’m using an antenna for this example, I’ll pick Antenna, then Local Over the Air Broadcast

Check over the channel list to make sure that all the channel’s that you want are there. If there are some that are missing, you can create a second zap2it account and put in a different postal or zip code to find the missing channels and combine the results with zap2xml. I haven’t tried it, but you should also be able to combine antenna and cable epg’s if you need to.

As an example, I live in the niagara region and we get a number of stations from Toronto, Hamilton and Buffalo. When i put my postal code in zap2it, there were some channels that were not showing up that were coming from Buffalo, so I googled buffalo zip code’s, then created a 2nd zap2it account and copied in the zip code and got a EPG listing with the missing channels.

You’ll also notice on the zap2it site that you can “star” channels…this if like making it as a favorite. later when we use the zap2xml script, you’ll have the choice of exporting information for all channels, or just the one’s you’ve starred. If you combine 2 or more EPG sources from zap2it, i’d recommend using this to avoid getting duplicate data since there may very well be overlapping information.

so now we’ve downloaded zap2xml and set up our zap2it account(s)…so the next step would be to create a folder to place the zap2ml script in, as well as the config files and output files that it’ll generate.

I called my epg. On ubuntu you can do something like
mkdir ~/epg
On windows, create a folder like c:\epg

if using ubuntu/linux copy the zap2xml.pl file to ~/epg
Depending on how your system is set up, you may achieve this differently, but since i’m using ubuntu server and have no GUI, i used the filezilla ftp client to transfer the file from my pc over to this folder.

next you’ll need to modify the permission on zap2xml.pl to allow the script to be executed.
chmod 755 ~/epg/zap2xml.pl

If you’re using windows, you’ll just need to run the zap2xml-install.exe file that you downloaded, and choose the epg folder you created as the extract location.

Next you’ll need to create a config file with the settings that zap2xml will use to download the epg. I have one file for each zap2it account. I named my files canada.zap2xmlrc & usa.zap2xmlrc

Just a tip if you’re creating this file from a windows pc, but will be transfering it to ubuntu/linux, do not use notepad, either use something like Notepad++ or learn to use a text editor on linux like vi, nano or even use cat to create the file.

You can find a list of all the option on the zap2xml website. Here is an example of my config canada.zap2xmlrc file

start=0
days=12
ncdays=0
ncsdays=2
retries=2
user=your_zap2it_username
pass=your_zap2it_password
cache=/home/userame/epg/cache1
lang=en
outfile=/home/username/epg/tv_grab_file.xmltv
outformat=xmltv

start if for offseting days, which i don’t want so it’s set to 0
days is for the number of days that you want to download epg data for.
ncdays is days not to cache information for from the end of the epg data.
ncsdays is days not to cache from the start of the epg data. I have this set to 2, so basically in case the lineup changed redownload the latest data and ignore the previously cached data for the today and tomorrow.
user and password if pretty self explanatory
cache if a folder to use to cache the programming guide data that the script downloads. this saves on time so that it doesn’t have to redownload everything every time, and only download new stuff or no cache days data.
outfile is the path of the output xmltv file that we’ll use in plex for the programming guide

if you are planning on downloading multiple epg source, it’s recommend using separate cache folder paths, and separate outfile path.

Next to simplify things we’re going to create a text file (script) with some additional setting that will call zap2xml. afterward you can simply run this script as needed, or create a scheduled task to run it at a specific time. On ubuntu i created named mine create_epg.sh and placed it in the epg folder. here is an example of the contents of the file.

#!/bin/bash
perl /home/username/epg/zap2xml.pl -C /home/username/epg/usa.zap2xmlrc -L -T -b
perl /home/username/epg/zap2xml.pl -C /home/username/epg/canada.zap2xmlrc -J /home/username/epg/usa.xmltv -L -T -a -b

it’s a simple bash script that call the zap2xml perl script, specified the path to the usa.zap2xmlrc config file, and tell zap2xml to output the live tag (-L), not to cache shows marked as to be announced (-T), and retain the channel order(-b). Next run the output for the canada output specified in the canada.zap2xmlrc config and combine it (-J) with the output we just generated (usa.xmltv) and for the canada one, also output all channel (-a)

You’ll also need to make sure that this create_epg.sh script is executable, which you can do with chmod, as we did before.

run this from a unbutu/linux command line type
sh ~/epg/create_epg.sh

If you’re using Windows, create a file named create_epg.bat in the epg folder with similar appropriate settings. Just make sure when your creating the file .bat file that it gets to correct extension and isn’t named create_epg.bat.txt. to avoid this don’t have windows hide file extension type, so you can easily see it and change it if necessary

C:\epg\zap2xml.exe -C C:\epg\usa.zap2xmlrc -L -T -b
C:\epg\zap2xml.exe -C C:\epg\canada.zap2xmlrc -J C:\epg\usa.xmltv -L -T -a -b

once finished double click create_epg.bat

if everything worked you’re epg data should be ready, so let’s test it with plex. Log into your plex media server, and go to settings, then Live TV & DVR. If you already have a DVR setup you’ll probably need to delete it and reset it up…so not any currently recording scheduling options that you have, cause you may loose them.

Next click on set up plex dvr

pick you device and click continue
perform a channel scan if necessary or accept what there if it’s already good, then click continue

Next click on “Have an XMLTV guide on your server? Click here to use that instead”

Then specify the path to you’re xmltv file that we output using zap2xml, and give your epg a title

Plex will try and auto map the channels to those in the zap2xml generated guide, so check it out and make any changes if necessary, then click Continue

At this point it should start loading the guide and you should be good.

If everything worked up to this point, the only thing left of to create a scheduled task to call our create_epg.sh or create_epg.bat script. In linux I used crontab -e to create the schedule job, and in windows i used task scheduler.

It’s getting pretty late, so i’m going to stop here for tonight, and finish explaining the step to create the scheduled jobs in more detail tomorrow.

7 Likes

this may seem like a dumb and only slightly related question, but I don’t see how to add the hdhomerun premium channels to zap2it ?

I don’t have HDHomeRun premium, so I haven’t been able to check it out, but I have a feeling that it’s channel list won’t show up in most online epg as a source, because it isn’t a standard source.

You could always find that premium channel in one of the cable or satellite sources maybe and try using that?

Very nice guide. I am also using zap2xml.

I created 2 zap2it accounts, but access them on separate browsers - there can be issues with the cookies getting confused. This may help someone.

1 Like

chrome > incognito mode :slight_smile:

1 Like

Is anyone else having problems with zap2xml the last few days?

It’s crashing out because it’s hitting some malformed JSON data. I haven’t been able to figure out what the problem is.

I haven’t noticed anything, but i’ll take a peek tonight. Are you getting an error message? if so can you post it. Also what options are you using with zap2xml, and are you using zap2it or tvguide as your source?

The error was that the JSON couldn’t be parsed i.e. the data was malformed.

I wasn’t able to pinpoint the exact problem, but it would always happen with the same .gz file in the cache. I worked around it by copying a good .gz file overtop of the one causing the error.

After that, I was able to grab a weeks worth of data for every one of DirecTVs 600 or so channels.

So it seems to have been a one-off in the data coming from zap2it. I suspect it might have been a special character or something like that. I did try the -e and -E options but that didn’t help.

If I can reproduce it, I’ll post some specifics.

So just as a head’s up, it the how to I said that with ubuntu perl was probably installed my default, but i just tried to build a new plex server with the 18.04.1 LTS build, and seem to be having an issue. There are alot less perl modules that come preinstalled than when i installed with ubuntu server 18.04.

Also on this new build, there are a number of sources / repositories missing, so i was unable to even install some of the perl modules. This post has more details on a possible work around https://askubuntu.com/questions/1065093/different-sources-list-entries-between-18-04-server-and-18-04-1-server-isos

I’m going to try installing it tonight with the traditional installer, instead of the new one to see if i get a different result, and if not then i’ll modify the sources.list file

btw if you try and run zap2xml and get an error message that a perl module is missing you can usually install it with sudo apt install libmodulename-perl where modulename is … you guessed it the name of the missing perl module. ex: sudo apt install libhttp-cookies-perl

I’ll update the main article once i test and find a fix

Just updated the first post with the fixes for installing the additional perl modules needed in ubuntu to run zap2xml

Hi @jrbilodeau

I encountered the error again. I am using zap2it as my source. zap2it is stumbling on a .gz file with size 0.

Command line: zap2xml.pl -u USEREMAIL-p USERPASSWD-U -o zap2it.xml -b -I -D -d 7 -b -e -q
malformed JSON string, neither tag, array, object, number, string or atom, at character offset 0 (before "(end of string)") at zap2xml.pl line 1404.

I captured the output to a log file - the last lines are:

[D] Parsing: EP014000660196
[D] Parsing: EP014000660058
[D] Parsing: EP014000660018
[D] Parsing: EP027781430210
[D] Parsing: EP016169680088
[D] Parsing: EP028822960017
[D] Parsing: MV000315670000
[D] Parsing: MV000238660000
[D] Parsing: EP000369550090
[D] Parsing: EP000369550091
[D] Parsing: EP000369550092
[D] Parsing: EP000369550093

 gunzip OEP000369550093.js.gz

gzip: OEP000369550093.js.gz: unexpected end of file

I copied OEP000369550092.js.gz over OEP000369550093.js.gz and re-ran zap2xml - no problem.

OEP000369550091.js.gz and OEP000369550092.js.gz are for the show “Martin” so I am confident that the 0 size file was also for the same show.

{"seriesTitle":"Martin","seriesDescription":"Stand-up comedian Martin Lawrence drives this irreverent sitcom as a sexist, cocky and wisecracking radio station talk show host. His girlfriend and eventual wife, Gina, puts up with him -- although clashes do occur. Martin's friends, Tommy and Cole, help him get into trouble.","seriesImage":"p184025_l_h9_aa","backgroundImage":"p184025_i_h6_aa","releaseYear":"0","seriesGenres":"Sitcom","overviewTab":{"cast":[{"priority":"1","type":"Cast","role":"Actor","characterName":"Martin Payne","assetId":"990_v9_ba","name":"Martin Lawrence"},{"priority":"2","type":"Cast","role":"Actor","characterName":"Gina Waters","assetId":"62761_v9_ba","name":"Tisha Campbell-Martin"},{"priority":"3","type":"Cast","role":"Actor","characterName":"Pam James","assetId":"234641_v9_bb","name":"Tichina Arnold"},{"priority":"4","type":"Cast","role":"Actor","characterName":"Tommy Strong","assetId":"158893_v9_ba","name":"Tommy Ford"},{"priority":"5","type":"Cast","role":"Actor","characterName":"Cole Brown","assetId":"173541_v9_ba","name":"Carl Anthony Payne"}],"crew":[{"priority":"6","type":"Crew","role":"Executive Producer","characterName":"","assetId":"","name":"Samm-Art Williams"},{"priority":"7","type":"Crew","role":"Executive Producer","characterName":"","assetId":"990_v9_ba","name":"Martin Lawrence"}],"upcomingEpisode":{"episodeTitle":"You've Got a Friend","seasonNumber":"2","episodeNumber":"8","releaseYear":"","synopsis":"Martin (Martin Lawrence) agrees to go shopping with Gina (Tisha Campbell) after she begins spending too much time with a male friend (Darryl Alan Reed).","programGenres":"Sitcom","isNew":false,"isLive":false,"isPremier":false,"isFinale":false,"tags":"STEREO|CC","originalAirDate":"1993-10-10T00:00Z","tmsID":"EP000369550038","displayRating":"TV-PG","timings":[{"time":"2018-11-22T01:00Z","channels":[{"time":"2018-11-22T01:00Z","prgSvcID":"10051","callSign":"BET","lcn":"329","channelAssetId":"s10051_h3_aa"},{"time":"2018-11-22T01:00Z","prgSvcID":"63236","callSign":"BETHD","lcn":"329","channelAssetId":"s63236_h3_aa"}]},{"time":"2018-11-25T08:30Z","channels":[{"time":"2018-11-25T08:30Z","prgSvcID":"10051","callSign":"BET","lcn":"329","channelAssetId":"s10051_h3_aa"},{"time":"2018-11-25T08:30Z","prgSvcID":"63236","callSign":"BETHD","lcn":"329","channelAssetId":"s63236_h3_aa"}]}],"ovdDetails":
...

I’m not sure how to map the episode EP000369550093 to zap2it.com but I’ll see if I can find it and see if there is something suspect in the metadata for that episode.

Well, that wasn’t too hard. I think this is the offending episode of - ahem - Martin.

It’s the one on VH1, I’m not picking up BET listings.

perl /home/username/epg/zap2xml.pl -C /home/username/epg/usa.zap2xmlrc -L -T -b
perl /home/username/epg/zap2xml.pl -C /home/username/epg/canada.zap2xmlrc -J /home/username/epg/usa.xmltv -L -T -a -b

C:\epg\zap2xml.exe -C C:\epg\usa.zap2xmlrc -L -T -b
C:\epg\zap2xml.exe -C C:\canada.zap2xmlrc -J C:\epg\usa.xmltv -L -T -a -b

I believe the second windows command needs to include epg\ in the path before canada

C:\epg\zap2xml.exe -C C:\epg\usa.zap2xmlrc -L -T -b
C:\epg\zap2xml.exe -C C:\epg\canada.zap2xmlrc -J C:\epg\usa.xmltv -L -T -a -b

Any perl gurus know how to modify zap2xml.pl so that it checks for a zero length json buffer and skips decoding it if found?

  if (-e $fn) {
    &pout("[D] Parsing: $cp\n");
    my $gz = gzopen($fn, "rb");
    my $buffer;
    $buffer .= $b while $gz->gzread($b, 65535) > 0;
    $gz->gzclose();
    my $t = decode_json($buffer);

There are a number of calls to decode_json() that should probably be considered - but the one listed above is where I’m running into a problem.

combining multiple EPGs from within plex is a pretty popular feature request. If you want this feature, go here and vote:

Hey @TopDog69,

I’d recommend sending a message to the developper. According to the website any comments or suggestion can be sent to zap2xml@gmail.com

That way, the fix is made available for everyone.

Hope you’re able to get it working.

Thanks for the heads up. I made the correction

I am using a similar method using mc2xml and Schedules Direct subscription to retrieve a merged xmltv guide file, which was the only way I could use both ATSC over-air channels along with the new Silicon Dust Premium TV streams fro the Quattro and Connect devices I have. Do you guys know how channel (network) logos could be tied to these stations for display in the grid view etc? They work using the normal Gracenote lineup method but I am not sure how I could assign logos to each. I have the actual logos locally as I use them this way fro NextPVR

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.