I have a question. Is anyone out there looking into writing a URL service for Yahoo Screens? I ask because I really enjoy the web series Burning Love and would love to be able to access that show in a channel.
I have looked at their site and at first glance, they seem to hide their videos quite well, so being a newbie at this, I know the URL service would not be an easy one for me to tackle. So I figured I would ask if anyone else is trying or has tried to create a URL service for them yet.
Update/Edit:
As mentioned below, there was a URL service and I created a channel.
The Yahoo Screen channel may be downloaded it at
https://github.com/shopgirl284/YahooScreen.bundle
As noted in this forum post and the readme file, Electric City (videos are hosted on another site) and Cybergeddon (the page only allows access to one video at a time) are not available through this channel.
Update/Edit:
09/12/2013:
There has been a complete rewrite of their website. I have just uploaded a new version that works and gives you access to the all the sections of their site again, so you can have access to the channel again. Due to the rewrite of the Yahoo Website, the navigation is much more difficult and cumbersome. I will be attempting to work on this and make changes to the channel over the next couple of days to see if I can improve the channel navigation. I will update everyone here when I have completed all the tweaks to the channel.
Funny you should ask. I actually hacked together an URL Service for Yahoo Screens about a week ago. I think it just went live within the last few days. You should be able to test it against one of your Burning Love video urls using the standard lookup url. When I was working on it, it looked like they have a lot of content. I would encourage you to make a channel for it.
If you run into troubles with the URL service, feel free to make changes. You can always submit a pull request to the official repository if you make any improvements.
You are the man! I am so excited to hear that. I didn't even think to look into the services bundle to see if there was a URL service there.
I will start working on seeing if I can get some kind of channel working for it. They have some very good web series. Many have been nominated and won awards, so it is definitely worth having a channel for it.
I am actually piecing together a channel with all my favorite web series in it from different places, (now that I have Web Therapy from L Studios). It is pretty rough right now. I haven't done alot of automation. Just hard keyed the series that I want into it and then pull the episodes from the webseries' page (like an from an RSS Feed or YouTube Playlist). Though I ran into issues trying to pull up Hulu original shows.
But I will definitely see what I can put together with your URL Service. Thanks again.
Just to let you know, your URL service is working great. I just threw together a basic xpath pull in my webisode channel for Burning Love. The channel is not that pretty yet, but your URL service works great.
What do I need to do to have these show up in my PMC?! I love this show and my wife dies laughing at this! Why is it only on yahoo! AHHHH! We were youtubing the first season and man what a hassle....
Currently, you can use the PlexIt bookmarklet on the Yahoo Screens website to add individual episodes to your myPlex queue for viewing on any Plex client. When shopgirl has a working channel which allows perusing their collection without using a web browser, I'm sure she'll be willing to share ;)
Yes I will. I have got a Yahoo Screen channel started, but it is still quite rough. Yahoo is not consistent in how each show's main page is put together on their site, so I have some tweaking to do.
I may just go ahead and make a separate Burning Love channel for now since that is the most popular show and I have that part of the channel working for the most part. Give me a day or two to see what I can figure out. I will post back here and let you know.
shopgirl, I think that if you watch the http traffic as the pages load, you'll be able to track down the underlying requests and JSON responses that power the website. Then you should be able to build a much more robust channel with a lot less code. You may need to dig through some javascript...
I can't resist...
This is a good place to start:
http://screen.yahoo.com/_xhr/slate-data/?list_id=7b1cbcc0-897a-4f19-a276-833534e9dd8d&start=3&count=21&u1_tlen=50&u1_slen=100&u2_tlen=100&u2_slen=200&u3_tlen=150&u3_slen=300&u4_tlen=200&u4_slen=400&u5_tlen=250&u5_slen=500&pc_starts=4,7,10,13,16,19,22&pc_layouts=3u2u_2u1u-2u1u,3u2u_2u1u-2u1u,3u2u_2u1u-2u1u,3u2u_2u1u-2u1u,3u2u_2u1u-2u1u,3u2u_2u1u-2u1u,3u2u_2u1u-2u1u&display_show_logo=0&spaceid=792883968&mit=TV%20Media%20Slate
If you change the "list_id" valut, you should be able to grab all sorts of different stuff. Messing with the other values might yield some benefit too.
The "list_id" for Burning Love is buried in the the source HTML of the main series page.
YUI.namespace("Media").CONTENT_ID = "776ceaf7-5465-3510-ac37-68e62a53537b"
Have fun :D
Call me a newb… But as an “interested modder” what do you do once you have these sources? I found where you got this by pulling the “Source” from the Burning Love page with firefox… But how do you use that to make a channel?
That is a very broad question. Basically, the channel code is responsible for creating a menu hierarchy. For more detail, I would suggest checking out the blog articles and docs at dev.plexapp.com
Thanks for not resisting.
I was trying to find some json for yahoo screens, but I was looking in the Yahoo Originals page at http://screen.yahoo.com/yahoo-originals/ since that was where all the original shows are listed and I couldn't seem to find any data held in javascript.
I actually was working on a new post to ask about the subject of JSON a little more and how to find it. I will definitely take a look at you link first. It may help me with some of my JSON questions. Then, I will start a new post on the subject of JSON once I know what exactly my questions are. Does that tell you how confused I am on the subject? :-)
The link I posted was requested during the loading of the screens.yahoo.com main page. I believe it’s used to populate the featured content, or whatever they call it. I would guess that there should be a similar request when populating the “originals” page. If not, there’s likely a content_id listed in the originals page source that can be used with that request to populate the JSON you’re looking for.
Re: getting to know JSON. Personally, I find JSON a lot easier to work with than HTML or XML. There’s no need to worry about namespaces or xpath. It’s pretty straight-forward.
I get it is alot easier, like pulling xml. I just am having trouble with how to find it and pull the source into my channel. Though it is starting to make a little more sense now.
The content id for the originals page will not work in that link you gave me. But I will play with it some more and see what I can get.
So if I populate the shows from the originals page using HTML and then use those links to the individual shows to get the content id for each, how would I tell it to pull the JSON data with each content id. I have looked at some a couple other pages and it seems they just call the URL of the page with JSON data or add a few commands to the end like &alt=json. So that has me confused
Can I just call it with:
data = JSON.ObjectFromURL(JSON_url)
where I just use the direct link and manipulate the address to have the correct SHOW_CONTENT_ID?
JSON_url = http://screen.yahoo.com/_xhr/slate-data/?list_id=&start=3&count=21&u1_tlen=50&u1_slen=100&u2_tlen=100&u2_slen=200&u3_tlen=150&u3_slen=300&u4_tlen=200&u4_slen=400&u5_tlen=250&u5_slen=500&pc_starts=4,7,10,13,16,19,22&pc_layouts=3u2u_2u1u-2u1u,3u2u_2u1u-2u1u,3u2u_2u1u-2u1u,3u2u_2u1u-2u1u,3u2u_2u1u-2u1u,3u2u_2u1u-2u1u,3u2u_2u1u-2u1u&display_show_logo=0&spaceid=792883968&mit=TV%20Media%20Slate
If that is the case, I get how it works. And if so, would I always call the link that contains the JSON data I want to pull (like the example above)?
And I will play with the link more to see what I can and cannot take out and make it still work.
Sounds to me like you're on the right track. A couple more things I noticed that might help you out,
- In the JSON URL there params for "start" and "count". By changing those values, you can decide where in the list your results should begin and how many items to load. I would try 0 and 100 (or some relatively big but manageable number).
- On the originals page, when you load the second page of the carousel, there's an HTTP request which includes a list_id. That's probably the one you want to use to get the list of original shows using the JSON URL
- Grabbing the carousel URLs and manipulating it might also be beneficial.
OK I almost have the channel working. But I have problem with a loop. Most of the json data does not have a full http address, so I have to add the beginning to most of the addresses, but I have one channel that has a few links with full http addresses. When I pull up that channel with full urls, it kicks it out of the loop and doesn't return object containers. And I cannot for the life of me figure out what I am doing wrong. It works fine for any other channels that do not have urls that start with http. I know that the problem channels are entering the loop because I had a log message in there that showed it pulled the url.
I know it has to be something simple that I am missing, but I think I am just a little burned out at this point and cannot see it. Any help is greatly appreciated.
Below is the code:
(BTW, some urls were coming up as false, so added the if url to fix that).
for video in data['items']:
url = video['link_url']
Log('url from json is %s '%url)
if url:
title = video['title_short']
thumb = video['image_thumb_url']
duration = video['duration']
duration = Datetime.MillisecondsFromString(duration)
date = video['date']
date = Datetime.ParseDate(date)
if not url.startswith('http://'):
url = YahooURL + url
oc.add(VideoClipObject(
url = url,
title = title,
thumb = thumb,
duration = duration,
originally_available_at = date))
I think I may have figured out my problem. Will JSON data pulls kick you out of the process if one of the fields you are trying to pull is empty?
If the ['key'] you're looking for in the JSON is missing for some entries that will cause an error. If the value of the key is empty and you try to manipulate it, you will often end up with an error having to do with trying to manipulate "None" (or some such). You will likely have to include some error checking to catch those situations. When I'm having those sorts of issues and having trouble figuring out where the error is being generated, I will add A LOT of logging, like the value of every variable being assigned in the For-loop. Then that I can track each one through the loop and compare it to the JSON (or XML or HTML) until I can see where the error is coming from.
With the error above, I ended up finding that the ones without values for certain data were of a different type, so I just separated it into two different pulls based on the type.
I have the channel working, but there were three shows that did not work with the JSON code and of course after working on separate code to handle them I have discovered there are issues with them and the URL service.
Three of the shows give me errors when trying to produce a list of videos with the URL service. The channel log says no service found for URL. It shows three attempts (I am guessing because there are three lines in the code that say no service found for url), then give an error "AttributeError: 'NoneType' object has no attribute 'sandbox'"
I have tried to run some of the urls for the shows through manually with the test address you gave me back when I was writing my URL service. Two of the shows do not work, so there is an issue with them working with the URL service.
But one, Yahoo Animal Allstars works when I take the url in the log and test it through the URL service address, though it just starts playing the video vs returning a video file name. Why would this happen?
