\
**Category:** Dev/API Corner\
**Tags:** plugin-dev\
**Created:** [July 11, 2016, 7:07am UTC](https://forums.plex.tv/t/html-elementfromurl-execute-javascript/151118 "2016-07-11T07:07:31Z")\
**Posts on this page:** 8\
**Page:** 1
**Author:**  [@robyice](https://forums.plex.tv/u/robyice)\
**Post date:** [July 11, 2016, 7:07am UTC](https://forums.plex.tv/t/html-elementfromurl-execute-javascript/151118/1 "2016-07-11T07:07:31Z")
Hi I am trying to work out the video URL but the site is using javascript to create an Iframe then inside the iframe more javascript creates the video url so with using ElmentFromURL its not even creating the iframe and not executing any of the javascript. I can easily work out what the Iframe url should be but if I try and link to this URL it detects that it has not been loaded from the previous page so don’t display anything. so if I can get it to run all the javascript and build the page correctly getting the video URL will be easy.
---
**Author:**  [@Twoure](https://forums.plex.tv/u/Twoure)\
**Post date:** [July 11, 2016, 8:14am UTC](https://forums.plex.tv/t/html-elementfromurl-execute-javascript/151118/2 "2016-07-11T08:14:37Z")
If your final video URL requires the `Referer` value set within the request header, then you are out of luck. Plex Framework does not support the `Referer` input when playing streams. It only allows the `User-Agent` and `Cookies` to be set. Note: I said **Final** video URL.
Sometimes a site doesn’t need the `Referer`. Many times you can get away with only specifying a browser like `User-Agent` that is consistent between calls.
A good practice is to test what headers you’ll need is by using `curl` in a terminal or cmd (`-I` so only response headers are read).
```auto
curl -I "final_video_url"
```
Normally I setup a test script outside Plex Framework to get the video URL (as-well-as other channel functions before they are Plexified). Then I test the final video URL with `curl` and then with `curl` again but within [https://codeanywhere.com](https://codeanywhere.com) so my IP is different. If it fails, then I start adding in headers one-by-one so I know which are absolutely needed. Also by changing my request IP, I find out if the final video URL has a generate hash tied to my IP. If so, then the URL can only be direct play/stream within the Plex Media Server’s home network.
As-far-as JavaScript, Plex is built off of a modified version of Python 2.7 and uses no JavaScript locally. If you really need to use JavaScript, then look into [PyExecJS](https://github.com/doloopwhile/PyExecJS). Note you will need to specify to your user base that a valid JavaScript Runtime Engine will be need while the server is running.
Please note I’m no official developer, these have been my observations over the past year of Plex channel development. If anyone has corrections to what I’ve said, then please let me know, as the info would benefit me as well.
---
**Author:**  [@robyice](https://forums.plex.tv/u/robyice)\
**Post date:** [July 12, 2016, 4:42am UTC](https://forums.plex.tv/t/html-elementfromurl-execute-javascript/151118/3 "2016-07-12T04:42:47Z")
> @Twoure said:
> If your final video URL requires the `Referer` value set within the request header, then you are out of luck. Plex Framework does not support the `Referer` input when playing streams. It only allows the `User-Agent` and `Cookies` to be set. Note: I said **Final** video URL.
>
> Sometimes a site doesn’t need the `Referer`. Many times you can get away with only specifying a browser like `User-Agent` that is consistent between calls.
>
> A good practice is to test what headers you’ll need is by using `curl` in a terminal or cmd (`-I` so only response headers are read).
>
> ```sh
> curl -I "final_video_url"
>
> ```
>
> Normally I setup a test script outside Plex Framework to get the video URL (as-well-as other channel functions before they are Plexified). Then I test the final video URL with `curl` and then with `curl` again but within [https://codeanywhere.com](https://codeanywhere.com) so my IP is different. If it fails, then I start adding in headers one-by-one so I know which are absolutely needed. Also by changing my request IP, I find out if the final video URL has a generate hash tied to my IP. If so, then the URL can only be direct play/stream within the Plex Media Server’s home network.
>
> As-far-as JavaScript, Plex is built off of a modified version of Python 2.7 and uses no JavaScript locally. If you really need to use JavaScript, then look into [PyExecJS](https://github.com/doloopwhile/PyExecJS). Note you will need to specify to your user base that a valid JavaScript Runtime Engine will be need while the server is running.
>
> Please note I’m no official developer, these have been my observations over the past year of Plex channel development. If anyone has corrections to what I’ve said, then please let me know, as the info would benefit me as well.
Hi Twoure, Thanks for the help, the end url is actually a google video so the stream dont need to pass referer over, the problem I was getting is this.
the main page uses javascript to load an Iframe with a an id of the video. the iframe’s page javascript takes this and has a hash value in the javascript the var name changes daily but its all ways the 3rd one in the javascript, a md5 hash is then made of the video var and then this other hash value from the javascript is then put in front of this a new md5 hash is created but with some elements of its location shifted around, and then the original video var and now this new secirity var are passed to a different web site and this then returns a google video.
I can now get the video var to pass over and I have managed to re write what the javascript is doing to make the security var, the problem I have at the moment is the Iframe page that contains the javascript with the hash in the javascript wont fetch with http.request or html.elementfromurl, from testing with firefox, the only requirement for the page to load is I need to set the referer as the correct page that loads it, no cookies or other values are needed to be passed.
I tried to add , headers={‘Referer’: ‘[http://TheUrlOfCallingPage](http://TheUrlOfCallingPage)’} in the HTTP.Request but it dont seem to do anything.
considering writing a php file on one of my web servers that I can call to handle this and pass the value back, unless someone can think of a better solution
---
**Author:**  [@Twoure](https://forums.plex.tv/u/Twoure)\
**Post date:** [July 12, 2016, 6:13am UTC](https://forums.plex.tv/t/html-elementfromurl-execute-javascript/151118/4 "2016-07-12T06:13:21Z")
> @robyice said:
> I can now get the video var to pass over and I have managed to re write what the javascript is doing to make the security var, the problem I have at the moment is the Iframe page that contains the javascript with the hash in the javascript wont fetch with http.request or html.elementfromurl, from testing with firefox, the only requirement for the page to load is I need to set the referer as the correct page that loads it, no cookies or other values are needed to be passed.
>
> I tried to add , headers={‘Referer’: ‘[http://TheUrlOfCallingPage](http://TheUrlOfCallingPage)’} in the HTTP.Request but it dont seem to do anything.
Just to be sure (sorry), did you test the URL in question via `urllib2` or `requests` in a separate script outside the Plex Framework (i.e. a script separate from your channel and PMS altogether)? Plex uses `urllib2` to build its requests, but `requests` can be easier for a quick test (just depends on your preference).
Doing a simple test with pure python (no Plex) should give you a better idea of what headers are needed. Not sure how you’re testing for headers, but Firefox and Chrome both cache header values and do not always show the info (or indicate that info was cached) while using the developer tools.
Some other methods for retrieving video URLs:
1. If page has an embed address, sometimes can skip obfuscation and `POST` requests when navigating to an embed link
2. If the page has an app released with it, sometimes the way the app retrieves video links is different and may be more straight forward.
3. Change the `User-Agent` to look like a mobile device. Many websites serve mobile devices different stream information as-well-as bypass some of the obfuscation
Otherwise there is not much more I can do without seeing an example webpage/script.
---
**Author:**  [@Twoure](https://forums.plex.tv/u/Twoure)\
**Post date:** [July 16, 2016, 10:57pm UTC](https://forums.plex.tv/t/html-elementfromurl-execute-javascript/151118/5 "2016-07-16T22:57:29Z")
If you need to catch the redirected URL, then a modified version of my code [here](https://github.com/Twoure/UnSupportedServices.bundle/blob/48f07be32288354b00914f083c2ac6bb2eebfeba/Contents/Service%20Sets/com.plexapp.plugins.filerev/URL/FileRev/ServiceCode.pys#L74) can be used as follows:
```auto
import urllib2
import contextlib
def get_url_redirect(url, http_headers):
class NoRedirection(urllib2.HTTPErrorProcessor):
def http_response(self, request, response):
return response
https_response = http_response
opener = urllib2.build_opener(NoRedirection)
opener.addheaders = [(k, http_headers[k]) for k in http_headers.keys()]
try:
with contextlib.closing(opener.open(url)) as resp:
h = resp.headers
return h['Location'] if 'Location' in h.keys() else (h['location'] if 'location' in h.keys() else url)
except urllib2.HTTPError, e:
Log.Error('* get_url_redirect: %s' %str(e))
return url
```
Original code taken from 2nd part of the [answer here](http://stackoverflow.com/questions/554446/how-do-i-prevent-pythons-urllib2-from-following-a-redirect#answer-11744894).
Make sure the `User-Agent` and `Referer` are both set (`urllib2` will need at least the `User-Agent` set to work properly), as-well-as any other headers that may be needed to access the redirected URL.
---
**Author:**  [@sander1](https://forums.plex.tv/u/sander1)\
**Post date:** [August 6, 2016, 1:23pm UTC](https://forums.plex.tv/t/html-elementfromurl-execute-javascript/151118/6 "2016-08-06T13:23:57Z")
If you know you can expect a redirect URL and you want the value of that URL (instead of simply following it and grab the contents), you can also do it with framework only functions (without urllib):
```auto
try:
page = HTTP.Request(url, follow_redirects=False).content
except Ex.RedirectError, e:
if 'Location' in e.headers:
new_url = e.headers['Location']
```
---
**Author:**  [@czukowski](https://forums.plex.tv/u/czukowski)\
**Post date:** [August 27, 2016, 10:10am UTC](https://forums.plex.tv/t/html-elementfromurl-execute-javascript/151118/7 "2016-08-27T10:10:56Z")
I know this is resolved now, but I’d like to mention, for anybody who might be still looking to execute Javascript, I’ve stumbled upon [Js2Py library](https://github.com/PiotrDabkowski/Js2Py "Js2Py") which can parse and interpret (almost) any Javascript code without any other dependencies, which, I think, is just amazing! That said, I didn’t try to run the whole web page with it, but it worked perfectly with just a portion of it where the page decodes the video URL using some weird algorithm.
---
**Author:**  [@system](https://forums.plex.tv/u/system)\
**Post date:** [January 8, 2020, 7:58pm UTC](https://forums.plex.tv/t/html-elementfromurl-execute-javascript/151118/8 "2020-01-08T19:58:36Z")