http headers for stream

plugin-dev

#1

Hi,

Is there a way to set headers like refer for a stream? I'm working on a chennel and the final stream url needs to be requested with a refer header or it returns 403.

I've searched the forum and found the following 2 threads. But it seems this has not been solved yet.
https://forums.plex.tv/discussion/100246/individual-request-headers-for-streaming-requests
https://forums.plex.tv/discussion/215815/http-headers-when-opening-a-stream

Regards,
Tony


#2

Try something like this:
Header = {'Referer': 'https://www.example.com'}
Reqest = HTTP.Request('https://www.example.com/streamurl...', headers=Header)


#3

I tried the following 3 methods.
1. put it in PartObject
MediaObject(
parts = [PartObject(key=Callback(PlayVideo1, url=finalurl, headers = headers), http_headers=headers)],
2. put it in IndirectResponse
@indirect
def PlayVideoWithHeader(url=None, headers=None):
return IndirectResponse(VideoClipObject, key= url, http_headers=headers)
3. put it in ObjectContainer
@indirect
def PlayVideo1(url, headers=None):
oc = ObjectContainer(http_headers=headers)
oc.add(VideoClipObject(
items = [
MediaObject(
parts = [PartObject(key=url)]
)
]
))
return oc

None of the above works

Nov 05, 2017 18:45:02.905 [4516] ERROR - [Transcoder] https://yun.fantasy.tv/video/39/146/1508755305440111.mp4?_upt=1ce50eac1509875692: Server returned 403 Forbidden (access denied)


#4

Look at this:
https://github.com/plexinc-plugins/Services.bundle/blob/fb1ecfc63249b7ab1277afbcde1161c3e8ba4046/Contents/Service%20Sets/com.plexapp.plugins.comedycentral/URL/Comedy%20Central/ServiceCode.pys


#5

@flow, Thank you for sharing the url. But I think it's a different case.
The source you mentioned uses header to get the final stream url.
In my case, final file url has already been got, but to access that file, it needs refer in header.


#6

@TonyAtTokyo said:
@flow, Thank you for sharing the url. But I think it's a different case.
The source you mentioned uses header to get the final stream url.
In my case, final file url has already been got, but to access that file, it needs refer in header.

Try

return IndirectResponse(VideoClipObject, key=video_url, http_headers=HTTP_HEADERS)

Ref: https://github.com/plexinc-plugins/Services.bundle/blob/master/Contents/Service%20Sets/com.plexapp.plugins.amt/URL/iTunes%20Movie%20Trailers/ServiceCode.pys#L104


#7

@coder-alpha said:
Try

return IndirectResponse(VideoClipObject, key=video_url, http_headers=HTTP_HEADERS)

Ref: https://github.com/plexinc-plugins/Services.bundle/blob/master/Contents/Service%20Sets/com.plexapp.plugins.amt/URL/iTunes%20Movie%20Trailers/ServiceCode.pys#L104

Thanks for your suggestion, but I've already tried it in my second approach.

  1. put it in IndirectResponse @indirect def PlayVideoWithHeader(url=None, headers=None): return IndirectResponse(VideoClipObject, key= url, http_headers=headers)

Anyway, I will try it again according to the sample you provided. Thank you.


#8

@TonyAtTokyo said:

@coder-alpha said:
Try

return IndirectResponse(VideoClipObject, key=video_url, http_headers=HTTP_HEADERS)

Ref: https://github.com/plexinc-plugins/Services.bundle/blob/master/Contents/Service%20Sets/com.plexapp.plugins.amt/URL/iTunes%20Movie%20Trailers/ServiceCode.pys#L104

Thanks for your suggestion, but I've already tried it in my second approach.

  1. put it in IndirectResponse @indirect def PlayVideoWithHeader(url=None, headers=None): return IndirectResponse(VideoClipObject, key= url, http_headers=headers)

Anyway, I will try it again according to the sample you provided. Thank you.

Sorry missed that. I have had issues in some cases based on the string passed in the headers. They would not encode properly what Plex seems to accept. So you might want to try your referrer header making an HTTP.Request and make sure you're getting a 200 response.


#9

@TonyAtTokyo the approach with IndirectResponse should generally work.

If you try to use it and look at the generated XML, does it contain all headers, particularly cookies, that the website is setting when the video is watched in web browser?

And the second thing to check is whether the client (I guess transcoder in your case) is actually sending those headers to the website. Not sure how to do it with the transcoder, but maybe you could try playing it locally without transcoding and search the Plex client logs and if you see the difference (ie if it works in one case, but not the other), then perhaps you could isolate the problem.


#10

@czukowski @coder-alpha
I tried again and found the following issues:
1. In xml, the referer url will be encoded. e.g. https%3A%2F%2F
2. In Android player, it just ignores such headers.

Well, I give up. I decide to use plex for local materials only and do such funny things with kodi addon which is much more flexiable.

Anyway, thanks for your support.


#11

I don't think encoded value itself should be a problem, if I recall correctly, the headers are sent to the clients as a part of the media URL and as such, they should be encoded. But I hear different Plex clients may or may not supports different aspects of Plex channels API. I think this is a likely cause. I don't use Android player personally, but I've used a few channels that employed similar technique with media request headers and they worked in Plex Home Theater.


#12

@czukowski @TonyAtTokyo

I revisited my issue and I actually can't seem to get it to show the referrer. My url is a .m3u8 so I tried using

return IndirectResponse(VideoClipObject, key=HTTPLiveStreamURL(url), http_headers=http_headers, http_cookies=http_cookies)

that didn't seem to work and then came across objectkit.py and added post_headers which also didn't work.

 --------------------------- objectkit.py ---------------------------
 # Make sure we don't have an extension already.
      if path.split('/')[-1].find('.') == -1:
        path += '.m3u8'
        post_url = url.post_url if hasattr(url, 'post_url') else None
        post_headers = url.post_headers if hasattr(url, 'post_headers') else {}

        url = type(url)(path if query == None else path + query)
        url.post_url = post_url
        url.post_headers = post_headers

      hls_url = IndirectHLSURL(url) if isinstance(url, indirect_callback_string) else CallbackHLSURL(url)

I ended up using

return IndirectResponse(VideoClipObject, key=url, http_headers=http_headers, http_cookies=http_cookies)

which still does not work for the item.

The following is the VLC test file that does play the item fine. The url in the example is expired now so wont wont unless updated.

#EXTM3U
#EXTINF:0,Test
#EXTVLCOPT:http-referrer=http://mycloud.to
#EXTVLCOPT:http-user-agent=Mozilla
https://mdz3ek.mcloud.to/1/CSoCFipfHyyNlzUmug8xkw/0/b/v/a/3rywj2/list.m3u8?e=1510268400

And here is the xml from Plex

!
!
!


I'm using Plex-web for testing and do remember this used to work on it before on the previous Plex-web version (probably 2.xx). Any suggestions or working examples you could point me at. Thanks !


#13

After spending some time on kodi, I found it's very easy to implement on kodi. Just put "|Referer=http://xxxxxx" after the real url and it works.
Plex team should consider to add this feature.


#14

@TonyAtTokyo said:
After spending some time on kodi, I found it's very easy to implement on kodi. Just put "|Referer=http://xxxxxx" after the real url and it works.
Plex team should consider to add this feature.

I personally consider channels are dead. No development from Plex since YEARS. They just were to lazy to add the Plex Home (user restrictions if you want) functionality to it or to publish any newer documentation. I ranted about that circumstance in 2015 but gave it up arguing as the reading comprehension was zero.

Also their channel system works completely different from Kodis.


#15

Yeah the playback of online media in Plex is completely different than Kodi. Kodi has its own player software. But, because Plex works on such a wide variety of players/devices, it must depend on the native player software of each player/device. So there are many situations where it is much easier to make online media play on Kodi than it is on Plex.

But I do share the frustrations of other Plex plugin developers at the lack of response to plugin framework issues. That is why I created the pinned document with all the known issues, Known Plex player app and PMS issues supporting Plugin Framework. I cannot say it will bring about any changes, but I am still submitting issues and including a link to that pinned thread in any discussions I have with Plex app developers. (Maybe the constant reminder and a publicly available list of these issues may encourage change)

So, if you can specify and provide data about a specific issue, please respond to my call for input on known issues at Input for Plex player app/PMS issues with Plugin Framework.


#16

@TonyAtTokyo said:
After spending some time on kodi, I found it's very easy to implement on kodi. Just put "|Referer=http://xxxxxx" after the real url and it works.
Plex team should consider to add this feature.

Yes, Kodi does have quite a few positives but the one-server concept/many clients support of Plex trumps it for many user cases. And this is something easy to be implemented, so I'm hoping for good things to come.

@flow

I'd say lets give it another shot, collectively.

@shopgirl284
Thanks for that thread. What other information can I provide other that what I posted in my above post. I could (if I can find my old PMS installers) try it on an older version where I'm pretty sure this used to work.


#17

I think it must be player-dependent. I've checked the sources of one of my older channels and found this:

From init.py-170:174

    return IndirectResponse(
        VideoClipObject,
        key=media_url,
        http_headers={k: v for k, v in headers.iteritems() if v is not None}
    )

XML generated (notice httpCookies and httpHeaders attributes which I haven't found in @coder-alpha reply):

<MediaContainer
    title1="M*A*S*H online"
    size="1"
    identifier="com.plexapp.plugins.mashonline"
    sourceTitle="M*A*S*H online"
    mediaTagPrefix="/system/bundle/media/flags/"
    httpCookies="__cfduid=d2388eb066e0f8e98231b4042846e76161490998205"
    httpHeaders="Cookie=__cfduid%3Dd2388eb066e0f8e98231b4042846e76161490998205&Referer=http%3A//vidzi.tv/embed-tcdv9gtn9io3-630x475.html" prefsKey="/:/plugins/com.plexapp.plugins.mashonline/prefs">
    <Video sourceIcon="http://resources-cdn.plexapp.com/image/source/com.plexapp.plugins.mashonline.jpg?h=None" key="https://srv31.vidzi.tv/m5uqje4k242z5d7ga4sdtvfv6hka4gpkcqr552t5t7qedz7nikbur4ymvuuq/v.mp4" type="clip">
        <Media optimizedForStreaming="0" height="480" width="640" container="mp4" audioCodec="aac" aspectRatio="1.33" videoCodec="h264" videoResolution="sd">
            <Part container="mp4" key="https://srv31.vidzi.tv/m5uqje4k242z5d7ga4sdtvfv6hka4gpkcqr552t5t7qedz7nikbur4ymvuuq/v.mp4" file="" optimizedForStreaming="0">
                <Stream index="0" selected="1" streamType="1" height="480" width="640" codec="h264" id="1"/>
                <Stream index="1" selected="1" streamType="2" codec="aac" id="2"/>
            </Part>
        </Media>
    </Video>
</MediaContainer>

Then there are lines like this in Plex Home Theater.log:

16:38:30 T:19064  NOTICE: DVDPlayer: Opening: https://srv31.vidzi.tv/m5uqje4k242z5d7ga4sdtvfv6hka4gpkcqr552t5t7qedz7nikbur4ymvuuq/v.mp4|Referer=http%3A//vidzi.tv/embed-tcdv9gtn9io3-630x475.html

So it may actually work like @TonyAtTokyo has posted (|Referer=http://xxxxxx) behind the scenes. I also know the player must take cookies into account, even though they're not mentioned in PHT logs, because the video wouldn't load without the Cloudflare session ID (although it's probably related to DDoS protection rather than keeping nosy users from streaming their hosted media directly).

Note: the same channel can also work in with m3u8 playlists, selectable in channel properties, but as of today's testing, PHT couldn't open them, even though generated XML and player logs were very similar to above.


#18

I think it must be player-dependent. I've checked the sources of one of my older channels and found this:

From init.py-170:174

    return IndirectResponse(
        VideoClipObject,
        key=media_url,
        http_headers={k: v for k, v in headers.iteritems() if v is not None}
    )

XML generated (notice httpCookies and httpHeaders attributes which I haven't found in @coder-alpha reply):

<MediaContainer
    title1="M*A*S*H online"
    size="1"
    identifier="com.plexapp.plugins.mashonline"
    sourceTitle="M*A*S*H online"
    mediaTagPrefix="/system/bundle/media/flags/"
    httpCookies="__cfduid=d2388eb066e0f8e98231b4042846e76161490998205"
    httpHeaders="Cookie=__cfduid%3Dd2388eb066e0f8e98231b4042846e76161490998205&Referer=http%3A//vidzi.tv/embed-tcdv9gtn9io3-630x475.html" prefsKey="/:/plugins/com.plexapp.plugins.mashonline/prefs">
    <Video sourceIcon="http://resources-cdn.plexapp.com/image/source/com.plexapp.plugins.mashonline.jpg?h=None" key="https://srv31.vidzi.tv/m5uqje4k242z5d7ga4sdtvfv6hka4gpkcqr552t5t7qedz7nikbur4ymvuuq/v.mp4" type="clip">
        <Media optimizedForStreaming="0" height="480" width="640" container="mp4" audioCodec="aac" aspectRatio="1.33" videoCodec="h264" videoResolution="sd">
            <Part container="mp4" key="https://srv31.vidzi.tv/m5uqje4k242z5d7ga4sdtvfv6hka4gpkcqr552t5t7qedz7nikbur4ymvuuq/v.mp4" file="" optimizedForStreaming="0">
                <Stream index="0" selected="1" streamType="1" height="480" width="640" codec="h264" id="1"/>
                <Stream index="1" selected="1" streamType="2" codec="aac" id="2"/>
            </Part>
        </Media>
    </Video>
</MediaContainer>

Then there are lines like this in Plex Home Theater.log:

16:38:30 T:19064  NOTICE: DVDPlayer: Opening: https://srv31.vidzi.tv/m5uqje4k242z5d7ga4sdtvfv6hka4gpkcqr552t5t7qedz7nikbur4ymvuuq/v.mp4|Referer=http%3A//vidzi.tv/embed-tcdv9gtn9io3-630x475.html

So it may actually work like @TonyAtTokyo has posted (|Referer=http://xxxxxx) behind the scenes. I also know the player must take cookies into account, even though they're not mentioned in PHT logs, because the video wouldn't load without the Cloudflare session ID (although it's probably related to DDoS protection rather than keeping nosy users from streaming their hosted media directly).

Note: the same channel can also work in with m3u8 playlists, selectable in channel properties, but as of today's testing, PHT couldn't open them, even though generated XML and player logs were very similar to above.


#19

@czukowski
Thanks for the info but which client/version are you using for the xml ?

I installed your channel and using Plex-web (Version 3.20.8) I get a very different xml (below). I see the HTTP.CookiesForURL error (you've already posted about) in my log (included further below), probably due to which the http headers are not showing up but also guessing since you don't get those, probably you are using an older version where I do suspect it used to work.

<MediaContainer title1="M*A*S*H online" noCache="1" size="1" identifier="com.plexapp.plugins.mashonline" sourceTitle="M*A*S*H online" mediaTagPrefix="/system/bundle/media/flags/" prefsKey="/:/plugins/com.plexapp.plugins.mashonline/prefs">
<Video sourceIcon="http://resources-cdn.plexapp.com/image/source/com.plexapp.plugins.mashonline.jpg?h=None" key="/video/mashonline/:/function/CreateVideoObjectContainer?function_args=Y2VyZWFsMQoxCmRpY3QKNAppMApzMTIKc2Vhc29uX2luZGV4aTAKczEzCmVwaXNvZGVfaW5kZXhzMQowczE1CmFzeW5jQ2hlY2tGaWxlc3MxCjFzMTAKY2hlY2tGaWxlc3IwCg__" type="clip" title="Episode 1 - The Pilot" summary=" About this MASH episode: MASH - Pilot is the 1st episode of season one and of course of the series. Returning to the Swamp after a long session in the OR, Hawkeye receives a letter announcing that Ho-Jon has been accepted into his alma mater, though he and Trapper still have the task of coming up with the $2,000 for travel and tuition. Hawkeye convinces Trapper that they can accomplish it by raffling off a weekend pass to Tokyo with the company of a gorgeous nurse. They go to Col. Blake's office to propose the idea to him, and he nervously gives them permission. Later on, Hawkeye and Trapper get into an argument with Frank Burns, who, in a fit of rage, destroys their still. Furious with him, they put a bag over his head and throw him out of the tent. When Blake hears about it, he withdraws the passes and cancels the party they had planned to throw for fear that Burns will complain to General Hammond. He adds that he has to see Hammond in Seoul and was unhappy about the party taking place in his absence. However, as Blake is leaving, Radar reveals that he tricked him into signing two passes, so the party can take place. Unfortunately, Hawkeye's and Trapper's happiness is short lived as they discover that Frank Burns was made temporary commander. To get rid of him so they can have their party, Hawkeye injects him with a sedative and wraps a bandage around his face, prescribing that Frank should be sedated every hour on the hour. During the party, Margaret expresses her inability to find Frank. Suspicious of the activities of Hawkeye and Trapper, she calls General Hammond, who is so excited to hear from her he leaves at once. Meanwhile, Hawkeye announces that they have raised $1800 and then has the nurse draw a name for the raffle. Knowing that the nurse in question, Lt. Dish, is engaged, he announces that Father Mulcahy is the winner, but unfortunately, he does so just as General Hammond walks in. While the infuriated general questions Hawkeye and Trapper, Margaret walks in with Burns, still sedated and with bandages around his head, and screams at the two of them. Hammond demands that they be arrested, but just in the nick of time, choppers arrive loaded with casualties. After the session, which Hammond participates in, he tells Blake that Pierce and McIntyre are two of the best surgeons he has ever seen and, for that reason, he is dropping the charges." ratingKey="/2013/02/pilot.html" thumb="/video/mashonline/episodes/s1e1.jpg">
<Media container="mp4" optimizedForStreaming="0" height="480" width="640" audioCodec="aac" aspectRatio="1.33" videoCodec="h264" videoResolution="sd" indirect="1">
<Part container="mp4" key="/video/mashonline/:/function/GetActualVideoUrl?function_args=Y2VyZWFsMQoxCmRpY3QKMgppMApzMTIKc2Vhc29uX2luZGV4aTAKczEzCmVwaXNvZGVfaW5kZXhyMAo_&indirect=1&mediaInfo=%7B%22audio_channels%22%3A%20null%2C%20%22protocol%22%3A%20null%2C%20%22optimized_for_streaming%22%3A%20false%2C%20%22video_frame_rate%22%3A%20null%2C%20%22duration%22%3A%20null%2C%20%22height%22%3A%20480%2C%20%22width%22%3A%20640%2C%20%22container%22%3A%20%22mp4%22%2C%20%22audio_codec%22%3A%20%22aac%22%2C%20%22aspect_ratio%22%3A%20%221.33%22%2C%20%22video_codec%22%3A%20%22h264%22%2C%20%22video_resolution%22%3A%20%22sd%22%2C%20%22bitrate%22%3A%20null%7D" file="" optimizedForStreaming="0">
<Stream index="0" selected="1" streamType="1" height="480" width="640" codec="h264" id="1"/>
<Stream index="1" selected="1" streamType="2" codec="aac" id="2"/>
</Part>
</Media>
</Video>
</MediaContainer>

! 2017-11-14 17:17:01,286 (3a64) : DEBUG (runtime:125) - Calling function 'GetActualVideoUrl'
! 2017-11-14 17:17:01,313 (3a64) : DEBUG (networking:161) - Fetching 'http://mashonline.blogspot.com/2013/03/hey-look-me-over.html' from the HTTP cache
! 2017-11-14 17:17:01,381 (3a64) : DEBUG (networking:166) - Requesting 'http://vidzi.tv/embed-bswmqy4tn9g3-630x475.html'
! 2017-11-14 17:17:04,039 (3a64) : ERROR (logkit:22) - Error occurred on retrieving episode video URL.
! 2017-11-14 17:17:04,039 (3a64) : CRITICAL (core:574) - get_cookies_for_url() takes exactly 3 arguments (2 given) (most recent call last):
! File "C:\Users\CA\AppData\Local\Plex Media Server\Plug-ins\mashonline.bundle\Contents\Code\loader.py", line 38, in episode_video_url
! 'cookie': HTTP.CookiesForURL(url),
! File "C:\Program Files (x86)\Plex\Plex Media Server\Resources\Plug-ins-46276db8d\Framework.bundle\Contents\Resources\Versions\2\Python\Framework\api
etworkkit.py", line 207, in CookiesForURL
! return self.core.networking.get_cookies_for_url(url)
! TypeError: get_cookies_for_url() takes exactly 3 arguments (2 given)
!

! 2017-11-14 17:17:04,039 (3a64) : DEBUG (logkit:13) - Decoded player data:
! 2017-11-14 17:17:04,039 (3a64) : DEBUG (logkit:13) - {u'hls_maxbufferlength': 300, u'hls_startfromlevel': 1, u'width': u'100%', u'image': u'https://srv27.vidzi.tv/i/01/00131/bswmqy4tn9g3.jpg', u'hls_maxbackbufferlength': 900, u'height': u'100%', u'sources': [{u'file': u'https://srv27.vidzi.tv/hls2/dnuqjj4f242qedz7nikb5gjximaey546g6j6rpoeg,jr6nuklc77oexpo4wrq,ct6nuklc77acr7pi2hq,336nuklc77jhsva3fla,.urlset/master.m3u8'}, {u'file': u'https://srv27.vidzi.tv/dnuqjj4f242qedz7nikb5gjximaey546g6j6rpoeg336nuklc77jhsva3fla/v.mp4'}], u'tracks': [{u'default': True, u'kind': u'subtitles', u'file': u'https://vidzi.tv/srt/00131/bswmqy4tn9g3_English.vtt', u'label': u'English'}, {u'kind': u'subtitles', u'file': u'https://vidzi.tv/srt/00131/bswmqy4tn9g3_Spanish.vtt', u'label': u'Spanish'}, {u'kind': u'thumbnails', u'file': u'https://vidzi.tv/bswmqy4tn9g3.vtt'}], u'base': u'https://vidzi.tv/player7/', u'captions': {u'color': u'FFFFFF', u'fontFamily': u'Verdana', u'fontsize': 20, u'back': False}}
! 2017-11-14 17:17:04,042 (3a64) : CRITICAL (core:574) - Exception when calling function 'GetActualVideoUrl' (most recent call last):
! File "C:\Program Files (x86)\Plex\Plex Media Server\Resources\Plug-ins-46276db8d\Framework.bundle\Contents\Resources\Versions\2\Python\Framework\code\sandbox.py", line 294, in call_named_function
! result = f(*args, **kwargs)
! File "C:\Program Files (x86)\Plex\Plex Media Server\Resources\Plug-ins-46276db8d\Framework.bundle\Contents\Resources\Versions\2\Python\Framework\components\runtime.py", line 85, in __call
_
! return self.f(*args, **kwargs)
! File "C:\Users\CA\AppData\Local\Plex Media Server\Plug-ins\mashonline.bundle\Contents\Code__init
_.py", line 159, in GetActualVideoUrl
! video_urls = loader.episode_video_url(iframe_url, source_type)
! File "C:\Users\CA\AppData\Local\Plex Media Server\Plug-ins\mashonline.bundle\Contents\Code\loader.py", line 48, in episode_video_url
! raise Ex.MediaNotAvailable
! MediaNotAvailable: (2001, 'This media is not currently available.')


And in order to get the Channel working to see if any headers changed I edited the line 38 to 'cookie': 'testcookie',#HTTP.CookiesForURL(url), but no change in xml but the video does play fine. Probably it doesn't need that info anymore ?

EDIT: PMS Version 1.9.5.4339 Win7-64 for sake of completeness


#20

@coder-alpha, I have updated the first post of the Input for Plex player app/PMS issues with Plugin Framework thread to include the specific information to include about an issue.