SearchServiceRecord Services and DirectoryObject error...

Dev Community Dev/API Corner
41

Thanks!!! "Summary Page" sounds good...

I still haven't figure out what is malformed in inbox page... Might just use split() like I have to get the job done but like I said using HTML.ElementFromString() elsewhere worked with Requests.

On another note, I thought I read where Plex Media Sever is suppose to use your Local Network Browser settings to process webpages.  I am testing on a Windows 7 64bit machine and I changed the settings to use socks=127.0.0.1:9050 which is the Tor network and yes Tor is running. :huh:    This is so I can see if I can get my code to process StreamCloud.eu site since it blocks a lot of outside countries such as the USA.  To say all that, I still get a HTTPError: HTTP Error 404: Not Found in the plugin log.  But if I stick the URL in the address bar of Internet Explorer with the Proxy settings and it will pull up the site.  I even reset Plex Media Server to make sure...

I thought about adding a setting in my Plugin Prefernces to enable and disable Tor network Proxy settings.  I found a nice Piece of Python code for turning on and off proxy usage in Windows registry....  I will need to add OS checking for Linux and MAC later......

Thanks again for all the help...

42

In my experience, using proxies with PMS does not work well. I wish you luck on that front.

43

You Might be interested in this...  Looking into this code to be able to control socks to socket in python apps...

https://github.com/pagekite/PySocksipyChain/   This is version 2.0 of the socks.py

I was reading in the forum threads where people where asking for a way to control proxy per plugin  and this might be a way.

I was watching this one guys youtube 2 part vids about this and Tor network.

Here is what I have come up with so far....

I tried to call  EnableTorConnect() from  def Start():  but I noticed since the socket server wasn't finished loading it would time out loading the plugin.  So I called it from def MainMenu():  I have added an Enable and Disable in Preferenances...  I need to figure out using this how to tell it not to proxy 127.0.0.1:32400  I think it does not like that because I am getting  loading time out error.    Do you have any ideas on this?  Again, thanks for your help!!!!
 

import socket
import httplib

Import SocksiPy


import sockschain as socks

def DEBUG(msg): Log(msg)

socks.DEBUG = DEBUG


####################################################################################################


Set Up Tor Network Proxy


This will be be used for Hosting sites that Block Countries - StreamCloud.eu & TheFile.me



####################################################################################################

def connectTor():

socks.setdefaultproxy(socks.PROXY_TYPE_SOCKS5, “127.0.0.1”, 9050, True)

socket.socket = socks.socksocket


####################################################################################################

def newIdentity():

socks.setdefaultproxy()

s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

s.connect(“127.0.0.1”, 9051)

s.send("AUTHENTICATE

")

response = s.recv(128)

if response.startswith(“250”):

s.send("SIGNAL NEWNYM

")

s.close()

connectTor()


####################################################################################################

def EnableTorConnect():

# Connect to Tor Proxy Network


CheckProxy = Prefs['disabledTorProxy']

if CheckProxy != 'Disabled':
	connectTor()
	conn = httplib.HTTPConnection("my-ip.heroku.com")
	conn.request("GET", "/")
	response = conn.getresponse()
	Log("MY Tor Proxy IP Address: " + response.read())
44

Ok Guess What, I figured it out with tons of research the Proxy Per Plugin Idea....  After figuring it out, the streamcloud site does not like the post data and sends me back to the same page... I don't think it has anything to do with the proxy just urllib2 from my research and malformed headers or something that the HTTP server is not liking.  Need to do more research see if there is a fix. Or a monkey patch...

Now for the good stuff:
 

####################################################################################################
# Set Up Tor Network Proxy
# This will be be used for Hosting sites that Block Countries - StreamCloud.eu & TheFile.me
####################################################################################################
def create_connection(address, timeout=None, source_address=None):
	sock = socks.socksocket()
	sock.connect(address)
	return sock

####################################################################################################

def connectTor():

socks.usesystemdefaults()

socks.setdefaultproxy(socks.PROXY_TYPE_SOCKS5, “127.0.0.1”, 9150, True)

# patch the socket module

socket.socket = socks.socksocket

socket.create_connection = create_connection


####################################################################################################

def newIdentity():

socks.setdefaultproxy()

s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

s.connect((“127.0.0.1”, 9151))	

s.send("AUTHENTICATE

")

response = s.recv(128)

if response.startswith(“250”):

s.send("SIGNAL NEWNYM

")

else:

Log(“Error: Could not Change Tor IP”)

s.close()

connectTor()


####################################################################################################

def EnableTorConnect():

# Connect to Tor Proxy Network


CheckProxy = Prefs['disabledTorProxy']
ChangeIP = Prefs['disabledgetTorIP']

if CheckProxy != 'Disabled':
	if ChangeIP != 'Disabled':
		newIdentity()
	else:
		connectTor()
	conn = httplib.HTTPConnection("my-ip.heroku.com")
	conn.request("GET", "/")
	response = conn.getresponse()
	Log("MY Tor Proxy IP Address: " + response.read())

I did add in Plex Media Server address to the sockschain function usesystemdefaults():
no_proxy = ['localhost', 'localhost.localdomain', '127.0.0.1', '127.0.0.1:32400']

This fixed the socks5 issue I was having it trying to proxy Plex localhost HTTP server through Tor network. 

This can not be called from Start() because socket server loading...
Call it from:

def MainMenu():

          # Enable Tor Proxy
          EnableTorConnect()

         rest of your function routines......

Any feedback would be nice or thoughts...

 

45

You are probably thinking how do I call this sockschain Python Module.  I put the folder under my URL Service.  and use the following at the top of my _init_.py

import os, sys
path = os.getcwd().split("?\\")[1].split('Plug-in Support')[0]+"Plug-ins\MOVIE2K.bundle\Contents\Services\URL\MOVIE2K"
sys.path.append(path)

This is the path I created for my plugin...  There might be a better way in Plex plugin design just not sure....  I know you can put the in python2.6.zip file under the Plex Media Server install folder but I know people might not like doing that...

Just an update...  Feedback and ideas wanted B)

46

Update had to add a fix to the PySocksipyChain error Socks5 checker that I found online from another guy that ran into the same issue:

From first box to second box:

        # Do a minimal input check first
        if ((not type(destpair) in (list, tuple)) or
            (len(destpair) < 2) or (type(destpair[0]) != type('')) or
            (type(destpair[1]) != int)):
            raise GeneralProxyError((5, _generalerrors[5]))

        # Do a minimal input check first
        if not isinstance(destpair, (list, tuple)) and len(destpair) != 2 and\
                    not isinstance(destpair[0], basestring) and\
                    not isinstance(despair[1], int):
                    raise GeneralProxyError((5, _generalerrors[5] + \
                    '
destpair must be a list or tuple of length two '\
                    'with the first being a string and the second being and int
'\
                    'destpair = %r' % (str(destpair),)))

On another note, does anybody know if you can send header or cookie information for the transcoder to use if a site is looking for something to allow it to play the video from a site?  I think I might have a few sites that might be looking for a certain Host and/or Referer because when it goes to transcode the video it just pops back an error.  I know the video is good because it will play it from the site.  I also have tested with VLC and it does the same thing as the transcoder errors...

Thanks again for your help!

47

I don't believe you can include custom headers but you can include user_agent & cookies with the final video URL if you return it via an @indirect callback rather than a simple Redirect().  Basically, by setting the indirect decorator on your PlayVideo function, you tell clients that they should expect an ObjectContainer as a response instead of a video url. Then when you build the ObjectContainer to return, you can set attributes such as cookies and user_agent which get passed to the clients. I'm not 100% sure if/how the transcoder handles those attributes but I'll do some digging and see what I can find out.

In the end, your code might look something like this:

@indirect
def PlayVideo(url):
  ''' perform whatever url manipulation necessary to get the direct video_url '''
  return ObjectContainer(
    user_agent = "YOUR_USER_AGENT_STRING",
    http_cookies = "YOUR_COOKIES",
    objects = [
      VideoClipObject(
        items = [
          PartObject(key = video_url)
        ]
      )
    ]
  )

In general, if you're not adding user_agent/cookies to the response, you can use the short form IndirectResponse()

return IndirectResponse(VideoClipObject, key=video_url)
48

You can set HTTP headers as well (which can include User Agent and Cookies as well).

  return ObjectContainer(
    http_headers = {'User-Agent': 'Blah', 'Referer': 'http://www.blah.com/'},
    objects = [
      VideoClipObject(
        items = [
          PartObject(key = video_url)
        ]
      )
    ]
  )

Apple Movie Trailers is one of the few URL Services that use this: https://github.com/plexinc-plugins/Services.bundle/blob/master/Contents/Service%20Sets/com.plexapp.plugins.amt/URL/Apple%20Movie%20Trailers/ServiceCode.pys#L181

49

Remind you I am using Service URL to process the video URLs...

When when calling PlayVideo from:

parts.append(PartObject(key=Callback(PlayVideo, url=VideoStream_URL*)))

obj = [MediaObject(
                parts = parts,

I get the following Error message in plug-in log:
 

2013-05-07 13:24:52,400 (764) :  CRITICAL (core:561) - Exception when calling function 'PlayVideo' (most recent call last):
  File "C:\Documents and Settings\Administrator\Local Settings\Application Data\Plex Media Server\Plug-ins\Framework.bundle\Contents\Resources\Versions\2\Python\Framework\code\sandbox.py", line 294, in call_named_function
    result = f(*args, **kwargs)
  File "C:\Documents and Settings\Administrator\Local Settings\Application Data\Plex Media Server\Plug-ins\Framework.bundle\Contents\Resources\Versions\2\Python\Framework\components\runtime.py", line 82, in __call__
    return self._f(*args, **kwargs)
  File "C:\Documents and Settings\Administrator\Local Settings\Application Data\Plex Media Server\Plug-ins\MOVIE2K.bundle\Contents\Services\URL\MOVIE2K\ServiceCode.pys", line 211, in PlayVideo
    objects = [VideoClipObject(items = [PartObject(key = video_url)])]
  File "C:\Documents and Settings\Administrator\Local Settings\Application Data\Plex Media Server\Plug-ins\Framework.bundle\Contents\Resources\Versions\2\Python\Framework\modelling\objects.py", line 215, in __init__
    Container.__init__(self, **kwargs)
  File "C:\Documents and Settings\Administrator\Local Settings\Application Data\Plex Media Server\Plug-ins\Framework.bundle\Contents\Resources\Versions\2\Python\Framework\modelling\objects.py", line 128, in __init__
    self.add(obj)
  File "C:\Documents and Settings\Administrator\Local Settings\Application Data\Plex Media Server\Plug-ins\Framework.bundle\Contents\Resources\Versions\2\Python\Framework\modelling\objects.py", line 162, in add
    raise Framework.exceptions.FrameworkException("Object of type '%s' cannot be added to this container." % str(type(obj)))
FrameworkException: Object of type '' cannot be added to this container.

2013-05-07 13:24:52,414 (764) :  CRITICAL (core:561) - Exception (most recent call last):

File “C:\Documents and Settings\Administrator\Local Settings\Application Data\Plex Media Server\Plug-ins\Framework.bundle\Contents\Resources\Versions\2\Python\Framework\components\runtime.py”, line 837, in handle_request

result = f(**d)

File “C:\Documents and Settings\Administrator\Local Settings\Application Data\Plex Media Server\Plug-ins\Framework.bundle\Contents\Resources\Versions\2\Python\Framework\handlers\services.py”, line 150, in call_service_function

result = self._core.services._call_named_function_in_service(f_name, service, f_args, f_kwargs)

File “C:\Documents and Settings\Administrator\Local Settings\Application Data\Plex Media Server\Plug-ins\Framework.bundle\Contents\Resources\Versions\2\Python\Framework\components\services.py”, line 588, in _call_named_function_in_service

return self._call_named_function_in_sandbox(service.sandbox, fname, None, f_args, f_kwargs, allow_deferred, raise_exceptions, f_optional)

File “C:\Documents and Settings\Administrator\Local Settings\Application Data\Plex Media Server\Plug-ins\Framework.bundle\Contents\Resources\Versions\2\Python\Framework\components\services.py”, line 572, in _call_named_function_in_sandbox

result = sandbox.call_named_function(fname, allow_deferred=allow_deferred, raise_exceptions=raise_exceptions, args=f_args, kwargs=f_kwargs, mod_name=mod_name, optional_kwargs=f_optional)

File “C:\Documents and Settings\Administrator\Local Settings\Application Data\Plex Media Server\Plug-ins\Framework.bundle\Contents\Resources\Versions\2\Python\Framework\code\sandbox.py”, line 294, in call_named_function

result = f(*args, **kwargs)

File “C:\Documents and Settings\Administrator\Local Settings\Application Data\Plex Media Server\Plug-ins\Framework.bundle\Contents\Resources\Versions\2\Python\Framework\components\runtime.py”, line 82, in call

return self._f(*args, **kwargs)

File “C:\Documents and Settings\Administrator\Local Settings\Application Data\Plex Media Server\Plug-ins\MOVIE2K.bundle\Contents\Services\URL\MOVIE2K\ServiceCode.pys”, line 211, in PlayVideo

objects = [VideoClipObject(items = [PartObject(key = video_url)])]

File “C:\Documents and Settings\Administrator\Local Settings\Application Data\Plex Media Server\Plug-ins\Framework.bundle\Contents\Resources\Versions\2\Python\Framework\modelling\objects.py”, line 215, in init

Container.init(self, **kwargs)

File “C:\Documents and Settings\Administrator\Local Settings\Application Data\Plex Media Server\Plug-ins\Framework.bundle\Contents\Resources\Versions\2\Python\Framework\modelling\objects.py”, line 128, in init

self.add(obj)

File “C:\Documents and Settings\Administrator\Local Settings\Application Data\Plex Media Server\Plug-ins\Framework.bundle\Contents\Resources\Versions\2\Python\Framework\modelling\objects.py”, line 162, in add

raise Framework.exceptions.FrameworkException(“Object of type ‘%s’ cannot be added to this container.” % str(type(obj)))

FrameworkException: Object of type ‘<class ‘Framework.modelling.objects.PartObject’>’ cannot be added to this container.

UPDATE: I kind of understand what I should do now looking more at the Apple Trailers that Castle Keeper provided.   Where I am at in the ObjectContainer is the items = [MediaObject(
                parts = parts, ect....] which is the MediaObject Part of the URL Service.  Being this far down in the cycle how do I append the current ObjectContainer with the values of http_headers and http_cookies?

Also, trying to use:  return IndirectReponse(VideoClipObject, key=url)
I get a Log error of:   NameError: global name 'IndirectReponse' is not defined

Any ideas?  Thanks agin for the info in Headers, Cookies and Useragent!

50

Also, trying to use:  return IndirectReponse(VideoClipObject, key=url)

I get a Log error of:   NameError: global name 'IndirectReponse' is not defined

Sorry. Typo on my part. That should be IndirectResponse(). I've corrected the previous post.

51

I didn't even think to look for spelling issue...  :blink:

Ummm, any ideas on how to update the ObjectContainer with the http_headers, http_cookies or user_agent info since I am updating the items part of the VideoObject from with in this part of the URL Service function MediaObjectsForURL(url) .   From within this function how can I back track to update the ObjectContainer with the http_headers, http_cookies or user_agent info?  I call from within this function to get the Hosting site video URL.  OOOOOOOOOOO, ummmm,  I would have to rethink where to call this from....  I would have to call it from the function where I do oc.add(EpisodeObject( or oc.add(MovieObject( then call and pass the Hosting site video url then instead of Movie2k page url............  Talk about moving a lot of code around..... :huh:  I am not sure if this will cause time out issues with the Roku because this why I split it up the way I did.  Now my head hurts!?!?! <_<

52

Either you didn't quite grasp the concept of the using the @indirect, or I don't follow your line of thinking/coding, possibly both.

I'll try to clarify...

When using the @indirect (whether returning an IndirectResponse or a custom-built ObjectContainer), the idea is to pass that from a function callback in the MediaObject. So,

### Plugin Code (__init__.py) ###

def MyVideoMenu():
  '''grab a list of videos and build a directory'''
  oc = ObjectContainer() ### This is not the object container we're adding the headers/cookies to. Although, you could if you needed to do so.
  for video in my_list_of_videos:  
    oc.add(VideoClipObject(url, title, summary, ...))### When the user selects a videos in the menu, the url associated with it is used to call the appropriate URL service.
  return oc


### URL Service Code (ServiceCode.pys) ###
def MetadataObjectForURL(url):
  '''build a MetadataObject for the given URL'''
  return VideoClipObject(title, summary, blahblah)

def MediaObjectsForURL(url):
  '''build a list of MediaObjects for the given url'''
  ### This is the next link in the chain which started with the user selecting a video in the channel ###
  ''' ideally this should be a static list '''
  return [
    MediaObject(
      parts = [ PartObject(key=Callback(PlayVideo, url=url))] ### This line adds a new link to the chain.
      )
    ]

@indirect ### This decorator tells the client to expect an ObjectContainer rather than a direct video url as a response from this function.
def PlayVideo(url):
  ''' add whatever code is necessary to grab the custom cookies/headers required for playback '''
  return ObjectContainer(
    http_headers = My_custom_headers,
    http_cookies = My_session_cookies,
    objects = [
      VideoClipObject(
        items[
          PartObject(key=video_url)
        ]
      )
    ]
  )

If you are unable to execute the code necessary to grab the headers/cookies based solely on the given url and any values stored in the ServicePrefs (username, password, etc.), and they cannot be added statically to the code, then it's probably best not to use an URL Service for your channel. There are a few channels around that use the current framework methods without using URL Services (for one reason or another). Netflix is one, PBS Kids is another, and so is the unofficial IMDb Trailers channel. Take a look at the code for those and it might help you work through the rationale for your situation.

53

Actually come to find out after reading your last code segment I did it this way before when I received the Log error message I posted last:


FrameworkException: Object of type ‘<class ‘Framework.modelling.objects.PartObject’>’ cannot be added to this container.


And I am still getting the above error message in my plug-in log file… I built it exactly like you have it except items = […


I know the following Roku Error Message is caused by the above Python error:

Video Unavailable… Sorry, but we can’t play this video. The original video may no longer be available, or it may be in a format that isn’t supported.


So, we were on the same page and the Plex Media Server I guess is not…


I am going to zip up the ServiceCode.psy and send it to your mail account here on Plex forums… So you can look over at the code and see I did it exactly like you said but maybe I overlooked something again… Thanks for everyone’s help…

54

While waiting on the results and thoughts of the other issues and questions...

EpisodeObject I noticed I can't set generes or MPAA TV ratings (TV-G, TV-PG, TV-14, TV-MA, ect...) they are blank.

Any ideas thanks again!!!

Changing the subject back to one of my older posts.  I was looking on the Amazon channel for the Roku and they have these ratings for their TV shows.  So I guess the Plex Channel does not have these programmed into their Plex Roku channel code yet.

55

Ok, I figured out the malfunction of my error:

@indirect
def PlayVideo(url):
	oc = ObjectContainer()

	try:
		video_url = url.split("?")[0]
		http_cookies = String.Unquote(url.split("cookies=")[1].split("&")[0], usePlus=True)
		http_headers = String.Unquote(url.split("headers=")[1], usePlus=True)
		user_agent = ""
	except:
		video_url = url
		http_cookies = {}
		http_headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.22 (KHTML, like Gecko) Chrome/25.0.1364.152 Safari/537.22'}

	oc.http_headers = http_headers
	oc.http_cookies = http_cookies
	oc.add(VideoClipObject(
		items = [
				MediaObject(
					parts = [PartObject(key=video_url)]
				)
        		]
	))

	return oc

if you Notice the MeidaObject part of the ObjectContainer was missing so it was throwing the error:
FrameworkException: Object of type '' cannot be added to this container.

I do have one question about the http_cookies.  From trying to get it to work it's a string and not a dictionary like http_headers.
So what is the string format the cookies need to be in so the Plex media server uses them.  Like this?:

http_cookies = "pageredir=http%3A%2F%2Fmegavideoz.eu%2Fvideo%2FRY8URWD9AHRH%2FOblivion2013HDCamFOXCD1; PHPSESSID=7gfhq7s3u62ov6qgar42a18fb7"

So thanks again to both of your help now I can go and start working on the sites the need a little extra info to play the videos!!!!!


 

56

I do have one question about the http_cookies.  From trying to get it to work it's a string and not a dictionary like http_headers.
So what is the string format the cookies need to be in so the Plex media server uses them.  Like this?:

http_cookies = "pageredir=http%3A%2F%2Fmegavideoz.eu%2Fvideo%2FRY8URWD9AHRH%2FOblivion2013HDCamFOXCD1; PHPSESSID=7gfhq7s3u62ov6qgar42a18fb7"

That looks correct. Although the values do not have to be URL encoded, according to http://www.nczonline.net/blog/2009/05/05/http-cookies-explained/

57

Good Day to all Plex users out there!!!

I apologize haven't posted sooner but I was working on ideas and fixes for the plug-in. :ph34r:


Thanks for the cookie info, helps a lot.  My next step I am going to try and get this plug-in over on Git-hub to get your eyes on it.  Still have the issue with Plex Web side of it when I try to get to the "Summary Page". 


Also, how can I test to see why a server kicks you back after you post form data to the site.  I have a cookie session set up using requests Python module to get the php cookie sessions and then manually add other cookie values on those the Hosting Sites that are having issues.  I have about four or five Hosting sites that kick me back to the posting page instead of to the next page.  Only, thing I have gleamed from the web is that something to do with malformed headers and the HTTP server kicks it back or Content-Type or Content-Length.  I pass pass 0 for the Content-Length so it will have a value but if I put any other value size in it will cause other posting errors.  For Content-Type I pass value of application/x-www-form-urlencoded.

Here is a good Python Web-Scraping issue as mine:  http://stackoverflow.com/questions/15733078/extremely-strange-web-scraping-issue-post-request-not-behaving-as-expected

But as noted I am using Requests and it does not solve this issue for me.

Can the order of the post data and/or headers cause this issue too?  I am trying to see if importing the Python module OrderedDict can help:  https://pypi.python.org/pypi/ordereddict

Thanks for the Great Forum!

58

Update on form issue.  It looks like StreamCloud looks to see if you have waited the full 10 seconds before proceeding to the next page upon requesting the page before posting the form.  Faststream requires a 5 second wait.  The other sites that I have the issue with returning to the  same page I have not figured out their issues yet.

But, now I have another issue and I know you said Plex Media Server is not friendly with proxy setup.  I have no issues with the Tor Socks 5 proxy network working with the http processing now.  However, when I go to play/transcode the video will not play.  And this is do to the fact of different IP addresses.  The sites will throw up Wrong IP.  The reason is the transcoder is not talking to the Tor network.  So I went in and set up the lan settings in Internet Options to use Tor.  IE, Chrome work fine with it. 

Where does the Plex transcoder get it's internet connection info from?  I did reset the Plex Media Server after I changed Internet Options to do the tests.

Thanks for your great help!

59

Update:

I am playing with a solution for the Plextranscoder called Proxifier for Windows:
http://www.sharewareplaza.com/proxifier-download_22648.html

It watches applications internet connection and routes them through your proxy.  So I set it up to work with Tor Network Proxy.  It works great with the Proxy Socks code in the plugin and when the transcoder kicks up it uses the Proxy.  I am still testing see how reliable it will be.
 

Hope this helps somebody that wants to test Proxy with Plex Media Server and plugin development

60

Update,

Proxifier works well with the PlexTranscoder but it does add it seems a little overhead when you are trying to browse the plugin.  So I launch Proxifier when I want to go play the video if I have Tor enabled in the Plugin.  I might try to see about automating this some.  Hope this helps somebody wanting to try this in your plugin...

Also, just a reminder I did post the plugin to my github account:  https://github.com/Joecowboy/MOVIE2K.bundle

Current major issues:  Does not work with Expermental Transcoder for some reason on the Roku but change the transcoder settings on Plex Channel it will play.  Next, when you try to go to the "Summary Page" on Plex Web it errors out maybe these two issues are related.

I have issues with Filego and Vidhog returning me to the same page upon posting form not sure why but I did put a timer delay for Vidhog of 15 seconds, still no go.  Clicktoview, Vidbux, Vidxden and 180upload use captcha.  So these will be flaky.

Maybe you can check out and see why the two major issues.  I have racked my brain on them. 

Thanks again for all the support!!!