Threading question from Unsuppored App Store

[joecowboy]

@parallelize
def GetUpdateList():
   for num in range(len(Dict['plugins'])):
          @task
          def GetRSSFeed(num=num):


Can you explain what @parallelize and @task do for threading. These are not listed on the threading link.

I am wondering can I use this same logic as you have used inside my TheMovieListings at the while CreatePage:

https://github.com/Joecowboy/MOVIE2K.bundle/blob/master/Contents/Code/__init__.py#L1647

Example:
def TheMovieListings(title, page, date, dateadd, thumb, type, PageOfHosts, Host=None):
     oc = ObjectContainer(title2=title)

     DO STUFF HERE.....
     @parallelize
     def GetMovieList(parameters=parameters):
          while CreatePage:
                @task
                def GetMovieObjects(parameters=parameters):
                     oc.add(stuff.....)
                     CreatPage = True or False

     return oc

Question how do I get CreatePage and oc returned from GetMovieObjects? And would this logic help with the issue of timeouts?  <_<

Thanks for your help as always

[joecowboy]

So still learning some of the ins and outs of Python.  So what you are doing is nested functions.  And the scope of the inner most definition function can see the outer scope but not the other way around.  So now is the trick of being able to update my counters.  I was reading about using lists aka arrays or dictionary to update namespace of the outer scope variable.  But I would only need one element from the array or put all my counters in a dictionary.

So if I were to initialize i = 0 instead use i = [0].  Then in my inner scope i[0] = i[0] + 1. 

So if I were to initialize out = {} out.i = 0 or out = {'i': 0} .  Then in my inner scope out.i =  out.i + 1.

Well I guess Python 2.5 will not let you use the dot operator.  So have to use out['i'] = out['i'] + 1

Reason being from what I have read you can update the namespace of the outer variables.

Question still stands, what does @parallelize and @task do for threading?  <_<

[joecowboy]

def TheMovieListings(title, page, date, dateadd, thumb, type, PageOfHosts, Host=None):
     oc = ObjectContainer(title2=title)

     @parallelize
     def GetMovieList(title=title, page=page, date=date, dateadd=dateadd, thumb=thumb, type=type, PageOfHosts=PageOfHosts, Host=Host):

          DO STUFF HERE.....
 

          out = {'CreatePage': True}

          while out['CreatePage']:
                @task
                def GetMovieObjects():

                      DO STUFF HERE.....

                      if done:

                            out['CreatePage'] = False

     return oc

Question:

Why will not the while loop enter the def GetMovieObjects(): ???  For a test I can change the while to if statement and it will enter def GetMovieObjects():  Are there issues with a while loop wrapped around nested def function?  :blink:

[joecowboy]

I know you all are waiting with baited breath to see if I figured this out...  :P

for num in range(0,TotalHosts):
     if not out['CreatePage']:
          break

The above statement worked in place of the while loop. The variable num is not used but just for an arbitrary counter to make the loop work.

Feed back for the Threading: 

So far so good.  Seems to process the hosts a bit faster with no timeouts.  I have not timed the wait time between threading and non threading version. 

Update:
Went ahead and used the variable num and passed it in  def GetMovieObjects(num=num):  This in turn fixed issue with one of my main counters skipping count.
So new logic layout:

def TheMovieListings(title, page, date, dateadd, thumb, type, PageOfHosts, Host=None):
     oc = ObjectContainer(title2=title)

     @parallelize
     def GetMovieList(title=title, page=page, date=date, dateadd=dateadd, thumb=thumb, type=type, PageOfHosts=PageOfHosts, Host=Host):

          DO STUFF HERE.....
 

          out = {'CreatePage': True}

          for num in range(0,TotalHosts):
                @task
                def GetMovieObjects(num=num):

                      if out['CreatePage']:

                           DO STUFF HERE.....

                           if done:

                                 out['CreatePage'] = False

     return oc

 

[sander1]

This is just a guess, but I think that in order to use @parallelize/@task you need to know the number of tasks that need to be created up front. So a "for" loop will work, because you know how many items need to be processed, but this information is not known when using a "while" loop.

[joecowboy]

This is just a guess, but I think that in order to use @parallelize/@task you need to know the number of tasks that need to be created up front. So a "for" loop will work, because you know how many items need to be processed, but this information is not known when using a "while" loop.

You are right sander.  The for loop did fix it.  However, I am wondering does Threading and PlexConnect and Apple TV3 work together.  I am getting institute11 to run it on his OSX setup and PlexConnect and Apple TV.  There is another member having issues with my plugin now with that setup.  Fun fun..... :wacko: 
 

[Sengian]

Just a question about this, as I tried to read your code but it is a little bit too heavy for casual reading.

Using @task, your are in fact able to thread a function but how do you return data to the main thread?

[mikedm139]

Just a question about this, as I tried to read your code but it is a little bit too heavy for casual reading.

Using @task, your are in fact able to thread a function but how do you return data to the main thread?

Well, I'm no expert on threading. In the UAS, there are two scenarios that involve threading.

First (sequentially but implemented much later chronologically) is the background updater. It never returns anything to the main thread. It just continues on in the background and does not interact with the main thread after it is spawned.

Second, the parallelization of checking for updates. In that case, a separate thread/task is spawned for each installed channel to check for updates and install them as required. Each of those threads only returns once the task is complete.

I believe that it _should_ be possible to pass events from a thread back to the parent process but, i've never played much with it. I think that's one section of the docs that's relatively complete and up to date butI don't know of any examples anywhere.

[Sengian]

I believe that it _should_ be possible to pass events from a thread back to the parent process but, i've never played much with it. I think that's one section of the docs that's relatively complete and up to date butI don't know of any examples anywhere.

You are right about the doc and I know how to do this. Classically, you just create a derived class of Threading with a method returning the value you want.

I am just wondering about @parallelize/@task decorators because if I can do it without the (relative) bother to implement the threading class, I would be happy to do it.