I'm completely lost here, so far this issue makes no sense whatsoever...?
I stored a dict containing sub-dicts of data using the Data() function. The data stores fine, and appears to retrieve fine as well. The problem I am running into is that when I pull a list of keys from a sub-dict, then iterate through it, I keep running into errors stating the key cannot be found? The keys were directly pulled from the dict so I do not understand how this makes any sense? When I do a printout of the dict and failed key names, they are all present in the list. "has_key()" & "if key in dict" suffer from the same issues as well.
It really makes no sense at all, if the list of keys came from the exact same source then how could they not match?
Data Structure (data > genre > item title > item data):
# (Note that title1, title2, etc. are generally unique values, there is no # direct relevance towards the ObjectContainer arguments of the same name)items = {“data”: {“Science”: {“title1”: {“url”: “http://some.url”, “title”: “Some Title”, “rating”: 5.5, etc…},
“title2”: {“url”: “http://some.url”, “title”: “Some Title”, “rating”: 5.5, etc…},
etc…},
“Space”: {“title1”: {“url”: “http://some.url”, “title”: “Some Title”, “rating”: 5.5, etc…},
“title2”: {“url”: “http://some.url”, “title”: “Some Title”, “rating”: 5.5, etc…},
etc…},
etc…
}
}
With some genres, I might have up to +500 values so to save load time I created a per-page formula to limit the number of results returned.
# Assuming that... # ----------------- # genre = "Science" # page = 1 # limit = 20 # -----------------@route("/video/docuheavencom2/categorymenu/documentarymenu", genre=str, page=int, limit=int)
def DocumentaryMenu(genre, page=1, limit=20):
Log(“DocumentaryMenu Started!”)
oc = ObjectContainer(view_group=“List”, title2=“DocumentaryHeaven2”, no_cache=True)
# Load data
items = Data.LoadObject(“data”)
# create starting index using page/limit constraints
index = (page - 1) * limit
# get list of keys for genre
keys = items[“data”][genre].keys()
# Sort key list alphabetically
keys.sort()
# the key ‘count’ is not a data item, move it to the end…
keys.insert(len(keys), keys.pop(keys.index(“count”)))
# Now limit ‘keys’ list to all but the ‘count’ key at the end to
# avoid future errors, followed up by applying the page/limit
# conditions.
keys = keys[:-1][index:index+limit]
# Log the item data by referencing the keys list directly to ensure
# the key is found. Oddly enough, this always seem to find the key,
# yet keys[1] is a key that always fails to be found in the loop?
Log(items[“data”][genre][keys[1]])
# Problem starts from second iteration in list, tried multiple ways
# to find the key but all fail to find most of them…
for key in keys:
#if items[“data”][genre].has_key(key):
#if key in items[“data”][genre]:
if key in items[“data”][genre].keys():
#html = HTML.ElementFromURL(items[“data”][genre].get(k)[“url”])
html = HTML.ElementFromURL(items[“data”][genre][key][“url”])
# All xpaths are tested and working fine…
title = html.xpath("//meta[@property=‘og:title’]")[0].get(“content”).encode(“utf-8”).strip()
thumb_url = html.xpath("//meta[@property=‘og:image’]")[0].get(“content”).encode(“utf-8”).strip()
summary = html.xpath("//meta[@property=‘og:description’]")[0].get(“content”).encode(“utf-8”).strip()
rating = re.findall(“Rating: (.?)/", html.xpath("//div[starts-with(@id, ‘gdr_text_a’)]/text()")[0].encode(“utf-8”).strip())[0]
votes = re.findall(".?((.*?) votes”, html.xpath("//div[starts-with(@id, ‘gdr_text_a’)]/text()")[1].encode(“utf-8”).strip())[0]
genres = “, “.join([genre.encode(“utf-8”).strip() for genre in html.xpath(”//div[@class=‘singleCats’]/a/text()”)])
try:
url = html.xpath("//div[@class=‘video’]/iframe")[0].get(“src”).encode(“utf-8”).strip()
except IndexError:
url = html.xpath("//div[@class=‘video’]//embed")[0].get(“src”).encode(“utf-8”).strip()
except IndexError:
url = None
# Some messy logging to see lots of detail
Log(items[“data”][genre][key][“url”])
Log(title)
Log(rating)
Log(votes)
Log(url)
Log(genres)
Log(summary)
Log("")
Log(genre)
Log(index)
Log(limit)
Log(keys) # log keys list to compare with failed keys
else:
# If key is not found, log a message with the current key
Log(“Cannot find key?”)
Log([key])
Log("")
return oc
Log File:
2014-03-15 13:08:51,213 (1b84) : INFO (logkit:16) - DocumentaryMenu Started!
2014-03-15 13:08:52,302 (b58) : DEBUG (networking:172) - Requesting 'http://documentaryheaven.com/watch-online/'
2014-03-15 13:08:52,312 (1b84) : INFO (logkit:16) - HERE!!
2014-03-15 13:08:52,312 (1b84) : INFO (logkit:16) - {'url': 'http://documentaryheaven.com/911-chronicles-truth-rising/', 'summary': '', 'thumb': '', 'videos': {}, 'title': '9/11 Chronicles: Truth Rising'}
2014-03-15 13:08:52,315 (1b84) : DEBUG (networking:172) - Requesting 'http://documentaryheaven.com/102-minutes-that-changed-america/'
2014-03-15 13:08:52,417 (1b84) : INFO (logkit:16) - http://documentaryheaven.com/102-minutes-that-changed-america/
2014-03-15 13:08:52,418 (1b84) : INFO (logkit:16) - 102 Minutes That Changed America
2014-03-15 13:08:52,420 (1b84) : INFO (logkit:16) - 9.8
2014-03-15 13:08:52,421 (1b84) : INFO (logkit:16) - 19
2014-03-15 13:08:52,421 (1b84) : INFO (logkit:16) - http://www.putlocker.com/embed/F2BFF69FCDF1D086
2014-03-15 13:08:52,423 (1b84) : INFO (logkit:16) - 911, Society
2014-03-15 13:08:52,424 (1b84) : INFO (logkit:16) - 102 Minutes That Changed America is a 102-minute American television special documentary film that was produced by History and premiered commercial-free on September 11, 2008, marking the seventh...
2014-03-15 13:08:52,424 (1b84) : INFO (logkit:16) -
2014-03-15 13:08:52,426 (1b84) : INFO (logkit:16) - ['102 Minutes That Changed America', '9/11 Chronicles: Truth Rising', '9/11 Eyewitness', '9/11 False Flag', '9/11 Mysteries', '9/11 Revisited, Were Explosives Used?', '9/11 The Explosive Reality', '9/11 The Firemens Story', '9/11: 10 Years Later', '9/11: THE SENSIBLE DOUBT', '9/11: The Falling Man', '9/11: The Road To Tyranny', 'Anthrax War', 'Architects & Engineers: Solving the Mystery of WTC 7', 'Confronting The Evidence: A Call To Reopen The 9/11 Investigation', 'Deceptions', 'Elephant in the Room', 'Everybody\xe2\x80\x99s Gotta Learn Sometime', 'Fabled Enemies', 'Fahrenheit 9/11']
2014-03-15 13:08:52,427 (1b84) : INFO (logkit:16) - Society
2014-03-15 13:08:52,427 (1b84) : INFO (logkit:16) - 0
2014-03-15 13:08:52,428 (1b84) : INFO (logkit:16) - 20
2014-03-15 13:08:52,430 (1b84) : INFO (logkit:16) - Cannot find key?
2014-03-15 13:08:52,430 (1b84) : INFO (logkit:16) - ['9/11 Chronicles: Truth Rising']
2014-03-15 13:08:52,431 (1b84) : INFO (logkit:16) -
2014-03-15 13:08:52,433 (1b84) : INFO (logkit:16) - Cannot find key?
2014-03-15 13:08:52,434 (1b84) : INFO (logkit:16) - ['9/11 Eyewitness']
2014-03-15 13:08:52,434 (1b84) : INFO (logkit:16) -
2014-03-15 13:08:52,437 (1b84) : DEBUG (networking:172) - Requesting 'http://documentaryheaven.com/911-false-flag/'
2014-03-15 13:08:52,530 (1b84) : INFO (logkit:16) - http://documentaryheaven.com/911-false-flag/
2014-03-15 13:08:52,532 (1b84) : INFO (logkit:16) - 9/11 False Flag
2014-03-15 13:08:52,533 (1b84) : INFO (logkit:16) - 6.6
2014-03-15 13:08:52,535 (1b84) : INFO (logkit:16) - 50
2014-03-15 13:08:52,536 (1b84) : INFO (logkit:16) - http://www.youtube.com/embed/_9_GRH5V9-E
2014-03-15 13:08:52,536 (1b84) : INFO (logkit:16) - 911, Conspiracy, Society
2014-03-15 13:08:52,539 (1b84) : INFO (logkit:16) - The world has changed after September 11th. It’s changed because we’re no longer safe. These words were used by the George W. Bush, elected President of the United States in 2000, to dictate...
2014-03-15 13:08:52,539 (1b84) : INFO (logkit:16) -
2014-03-15 13:08:52,540 (1b84) : INFO (logkit:16) - ['102 Minutes That Changed America', '9/11 Chronicles: Truth Rising', '9/11 Eyewitness', '9/11 False Flag', '9/11 Mysteries', '9/11 Revisited, Were Explosives Used?', '9/11 The Explosive Reality', '9/11 The Firemens Story', '9/11: 10 Years Later', '9/11: THE SENSIBLE DOUBT', '9/11: The Falling Man', '9/11: The Road To Tyranny', 'Anthrax War', 'Architects & Engineers: Solving the Mystery of WTC 7', 'Confronting The Evidence: A Call To Reopen The 9/11 Investigation', 'Deceptions', 'Elephant in the Room', 'Everybody\xe2\x80\x99s Gotta Learn Sometime', 'Fabled Enemies', 'Fahrenheit 9/11']
2014-03-15 13:08:52,542 (1b84) : INFO (logkit:16) - Society
2014-03-15 13:08:52,542 (1b84) : INFO (logkit:16) - 0
2014-03-15 13:08:52,543 (1b84) : INFO (logkit:16) - 20
2014-03-15 13:08:52,545 (1b84) : INFO (logkit:16) - Cannot find key?
2014-03-15 13:08:52,545 (1b84) : INFO (logkit:16) - ['9/11 Mysteries']
2014-03-15 13:08:52,546 (1b84) : INFO (logkit:16) -
2014-03-15 13:08:52,546 (1b84) : INFO (logkit:16) - Cannot find key?
2014-03-15 13:08:52,549 (1b84) : INFO (logkit:16) - ['9/11 Revisited, Were Explosives Used?']
2014-03-15 13:08:52,552 (1b84) : INFO (logkit:16) -
2014-03-15 13:08:52,553 (1b84) : INFO (logkit:16) - Cannot find key?
2014-03-15 13:08:52,555 (1b84) : INFO (logkit:16) - ['9/11 The Explosive Reality']
2014-03-15 13:08:52,558 (1b84) : INFO (logkit:16) -
2014-03-15 13:08:52,559 (1b84) : INFO (logkit:16) - Cannot find key?
2014-03-15 13:08:52,561 (1b84) : INFO (logkit:16) - ['9/11 The Firemens Story']
2014-03-15 13:08:52,562 (1b84) : INFO (logkit:16) -
2014-03-15 13:08:52,563 (1b84) : INFO (logkit:16) - Cannot find key?
2014-03-15 13:08:52,565 (1b84) : INFO (logkit:16) - ['9/11: 10 Years Later']
2014-03-15 13:08:52,565 (1b84) : INFO (logkit:16) -
2014-03-15 13:08:52,566 (1b84) : INFO (logkit:16) - Cannot find key?
2014-03-15 13:08:52,569 (1b84) : INFO (logkit:16) - ['9/11: THE SENSIBLE DOUBT']
2014-03-15 13:08:52,569 (1b84) : INFO (logkit:16) -
2014-03-15 13:08:52,572 (1b84) : DEBUG (networking:172) - Requesting 'http://documentaryheaven.com/911-the-falling-man/'
2014-03-15 13:08:52,671 (1b84) : INFO (logkit:16) - http://documentaryheaven.com/911-the-falling-man/
2014-03-15 13:08:52,674 (1b84) : INFO (logkit:16) - 9/11: The Falling Man
2014-03-15 13:08:52,676 (1b84) : INFO (logkit:16) - 8.9
2014-03-15 13:08:52,677 (1b84) : INFO (logkit:16) - 30
2014-03-15 13:08:52,680 (1b84) : INFO (logkit:16) - //www.youtube.com/embed/NyR173pp1cU
2014-03-15 13:08:52,681 (1b84) : INFO (logkit:16) - 911, News & Politics, Society
2014-03-15 13:08:52,684 (1b84) : INFO (logkit:16) - The Falling Man, the story of the 'jumpers' of 9/11. It follows the trail of the photographer who nearly lost his life shooting the image; of the citizens who decried it, arguing that it was...
2014-03-15 13:08:52,687 (1b84) : INFO (logkit:16) -
2014-03-15 13:08:52,690 (1b84) : INFO (logkit:16) - ['102 Minutes That Changed America', '9/11 Chronicles: Truth Rising', '9/11 Eyewitness', '9/11 False Flag', '9/11 Mysteries', '9/11 Revisited, Were Explosives Used?', '9/11 The Explosive Reality', '9/11 The Firemens Story', '9/11: 10 Years Later', '9/11: THE SENSIBLE DOUBT', '9/11: The Falling Man', '9/11: The Road To Tyranny', 'Anthrax War', 'Architects & Engineers: Solving the Mystery of WTC 7', 'Confronting The Evidence: A Call To Reopen The 9/11 Investigation', 'Deceptions', 'Elephant in the Room', 'Everybody\xe2\x80\x99s Gotta Learn Sometime', 'Fabled Enemies', 'Fahrenheit 9/11']
2014-03-15 13:08:52,694 (1b84) : INFO (logkit:16) - Society
2014-03-15 13:08:52,696 (1b84) : INFO (logkit:16) - 0
2014-03-15 13:08:52,697 (1b84) : INFO (logkit:16) - 20
2014-03-15 13:08:52,700 (1b84) : INFO (logkit:16) - Cannot find key?
2014-03-15 13:08:52,703 (1b84) : INFO (logkit:16) - ['9/11: The Road To Tyranny']
2014-03-15 13:08:52,704 (1b84) : INFO (logkit:16) -
2014-03-15 13:08:52,707 (1b84) : INFO (logkit:16) - Cannot find key?
2014-03-15 13:08:52,710 (1b84) : INFO (logkit:16) - ['Anthrax War']
2014-03-15 13:08:52,711 (1b84) : INFO (logkit:16) -
2014-03-15 13:08:52,713 (1b84) : INFO (logkit:16) - Cannot find key?
2014-03-15 13:08:52,716 (1b84) : INFO (logkit:16) - ['Architects & Engineers: Solving the Mystery of WTC 7']
2014-03-15 13:08:52,719 (1b84) : INFO (logkit:16) -
2014-03-15 13:08:52,721 (1b84) : INFO (logkit:16) - Cannot find key?
2014-03-15 13:08:52,723 (1b84) : INFO (logkit:16) - ['Confronting The Evidence: A Call To Reopen The 9/11 Investigation']
2014-03-15 13:08:52,726 (1b84) : INFO (logkit:16) -
2014-03-15 13:08:52,730 (1b84) : DEBUG (networking:172) - Requesting 'http://documentaryheaven.com/deceptions/'
2014-03-15 13:08:52,838 (1b84) : INFO (logkit:16) - http://documentaryheaven.com/deceptions/
2014-03-15 13:08:52,841 (1b84) : INFO (logkit:16) - Deceptions
2014-03-15 13:08:52,844 (1b84) : INFO (logkit:16) - 7.0
2014-03-15 13:08:52,845 (1b84) : INFO (logkit:16) - 102
2014-03-15 13:08:52,848 (1b84) : INFO (logkit:16) - http://www.youtube.com/p/27A96CADB973D963?version=3&hl=en_US&fs=1
2014-03-15 13:08:52,851 (1b84) : INFO (logkit:16) - 911, Conspiracy, News & Politics, Society
2014-03-15 13:08:52,852 (1b84) : INFO (logkit:16) -
2014-03-15 13:08:52,855 (1b84) : INFO (logkit:16) -
2014-03-15 13:08:52,858 (1b84) : INFO (logkit:16) - ['102 Minutes That Changed America', '9/11 Chronicles: Truth Rising', '9/11 Eyewitness', '9/11 False Flag', '9/11 Mysteries', '9/11 Revisited, Were Explosives Used?', '9/11 The Explosive Reality', '9/11 The Firemens Story', '9/11: 10 Years Later', '9/11: THE SENSIBLE DOUBT', '9/11: The Falling Man', '9/11: The Road To Tyranny', 'Anthrax War', 'Architects & Engineers: Solving the Mystery of WTC 7', 'Confronting The Evidence: A Call To Reopen The 9/11 Investigation', 'Deceptions', 'Elephant in the Room', 'Everybody\xe2\x80\x99s Gotta Learn Sometime', 'Fabled Enemies', 'Fahrenheit 9/11']
2014-03-15 13:08:52,861 (1b84) : INFO (logkit:16) - Society
2014-03-15 13:08:52,862 (1b84) : INFO (logkit:16) - 0
2014-03-15 13:08:52,865 (1b84) : INFO (logkit:16) - 20
2014-03-15 13:08:52,868 (1b84) : INFO (logkit:16) - Cannot find key?
2014-03-15 13:08:52,871 (1b84) : INFO (logkit:16) - ['Elephant in the Room']
2014-03-15 13:08:52,872 (1b84) : INFO (logkit:16) -
2014-03-15 13:08:52,875 (1b84) : INFO (logkit:16) - Cannot find key?
2014-03-15 13:08:52,877 (1b84) : INFO (logkit:16) - ['Everybody\xe2\x80\x99s Gotta Learn Sometime']
2014-03-15 13:08:52,878 (1b84) : INFO (logkit:16) -
2014-03-15 13:08:52,881 (1b84) : INFO (logkit:16) - Cannot find key?
2014-03-15 13:08:52,882 (1b84) : INFO (logkit:16) - ['Fabled Enemies']
2014-03-15 13:08:52,884 (1b84) : INFO (logkit:16) -
2014-03-15 13:08:52,888 (1b84) : INFO (logkit:16) - Cannot find key?
2014-03-15 13:08:52,890 (1b84) : INFO (logkit:16) - ['Fahrenheit 9/11']
2014-03-15 13:08:52,891 (1b84) : INFO (logkit:16) -
2014-03-15 13:08:52,905 (1b84) : DEBUG (runtime:106) - Sending packed state data (1074 bytes)
2014-03-15 13:08:52,907 (1b84) : DEBUG (runtime:918) - Response: [200] MediaContainer, 423 bytes
2014-03-15 13:08:53,924 (b58) : DEBUG (base:123) - Checking if com.plexapp.plugins.documentaryheaven is broken
2014-03-15 13:08:53,927 (b58) : DEBUG (networking:172) - Requesting 'http://127.0.0.1:32400/:/plugins/com.plexapp.system/messaging/function/X1N0b3JlU2VydmljZTpJc0NoYW5uZWxCcm9rZW4_/Y2VyZWFsMQoxCmxpc3QKMApyMAo_/Y2VyZWFsMQoxCmRpY3QKMQpzMzcKY29tLnBsZXhhcHAucGx1Z2lucy5kb2N1bWVudGFyeWhlYXZlbnMxMAppZGVudGlmaWVycjAK'
2014-03-15 13:08:53,961 (b58) : DEBUG (runtime:106) - Sending packed state data (618 bytes)
2014-03-15 13:08:53,963 (b58) : DEBUG (runtime:918) - Response: [200] MediaContainer, 21653 bytes
2014-03-15 13:08:54,147 (454) : DEBUG (runtime:717) - Handling request GET /video/docuheavencom2
2014-03-15 13:08:54,148 (454) : DEBUG (runtime:49) - Received packed state data (580 bytes)
2014-03-15 13:08:54,151 (454) : DEBUG (runtime:814) - Found route matching /video/docuheavencom2
2014-03-15 13:08:54,151 (454) : INFO (logkit:16) - MainMenu Started!
2014-03-15 13:08:54,153 (454) : INFO (logkit:16) - CategoryMenu Started!
2014-03-15 13:08:54,503 (454) : DEBUG (networking:172) - Requesting 'http://documentaryheaven.com/watch-online/'
2014-03-15 13:08:56,177 (454) : DEBUG (base:123) - Checking if com.plexapp.plugins.documentaryheaven is broken
2014-03-15 13:08:56,180 (454) : DEBUG (networking:172) - Requesting 'http://127.0.0.1:32400/:/plugins/com.plexapp.system/messaging/function/X1N0b3JlU2VydmljZTpJc0NoYW5uZWxCcm9rZW4_/Y2VyZWFsMQoxCmxpc3QKMApyMAo_/Y2VyZWFsMQoxCmRpY3QKMQpzMzcKY29tLnBsZXhhcHAucGx1Z2lucy5kb2N1bWVudGFyeWhlYXZlbnMxMAppZGVudGlmaWVycjAK'
2014-03-15 13:08:56,217 (454) : DEBUG (runtime:106) - Sending packed state data (618 bytes)
2014-03-15 13:08:56,219 (454) : DEBUG (runtime:918) - Response: [200] MediaContainer, 21653 bytes
Now if you search the keys list printout for the missing keys, you will notice that all the missing keys are there...so am I going crazy here or what? :(
What really doesn't make sense is how prior to the loop, referencing the key from the keys list to print out one of the failed items works perfectly fine, but two lines later the same type of reference fails once its iteration starts..?
Successful: Log(items["data"][genre][keys[1]]) reference prior to the loop starting:
2014-03-15 13:08:52,312 (1b84) : INFO (logkit:16) - {'url': 'http://documentaryheaven.com/911-chronicles-truth-rising/' 'summary': '', 'thumb': '', 'videos': {}, 'title': '9/11 Chronicles: Truth Rising'}
Successful: Log(keys) results during loop:
2014-03-15 13:08:52,426 (1b84) : INFO (logkit:16) - ['102 Minutes That Changed America', '9/11 Chronicles: Truth Rising', '9/11 Eyewitness', '9/11 False Flag', '9/11 Mysteries', '9/11 Revisited, Were Explosives Used?', '9/11 The Explosive Reality', '9/11 The Firemens Story', '9/11: 10 Years Later', '9/11: THE SENSIBLE DOUBT', '9/11: The Falling Man', '9/11: The Road To Tyranny', 'Anthrax War', 'Architects & Engineers: Solving the Mystery of WTC 7', 'Confronting The Evidence: A Call To Reopen The 9/11 Investigation', 'Deceptions', 'Elephant in the Room', 'Everybody\xe2\x80\x99s Gotta Learn Sometime', 'Fabled Enemies', 'Fahrenheit 9/11']
Failed: Log("Cannot find key?") & Log([key]) results during loop when key is not found (key is in a list for printout only):
2014-03-15 13:08:52,430 (1b84) : INFO (logkit:16) - Cannot find key?
2014-03-15 13:08:52,430 (1b84) : INFO (logkit:16) - ['9/11 Chronicles: Truth Rising']
So as you can clearly see, prior to the iteration, "9/11 Chronicles: Truth Rising" is found just fine...yet once the iteration begins that same key is unable to be found within the list of keys...while the printout of the keys clearly shows otherwise...
The only ideas I had was to check the data types to see if there was a discrepancy but the built in types() & help() function are not available in your custom build. The only other idea is that the string encoding types might not match? All entries within the dict are manually encoded to utf-8 before being saved to Data(), does Plex do any type of string encoding on the dict entries when storing/retrieving data with Data.LoadObject() & Data.SaveObject() that might be interfering?
Although I cannot really tell if this is a bug or my own stupidity, I thought I'd still mention this has happened with the last two server versions.
All thoughts are appreciated!
