HTTP.Request loads early

matthewdias · September 28, 2018, 9:19pm

I’ve encountered something pretty strange. When I set the data field for HTTP.Request, the request loads early, without the specified body. If I don’t set data, or I set it to an empty string, it initializes as expected, and doesn’t load until I call request.load() later as it should. The weirder part is I set data in a different request elsewhere and it doesn’t load prematurely.

    request = HTTP.Request(
        OAUTHURL,
        data = 'auth data here'
    )
    try:
        request.load()
        token = JSON.ObjectFromString(request.content)

2018-09-28 16:16:52,208 (700009d41000) :  DEBUG (networking:166) - Requesting 'https://kitsu.io/api/oauth/token'
2018-09-28 16:16:53,256 (700009d41000) :  ERROR (networking:219) - Error opening URL 'https://kitsu.io/api/oauth/token'
2018-09-28 16:16:53,257 (700009d41000) :  CRITICAL (agentkit:1014) - Exception in the search function of agent named 'Kitsu', called with keyword arguments {'openSubtitlesHash': '96518cf312653bd8', 'episode': '1', 'episodic': '1', 'show': 'Cowboy Bebop', 'season': '1', 'filename': '%2FUsers%2FMatthew%2FDownloads%2FPlex%2FCowboy%20Bebop%2F1%2Emp4', 'plexHash': '997b5a78c7a8cbc7ae3f0d1b80f60a418462a901', 'duration': '140054', 'id': '145'} (most recent call last):
  File "/Applications/Plex Media Server.app/Contents/Resources/Plug-ins-21ab172de/Framework.bundle/Contents/Resources/Versions/2/Python/Framework/api/agentkit.py", line 1007, in _search
    agent.search(*f_args, **f_kwargs)
  File "/Users/Matthew/Library/Application Support/Plex Media Server/Plug-ins/Kitsu.bundle/Contents/Code/__init__.py", line 18, in search
    search_anime('tv', results, media, lang)
  File "/Users/Matthew/Library/Application Support/Plex Media Server/Plug-ins/Kitsu.bundle/Contents/Code/search.py", line 19, in search_anime
    'X-Algolia-API-Key': algolia_key()
  File "/Users/Matthew/Library/Application Support/Plex Media Server/Plug-ins/Kitsu.bundle/Contents/Code/kitsu.py", line 90, in algolia_key
    return anon_key
  File "/Users/Matthew/Library/Application Support/Plex Media Server/Plug-ins/Kitsu.bundle/Contents/Code/kitsu.py", line 21, in authenticate
    return login(username, password)
  File "/Users/Matthew/Library/Application Support/Plex Media Server/Plug-ins/Kitsu.bundle/Contents/Code/kitsu.py", line 42, in login
    OAUTHURL,
  File "/Applications/Plex Media Server.app/Contents/Resources/Plug-ins-21ab172de/Framework.bundle/Contents/Resources/Versions/2/Python/Framework/api/networkkit.py", line 194, in Request
    method=method,
  File "/Applications/Plex Media Server.app/Contents/Resources/Plug-ins-21ab172de/Framework.bundle/Contents/Resources/Versions/2/Python/Framework/api/networkkit.py", line 67, in _http_request
    req = self._core.networking.http_request(url, *args, **kwargs)
  File "/Applications/Plex Media Server.app/Contents/Resources/Plug-ins-21ab172de/Framework.bundle/Contents/Resources/Versions/2/Python/Framework/components/networking.py", line 370, in http_request
    return HTTPRequest(self._core, url, data, h, url_cache, encoding, errors, timeout, immediate, sleep, opener, follow_redirects, method)
  File "/Applications/Plex Media Server.app/Contents/Resources/Plug-ins-21ab172de/Framework.bundle/Contents/Resources/Versions/2/Python/Framework/components/networking.py", line 141, in __init__
    self.load()
  File "/Applications/Plex Media Server.app/Contents/Resources/Plug-ins-21ab172de/Framework.bundle/Contents/Resources/Versions/2/Python/Framework/components/networking.py", line 181, in load
    f = self._opener.open(req, timeout=self._timeout)
  File "/Applications/Plex Media Server.app/Contents/Resources/Plug-ins-21ab172de/Framework.bundle/Contents/Resources/Platforms/Shared/Libraries/urllib2_new.py", line 444, in open
    response = meth(req, response)
  File "/Applications/Plex Media Server.app/Contents/Resources/Plug-ins-21ab172de/Framework.bundle/Contents/Resources/Platforms/Shared/Libraries/urllib2_new.py", line 557, in http_response
    'http', request, response, code, msg, hdrs)
  File "/Applications/Plex Media Server.app/Contents/Resources/Plug-ins-21ab172de/Framework.bundle/Contents/Resources/Platforms/Shared/Libraries/urllib2_new.py", line 482, in error
    return self._call_chain(*args)
  File "/Applications/Plex Media Server.app/Contents/Resources/Plug-ins-21ab172de/Framework.bundle/Contents/Resources/Platforms/Shared/Libraries/urllib2_new.py", line 416, in _call_chain
    result = func(*args)
  File "/Applications/Plex Media Server.app/Contents/Resources/Plug-ins-21ab172de/Framework.bundle/Contents/Resources/Platforms/Shared/Libraries/urllib2_new.py", line 565, in http_error_default
    raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
HTTPError: HTTP Error 401: Unauthorized

Elsewhere I have the following which works as expected

    request = HTTP.Request(
        'https://' + ALGOLIA_APP_ID + '-dsn.algolia.net/1/indexes/production_media/query',
        headers = {
            'Content-Type': 'application/json',
            'X-Algolia-Application-Id': ALGOLIA_APP_ID,
            'X-Algolia-API-Key': algolia_key()
        },
        data = '{"params":"query=' + query + '&facetFilters=[' + filters + ']"}'
    )
    try:
        request.load()

Can anyone shed some light on this?

czukowski · September 29, 2018, 6:38am

@matthewdias I don’t think the request loads early, according to the log it’s failed with “HTTP Error 401: Unauthorized”. Usually this means the server has not received any authentication headers it was expecting.

matthewdias · September 29, 2018, 6:56am

Yes, but if it was loading when it was supposed to, it would get caught in the try block, and wouldn’t log the trace. Which to me indicates that the 401 is coming because the authentication data doesn’t even get set before the request is sent. (My snippet above is just example code of course, I’ve tried it with a correct request and it still errored.)

czukowski · September 29, 2018, 12:35pm

Okay, looking more closely:

...
  File "/Applications/Plex Media Server.app/Contents/Resources/Plug-ins-21ab172de/Framework.bundle/Contents/Resources/Versions/2/Python/Framework/components/networking.py", line 370, in http_request
    return HTTPRequest(self._core, url, data, h, url_cache, encoding, errors, timeout, immediate, sleep, opener, follow_redirects, method)
  File "/Applications/Plex Media Server.app/Contents/Resources/Plug-ins-21ab172de/Framework.bundle/Contents/Resources/Versions/2/Python/Framework/components/networking.py", line 141, in __init__
    self.load()
...

And Framework.bundle/Contents/Resources/Versions/2/Python/Framework/components/networking.py:

140       if immediate:
141         self.load()

Apparently, the immediate parameter seems to be the culprit as it’s somehow made its way into HTTP.Request parameters.

Why won’t you put HTTP.Request initialization into the try block as well? If you don’t intend to delay executing the request by a substantial amount of time, you might as well execute it immediately. By the way, request.content executes the load method implicitly

matthewdias · September 29, 2018, 4:35pm

I have tried setting immediate to false, even tho the default is true, to no avail. Of course I could move the initialization to the try block, which would catch the error, but that still doesn’t solve the problem of the request sending before the request instance is returned. Because of that I can’t check the validity of the request after it errors. I’ve found that using values instead of data works so I’m just going to use that instead. The auth endpoint I’m hitting accepts form data too.

czukowski · September 29, 2018, 8:55pm

Albeit it’s very tedious, you could edit networking.py temporarily and add logging calls to see the function arguments and gradually work your way up the stack trace to find out where the immediate parameter comes from (if I recall correctly, it was false by default). It is also possible to connect a Python debugger, but that involves even more work to set up.

But if what you’re doing is essentially this:

    request = HTTP.Request(...)
    try:
        request.load()
        token = do_something_with(request.content)

Then you can safely add HTTP.Request to the try block. Apart from the possible immediate execution, it’s effectively same as:

    try:
        token = do_something_with(HTTP.Request(...).content)

I can’t see your except block, but the underlying Plex code uses urllib2 which raises standard URLErrors or HTTPErrors, so it’s entirely possible to examine HTTP response codes in case of errors. Always worked for me at least

matthewdias · September 29, 2018, 11:28pm

I tried catching HTTPError before and I’m pretty sure it gave me an error on interpret. But if I experience this again elsewhere I’ll try again and also take a look at debugging networking.py. Thanks for the tips.

system · December 28, 2018, 11:28pm

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Get http headers in v2 requests Dev/API Corner plugin-dev	7	143	December 20, 2019
HTTP Request POST Data Dev/API Corner plugin-dev	5	378	December 20, 2019
Read current X-Plex-Token in an agent & ensure that a HTTP.Request gets executed exactly once Dev/API Corner scanner-agent-dev	5	317	January 8, 2020
Metadata agent suddenly stops working Dev/API Corner scanner-agent-dev	5	145	December 20, 2019
HTTP.SetPassword Error Dev/API Corner other-dev	4	93	December 20, 2019

HTTP.Request loads early

Related topics