[Cloud Sync] ACD Support removed, last working build is 1.1.3.2700

Plex Media Server Plex Pass Betas
41

@StSimm1 said:
Hi, thanks for all the info. I’ll take a proper look at the logs for both 1.1.4 and 1.2.7 shortly.

However, most of the above snippets do indicate the root cause is Amazon rate limiting (we are working hard to remedy this). It may be that 1.2.X builds of PMS are more sensitive to the rate limit, so Ill check that too.

Have you checked other providers? Are they able to sync in 1.2.X?

Only have Amazon currently, used to use Google Drive and even Dropbox, but completely switched over earlier this year with the unlimited plan that Amazon has. From what I’ve read though, these issues seem to only affect Amazon. Prior to Plex Cloud being released to the beta testers, 1.1.4 uploaded perfectly fine with no issue, so it all does seem to be caused by something on the back-end with how Amazon handles uploads.

I am currently back on 1.1.4 as I’ve been trying to get some files to finish syncing, noticed it even having issues with files less than 20MB (will upload, but not sync).

Additionally, synced content will always have a thumbnail when viewed through Amazon’s Cloud website, and the majority of content “Waiting to be uploaded” (when it really did upload) will not have a thumbnail.

So, while most of the time it can be easy to tell what hasn’t synced merely by looking through the Amazon Drive, occasionally some content with the “Waiting to be uploaded” flag will have a thumbnail, too. All synced content, however, will have a thumbnail from what I’ve seen.

Another thing that stands out is that if I were to try and upload 50 items in groups of 10 each, it would take much longer to sync all 50 than simply doing 50 in one group. More items within a group appears to give a higher probability that more will upload / sync.

Thanks for looking into all of this, hope this can help get things fixed up.

42

@xSean said:
Only have Amazon currently, used to use Google Drive and even Dropbox, but completely switched over earlier this year with the unlimited plan that Amazon has. From what I’ve read though, these issues seem to only affect Amazon. Prior to Plex Cloud being released to the beta testers, 1.1.4 uploaded perfectly fine with no issue, so it all does seem to be caused by something on the back-end with how Amazon handles uploads.

I am currently back on 1.1.4 as I’ve been trying to get some files to finish syncing, noticed it even having issues with files less than 20MB (will upload, but not sync).

Additionally, synced content will always have a thumbnail when viewed through Amazon’s Cloud website, and the majority of content “Waiting to be uploaded” (when it really did upload) will not have a thumbnail.

So, while most of the time it can be easy to tell what hasn’t synced merely by looking through the Amazon Drive, occasionally some content with the “Waiting to be uploaded” flag will have a thumbnail, too. All synced content, however, will have a thumbnail from what I’ve seen.

Another thing that stands out is that if I were to try and upload 50 items in groups of 10 each, it would take much longer to sync all 50 than simply doing 50 in one group. More items within a group appears to give a higher probability that more will upload / sync.

Thanks for looking into all of this, hope this can help get things fixed up.

I can attest to there being some kind of issue with how Amazon processes videos after upload. I find that often times the videos uploaded via Cloud Sync don’t show up for hours, sometimes days (if at all) even though I can confirm the upload actually finished in many (not all) cases. The issue of the thumbnail is also a good indicator of which videos had a problem. I think since the newer interface to Prime Photos was released, there are issues with video processing. I did call Amazon myself asking if one could turn off Prime Photos and tell Plex to not do video processing, ie, just store the files. I was told they cannot turn this off.

This makes me wonder if Plex could do something with changing the file extension or somehow making the videos not look like videos to ACD to resolve these other issues.

43

Just to understand. Only plex pass users can read this thread?
I’ve understood the situation. I just would like to know if I can monitor the evolution of this problem (hope to be fixed in a near future) reading this thread without paying my plex pass.
Thanks

44

@dlogrieco said:
Just to understand. Only plex pass users can read this thread?
I’ve understood the situation. I just would like to know if I can monitor the evolution of this problem (hope to be fixed in a near future) reading this thread without paying my plex pass.
Thanks

Only Plex Pass users can see this topic, but I just keep on checking the Release Announcements for any relevant notes. I’m not sure on this, but you may be able to get any email updates (which include contents of every post) by subscribing to this topic even when you don’t have Plex Pass.

There was another update to PMS (1.3.0), but not sure if it tackled the syncing issues to Amazon since it just references Photo Sync, issues with multiple clients, and playlists.

I’m sticking to 1.1.4 until I see that an update fixed this issue (or until 1.1.4 starts syncing everything).

In other news, tried out the 4.2.4 version of Plex (Preview), still cannot play any Cloud Sync content on any of my Roku 3s. Always getting the message about not supporting transcoding (even when forcing Direct Play). I do know the Roku 3 itself does support transcoding as any locally streamed content still works, just nothing in Cloud Sync. All other devices still can play Cloud Sync content without issue.

45

Tried something new (unrelated to PMS not uploading content to Amazon), so I’ve been uploading all of my content to Amazon Drive directly to be ready for Plex Cloud whenever that opens up further.

I then installed ExpanDrive and let Plex look at one of my media folders and create a new library from it. From there, I went to one of my Roku 3s and played one of the videos that were on Amazon Drive and it played perfectly.

Buffering was quick and resource usage on my computer (which is where PMS was running) was almost nonexistent. Basically had it acting like Plex Cloud, but just with the caveat of needing my PC to be running PMS to act as a line for the Roku to access the content on the Amazon Drive.

So while Cloud Sync simply doesn’t work at all on my Roku 3, streaming content from Amazon Drive through my PC does work just fine.

46

@xSean said:
Tried something new (unrelated to PMS not uploading content to Amazon), so I’ve been uploading all of my content to Amazon Drive directly to be ready for Plex Cloud whenever that opens up further.

I then installed ExpanDrive and let Plex look at one of my media folders and create a new library from it. From there, I went to one of my Roku 3s and played one of the videos that were on Amazon Drive and it played perfectly.

Buffering was quick and resource usage on my computer (which is where PMS was running) was almost nonexistent. Basically had it acting like Plex Cloud, but just with the caveat of needing my PC to be running PMS to act as a line for the Roku to access the content on the Amazon Drive.

So while Cloud Sync simply doesn’t work at all on my Roku 3, streaming content from Amazon Drive through my PC does work just fine.

That is fascinating. What kind of computer / network connection are you running? I’d be interested in doing that myself. ExpanDrive has been very flaky with me on my Mac, but I’m going to assume their Windows development is far far superior.

47

@TitanShadow said:

@xSean said:
Tried something new (unrelated to PMS not uploading content to Amazon), so I’ve been uploading all of my content to Amazon Drive directly to be ready for Plex Cloud whenever that opens up further.

I then installed ExpanDrive and let Plex look at one of my media folders and create a new library from it. From there, I went to one of my Roku 3s and played one of the videos that were on Amazon Drive and it played perfectly.

Buffering was quick and resource usage on my computer (which is where PMS was running) was almost nonexistent. Basically had it acting like Plex Cloud, but just with the caveat of needing my PC to be running PMS to act as a line for the Roku to access the content on the Amazon Drive.

So while Cloud Sync simply doesn’t work at all on my Roku 3, streaming content from Amazon Drive through my PC does work just fine.

That is fascinating. What kind of computer / network connection are you running? I’d be interested in doing that myself. ExpanDrive has been very flaky with me on my Mac, but I’m going to assume their Windows development is far far superior.

My computer has an i7-4790k, 16GB of RAM, and Windows 10 x64, but resource usage for everything were low when I had content being streamed through it, even had a person stream through the library set to access Amazon Drive from 50 miles away without any problem. For my internet speeds, I have 300Mb/s down and 15 Mb/s up, and I know the people streaming cap out at about 50Mb/s down.

I’ve had a few issues so far with ExpanDrive, mostly when trying to upload a lot through it (since I’m copying everything from my local library to it), the connection to Amazon Drive crashes, but not the program itself.

There were some problems going on at Amazon earlier today as Amazon Drive just didn’t work at all; from what I’ve read, that affected pretty much everyone.

48

After about a week of uploading things normally with minimal issues in 1.1.4, I decided to update to 1.3.2.3112…
The latest version of PMS still does not allow any content to upload to Amazon.

Again: This was after a week of uploading while in the build that was PRE-“STREAMING BRAIN”.

The only issues that I have had uploading was one of an older issue wherein a file that takes too long to upload will not sync (in my case, files that are above 6.5GB after conversion). Not sure if that was fixed in builds post-1.1.4, but cannot check for obvious reasons.

I was able to upload / sync over 30 files with no issue and without “bad duplicates” (ones that are missing thumbnails and not associated with any synced content) on the Amazon Drive itself while in 1.1.4.

There are of course the files that did not sync, but were uploaded that were greater than 6.5GB. That bug has been described in detail within the above link.

49

More logs, this time in 1.3.3.3148.
Same issue as before with nothing uploading.

Soon after it finishes the conversion process, it will attempt to upload, but only getting as far as 1% before stopping. Multiple instances of it failing can be seen within the log files.

Dec 16, 2016 03:02:49.503 [10508] DEBUG - CloudSync: Uploading D:\Users\Sean\AppData\Local\Plex Media Server\Cache\Transcode\Sync+\9428839\10\467493.mp4 to https://content-na.drive.amazonaws.com/cdproxy/nodes?suppress=deduplication
Dec 16, 2016 03:02:49.503 [10508] DEBUG - HTTP requesting POST https://content-na.drive.amazonaws.com/cdproxy/nodes?suppress=deduplication
Dec 16, 2016 03:02:49.696 [10508] DEBUG - HTTP 400 response from POST https://content-na.drive.amazonaws.com/cdproxy/nodes?suppress=deduplication
Dec 16, 2016 03:02:49.696 [10508] DEBUG - CloudSync: Finished uploading D:\Users\Sean\AppData\Local\Plex Media Server\Cache\Transcode\Sync+\9428839\10\467493.mp4 to https://content-na.drive.amazonaws.com/cdproxy/nodes?suppress=deduplication with status 400
Dec 16, 2016 03:02:49.696 [10508] DEBUG - CloudSync: Upload failure response body: {“message”:“Multipart request is incomplete: Content required.”}
Dec 16, 2016 03:02:49.696 [10508] DEBUG - CloudSync: Retrying after failed upload. Retry count is now 2

Even though it does say “Finished uploading”, looking at the drive itself shows there is nothing new actually added from that session.

50

Thanks for keeping us in the loop with your findings @xSean! I’m still running 1.1.3 but because of the unpredictable upload success (versus zero upload success on post-streaming brain versions), about 2 weeks ago, I temporarily added Google Drive as a sync option.
Since seeing your post about having a weeks’ worth of success uploading to ACD running pre-streaming brain versions, I am encouraged to try uploading to Amazon again.

Soooo, I’m trying to put ACD as the first choice in my sync order but Plex isn’t remembering this change…It keeps Google Drive as the first option. I started a thread documenting that issue.

Would rather have my stuff synced to the unlimited Amazon Drive versus my smaller, more costly Google Drive.

51

@amccook said:
Soooo, I’m trying to put ACD as the first choice in my sync order but Plex isn’t remembering this change…It keeps Google Drive as the first option. I started a thread documenting that issue.
https://forums.plex.tv/discussion/250128/cloud-sync-re-order-sync-providers#latest
I was able to reproduce this. I believe it is a bug. I will be filing the issue internally.

52

@MovieFan.Plex said:

@amccook said:
Soooo, I’m trying to put ACD as the first choice in my sync order but Plex isn’t remembering this change…It keeps Google Drive as the first option. I started a thread documenting that issue.
https://forums.plex.tv/discussion/250128/cloud-sync-re-order-sync-providers#latest
I was able to reproduce this. I believe it is a bug. I will be filing the issue internally.

…but nothing on everything else documented within this topic?
It is definitely an issue and one that only affects the newer builds.

53

It’s already been mentioned that there are problems with Plex using Amazon Cloud Drive. It’s possible that there was a change in recent versions that made the problem more apparent, but the underlying problem still exists.

54

@MovieFan.Plex said:
It’s already been mentioned that there are problems with Plex using Amazon Cloud Drive. It’s possible that there was a change in recent versions that made the problem more apparent, but the underlying problem still exists.

That simply isn’t the case though.
With the separation of Plex Cloud from ACD, older builds can once again sync normally.

It is only the newer builds post-“Streaming Brain” that still have the issue with uploading.

The only issue in the older builds is a much older issue where the upload will timeout if it takes too long and not sync with the uploaded file, most noticeably on files upwards of 6.5GB (on my upload speed of 15Mb/s).

Honestly now I’m not even sure on the point of trying to document what I’m seeing if it is just being outright ignored.

I’ll summarize everything right here:

Build 1.1.4 and older:

  • Can upload normally
  • Only issue is where large files will timeout
  • This precedes the issue with ACD by several months

Builds newer than 1.1.4 (including latest release):

  • Files will not upload
  • Files will only begin uploading and then immediately stop
  • As the issue is unique to these builds, it is only these builds that have the following line in the log:
    {“message”:“Multipart request is incomplete: Content required.”}
55

It is interesting to note that the amount of details, logs, and observations that I have posted into this topic that just how easy all of that is just pushed to the side for whatever reason.

This is obviously a Plex issue as it only occurs within new builds. Old builds do not have this issue.
This can be consistently reproduced, gives a unique message in the log, and even when the older builds did have an issue it wasn’t the same one.

So please, anyone at Plex, read the posts. Don’t skim or skip through them, just fully read them to get a complete understanding of the issue.

56

This thread by @xSean has been extremely thorough in documenting the issue that Cloud Sync users are experiencing when using Amazon Cloud Drive.
Please spend some time to read though these findings.

57

Unfortunately it seems like they intend to continue claiming that it is on the fault of Amazon without really saying as to why or how.

If something works in an older build, but doesn’t work in a newer one, then there’s obviously something wrong in the newer one. The constant dismissal of the issue only prolongs the length of time that it is active.

It’s not even like it isn’t possible to switch to the older build either, it is all easy to reproduce right now with a simple downgrade.

58

Quite a while for Plex to just ignore Cloud Sync.

59

As I mentioned before, there is nothing that was changed to explain this. If you would like to provide evidence for us to look at please do the following.

  • Restart PMS, sync 1 movie or 1 tv episode you have not synced before and let it fail/finish, delete the sync job, and get me the entire PMS log set.

  • Repeat using the other PMS

Please make sure you have debug logging enabled.

60

I’ve done this so many times it isn’t funny.

As I’ve said in this topic which summed up everything quite well, there is a clear difference in what happens from the older builds vs any later build.