AI Upscaling

AI upscaling on shows up to selected resolution. Either automatic through Library settings or through the optimize menu would be beneficial.

It has been used on old shows to raise the resolution to 4k without any artifacting and it would be nice to utilize all of the new AI upscaling technology on a frame by frame upscale of low resolution shows.

I suppose what you’re asking for is already covered in this existing feature suggestion (AI part is in the description, not in the title). I suggest you comment/vote in that thread in order to avoid distracting/dispersing votes/attention.

Therefore closing this thread as a duplicate.
Edit: corrected

That one is upscaling during transcoding, this one would be utilizing algorithms such as ESRGAN, SFTGAN, or other algorithms on the existing video to upscale the image before the transcoding began to convert 480p into 1080p or 4k videos saved locally. Would not require DLSS or any other product and can be run via CPU or on Raspberry Pis

Would you mind elaborating… maybe it’s just me but I’m not yet fully following your argument – but eager to better understand it :wink:

Absolutely.

Recently there has been a lot of new advances in terms of upscaling images and increasing their resolution. The same technology has been started to apply to old videos and TV shows to increase them to 4k without any artifacts and it looks like it was created that way.

Sorry for posting a link but this is the best visual example I can think of. Here is a 4k machine learning upscale of Ed, Edd, n Eddy that was rendered Ed, Edd n' Eddy 4K AI Upscale Announcement and Release! - YouTube</title

It would be awesome if as a feature we could setup the saved videos that are at a lower resolution and upscale them using an open source algorithm. I am guessing it would involve needing to break out the frames and individually upscaling each one since I don’t know if an open source video algorithm exists for video or if would run on a Raspberry pi as effectively as existing tools.

Just to get you right: you are asking for a feature in terms of upscaling a video before it is played? This upscaled video should then be stored locally? So you actually do not ask for common upscaler (like the Nvidia Shield Pro 2019 “AI upscaler” ) which do their job “on the fly”?
In this case you would need a powerful(!) GPU. I know these upscaling projects (e.g., a big upscaling project of Star Trek Voyager/DS9) and I know that this process is definitely time and resources consuming (up to 10h per SD to 4K upscale using modern/most recent Nvidia GPUs). I could not imagine how this should be implemented in plex beside the fact that I assume that this could be out of the developer’s scope

You cannot change the video picture in any way (that includes increasing its resolution) without decoding/decompressing it first. If you want to store the result of the upscaling, you need to compress the video picture again. Which means this is a complete transcoding process.

Most playback devices or tv screens nowadays already include their own hardware upscalers. Some of them are quite sophisticated, like the above mentioned nVidia Shield “AI upscaler”.
I don’t see any advantage in Plex doing the upscaling, because realistically it will in most cases be inferior in terms of performance and CPU requirements.

1 Like

How obout this “new” feature from nvidia:

Nvidia NGX Technology

and here a documentation.

This sounds like a decent possibility for upscaling videos on the fly and reach a better quality than with regulars upscalers of TVs or the nvidia shield :slight_smile: would be awesome if plex could make use of this feature in future…

1 Like

Another “new feature” from Nvidia wich could improve image quality of old videos:

Nvidia RTX Video Super Resolution (VSR)

This should allready work on the webplayer with accordingly hardware (RTX 30xx / 40xx).

Would be really nice to implement this on server side :slight_smile:

2 Likes