@cjcox said:
Also, seem to be some confusion between integrated GPU acceleration and features like QuickSync. Not all Intel CPUs have QuickSync, and GPU acceleration will vary abit depending on CPU. So… the original thread is about QuickSync, so for those that don’t have good hw decoding, I’d start a different thread for that.
Just two I found, I am sure others are there, just not going to copy and paste all of them. I’ll look later if any of these or the other finds via search shed more light on this. But I suspect I will just find excuses and reasons why this isn’t even touched at all.
@cayars said:
Anything over FFmpeg 2.8 already has QuickSync built in. It’s just a matter of Plex wanting to use it or not.
It’s not 100% straight forward as you can’t just take the command line you have and switch the encoder. Some of the options have to change as well.
And so much more than one usually thinks about. There are always tons of hold ups in projects like this. Using special features like this likely needs a new compiler which in turn requires big or small changes in the entire toolchain.
Nah, no new compiler would be needed. Nothing changes code compiler wise to cause this. It’s just a matter of linking in the new Intel libraries. It’s already part of FFmpeg so the hard part of this is already done for Plex (to build the exe). Of course this isn’t available on all platforms but many of the popular ones.
What will be harder for Plex and will require some thought and testing is the differences in the command lines sent to FFmpeg/transcoder. The logic used by Plex to build the command lines will need to branch based on the type of encode (CPU, QuickSync, Nvidia, ATI). That in itself won’t be that difficult but they may run into some issues on live streaming.
Since the command lines used for transcoding to files (sync and MO) are on the easier side, what they could do as PHASE1 is to incorporate a flag “Use Experimental Quick Sync Encoding” in the admin panel. Then start by incorporating Quick Sync in MP4 creation for Media Optimizer, Client Sync, Cloud Sync engines. These may be much more simple than a live streaming situation to start with since they are creating FILE BASED versions and the command lines aren’t that much different in use. This isn’t much different than the batch files and scripts we use offline to transcode our files.
It would be a start and could save many CPU resources for those using MO and Sync stuff. This is certainly the approach I would take to start to incorporate this functionality, but that’s just me.
@cayars said:
Anything over FFmpeg 2.8 already has QuickSync built in. It’s just a matter of Plex wanting to use it or not.
It’s not 100% straight forward as you can’t just take the command line you have and switch the encoder. Some of the options have to change as well.
And so much more than one usually thinks about. There are always tons of hold ups in projects like this. Using special features like this likely needs a new compiler which in turn requires big or small changes in the entire toolchain.
Atrus,
I thinks we all understand that this is not a simple project and not something that one can slap on and ducktape on code wise. However what troubles me and others is the fact that there is no recent feedback AT ALL from any Plex Employee on this topic. At least a statement or update or show of any interest would help. Even if it bad news. (correct me if I am wrong here). We are all speculating, hoping and fighting with each other in terms of who is right or wrong. And all during this time not a peep from Plex.
We don’t want excuses or deflections. Just be honest and frank with us and we will respect that more than more of the same silence. I think we all want a win win solution here. Hell, even if this means Plex saying we need more money for more developers and raising prices. Most of us know time is money and are willing to pay for it.
What concerns me is you guys will be rolled over by all these other guys that are enabling GPU transcoding. I would hate to move to another solution, but would not hesitate once the other options are more appealing.
Most people here are probably technical enough to understand that supporting QS requires some significant work on the part of Plex. Also that there are many features that are being asked for, and that not all of them can be implemented at once. That said, the lack of engagement on this is still quite odd. QS has the potential to improve Plex’s core function - making media available from anywhere - tremendously. It has the potential to expand Plex’s market, as customers would no longer need serious hardware for HD transcoding. (People running an Intel powered NAS, or a small BayTrail server would become likely PlexPass customers) How a small company can ignore something that both improves its product’s core functionality and expands its market for so long is beyond me.
We also know that both Emby and QNAP already have QS support for their media streaming products. (And QNAP has had it for some time now) If Plex’s competitors are able to do this, the excuses heard here seem like grasping at straws.
@cayars said:
Anything over FFmpeg 2.8 already has QuickSync built in. It’s just a matter of Plex wanting to use it or not.
It’s not 100% straight forward as you can’t just take the command line you have and switch the encoder. Some of the options have to change as well.
And so much more than one usually thinks about. There are always tons of hold ups in projects like this. Using special features like this likely needs a new compiler which in turn requires big or small changes in the entire toolchain.
Atrus,
I thinks we all understand that this is not a simple project and not something that one can slap on and ducktape on code wise. However what troubles me and others is the fact that there is no recent feedback AT ALL from any Plex Employee on this topic. At least a statement or update or show of any interest would help. Even if it bad news. (correct me if I am wrong here). We are all speculating, hoping and fighting with each other in terms of who is right or wrong. And all during this time not a peep from Plex.
We don’t want excuses or deflections. Just be honest and frank with us and we will respect that more than more of the same silence. I think we all want a win win solution here. Hell, even if this means Plex saying we need more money for more developers and raising prices. Most of us know time is money and are willing to pay for it.
What concerns me is you guys will be rolled over by all these other guys that are enabling GPU transcoding. I would hate to move to another solution, but would not hesitate once the other options are more appealing.
Sorry about the edits of this, just rambling on…
I do not agree. Not long before christmas the main guy for all things ffmpeg and transcoder for Plex very much said that it would take a long time for it to be integrated (in another thread about the same exact subject). It was mentioned, by me, in this thread (yesterday) that he had said just that. If an employee says it will take a long time, he/she will not be going through the countless threads about adding support for it as he/she has already made a statement about it.
@rcombs said:
Sorry about the confusion earlier, Atrus was slightly confused regarding the differences between x264’s OpenCL support and Intel’s ISMD and QuickSync video, and asked me to drop in and explain the details.
OpenCL support in x264 is only able to offload the lookahead process to the GPU; lookahead isn’t the largest part of H.264 encoding, but it’s the easiest to parallelize, and thus the best-suited to being performed on the GPU. It never produces any quality degradation. Since lookahead isn’t as important, it only gets about a 13% improvement (which Atrus mentioned), and this would be even smaller using the settings Plex Media Server generally uses (which are optimized for transcoder speed). This support probably isn’t worth much to the end-user, but may end up usable (with no user interaction) in a future Plex Media Server version simply due to ffmpeg and x264 being updated and some minor additional tweaks, but I can’t provide a confirmation or timeline on that.
Intel SMD (Streaming Media Drivers) are hardware audio/video decode/encode and other processing systems often seen on NAS machines, and QuickSync Video is a similar system seen on newer mainstream processors (starting with Sandy Bridge, expanded on Ivy Bridge, and available on almost all Haswells). These 2 systems (which have entirely separate SDKs) allow, in the best case, for the entire transcoding process to be performed on dedicated decoding and encoding hardware on-die with the CPU, which can create a huge speed advantage, especially if the CPU is already loaded (e.g. with another transcode, or some other program the user is running) or was weak to begin with, but they’re limited to a fixed number of concurrent transcodes, and usually result in significantly lower quality than x264’s output. Still, the process is worth while for many users’ servers, so we may be looking into integrating these systems into Plex Media Server at some point in the future, but I also can’t provide any confirmation or timeline. If I was Adam Savage, I’d say “Plausible”.
Lastly, Intel Clear Video and similar technologies allow most or all of the H.264 decoding process to be performed on a general-purpose GPU. This is more useful for reducing power usage during playback on laptops and similar, but could provide some (no idea how much) performance boost in some cases for the transcoder. To be honest, I haven’t even begun to research whether or not it’d be feasible to use GPU decoding across our supported platforms in ffmpeg, so I don’t even know if it’s remotely probable as a future feature.
All of the above features are things that we could look into in more detail for the transcoder in the future, but our dev team is small, and we can’t focus on the transcoder as much as we’d like to. However, the transcoder itself is a fork of the open-source project ffmpeg, to which anyone can submit feature requests or patches to implement said new features. If you’d like better support for hardware transcoding, upstream is always a good place to look.
Ok this was said almost two years ago. I know time flies, but was this “folder” opened again? To take another look at the newest ffmpeg and other developments?
@atrus said:
I do not agree. Not long before christmas the main guy for all things ffmpeg and transcoder for Plex very much said that it would take a long time for it to be integrated (in another thread about the same exact subject). It was mentioned, by me, in this thread (yesterday) that he had said just that. If an employee says it will take a long time, he/she will not be going through the countless threads about adding support for it as he/she has already made a statement about it.
Wow, I must have missed that post, Can you give us a link to that @atrus?
@atrus said:
I do not agree. Not long before christmas the main guy for all things ffmpeg and transcoder for Plex very much said that it would take a long time for it to be integrated (in another thread about the same exact subject). It was mentioned, by me, in this thread (yesterday) that he had said just that. If an employee says it will take a long time, he/she will not be going through the countless threads about adding support for it as he/she has already made a statement about it.
Wow, I must have missed that post, Can you give us a link to that @atrus?
I remember that post but that didn’t seem to be what I thought you were just recently talking about. To me what rcombs posted had more to do about the “legalities” of using the MPX libraries than the technical part of using them.
BTW, you guys should email/contact Intel on this. I emailed them and got a fast response that didn’t throw any obstacle in that I saw. It was pretty much what I speculated earlier. But Plex should clear this for themselves of course. Many other companies are using MPX in commercial/shareware/freeware and it’s allowed/encouraged.
When you think about it this only makes sense. Why would Intel put out libraries that you couldn’t use or distribute in YOUR works? That would make no sense at all. The more people using Quick Sync the better for Intel as AMD can’t support it. It gives them a “lock” on the CPU/GPU used.
@cayars said:
I remember that post but that didn’t seem to be what I thought you were just recently talking about. To me what rcombs posted had more to do about the “legalities” of using the MPX libraries than the technical part of using them.
BTW, you guys should email/contact Intel on this. I emailed them and got a fast response that didn’t throw any obstacle in that I saw. It was pretty much what I speculated earlier. But Plex should clear this for themselves of course. Many other companies are using MPX in commercial/shareware/freeware and it’s allowed/encouraged.
When you think about it this only makes sense. Why would Intel put out libraries that you couldn’t use or distribute in YOUR works? That would make no sense at all. The more people using Quick Sync the better for Intel as AMD can’t support it. It gives them a “lock” on the CPU/GPU used.
If something breaks GPL, it breaks GPL. If they really wanted everyone to use that, then they would have made it available for everyone during all situations. With that said, you assume that Plex has not been in contact with Intel. I would not bet on that assumption to be honest.
There is a “hang up” on the GPL thing. YES you can’t distribute their library as part of a GPL program.
However you can write and use their libraries in YOUR program and distribute that (open or closed). You can distribute binaries of said programs because that is your work and not their work. You can distribute code that calls their libraries as that is YOUR work.
If someone needs to recompile (open source like FFmpeg) then they need to get the libraries from Intel just as you did or anyone else wanting to compile FFmpeg (or other software using the libs). This is NOT UNCOMMON in the software industry at all. This is also why you can download windows versions of FFmpeg that have QuickSync in them and why other parties are using versions (modified) of FFmpeg with their software and it’s allowed. However if the person needs to compile it themselves then they have to fetch the libraries and agree to Intel’s terms.
This isn’t a deal breaker at all.
Plex should contact Intel again on this (if they did in the past) for clarification.
@cayars said:
There is a “hang up” on the GPL thing. YES you can’t distribute their library as part of a GPL program.
However you can write and use their libraries in YOUR program and distribute that (open or closed). You can distribute binaries of said programs because that is your work and not their work. You can distribute code that calls their libraries as that is YOUR work.
If someone needs to recompile (open source like FFmpeg) then they need to get the libraries from Intel just as you did or anyone else wanting to compile FFmpeg (or other software using the libs). This is NOT UNCOMMON in the software industry at all. This is also why you can download windows versions of FFmpeg that have QuickSync in them and why other parties are using versions (modified) of FFmpeg with their software and it’s allowed. However if the person needs to compile it themselves then they have to fetch the libraries and agree to Intel’s terms.
This isn’t a deal breaker at all.
Plex should contact Intel again on this (if they did in the past) for clarification.
No you can’t.
For example, if a program consists only of own original custom software, or is combined with source code from other software components, then the own custom software components need not be licensed under GPL and need not make their code available; even if the underlying operating system used is licensed under the GPL, applications running on it are not considered derivative works. Only if GPLed parts are used in a program (and the program is distributed), then all other source code of the program needs to be made available under the same license terms.
This would need to be an other software that you download (like flash under linux) being separated.
No you can’t what? What part are you disagreeing with? Do you think everyone on the planet distributing an EXE version of FFmpeg 2.8.x that supports Quick Sync is in violation of this?
Have you written Intel for clarification?
https://ffmpeg.zeranoe.com/faq/ which is the official windows compiled version of FFmpeg.
"Why dont the builds include FAAC, FDK-AAC, libaacplus?
These libraries are not compatible with the GPL license and cannot be included without licensing the build as nonfree. A nonfree build cannot be publicly distributed."
Notice no mention of anything to do with Quick Sync in there? Zeranoe is doing it correctly and isn’t violating Intel’s license. But if you want to rebuild it yourself you will need to get the libraries as they are not distributed as part of the source package for FFmpeg OR change a flag to tell it not to compile/link those libraries (as you don’t have them).
@cayars said:
No you can’t what? What part are you disagreeing with? Do you think everyone on the planet distributing an EXE version of FFmpeg 2.8.x that supports Quick Sync is in violation of this?
Have you written Intel for clarification?
https://ffmpeg.zeranoe.com/faq/ which is the official windows compiled version of FFmpeg.
"Why dont the builds include FAAC, FDK-AAC, libaacplus?
These libraries are not compatible with the GPL license and cannot be included without licensing the build as nonfree. A nonfree build cannot be publicly distributed."
Notice no mention of anything to do with Quick Sync in there? Zeranoe is doing it correctly and isn’t violating Intel’s license. But if you want to rebuild it yourself you will need to get the libraries as they are not distributed as part of the source package for FFmpeg OR change a flag to tell it not to compile/link those libraries (as you don’t have them).
You cannot distribute with a GPL program (I was agreeing with you).
You can distribute FFmpeg with libmfx with the --enable-nonfree flag, as mentioned here .
If you add the non-free, it breaks the GPL if you distribute.
At that point, you have to use LGPL.
Side note: there is an error on the main webpage of Zeranoe.
His builds are LGPL and not GPL 3.0.
FFmpeg is licensed under the GNU Lesser General Public License (LGPL) version 2.1 or later. However, FFmpeg incorporates several optional parts and optimizations that are covered by the GNU General Public License (GPL) version 2 or later. If those parts get used the GPL applies to all of FFmpeg.
FFmpeg is licensed under the GNU Lesser General Public License (LGPL) version 2.1 or later. However, FFmpeg incorporates several optional parts and optimizations that are covered by the GNU General Public License (GPL) version 2 or later. If those parts get used the GPL applies to all of FFmpeg.
Exactly. So when you use other parts/libs in FFmpeg you need to apply the full GPL not just the light version. And the link I just gave is the GPL and sections 5 through 7 specifically show why this would allowed for use by companies like Plex.
I really don’t understand why anyone thinks Plex can’t use a modified version of FFmpeg and claims it violated the GPL when the GPL is very clear on this matter.
FFmpeg is licensed under the GNU Lesser General Public License (LGPL) version 2.1 or later. However, FFmpeg incorporates several optional parts and optimizations that are covered by the GNU General Public License (GPL) version 2 or later. If those parts get used the GPL applies to all of FFmpeg.
Exactly. So when you use other parts/libs in FFmpeg you need to apply the full GPL not just the light version. And the link I just gave is the GPL and sections 5 through 7 specifically show why this would allowed for use by companies like Plex.
I really don’t understand why anyone thinks Plex can’t use a modified version of FFmpeg and claims it violated the GPL when the GPL is very clear on this matter.
Bottom line: Quicksync will not be implemented until there is a GPL implementation of it.
Intel is the one at fault, not Plex.
