Welcome to our forums! Please take a few moments to read through our Community Guidelines (also conveniently linked in the header at the top of each page). There, you'll find guidelines on conduct, tips on getting the help you may be searching for, and more!

Flex TV: Google Assistant/Alexa + Plex Integration (with support for Sonarr, Couch, Sick, etc.)

1356756

Answers

  • drummondstdrummondst Members, Plex Pass Posts: 32 Plex Pass

    @digitalhigh said:
    Just finished up basic genre searches for "I want to watch a horror movie" or "I want to watch a comedy". Now I'm going to try chaining genre + year + cast in one search and see if I can parse out all those commands without **** up other title searches. This becomes further compounded if I want to be nice and include checking for individual decades (a comedy made in the 80's", which then takes my "year" parameter and makes it a range of years, and now a function that needs to be able to juggle five different possible input variables and return an accurate result.

    Hehe. It kind of makes me wonder if using a conversation API would even be possible, considering as how much back and forth with the Plex Server I have to do to be able to generate a result.

    Oh, also - I'm going to start making considerations for a handful of beta testers soon. Ideally, I want to test IFTTT against the following search assistants and player devices, so if you have one of these, please let me know which ones you have, as well as a refresher on your technical knowledge in terms of web debugging and general setup and diagnostics of network/web applications.

    The List:

    Siri
    Alexa/Echo Dot
    Fire TV/Fire Stick
    Roku-Based Devices
    Apple TV
    any other kind of random device that may have a one-off version of the Plex Player app.
    An Android-TV based Smart TV
    A Non-Android TV based Smart TV
    A Linux box running PMP
    Is there plex for RasPi? Then that.

    I don't need help testing the win10 app, I know that works fine.

    If you have one of the above devices and feel like playing, let me know. You'll need to be comfortable setting up your own webserver with PHP+curl, as well as port forwarding to your page, etc. so that it's publicly accessible via IFTTT. Eventually, I'll be encapsulating this in some kind of "set and forget" webserver package, but for now, I just want to make it work.

    It's time for tryouts. :D

    image

    I have
    Siri
    Alexa
    Fire Stick

    I am a QA lead with 7 years of Software/hardware qa experience. I have experience working with webservers and portforwarding. Currently use IFTTT and Tasker in a few different areas. My personal favorite is one that I wrote that grabs the URL of the latest video of a daily youtube show and automatically turns on my tv (harmony hub) and casts the video onto the tv to act as my alarm clock.

  • digitalhighdigitalhigh Members, Plex Pass Posts: 288 Plex Pass

    @drummondst said:

    @digitalhigh said:
    Just finished up basic genre searches for "I want to watch a horror movie" or "I want to watch a comedy". Now I'm going to try chaining genre + year + cast in one search and see if I can parse out all those commands without **** up other title searches. This becomes further compounded if I want to be nice and include checking for individual decades (a comedy made in the 80's", which then takes my "year" parameter and makes it a range of years, and now a function that needs to be able to juggle five different possible input variables and return an accurate result.

    Hehe. It kind of makes me wonder if using a conversation API would even be possible, considering as how much back and forth with the Plex Server I have to do to be able to generate a result.

    Oh, also - I'm going to start making considerations for a handful of beta testers soon. Ideally, I want to test IFTTT against the following search assistants and player devices, so if you have one of these, please let me know which ones you have, as well as a refresher on your technical knowledge in terms of web debugging and general setup and diagnostics of network/web applications.

    The List:

    Siri
    Alexa/Echo Dot
    Fire TV/Fire Stick
    Roku-Based Devices
    Apple TV
    any other kind of random device that may have a one-off version of the Plex Player app.
    An Android-TV based Smart TV
    A Non-Android TV based Smart TV
    A Linux box running PMP
    Is there plex for RasPi? Then that.

    I don't need help testing the win10 app, I know that works fine.

    If you have one of the above devices and feel like playing, let me know. You'll need to be comfortable setting up your own webserver with PHP+curl, as well as port forwarding to your page, etc. so that it's publicly accessible via IFTTT. Eventually, I'll be encapsulating this in some kind of "set and forget" webserver package, but for now, I just want to make it work.

    It's time for tryouts. :D

    image

    I have
    Siri
    Alexa
    Fire Stick

    I am a QA lead with 7 years of Software/hardware qa experience. I have experience working with webservers and portforwarding. Currently use IFTTT and Tasker in a few different areas. My personal favorite is one that I wrote that grabs the URL of the latest video of a daily youtube show and automatically turns on my tv (harmony hub) and casts the video onto the tv to act as my alarm clock.

    Check your inbox, I've sent you my hangout/gmail info. Hit me up on hangouts if you have it, otherwise email is fine.

  • digitalhighdigitalhigh Members, Plex Pass Posts: 288 Plex Pass

    https://drive.google.com/open?id=0B0OpOqUNqeKkQ0Yyc1hlc2dmeUU

    Here's another video with parsing of more complex commands, genres, control commands, etc.

  • mrjudgesaysmrjudgesays Members, Plex Pass Posts: 1 Plex Pass
    edited January 20

    This is incredible. I would love to get my hands on this when it's ready for people like myself who aren't versed in coding. I've just ordered a raspberry pi to work on push notification via HA this weekend. Just out of interest, what are you using as your preferred method for viewing in your demo videos? I was thinking of getting an Nvidia Shield but I'm guessing you are using a pc? (My household is exclusively mac).

    Another question. Would this work with a remote server as my step-father suffers from a disease that has caused him to become 100% reliant on voice commands and he accesses the server at mine.

    On a side note I have the following devices for when (if) you do have time to hold my hand through setup:
    PlayStation 4 (original)
    Chromecast 2nd generation
    Sony 4K TV (not android. Possibly opera based?)
    AppleTV 4th Gen mainly used for reviews/HomeKit
    possible purchase of an Nvidia Shield this weekend

  • digitalhighdigitalhigh Members, Plex Pass Posts: 288 Plex Pass

    @mrjudgesays said:
    This is incredible. I would love to get my hands on this when it's ready for people like myself who aren't versed in coding. I've just ordered a raspberry pi to work on push notification via HA this weekend. Just out of interest, what are you using as your preferred method for viewing in your demo videos? I was thinking of getting an Nvidia Shield but I'm guessing you are using a pc? (My household is exclusively mac).

    My preferred set-top box these days is the Shield (2015), although I'd LOVE to get my hands on the new 2017 model. I've also got a Nexus Player in the bedroom, so "Android TV" as my platform of choice. I'd definitely recommend the shield to anybody thinking about getting one, I think it's an infinitely better value over a XBone or PS4...but I've also got a dedicated gaming rig in the basement to stream my games into my living room console-style. But that's a different discussion. Definitely - get a Shield.

    Another question. Would this work with a remote server as my step-father suffers from a disease that has caused him to become 100% reliant on voice commands and he accesses the server at mine.

    On a side note I have the following devices for when (if) you do have time to hold my hand through setup:
    PlayStation 4 (original)
    Chromecast 2nd generation
    Sony 4K TV (not android. Possibly opera based?)
    AppleTV 4th Gen mainly used for reviews/HomeKit
    possible purchase of an Nvidia Shield this weekend

    Right this second, no, it would not work with a remote server.

    But, that's only because I am specifically filtering non-owned servers from my results for testing.

    Once completed, yes, it absolutely should. I put a lot of work in last weekend making sure that all server and client resources are fetched from plex.TV using the credentials of the logged-in user.

    In fact, the only requirement I don't think I'll be able to get away from is that of needing to have a device running Phlex on the same network as the players you want to control - and to have Phlex be publicly accessible by means of port forward or proxy or whatever. Or, theoretically, you could set up a VPN between the players to control and the Phlex device.

    Of course, the term "webserver" sounds a lot scarier than it is...you could build a raspi with a PHP webserver on it, put Phlex on that, and plug it in somewhere and be set.

    http://www.instructables.com/id/Setup-a-Raspberry-Pi-PHP-web-server/

    And this is just as of right this second. Once I get all the parts working correctly, then I'll be looking at a better way of distributing this...maybe I can package it up in a jar or something and make it run on Java. Sky's the limit.

    Either way - I def. recommend a shield. It's hands-down the most powerful Android TV box you can get, and Android TV is a really awesome platform, even on the sad little Nexus Player.

  • XmemphistoXXmemphistoX Members, Plex Pass Posts: 23 Plex Pass
    edited January 20

    Can't wait. Where do we donate for this - I know you put a lot of your own time up for it. Wish you could get help from the Plex Devs as this only furthers their feature set. Maybe even hire you to tie it into a plexpass feature set.

  • digitalhighdigitalhigh Members, Plex Pass Posts: 288 Plex Pass

    @XmemphistoX said:
    Can't wait. Where do we donate for this - I know you put a lot of your own time up for it. Wish you could get help from the Plex Devs as this only furthers their feature set. Maybe even hire you to tie it into a plexpass feature set.

    If you're really inclined to, I have a donate address for paypal set up at donate.to.digitalhigh@gmail.com.

    It would be nice to get a reply from the Plex devs I've reached out to regarding the issues with PMP. It's frustrating that this works so amazingly with 2/3's of the Plex ecosystem, but just falls flat with PMP. I can even see that the one dev. I messaged is on here and active every single day.

    But, alas. No biggie. Good things to those who wait, right? Or blessed are those who help themselves? Make 'murica great again!

    On that note, today was largely about spiffing up the web UI a bit. I've been working on a "Now playing" footer that pops up when the player you've got selected has media going. I've still got a little bit of work to do yet, but it's got art, the title, synopsis, year, and a slider showing the current position in the video. I intend on making this slider work to adjust the offset in real time, as well as sticking the "control" commands directly in this footer.

    The other thing I'm working on is the "result cards" - the little boxes you see after you enter a command. Currently, these are generated in the session you're in when you enter a command - you can't see, say, a voice command as you use it elsewhere. So, I'm re-writing this bit of code so that as commands come into the server, they get dumped into a JSON file that holds the last 10x commands used by you. This will be scraped and updated by the web viewer, so that when you use your home to do a voice command, not only will the "now playing" footer change to show the current media, but you'll also see that command come into the UI in near real-time, with relevant debugging data.

    Why all this work to display commands? Because, it's insanely useful in wallowing through this quagmire that is "manual language parsing". And while I am generating a much more comprehensive, verbose logfile that I can tail to see what's going on, having a nice pretty card pop onto my screen when I do something by voice is just plain cool. I mean...come on. Plus, the UI is already pinging Phlex once every couple of seconds for player data and stuff, why not get a little more info in the process. :D

    I've got links out to a handful of people now to test this, I'd just like to wait on feedback from them before moving on to what should be a much quicker process of implementing Couchpotato fetching. I've already got this script working, it just needs to be translated to PHP and integrated into the rest of it.

    Then, just a little vanity work to create pretty displays with prebuilt URL's that can be fed into IFTTT/Maker, some homework into a more or less universal way of distributing this for people...and we should be good to go.

    Oh, also worked on setting up for testing remote servers...although I really think this should just work. ;)

  • digitalhighdigitalhigh Members, Plex Pass Posts: 288 Plex Pass

    @mrjudgesays said:

    Another question. Would this work with a remote server as my step-father suffers from a disease that has caused him to become 100% reliant on voice commands and he accesses the server at mine.

    Specifically because of this question, I have now made sure that a remote server can be used by Phlex to trigger playback. I'll need to go back through and make sure I put the server address and stuff in the respective JSON objects for previous commands so the UI knows where to get the appropriate art from, but I was still able to trigger playback of Batman V Superman on my test VM from my buddy's server across town. :D

  • herrerae0723herrerae0723 Members, Plex Pass Posts: 3 ✭✭

    @digitalhigh said:
    Just finished up basic genre searches for "I want to watch a horror movie" or "I want to watch a comedy". Now I'm going to try chaining genre + year + cast in one search and see if I can parse out all those commands without **** up other title searches. This becomes further compounded if I want to be nice and include checking for individual decades (a comedy made in the 80's", which then takes my "year" parameter and makes it a range of years, and now a function that needs to be able to juggle five different possible input variables and return an accurate result.

    Hehe. It kind of makes me wonder if using a conversation API would even be possible, considering as how much back and forth with the Plex Server I have to do to be able to generate a result.

    Oh, also - I'm going to start making considerations for a handful of beta testers soon. Ideally, I want to test IFTTT against the following search assistants and player devices, so if you have one of these, please let me know which ones you have, as well as a refresher on your technical knowledge in terms of web debugging and general setup and diagnostics of network/web applications.

    The List:

    Siri
    Alexa/Echo Dot
    Fire TV/Fire Stick
    Roku-Based Devices
    Apple TV
    any other kind of random device that may have a one-off version of the Plex Player app.
    An Android-TV based Smart TV
    A Non-Android TV based Smart TV
    A Linux box running PMP
    Is there plex for RasPi? Then that.

    I don't need help testing the win10 app, I know that works fine.

    If you have one of the above devices and feel like playing, let me know. You'll need to be comfortable setting up your own webserver with PHP+curl, as well as port forwarding to your page, etc. so that it's publicly accessible via IFTTT. Eventually, I'll be encapsulating this in some kind of "set and forget" webserver package, but for now, I just want to make it work.

    It's time for tryouts. :D

    image

    I have a Samsung Smart TV, Xbone, PS4, and a rasplex device. I'd definitely like to test some of these out.

    Great work !

  • digitalhighdigitalhigh Members, Plex Pass Posts: 288 Plex Pass

    @herrerae0723 said:

    I have a Samsung Smart TV, Xbone, PS4, and a rasplex device. I'd definitely like to test some of these out.

    Great work !

    PM Me an email address I can contact you at. If you don't have a bitbucket account, set one up and send me those details as well, I'll get you access to the git to get testing. :D

  • digitalhighdigitalhigh Members, Plex Pass Posts: 288 Plex Pass

    Also, because I enjoy double-posting so much - I just verified that this will even work to trigger playback from Plex Cloud instances. :D

    Oh, and I've got the server-side of things working for Couchpotato additions now as well - I just need to finish up the return JSON so that it gives back an object that's more or less comparable to the results of the playback commands. Only issue I'm having this second is making my "command log parser" switch between the art path for stuff from Plex and using an art path from IMDB. And that should be easy enough to fix up.

  • lolcatzurulolcatzuru Members Posts: 9 ✭✭

    sent you a small donation, can't wait for some kind of release

  • digitalhighdigitalhigh Members, Plex Pass Posts: 288 Plex Pass

    Just got the basic functionality of triggering adding new media to Sonarr working!

    As usual, I still need to polish up the WebUI's parsing of the return JSON from this, but everything is in place for it to work correctly from API. Once that's ready, I think there will just be a bit more work on the genre+year searching, the generation of URL's for people to copy and paste into IFTTT, and then a bit of testing. :D

    @lolcatzuru said:
    sent you a small donation, can't wait for some kind of release

    Thanks very much for the generosity! As always, stay tuned for updates here...shouldn't be that much longer.

  • lolcatzurulolcatzuru Members Posts: 9 ✭✭

    @digitalhigh said:
    Just got the basic functionality of triggering adding new media to Sonarr working!

    As usual, I still need to polish up the WebUI's parsing of the return JSON from this, but everything is in place for it to work correctly from API. Once that's ready, I think there will just be a bit more work on the genre+year searching, the generation of URL's for people to copy and paste into IFTTT, and then a bit of testing. :D

    @lolcatzuru said:
    sent you a small donation, can't wait for some kind of release

    Thanks very much for the generosity! As always, stay tuned for updates here...shouldn't be that much longer.

    i wish i could donate more but money is tight for me right now, i eagerly await the release of this, this will be absolutely amazing!

  • digitalhighdigitalhigh Members, Plex Pass Posts: 288 Plex Pass

    Lot's of nice refinement work today and steps toward making it user friendly. Figured out some bugs that were causing issues setting it up in an instance of Xampp, documented how to make it go there - more or less.

    Set up basic text fields in the UI where you can grab prebuilt URL's pointing to your instance of Phlex once you get it set up and public facing.

    Squashed a bunch of issues with the newly added "fetch" set of commands. Got a few more to do, but movie adding is done, and adding a series in Sonarr just needs a few final touches.

    Also, now verified that this will work to control Plex running on Roku, so that's one more device added to the list. Unfortunately, it looks like this won't work right away with Alexa and IFTTT due to limitations there, but I am also tinkering with the actions API on API.AI, so it's possible that I'll be trying to utilize more intelligent responses and the ability to use it on alexa down the road.

    Either way, I can now say that this is actively being tested by more than just myself, and that for the most part, this has a basically complete feature set, with the rest of the work being focused on bug fixes, improvements, streamlining things, and just generally trying to release something that isn't going to embarrass the hell out of me when I put it into the wild. Make sure there's a logout button, and that cached files get regenerated appropriately, etc. Make sure I'm not making unnecessary web calls and wasting precious response time. Delete all the curse words out of my logging commands. :D

    But, it's getting there.

    @lolcatzuru said:

    i wish i could donate more but money is tight for me right now, i eagerly await the release of this, this will be absolutely amazing!

    Don't sweat it! Every little bit is very appreciated. I'm not doing this for money, I'm doing it because I enjoy the challenge, and I like sharing what I figure out with the world. :D

  • JasonMeudtJasonMeudt Members, Plex Pass Posts: 217 Plex Pass

    More of an FYI...

    Connect Third-Party APIs to Google Assistant using the Voice Assistant Webhook

    https://www.xda-developers.com/xda-spotlight-connect-third-party-apis-to-google-assistant-using-the-voice-assistant-webhook/

    • Intel Core i7 4790 @ 3.6 Mhz (Plex Server)
    • Netgear R7000 (Tomato by Toastman firmware)
    • 2x Google Chromecast V2 (Media Player)
    • 3x Roku (Media Player)
    • 10 TB of Storage Space
  • digitalhighdigitalhigh Members, Plex Pass Posts: 288 Plex Pass

    @JasonMeudt said:
    More of an FYI...

    Connect Third-Party APIs to Google Assistant using the Voice Assistant Webhook

    https://www.xda-developers.com/xda-spotlight-connect-third-party-apis-to-google-assistant-using-the-voice-assistant-webhook/

    Damn you!

    Now I have so much more work to do!!! :P

  • JasonMeudtJasonMeudt Members, Plex Pass Posts: 217 Plex Pass

    @digitalhigh said:

    @JasonMeudt said:
    More of an FYI...

    Connect Third-Party APIs to Google Assistant using the Voice Assistant Webhook

    https://www.xda-developers.com/xda-spotlight-connect-third-party-apis-to-google-assistant-using-the-voice-assistant-webhook/

    Damn you!

    Now I have so much more work to do!!! :P

    Lol... Should be much 'easier' now... ☺

    • Intel Core i7 4790 @ 3.6 Mhz (Plex Server)
    • Netgear R7000 (Tomato by Toastman firmware)
    • 2x Google Chromecast V2 (Media Player)
    • 3x Roku (Media Player)
    • 10 TB of Storage Space
  • Main PlexMain Plex Members, Plex Pass Posts: 222 Plex Pass

    @digitalhigh said:

    Also, now verified that this will work to control Plex running on Roku, so that's one more device added to the list. Unfortunately, it looks like this won't work right away with Alexa and IFTTT due to limitations there, but I am also tinkering with the actions API on API.AI, so it's possible that I'll be trying to utilize more intelligent responses and the ability to use it on alexa down the road.

    This is amazing I really hope to see an alexa integration as I'm in the uk and I can't get Google home :(

  • digitalhighdigitalhigh Members, Plex Pass Posts: 288 Plex Pass

    @Main Plex said:

    @digitalhigh said:

    Also, now verified that this will work to control Plex running on Roku, so that's one more device added to the list. Unfortunately, it looks like this won't work right away with Alexa and IFTTT due to limitations there, but I am also tinkering with the actions API on API.AI, so it's possible that I'll be trying to utilize more intelligent responses and the ability to use it on alexa down the road.

    This is amazing I really hope to see an alexa integration as I'm in the uk and I can't get Google home :(

    It just happens to be your lucky day. The tip from @JasonMeudt got me motivated into looking harder at API.AI. I then talked to the creator of AutoVoice for a bit, which filled me in nicely on how the API.AI system actually works.

    So, this does a few things for the scope of the project:

    One, for API.AI to work, you apparently need a valid, not self-signed SSL cert and domain for API.AI to talk to. This isn't horribly difficult to set up using XAMPP, but does take the setup to a slightly higher difficulty.

    Which means that instead of rewriting everything and throwing out all the language parsing I've already written - I'll instead dub that system of command interpretation "API V1".

    The new system of parsing and delivering commands to Phlex, dubbed API V2, should provide a LOT more extensibility to the language parsing. I should be able to now parse out a lot more language, and make everything almost natural-like in conversations. There can be prompts when more input is needed, and there can be custom-ish responses when delivering a success or error message.

    So, this means a lot more work on my part to incorporate parsing of the input from API.AI, but it also means a hell of a lot more power behind what I can do with the voice inputs.

    And the best part is, because this is wrapped in the API.AI, I should be able to hook it up to Alexa via an action that loops it into the intent parsing that Google does.

    But, for now, I'm just getting a handle on sorting out which API input is which, and wrapping my brain around how I need to redo things in order to make this all flow beautifully.

    Either way...suuuuuuper exciting. I can't wait till I can have my first conversation with this puppy. :D

Sign In or Register to comment.