[REQUEST] Move from SQLite to MySQL or Postgres

com.plexapp.plugins.library.blobs.db → 1.24 GB
com.plexapp.plugins.library.db → 1.12 GB
dump.sql of com.plexapp.plugins.library.db → 1.11 GB

Can’t believe that Plex performs so slow on your systems. It’s extremely fast here.

Plex Server is a Linuxserver.io docker container running on an Unraid server 6.9.2. The Plex ecosystem is stored on a NVMe M.2 single drive XFS pool.

I’ve written lot’s of Python scripts against the Plex SQLite3 database itself and other scripts against the Plex API. Both ways perform blinding fast here.

1 Like

Using MySQL or Postgres on the backend would make Plex quite a bit more extensible and scalable, for all the reasons the original poster said. I understand that the developers may not see this as necessary for Plex’s main use case, operating in a user’s home. But it would open up many options for users that perhaps are not used as much. Consider that many users operate Plex as a virtual machine and want to offload to the database storage from where plex is running (on a VM, NAS, raspberry pi, etc) to a more persistent location, such as a server with RAID. Backups and repair of the database could be done using the tools available in MySql or Postgres.

Thanks and keep up the great work.

3 Likes

MySQL instead of SQLite for example would be nice :slight_smile:
+1

2 Likes

+1 to MySQL. SQLite is too slow for such volumes.

2 Likes

+1 for external database.
My Plex is install in k3s cluster and need external DB to use Plex on any node :confused:

5 Likes

+1.
here is my 2 cents in this situation.
I’ve been a plex user for about 4 years now, and i have about 18 TB of content. Most of which is lossless music files. I struggle with “waiting 200ms for a busy DB”. errors every 2-3 weeks. Sad part is that my server is inaccessible until restart occurs. So, in my experience, SQLite db in definitely not a good solution. This feature request was started in 2014 (!) like 7 years ago! Meanwhile, developers tend to focus on “questionable” decisions like “plex arcade” (which I should buy separately for some reason". I see only option - crowdfund a solid plugin development, to support MySQL/MariaDB/Postgre databases. Who is in, fellows? Please upvote if you agree, to see how many people think it is necessary. Thank you.

3 Likes

Frankly, I don’t understand why they haven’t written in a maintainable API that could be used for making the correct database calls if you’re using SQLite, MySQL, Postre, SQL Server, or whatever. It would be much easier to maintain a set of APIs and tie feature sets to the APIs and the capabilities that have been implemented via the database of choice. And, the beautiful thing is that many of those commands are similar (or the same) between several flavors of SQL. It’s really quite a beautiful thing.

That being said, giving users the opportunity to offload their SQL server to an external (and dedicated) server would help promote plugin developers, and all kinds of new features that one could potentially offload from the core Plex install. Service/database clustering, database integrity is cleaner, and just a myriad of other things could be possible.

2 Likes

Exactly. But devs keep silence on this topic, with no visible reason for such a long time…

Yeah. I really don’t understand why they’re not writing versioned APIs to handle these things. It’s clean, it follows MVC paradigms, and is easily maintainable. And thanks to versioning, they can easily update them as necessary, and if users wanted to use an older version, they could with minimal loss of features. Why not use modern design standards?!

1 Like

So i got famous “Waiting 200 ms for a busy database” mesages again. My plex install is literally unusable for about 2 days now… Just a humble reminder to fix this enormous issue.

1 Like

If you need a quick way to modify a SQLLite DB, you can use this. I use it for Python libraries that ship with SQLLite as the back-end DB. Works well for quick and dirty things.

https://sqlitebrowser.org/

I’ve voted on their topic already but would also like to add my 2 cents.

My collection is also quite large and I definitely think they need to work on this. I really wish plex was more responsive to users. One of their biggest downfalls I’ve seen. It’s extremely rare that I’ve seen a response on the forums from the staff.

1 Like

It’s quite unbelievable to me that a tool used to organize and display media would overlook such a vital tool to manage the database. I’ve run into the 200ms error way too many times and have tried to macgyver my way around it for over a year. i’d love a solution for a better database personally.

1 Like

+1
I just tried moving my plex library/watch data from a ubuntu server to docker in unraid and it didnt work and now while waiting for it to rebuild my logs are flooding with sleep for 200. mysql/mariadb option would be nice for this.

This topic seems to never garner a response from Plex, other than something along the lines as they see supporting a real RDBMS leans towards “Enterprise” use, and that’s not the direction Plex is supposed to be used in. Which is a response that we, as users and supporters of Plex deserve better. After all, we are supposed to be the driving force behind Plex, and they purport to be giving us what they think we want. Regardless of what they claim, SQLite does not scale, and we’re forced to deal with slow, sometimes unaccessable services. From the “waiting 200ms for a busy DB” to corrupted databases that have to be rebuilt, it becomes annoying. And, whenever a user reports one of these problems, Plex loves to point the finger at the users, saying we did something wrong.

While support mySQL/mariadb, postgres, or whatever may not be something all users will want, there are enough of us to warrant making this change. And the response from Plex that supporting a true DBMS would require a complete rewrite of certain parts, is bogus, or the point could be made that it’s bad programming on their part.

This could be solved by using standard SQL, and creating a library that handles the database calls. This is how we’ve done it – we have our database abstraction layer that handles all calls to the database. The library defines methods like ‘ExecQuery’ that executes SQL that returns a result set (i.e. select statements) and ExecNonQuery that executes SQL that does not return a result set (i.e. update, insert, delete, etc). The code doesn’t care which DBMS that is being used, as the library takes care of establishing a connection to the database, etc. It handles any specifics for the various DBMS, etc.

I, not for a second, don’t believe that Plex doesn’t have a library or set of functions defined for communicating with their SQLite database and they just use those calls to communicate. It would be trivial to update this library to support another DBMS. Plex just doesn’t want too. They will come back wth ‘There are not enough users wanting this to warrant this’ when that is not the case. They just don’t want too. Plex has gotten to the point where they no longer listen to their users, but rather, do what they want to do. A perfect example is killing off Plugins. They claimed they were killing them off because the plugin system was a mess, and they were going to bring it back after they cleaned it up, and improved it. But it’s been a few years, and nothing. The hope was, they would kill it off, and slowly people would stop asking for it and just forget about it.

Plex no longer cares what their users want. They are bringing features that they want, features to bring in even more revenue to the company. Someone mentioned the Plex Arcade. I didn’t see many people requesting this, but they spent considerable resources developing it and one day just brought it out. Bad enough that a big portion of the userbase can’t use it because we run our servers on linux/freebsd – that right there should have been a non-starter. A feature that’s only available for users running their servers on Windows and Mac (and yes, I know there are clients availale for Amazon Fire, Android, etc).

Plex, how about this. Since you can bring something like Plex Arcade, even though it 1) was not asked for, 2) can’t be used by a big portion of your users, give us something we’are actually asking for – and have been asking for it seems, for YEARS (almost 10).

And, while this isn’t the right place for this, but while we’re asking for things – give us the ability to run our own authentication server and not have to authenticate using your servers. How hard would it be to release an authentication server that we can run ourselves and not need to use your servers. That way, when there’s a problem with your servers, we don’t loose some of our functionality.

There are enough of us who have the skill to run this ourselves so we’re not dependant upon you. Kinda the spirit of the self-hosting community. This is one of the few services, regardless of what I do, there’s no way to get around having to send requests off my network for a service. We can selfhost almost everything else and/or replace commonly used features with something we have 100% control over.

6 Likes

Wholeheartedly agree.

People want to run/distribute their databases across their network. You shouldn’t force data to be co-located. That’s just bad architecture.

3 Likes

And still, not a word from Plex. Really goes to show that Plex has stopped caring about their users, and more so their bottom line.

Hate to say it, but maybe it’s time to look seriously at Jellyfin. Being open source, we have the source code available and can contribute,

2 Likes

My bet is they don’t want to do it because they don’t want the product to scale. They don’t want people setting up huge Plex based streaming services to avoid unwanted attention from the MPAA

1 Like

Plex already uses SQL interface abstraction libraries… it’s not as simple as that.

My assumption is that SQLite is higher performance, lower latency, lower overhead, lower maintenance, more secure, more reliable, and a better fit for the engineering problem than an outboard RDBMS would be. The way Plex uses the database is the definition of a perfect fit for SQLite.

I suspect the tight coupling allows Plex to use database engine features for text search and indexing, to consistently manage schema migrations, and to perform backups and simple DB maintenance. These things would all be much more complicated if multiple engines were supported.

SQLite is available on all of the platforms Plex supports. It requires much less user support than a pluggable system would. Consistency has value.

I doubt Plex is actively trying to prevent the product from scaling; there are some very large Plex systems. I do imagine they’re focused on the majority use case.

4 Likes

Basically exactly what @Volts said.

I was going to jump in to counter some of the crazy conspiracy theories (“Plex doesn’t want the product to scale” or “Plex doesn’t care about their users”) but @Volts pretty much nailed it.

There are almost certainly decent optimizations we can do at the application layer before declaring SQLite dead. But even now, for most normal and even large (107,057 metadata items over here!) databases, when fed with appropriate RAM/SSD, SQLite does totally fine.

2 Likes