PMS becomes unavailable since v1.12.1.4885

after a reboot everything is working for about 24-48 hours.

@GodZZila said:
Same issue’s

https://forums.plex.tv/discussion/312201/issue-with-1-12-0-4829-6de959918-upgrade#latest

@GodZZila said:
after a reboot everything is working for about 24-48 hours.

May be the same issue - but you can see from posts here what would be required to investigate

Initially logs with debug logging enabled and depending on what is found in the logs then - if it is a thread deadlock situation - then we move to getting connections list, process dump and logs

@GodZZila said:
Same issue’s

https://forums.plex.tv/discussion/312201/issue-with-1-12-0-4829-6de959918-upgrade#latest

That topic post is for a different issue relating to thecus and change in opening web app url to modules/Plex/www/index.php - not a deadlock topic.

Suggest opening a new topic with debug logs - debug logging enabled before server restart and zipped logs when issue arises

Should I apply the version 1.12.2.4929 update?

@foz1977 said:
Should I apply the version 1.12.2.4929 update?

Yes - please do

At some point between 4.30am and 10am deadlocking occurred.
Will PM logs etc.

@foz1977 said:
At some point between 4.30am and 10am deadlocking occurred.
Will PM logs etc.

I will have to wait for next lockout because you said you forgot to capture a process dump to go with the logs and connections info

How frequent are the lockouts ?

This is the first lockout since March 28th. Prior to that it was every 6-10 hours. Fair to say that it’s pretty inconsistent

Good news. I’ve woken up properly and have found the dumps.

Also PMS has become unavailable (pre 503) again so links to be PM’d

Connections queue is approaching 400. Would you like me to wait until 503’s occur before rebooting and follow the same procedure?

@foz1977 said:
Connections queue is approaching 400. Would you like me to wait until 503’s occur before rebooting and follow the same procedure?

I was after multiple dumps for the same lockouts so - yes please
Wanted to see in the dumps the status of the threads over time

So at least for the one lockout, 2 sets of:
dump + logs + connections info

2 sets of post 503’s comung up

Well it looks from this feed that i’m not the only one that had 1.12.1.4929 update stop my plex server. I have tried to reboot, uninstall plex, install the previous stable version (1.12.1.4885), still not able to load plex server. tried to uninstall and install the 4929 update again, and still cannot connect to the server. from what i can tell from my AS7008t NAS, the plex service isn’t even running. Anyone have any ideas?

Here is what i have been able to find in the logs:
Apr 05, 2018 23:02:41.183 [0x7fcf97ffe700] INFO - Plex Media Server v1.12.2.4929-29f6d1796 - ubuntu PC x86_64 - build: linux-ubuntu-x86_64 asustor - GMT -04:00
Apr 05, 2018 23:02:41.183 [0x7fcf97ffe700] INFO - Linux version: 4.4.24 (#1 SMP Mon Mar 26 02:57:14 CST 2018), language: C
Apr 05, 2018 23:02:41.183 [0x7fcf97ffe700] INFO - Processor Intel® Core™ i3-4330 CPU @ 3.50GHz
Apr 05, 2018 23:02:41.183 [0x7fcf97ffe700] INFO - ./Plex Media Server
Apr 05, 2018 23:02:41.183 [0x7fcfa3e7f800] DEBUG - BPQ: [Idle] -> [Starting]
Apr 05, 2018 23:02:41.183 [0x7fcfa3e7f800] VERBOSE - BPQ: delaying processing 120 second(s)
Apr 05, 2018 23:02:41.185 [0x7fcfa3e7f800] DEBUG - Opening 20 database sessions to library (com.plexapp.plugins.library), SQLite 3.13.0, threadsafe=1
Apr 05, 2018 23:02:41.233 [0x7fcfa3e7f800] DEBUG - Running migrations.
Apr 05, 2018 23:02:41.354 [0x7fcfa3e7f800] ERROR - SQLITE3:0x10, 11, database corruption at line 62365 of [fc49f556e4]
Apr 05, 2018 23:02:41.354 [0x7fcfa3e7f800] ERROR - SQLITE3:0x10, 11, statement aborts at 4: [select max(max(metadata_items.changed_at),max(metadata_items.resources_changed_at)) from metadata_items] database disk image is malformed
Apr 05, 2018 23:02:41.354 [0x7fcfa3e7f800] ERROR - Database corruption: sqlite3_statement_backend::loadOne: database disk image is malformed
Apr 05, 2018 23:02:41.354 [0x7fcfa3e7f800] ERROR - Error: Unable to set up server: sqlite3_statement_backend::loadOne: database disk image is malformed (N4soci10soci_errorE)

@SteelZ691 said:
Well it looks from this feed that i’m not the only one that had 1.12.1.4929 update stop my plex server. I have tried to reboot, uninstall plex, install the previous stable version (1.12.1.4885), still not able to load plex server. tried to uninstall and install the 4929 update again, and still cannot connect to the server. from what i can tell from my AS7008t NAS, the plex service isn’t even running. Anyone have any ideas?

Here is what i have been able to find in the logs:
Apr 05, 2018 23:02:41.183 [0x7fcf97ffe700] INFO - Plex Media Server v1.12.2.4929-29f6d1796 - ubuntu PC x86_64 - build: linux-ubuntu-x86_64 asustor - GMT -04:00
Apr 05, 2018 23:02:41.183 [0x7fcf97ffe700] INFO - Linux version: 4.4.24 (#1 SMP Mon Mar 26 02:57:14 CST 2018), language: C
Apr 05, 2018 23:02:41.183 [0x7fcf97ffe700] INFO - Processor Intel(R) Core™ i3-4330 CPU @ 3.50GHz
Apr 05, 2018 23:02:41.183 [0x7fcf97ffe700] INFO - ./Plex Media Server
Apr 05, 2018 23:02:41.183 [0x7fcfa3e7f800] DEBUG - BPQ: [Idle] → [Starting]
Apr 05, 2018 23:02:41.183 [0x7fcfa3e7f800] VERBOSE - BPQ: delaying processing 120 second(s)
Apr 05, 2018 23:02:41.185 [0x7fcfa3e7f800] DEBUG - Opening 20 database sessions to library (com.plexapp.plugins.library), SQLite 3.13.0, threadsafe=1
Apr 05, 2018 23:02:41.233 [0x7fcfa3e7f800] DEBUG - Running migrations.
Apr 05, 2018 23:02:41.354 [0x7fcfa3e7f800] ERROR - SQLITE3:0x10, 11, database corruption at line 62365 of [fc49f556e4]
Apr 05, 2018 23:02:41.354 [0x7fcfa3e7f800] ERROR - SQLITE3:0x10, 11, statement aborts at 4: [select max(max(metadata_items.changed_at),max(metadata_items.resources_changed_at)) from metadata_items] database disk image is malformed
Apr 05, 2018 23:02:41.354 [0x7fcfa3e7f800] ERROR - Database corruption: sqlite3_statement_backend::loadOne: database disk image is malformed
Apr 05, 2018 23:02:41.354 [0x7fcfa3e7f800] ERROR - Error: Unable to set up server: sqlite3_statement_backend::loadOne: database disk image is malformed (N4soci10soci_errorE)

That is unrelated to this forum topic.
The database has been found to be corrupt. Updates do highlight pre-existing database corruptions when the database schema needs to be updated and this is what happened here.

Please have a look at the following support articles
https://support.plex.tv/articles/201100678-repair-a-corrupt-database/
https://support.plex.tv/articles/202485658-restore-a-database-backed-up-via-scheduled-tasks/

@sa2000 said:

@SteelZ691 said:
Well it looks from this feed that i’m not the only one that had 1.12.1.4929 update stop my plex server. I have tried to reboot, uninstall plex, install the previous stable version (1.12.1.4885), still not able to load plex server. tried to uninstall and install the 4929 update again, and still cannot connect to the server. from what i can tell from my AS7008t NAS, the plex service isn’t even running. Anyone have any ideas?

Here is what i have been able to find in the logs:
Apr 05, 2018 23:02:41.183 [0x7fcf97ffe700] INFO - Plex Media Server v1.12.2.4929-29f6d1796 - ubuntu PC x86_64 - build: linux-ubuntu-x86_64 asustor - GMT -04:00
Apr 05, 2018 23:02:41.183 [0x7fcf97ffe700] INFO - Linux version: 4.4.24 (#1 SMP Mon Mar 26 02:57:14 CST 2018), language: C
Apr 05, 2018 23:02:41.183 [0x7fcf97ffe700] INFO - Processor Intel(R) Core™ i3-4330 CPU @ 3.50GHz
Apr 05, 2018 23:02:41.183 [0x7fcf97ffe700] INFO - ./Plex Media Server
Apr 05, 2018 23:02:41.183 [0x7fcfa3e7f800] DEBUG - BPQ: [Idle] → [Starting]
Apr 05, 2018 23:02:41.183 [0x7fcfa3e7f800] VERBOSE - BPQ: delaying processing 120 second(s)
Apr 05, 2018 23:02:41.185 [0x7fcfa3e7f800] DEBUG - Opening 20 database sessions to library (com.plexapp.plugins.library), SQLite 3.13.0, threadsafe=1
Apr 05, 2018 23:02:41.233 [0x7fcfa3e7f800] DEBUG - Running migrations.
Apr 05, 2018 23:02:41.354 [0x7fcfa3e7f800] ERROR - SQLITE3:0x10, 11, database corruption at line 62365 of [fc49f556e4]
Apr 05, 2018 23:02:41.354 [0x7fcfa3e7f800] ERROR - SQLITE3:0x10, 11, statement aborts at 4: [select max(max(metadata_items.changed_at),max(metadata_items.resources_changed_at)) from metadata_items] database disk image is malformed
Apr 05, 2018 23:02:41.354 [0x7fcfa3e7f800] ERROR - Database corruption: sqlite3_statement_backend::loadOne: database disk image is malformed
Apr 05, 2018 23:02:41.354 [0x7fcfa3e7f800] ERROR - Error: Unable to set up server: sqlite3_statement_backend::loadOne: database disk image is malformed (N4soci10soci_errorE)

That is unrelated to this forum topic.
The database has been found to be corrupt. Updates do highlight pre-existing database corruptions when the database schema needs to be updated and this is what happened here.

Please have a look at the following support articles
https://support.plex.tv/articles/201100678-repair-a-corrupt-database/
https://support.plex.tv/articles/202485658-restore-a-database-backed-up-via-scheduled-tasks/

sa2000,

thanks for the input… It was really late for me to be messing with this issue last night… i looked at my error this morning and figured that out, about the same time you messaged… Thank you again for your input…

I was just watching something internally when the video froze. I tried to launch in another tab and there was no response.

I went through the process of dump, logs & connections. When I attempted to launch again it actually worked (for the first time since the issue started).

Will PM two sets of information.

@foz1977 said:
I was just watching something internally when the video froze. I tried to launch in another tab and there was no response.

I went through the process of dump, logs & connections. When I attempted to launch again it actually worked (for the first time since the issue started).

Will PM two sets of information.

could be different issue

The dumps you provided and logs etc show similar deadlock to the one I wanted the multiple dumps for. Appears to be to do with tidying transcode session temp directory, The evidence has been referred to the development team to supplement what was provided before.

Thank you very much :smile:

New set of logs to be PM’d

@sa2000 said:

@foz1977 said:
I was just watching something internally when the video froze. I tried to launch in another tab and there was no response.

I went through the process of dump, logs & connections. When I attempted to launch again it actually worked (for the first time since the issue started).

Will PM two sets of information.

could be different issue

The dumps you provided and logs etc show similar deadlock to the one I wanted the multiple dumps for. Appears to be to do with tidying transcode session temp directory, The evidence has been referred to the development team to supplement what was provided before.

I am making a pre-release build of version 1.13.0 available to you - it has a fix for this deadlock. The other deadlock is still under investigation