Plex server crashing on Debian

Plex Media Server
1

Plex server crashing, seems to coincide with errors in dmesg below. Looks like this is happening during playback.

Server Version#: 1.40.4.8679-424562606
Player Version#: 1.99.0.210-2d92b948

dmesg:
| >>> [root@plex ~]$ dmesg
[4756189.537415] TCP: request_sock_TCP: Possible SYN flooding on port 2647. Sending cookies. Check SNMP counters.
[4804773.780260] PMS GTP[1495643]: segfault at 10 ip 00007f49712947ca sp 00007f495859c918 error 4 in ld-musl-x86_64.so.1[7f4971284000+53000] likely on CPU 3 (core 3, socket 0)
[4804773.780290] Code: 14 66 85 d2 74 02 f4 9b 8b 57 f8 81 fa ff ff 00 00 7f 02 f4 9b 83 e1 1f 89 d0 c1 e0 04 48 98 48 29 c7 48 8b 47 f0 48 83 c7 f0 <48> 39 78 10 74 02 f4 9b 8b 70 20 83 e6 1f 39 ce 73 02 f4 9b 8b 78
[4883139.567158] traps: PMS GTP[1517482] general protection fault ip:7fdec9eba7d0 sp:7fdeb2f80918 error:0 in ld-musl-x86_64.so.1[7fdec9eaa000+53000]
[4948904.302494] PMS GTP[1537006]: segfault at 10 ip 00007efcb873f7ca sp 00007efca6d5f918 error 4 in ld-musl-x86_64.so.1[7efcb872f000+53000] likely on CPU 1 (core 1, socket 0)
[4948904.302502] Code: 14 66 85 d2 74 02 f4 9b 8b 57 f8 81 fa ff ff 00 00 7f 02 f4 9b 83 e1 1f 89 d0 c1 e0 04 48 98 48 29 c7 48 8b 47 f0 48 83 c7 f0 <48> 39 78 10 74 02 f4 9b 8b 70 20 83 e6 1f 39 ce 73 02 f4 9b 8b 78
[4949726.715986] PMS GTP[1537288]: segfault at 10 ip 00007f0413b327ca sp 00007f040a910918 error 4 in ld-musl-x86_64.so.1[7f0413b22000+53000] likely on CPU 3 (core 3, socket 0)
[4949726.715994] Code: 14 66 85 d2 74 02 f4 9b 8b 57 f8 81 fa ff ff 00 00 7f 02 f4 9b 83 e1 1f 89 d0 c1 e0 04 48 98 48 29 c7 48 8b 47 f0 48 83 c7 f0 <48> 39 78 10 74 02 f4 9b 8b 70 20 83 e6 1f 39 ce 73 02 f4 9b 8b 78
[5034521.420703] traps: PMS GTP[1561032] general protection fault ip:7fde0212b7dc sp:7fddeefae918 error:0 in ld-musl-x86_64.so.1[7fde0211b000+53000]
[5054955.965273] PMS GTP[1566492]: segfault at 10 ip 00007f368ebf57ca sp 00007f3683d91918 error 4 in ld-musl-x86_64.so.1[7f368ebe5000+53000] likely on CPU 0 (core 0, socket 0)
[5054955.965315] Code: 14 66 85 d2 74 02 f4 9b 8b 57 f8 81 fa ff ff 00 00 7f 02 f4 9b 83 e1 1f 89 d0 c1 e0 04 48 98 48 29 c7 48 8b 47 f0 48 83 c7 f0 <48> 39 78 10 74 02 f4 9b 8b 70 20 83 e6 1f 39 ce 73 02 f4 9b 8b 78
[5105521.061878] traps: PMS GTP[1584199] general protection fault ip:7f2dc5bb87ca sp:7f2db9435918 error:0 in ld-musl-x86_64.so.1[7f2dc5ba8000+53000]
[5157074.793806] systemd-journald[472782]: Oldest entry in /var/log/journal/82bfcabede124a4f93ac72dd1dbd65da/system.journal is older than the configured file retention duration (1month), suggesting rotation.
[5157074.793809] systemd-journald[472782]: /var/log/journal/82bfcabede124a4f93ac72dd1dbd65da/system.journal: Journal header limits reached or header out-of-date, rotating.
[5207761.932853] traps: PMS GTP[1614479] general protection fault ip:7f72678147d0 sp:7f7250bca918 error:0 in ld-musl-x86_64.so.1[7f7267804000+53000]
[5236705.696870] traps: PMS GTP[1622949] general protection fault ip:7f39f61b87d0 sp:7f39e7d9f918 error:0 in ld-musl-x86_64.so.1[7f39f61a8000+53000]
[5237215.996602] traps: PMS GTP[1623040] general protection fault ip:7fa41f7167ca sp:7fa416afd918 error:0 in ld-musl-x86_64.so.1[7fa41f706000+53000]

| >>> [root@plex ~]$ uname -a
Linux plex-server 6.1.0-21-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.1.90-1 (2024-05-03) x86_64 GNU/Linux

2

Have you considered

3

This looks promising thank you!

4

I am still seeing this issue in dmesg occasionally and the plex server will crash,

[6787897.043920] PMS GTP[2097258]: segfault at 7fa400006710 ip 00007fa4641b87ca sp 00007fa443266918 error 4 in ld-musl-x86_64.so.1[7fa4641a8000+53000] likely on CPU 3 (core 3, socket 0)
[6787897.043929] Code: 14 66 85 d2 74 02 f4 9b 8b 57 f8 81 fa ff ff 00 00 7f 02 f4 9b 83 e1 1f 89 d0 c1 e0 04 48 98 48 29 c7 48 8b 47 f0 48 83 c7 f0 <48> 39 78 10 74 02 f4 9b 8b 70 20 83 e6 1f 39 ce 73 02 f4 9b 8b 78

5

We’re going to need the server DEBUG logs ZIP which coincides with that please.

The problem with these is catching which part of PMS generated the SEGV
This should be in the PMS logs.

6

Attached, thank you.

Plex Media Server Logs_2024-09-01_14-39-12.zip (645.4 KB)

1 Like
7

I’ve also been getting this and I’m also on debian. I assume Plex monitors those crash events and sends crash reports to plex. I’m on PMS Version 1.41.0.8911

8

@The-Manager

Your logs are showing me some database issues.

Aug 28, 2024 20:12:22.396 [140343907343160] ERROR - [NSB/SSDP/Grabber] Exception inside transaction (inside=1) (/home/runner/actions-runner/_work/plex-media-server/plex-media-server/MediaProviders/Grabbers/MediaGrabber.cpp:171): sqlite3_statement_backend::loadOne: UNIQUE constraint failed: media_provider_resources.uuid
Aug 28, 2024 20:12:22.401 [140343907343160] ERROR - [NSB/SSDP/Grabber] SQLITE3:0x80000001, 2067, abort at 46 in [update media_provider_resources set parent_id=:media_provider_resources_parent_id,type=:media_provider_resources_type,status=:media_provider_resources_status,state=:media_provider_resources_state,identifier=:media_provider_resources_identifier,protocol=:media_provider_resources_protocol,uri=:media_provider_resources_uri,uuid=:media_provider_resources_uuid,extra_data=:media_provider_resources_extra_data,last_seen_at=:media_provider_resources_last_seen_at,created_at=:media_provider_resources_created_at,updated_at=:media_provider_resources_updated_at where id=:media_provider_resources_id]: UNIQUE constraint failed: media_provider_resources.uuid
Aug 28, 2024 20:12:22.402 [140343907343160] ERROR - [NSB/SSDP/Grabber] Exception inside transaction (inside=1) (/home/runner/actions-runner/_work/plex-media-server/plex-media-server/MediaProviders/Grabbers/MediaGrabber.cpp:171): sqlite3_statement_backend::loadOne: UNIQUE constraint failed: media_provider_resources.uuid
Aug 28, 2024 20:12:22.402 [140343945337656] WARN - [HttpClient/HCl#b9] HTTP error requesting GET http://localhost:34450/discover.json (7, Couldn't connect to server) (Failed to connect to localhost port 34450 after 0 ms: Couldn't connect to server)
Aug 28, 2024 20:12:22.403 [140343907343160] ERROR - [NSB/SSDP] DVR:Device: Error refreshing existing device device://tv.plex.grabbers.hdhomerun/2021-08-3XGL-8ZGZ30:5, marking as dead.
Aug 28, 2024 20:12:27.408 [140343907343160] ERROR - [NSB/SSDP] DVR:Device: Error refreshing existing device device://tv.plex.grabbers.hdhomerun/2021-08-LZ2T-QCHOEF:5, marking as dead.
Aug 28, 2024 20:12:27.410 [140343907343160] ERROR - [NSB/SSDP/Grabber] SQLITE3:0x80000001, 2067, abort at 46 in [update media_provider_resources set parent_id=:media_provider_resources_parent_id,type=:media_provider_resources_type,status=:media_provider_resources_status,state=:media_provider_resources_state,identifier=:media_provider_resources_identifier,protocol=:media_provider_resources_protocol,uri=:media_provider_resources_uri,uuid=:media_provider_resources_uuid,extra_data=:media_provider_resources_extra_data,last_seen_at=:media_provider_resources_last_seen_at,created_at=:media_provider_resources_created_at,updated_at=:media_provider_resources_updated_at where id=:media_provider_resources_id]: UNIQUE constraint failed: media_provider_resources.uuid
Aug 28, 2024 20:12:27.410 [140343907343160] ERROR - [NSB/SSDP/Grabber] Exception inside transaction (inside=1) (/home/runner/actions-runner/_work/plex-media-server/plex-media-server/MediaProviders/Grabbers/MediaGrabber.cpp:171): sqlite3_statement_backend::loadOne: UNIQUE constraint failed: media_provider_resources.uuid

Notice the constraint errors. These errors mean there is another entry (record) trying to use the same ID number. (constrained ID numbers guaranteeing uniqueness)

Whatever the cause, I have a way of cleaning that up at the DB level

  1. Stop Plex
  2. Ignore constraint errors
  3. Automatic cleanup
    — It will hiss & spit at you while doing this. Ignore the comments (the tool is)
  4. Start PMS
  5. STAY in the tool if you like
  6. When you see it’s working better, now you can exit.
    – Otherwise you can undo that last action.
9

Thanks for the update, so I have just done the following,

cd; wget https://raw.githubusercontent.com/ChuckPa/PlexDBRepair/master/DBRepair.sh
chmod +x DBRepair.sh
./DBRepair.sh

Options: 1, 42, 2, 7.

Output:

Automatic Check,Repair,Index started.

Checking the PMS databases
Check complete. PMS main database is OK.
Check complete. PMS blobs database is OK.

Exporting current databases using timestamp: 2024-09-02_09.54.27
Exporting Main DB
Exporting Blobs DB
Successfully exported the main and blobs databases.
Start importing into new databases.
Importing Main DB.
Importing Blobs DB.
Successfully imported databases.
Verifying databases integrity after importing.
Verification complete. PMS main database is OK.
Verification complete. PMS blobs database is OK.
Saving current databases with ‘-BACKUP-2024-09-02_09.54.27’
Making repaired databases active
Repair complete. Please check your library settings and contents for completeness.
Recommend: Scan Files and Refresh all metadata for each library section.

Backing up of databases
Backup current databases with ‘-BACKUP-2024-09-02_09.58.54’ timestamp.
Reindexing main database
Reindexing main database successful.
Reindexing blobs database
Reindexing blobs database successful.
Reindex complete.
Automatic Check, Repair/optimize, & Index successful.

I’ve cleared the logs at Library/Application Support/Plex Media Server/Logs/Plex Media Server.log and will review for further database issues and update on the status.

Everything seems to be okay so far.

10

More constraint issues in the logs just now,

| >>> [root@plex ~]$ grep -a SQLITE3 /var/lib/plexmediaserver/Library/Application\ Support/Plex\ Media\ Server/Logs/Plex\ Media\ Server.log
Sep 02, 2024 10:15:46.108 [140669045599032] ERROR - [Grabber] SQLITE3:0x80000001, 2067, abort at 46 in [update media_provider_resources set parent_id=:media_provider_resources_parent_id,type=:media_provider_resources_type,status=:media_provider_resources_status,state=:media_provider_resources_state,identifier=:media_provider_resources_identifier,protocol=:media_provider_resources_protocol,uri=:media_provider_resources_uri,uuid=:media_provider_resources_uuid,extra_data=:media_provider_resources_extra_data,last_seen_at=:media_provider_resources_last_seen_at,created_at=:media_provider_resources_created_at,updated_at=:media_provider_resources_updated_at where id=:media_provider_resources_id]: UNIQUE constraint failed: media_provider_resources.uuid
Sep 02, 2024 10:15:46.110 [140669045599032] ERROR - [Grabber] SQLITE3:0x80000001, 2067, abort at 46 in [update media_provider_resources set parent_id=:media_provider_resources_parent_id,type=:media_provider_resources_type,status=:media_provider_resources_status,state=:media_provider_resources_state,identifier=:media_provider_resources_identifier,protocol=:media_provider_resources_protocol,uri=:media_provider_resources_uri,uuid=:media_provider_resources_uuid,extra_data=:media_provider_resources_extra_data,last_seen_at=:media_provider_resources_last_seen_at,created_at=:media_provider_resources_created_at,updated_at=:media_provider_resources_updated_at where id=:media_provider_resources_id]: UNIQUE constraint failed: media_provider_resources.uuid
Sep 02, 2024 10:15:51.118 [140669045599032] ERROR - [Grabber] SQLITE3:0x80000001, 2067, abort at 46 in [update media_provider_resources set parent_id=:media_provider_resources_parent_id,type=:media_provider_resources_type,status=:media_provider_resources_status,state=:media_provider_resources_state,identifier=:media_provider_resources_identifier,protocol=:media_provider_resources_protocol,uri=:media_provider_resources_uri,uuid=:media_provider_resources_uuid,extra_data=:media_provider_resources_extra_data,last_seen_at=:media_provider_resources_last_seen_at,created_at=:media_provider_resources_created_at,updated_at=:media_provider_resources_updated_at where id=:media_provider_resources_id]: UNIQUE constraint failed: media_provider_resources.uuid
Sep 02, 2024 10:15:57.272 [140669028723512] ERROR - [NSB/SSDP/Grabber] SQLITE3:0x80000001, 2067, abort at 46 in [update media_provider_resources set parent_id=:media_provider_resources_parent_id,type=:media_provider_resources_type,status=:media_provider_resources_status,state=:media_provider_resources_state,identifier=:media_provider_resources_identifier,protocol=:media_provider_resources_protocol,uri=:media_provider_resources_uri,uuid=:media_provider_resources_uuid,extra_data=:media_provider_resources_extra_data,last_seen_at=:media_provider_resources_last_seen_at,created_at=:media_provider_resources_created_at,updated_at=:media_provider_resources_updated_at where id=:media_provider_resources_id]: UNIQUE constraint failed: media_provider_resources.uuid
Sep 02, 2024 10:15:57.273 [140669028723512] ERROR - [NSB/SSDP/Grabber] SQLITE3:0x80000001, 2067, abort at 46 in [update media_provider_resources set parent_id=:media_provider_resources_parent_id,type=:media_provider_resources_type,status=:media_provider_resources_status,state=:media_provider_resources_state,identifier=:media_provider_resources_identifier,protocol=:media_provider_resources_protocol,uri=:media_provider_resources_uri,uuid=:media_provider_resources_uuid,extra_data=:media_provider_resources_extra_data,last_seen_at=:media_provider_resources_last_seen_at,created_at=:media_provider_resources_created_at,updated_at=:media_provider_resources_updated_at where id=:media_provider_resources_id]: UNIQUE constraint failed: media_provider_resources.uuid
Sep 02, 2024 10:16:02.282 [140669028723512] ERROR - [NSB/SSDP/Grabber] SQLITE3:0x80000001, 2067, abort at 46 in [update media_provider_resources set parent_id=:media_provider_resources_parent_id,type=:media_provider_resources_type,status=:media_provider_resources_status,state=:media_provider_resources_state,identifier=:media_provider_resources_identifier,protocol=:media_provider_resources_protocol,uri=:media_provider_resources_uri,uuid=:media_provider_resources_uuid,extra_data=:media_provider_resources_extra_data,last_seen_at=:media_provider_resources_last_seen_at,created_at=:media_provider_resources_created_at,updated_at=:media_provider_resources_updated_at where id=:media_provider_resources_id]: UNIQUE constraint failed: media_provider_resources.uuid

11

Soemthing bashed your DB badly.

Look at your backups.

DBRepair will help you roll back to one of your previous ones (if it’s not corrupted at the SQLite level)

12

The oldest db backup is still a problem, any suggestions on how I could fix this manually?

13

Let me ask one more time to be certain I understand we’re in sync.

  1. You started DBRepair
  2. Enabled IGNORE mode (42)
  3. Ran “Auto” and ignored the constraint errors
  4. Started PMS
  5. Checked the logs and still found CONSTRAINT errors ?
14

That’s right, so Options: 1, 42, 2, 7. In that order.

15

I think I have fixed this now, ran delete from media_provider_resources and rebuilt the DVR fresh. Hopefully not broken anything else.

| >>> [root@plex Databases]$ sqlite3 com.plexapp.plugins.library.db
SQLite version 3.40.1 2022-12-28 14:03:47
sqlite> SELECT id FROM media_provider_resources;
51
52
50

16

that looks good.

See if it remains stable.

1 Like
17

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.