After some searching I discovered that Plex cannot handle HomePods as default audio out on my Apple TV. Pause and play happens with a 4 second audio delay. With is very annoying.
To be clear, while watching a video. When I press pause. The video stops but the audio continues to play for 4-5 seconds. When pressing play again, the video starts without the 4-5 seconds audio.
When I disable HomePods as speakers, this issue is gone. But, Plex is the only app that has this. Netflix, my TV app, ATV+ or YouTube don’t have this issue. So it seems a bug (or somewhere a wrong setting).