Your setup should be plenty sufficient, so you must be doing something wrong, or don’t have it configured correctly.
I am going to assume that you are using windows, if so you may need to disable tone mapping and see if that helps.
https://support.plex.tv/articles/hdr-to-sdr-tone-mapping/
See also [INFO] Plex, 4k, transcoding, and you - aka the rules of 4k