First, if you have not already done so, install the 64-bit version of Plex Media Server 1.29.0.6219 or later (download). That version added support for zero-copy transcoding with Nvidia GPUs, greatly increasing transcoding performance. It can be installed on top of the 32-bit version. The installer will detect and remove the 32-bit version before installing the 64-bit version.
Regarding the number of streams…
Simultaneous streams per Server
The number of streams per server is not capped. There will be real world limits, such as available uplink bandwidth for remote users, but Plex does not limit the number of streams for a server.
Simultaneous Remote Streams per User
The number of simultaneous remote streams per user can be configured in Settings → Network → Remote streams allowed per user.
Transcoding
This is what usually leads to confusion.
A transcode consists of two parts: (a) decoding from the current video format; and (b) encoding to the desired video format (Plex always encodes to H.264).
Nvidia limits their consumer GPUs, such as the GeForce GTX & RTX, to three simultaneous encodes. The number of simultaneous decodes is not limited.
Therefore, when using stock Nvidia drivers, the first three simultaneous transcodes will use the Nvidia GPU. The fourth and subsequent simultaneous transcodes will be decoded by the Nvidia GPU and encoded by the CPU.
The AMD Ryzen 9 3950X is a quite beefy CPU, with a Passmark of 39023. Even without a GPU, it should handle a 4K HDR transcode & tonemap or multiple 1080p transcodes. If you exceed three simultaneous transcodes it should be able to handle the load.
Additional Notes
Most Nvidia Quadro GPUs have no encode or decode limit (other than available memory).
The number of encodes/decodes supported by the various Nvidia GPUs is listed in Nvidia’s Video Encode and Decode GPU Support Matrix.
Elpamsoft has transcoding performance measurements for various Nvidia GPUs.
It is possible to patch the Nvidia drivers to remove the three encode limit. The patch is not supported by Nvidia. For additional information, see the Elpamsoft site and search for “session limits.”
Plex Docs:
Using Hardware Accelerated Streaming
What kind of CPU do I need for my Server?