Help troubleshooting low quality HW transcoding

The file is “bitrate-starved”. An average video bitrate of just 3.7mbps for a 1080p/H.264 stream is very low quality to begin with. It should never be transcoded again.
When you feed files such as this into a hardware transcoder, the results regularly look atrociously.

Another idea: Try disabling ‘hardware transcoding’, then play the file again.
If it looks much better that way, you might be hit by a driver issue. Corrupted video output stream with hardware transcoding