I’m unfamiliar with the Tizen Plex app but that sounds a lot like the quality limit setting on the other apps. I would say for in home use that you would definitely want to disable that. It should improve both quality and server performance. Your home network is likely capable of pretty high bandwidth especially if you are hardwired, so it is unlikely you would saturate and slow down your home network. But if you have aggressive bitrate limits set you could force the server into transcoding all kinds of content which will greatly alter quality and harm performance.
What else can I do to increase the audio and video quality of .mp4 and .mkv movies that are on the home network?
is there any specific configuration on the Plex Media Server?
Play a video on your TV and in the Plex Web GUI on your computer go to the dashboard and see if it is direct playing. If it’s direct playing you are getting the file as it is and the only thing you can do at that point is get better files or upgrade your equipment.