Audio Keep Alive Setting from Plex Home Theatre

Maybe you could experiment with the options
--audio-stream-silence=<yes|no>
and maybe
--audio-wait-open=<secs>
in the mpv.conf

See HTPC Tips and Tricks for general info on where to find the mpv.conf file.