Rare issue - degrading frame rate

So I tried Display Mode Switching option and issue persists whether it is off or on. Repro is simple:

  1. Play movie 4K HEVC HDR + AC3 in MP4 container in DirectPlay mode
  2. Press pause, wait 2 sec or
    you may try fast forward
  3. Press play
    Now frame rate is dropping for several seconds - several probably depends on specific stream. Sometimes it is 3-5 secs, sometimes it is 10-15 (previously there were cases when only restart helps). debug stats show 24p, but this is not what is actually displayed on the screen. My TV does not display actual frame rate it is receiving, unfortunately.