Video playback pauses and resumes only after 1+ minute



I've recently had the problem that videos playing via the Xbox One App pause for over a minute, after which they finally resume (though sometimes they don't or I run out of patience). This has occured only in the last 2-3 weeks, before that I usually didn't have issues. Most of the files that I play back are 720p files that are around 1GB, usually in x264, sometimes also x265.

It's really annoying and I have no clue how to solve this. Can anyone help?