This is totally normal.
At the beginning of playback, the file will be started to get transferred into the network buffer of the client as fast as possible – until that buffer is well filled. Further transfer is then throttled and only resumes as playback is progressing along the timeline.
Another possibility is missing or poor interleaving of the source file. If your source file is using the mp4/m4v container format, this is something you should investigate. Interleaving is pretty much essential to the ability to stream a video file.
See Capcut videos playback issues with plex - #2 by OttoKerner for more detail.
