You’re probably seeing the app go through and complete any missing downloads, which is different from refreshing.
(Not sure an album would ever re-download.)
Hmm, that doesn’t sound normal. Any chance you can upload Plexamp logs after it corrupts a playlist?