This is a long-standing complaint that I have been crusading against for several years now.
Part of the problem is that many tagging tools have the option (enabled by default in some apps) to preserve the ‘modified file date’ when they save tags to the media. Plex uses this date to decide whether to actually scan the file for metadata changes.
You could turn that option off, so when you change tags in the future, the file’s modified date will change, and Plex will re-scan it, but Plex really does need to change this, because you’re right, the ‘refresh metadata’ command doesn’t do what it’s supposed to do.