In Plexamp since version 4.8.2 yes, but only if you have local lyrics.
Note: The article above is a bit outdated (it’s specific to the legacy music agents). For the modern agent (Plex Music), you need to configure your music library to prefer local metadata. The file naming/organization is accurate though.