If indeed your album artwork is embedded in your files ensure within the advanced settings of your music library that album art is set to local files only. That way plex will ignore all its own stuff from their cloud and only use your local files.
I’ve done this and it works perfectly within my music library.
As you mention artist, the above setting does not work for artist artwork, they will pulled from the plex cloud. However, you can use your own local artwork if you name it artist-poster.ext and artist-background.ext placing the files at the artist level (not within albums). You will also need to ensure Prefer local metadata is used if you want them picked up as per → Local artist artwork not used when prefer local metadata is disabled.
Hope this helps you.
