I’ve tested this on iOS and Desktop.
If you are playing a track from a downloaded album, playlist, mix, etc., the album art will not be present in the “Now Playing” context menu (e.g. if you tap and hold on the song title from the “Now Playing” screen).

If you navigate to the non-downloaded version of the track the artwork will be present.

This does not affect the “Now Playing” bar at the bottom of the app as the artwork seems to be displayed correctly there in both cases.