There’s actually already a feature suggestion asking for an option to play music videos (if available) from album/shuffle/playlists. I suggest you vote/comment in that thread instead to avoid further distracting/dispersing of votes.
Therefore closing this thread.
