In the past, I have only used the combination of videos with my music library. But that has its limits. Nowadays, I would only use that if the only video content would be music videos, where I have the audio-version in my music library anyway.
(It would probably suit most of the content you may have found on older “CD Extra” releases, which contain both a CD Audio session and a CD Rom data session. The picture and audio quality on these videos is quite poor though, owing to the very limited data capacity of a CD and the fact that, back then, video codecs were also not very storage-space efficient.)
Nowadays, if there is a whole DVD/BR worth of video, chances are that it is a “Making Of” of some sort, or a live concert, or maybe the surround sound (re-)mix of an album. This “long-form” content is better suited for a dedicated movie library, IMHO.
Now admittedly, my own collection is still small, so I have no problems navigating through the plain list. I have filled the “Search Title” of all items to name the artist first, and only then append the name of the video. So if I use the jump list on the right screen border, I jump to the artist, rather than the title.
A “movies”-type of library provides the best feature set IMHO. Mainly because you can find quite a lot of metadata for such disc releases already on TheMovieDB.org, so Plex can fetch these automatically.
If your collection consists mainly of classic “music videos”, there might be a way to use a “TV shows”-library instead. This provides a hierarchy of Show > Season > Episode which you could utilize to fashion an Artist > Album > Song hierarchy.
(Just think of setting the Agent to “Plex Personal Media” in that library, because there are no metadata for music videos in the sources that Plex is using for TV shows. The mismatches would drive you crazy.)
Both Movie and TV Show libraries nowadays provide good local video asset support. So there is a way to accommodate the various bonus content you may find on such discs.
If I need to connect the video content with an musical artist, I add them both to a manually created collection. If the name of the collection is 100% the same in both the music and the video library, you can usually see the content together (depends a bit on the client type).
See this thread for additional pointers: Concert videos (both clips and full movies) in a single library?