Plex has no support for sorting or filtering by media format.
Plex will not read a metadata tag for collections. I don’t even think there is such a tag in the ID3 specification for mp3 files (there is an ID3 tag for ‘File Type,’ but it only allows for variants of the MP3 format), and Vorbis (used by FLAC), I believe, has fewer tags.
One thing you might consider is using the folder structure on your hard drive. This might be more work than it’s worth, but it would achieve the goal. Segregate your albums into folders by format. Create folders for MP3, FLAC, Etc., move the albums into the appropriate location, then in Plex, create a separate library for each format. Later on, when you’ve weeded out the duplicates, you can undo all that and make a single library. If you create those folders under a single parent folder, you could just leave it that way on the drive, and make a library pointing to the parent folder to create a single library with all formats.
Otherwise, I think you’re stuck with manually tagging your tracks in Plex. (This HowTo may help, if you aren’t already aware of it).
According to a post by a Plex Ninja, these are the tags Plex currently supports (there’s a feature request to add more thorough support for metadata, but I don’t think it would help your specific case much):
Artist
Album Artist
Album
Title
Year
Genre
Track #
Cover Art
