Put your album art into the folder of the record album and name it folder.jpg or folder.png, depending on the file format.
(Of course this only works if you store your files for each album into a separate folder. But that is recommended anyway.)
Once you have placed the coder files, perform a ‘Refresh Metadata’ on the album. The file will then appear for selection when you ‘edit’ an album.