From what I can see there are no N64 cores available for MacOS on the libretro site, which is maybe why you are getting an empty ZIP file?
After a lot of searching I found the Parallel N64 core for MacOS here.
I downloaded and installed it and working for me using my 2018 Mac mini PMS:
Hope this helps.
