I don’t have specifics, but can tell you it takes into account over 50 metrics, and plots them in n-dimensional space to find items that are closest together.
Timbre/instrumentation is definitely part of this, as you rightly guessed. In addition to that, tempo, rhythm, texture, tone, vocalisation, volume and more are considered.
For an example of what sort of data can be extracted in a similar process, check out any AcousticBrainz entry:
https://acousticbrainz.org/55be6e16-e321-4b06-8519-a20d85beac2b
To clarify: Plex does not use AcousticBrainz or the Essentia engine for Sonic Analysis; this serves only as an example of a similar service for anyone that might be curious
On top of this, the radios that make use of Sonic Analysis also go through the Plex smart shuffle process, which ‘dithers’ items to result in less repetition (true shuffle doesn’t feel random enough, see below), and favours popular (from last.fm) or highly rated tracks (your rating tags) that you haven’t heard recently by positioning them higher in a given queue.
There are definitely further secret herbs and spices that make up the final recipe, but hopefully this can satiate some of the curious cats in this topic 
The article from Spotify’s engineering team about randomness and shuffling is a very interesting read, and I highly recommend checking out the resources linked to at the bottom of that post if the intersect between psychology, maths, and music interests you.
