I use Handbrake and the following settings and my videos play on both the Logitech Revue and the Sony Google TV set top box.
VIDEO TAB:[list]Format MKV FileVideo Codec: H.264 (x264)Framerate: Same as sourceQuality: Constant Quality RF: 18 ( some friends and I found that this setting gave the best video quality for size )[/list]
AUDIO: [list]I choose all the audio tracks available (English, spanish, french etc)Codec is always AAC (CoreAudio)I also always do a stereo track and 5.1 trackSamplerate: AutoBitrate: 160DRC: default ( leave it alone )[/list]
[list]I choose all the audio tracks available (English, spanish, french etc)Codec is AC3 Passthru - all the other settings will grey out, this is good, it means there will be no re-encoding of the audio. We like that :)[/list]
SUBTITLES:[list]Here again I'll do all of the subtitles availablewith the defaults( just select the subtitle from the drop down and leave it )[/list]
The important thing right now with GoogleTV is H.264 video with AAC audio. (no H.264 in mkv container with mp3 audio)
You can download the plists of my recommended settings by following the link below. The video and audio will be handled automatically, but you should double check everything. You will have to add the extra audio tracks and subtitles that you want.
[list][*]Version 1.0.1 (AC3 Passthru Audio)[*]Version 1.0.0 (AAC Core Audio)[/list]
View the changelog here