If that doesn’t work, there are two potential issues:
Your video doesn’t have the language flag set on the file. I figure this one is unlikely, as any sources of anime I know of with dual-audio tracks flag them properly, but it’s worth checking.
You are playing a file you previously played, and left it on the Japanese audio track. Plex gets kinda sticky, and if you select any audio and/or subtitle track that normally wouldn’t be selected, it keeps that as your selection going forward for that file. You could test this by selecting a new episode you have NEVER played on Plex before
But yeah, it’s most likely you just need to check the box to enable automatic track selection.