The amber ones are great for those dull conditions where you need to cut the glare, but not make vision too dark.
For sunglasses I'd go for brown, not particularly for any better glare reduction, but they make all the greenery look a lot more vivid, and even the most boring swims appear quite lush!
Being serious I'd go for amber for dull conditions, and brown or grey for brighter light.