Should you ask the reports of your direct reports...

Say you're a senior manager and you have several managers with their teams reporting to you.

You have your own point of view on the performance of your direct reports (the managers), but how can you find out the point of view of the teams who report to them? How can you know/judge how the teams perceive their managers, and how important is it to you?
Is it advisable to get feedback straight from the team (in whatever way, possibly anonymously) or what consequences would you anticipate?

There are managers who are brilliant in managing up and making everyone above them think they are doing a fantastic job, while their teams are suffering from their incompetence of managing "down" and may leave, without ever telling their real reasons for leaving.

Have you come across this before? How would you solve this?

Anja