I think this is a double-edged sword. Sure, you might be able to go and work out issues with players, and I like to think I and most other GMs would as well. But it does open up the opportunity for some drama to erupt--GM A doesn't like Player B's responses, Player B gets called out...or Player B is the problem and gives GM A a nasty review that they'll have to explain because Player B's attitude is blowing up the team environment.
Though general trends in GM evaluations should come up, it's also important to note that outside of obvious cases one way or another there's going to be disagreement among different players and we'll see different responses. I know people who think I do a good job as GM and I can name a few others who (probably) think I'm awful and should be removed. I won't go into that, but I guess the point is that for any GM who doesn't have a good locker room and a good team, we'll see mixed reviews that will make it hard to evaluate whether or not they're actually an issue. Even Bush being completely offline for a couple months drew its share of protest when he was removed, as did the whole situation that led to me being hired (though in both cases the league was right to do what they did ultimately). Those were the obvious ones, so imagine how it might go over if someone who's actually active gets removed because they don't meet the standards of the blues or BoG (which themselves are going to be super hard to define so people are absolutely going to come out screaming about inconsistencies and "why didn't you fire x person instead" and such).
With that being said, though, I think the league should be more active in evaluating GMs. The league deserves GMs who are active contributors to the league as a whole and are able to run a team competently. In this day and age just one of those things doesn't exactly cut it with demand being as high as it is (yes, I know I'm saying that at my own risk). But any removal of a GM who isn't either straight-up inactive or not even trying to make their team good is going to be controversial, and that just raises the question of whether the league is willing to make a controversial move like that--and, in doing so, effectively change its policies by setting a new precedent.