Since the dawn of time, humanity has been struggling to find better ways to measure stuff. The big questions people were asking back then are the same questions they ask now: when, where, how long, how far, how much, how big, how small, how fast, and why is Bobina in the next cubicle paid more than me when I work twice as hard? Our obsession with measurements and metrics could even be one of the defining human characteristics. Without it, there could never be any progress, because if you can’t measure what you are doing, how can you decide when it improves?
Now, speaking with my engineering hat on, I love metrics as long as they are useful. Useful to me means something I can use to help improve my team’s performance. I think the contribution measurements in WAR are great – I don’t always agree with them and there are definitely things which they don’t (and possibly can’t) measure but I like that they are trying.
Everyone loves feedback. Even if the feedback isn’t 100% positive, it’s always a good thing to get feedback on what you are doing. It’s nice not to feel ignored, and nice to feel that the game gives you a pat on the head from time to time. It is a basic human need and most MMOs do it very poorly. As a healer, you know when a heal landed and stopped someone else from dying, sure it would be nice if they realised it also but even a basic UI will show you the effect you have on health bars. As a tank, you can tell when a monster is hitting you, and see very clearly when you pulled one off someone who it wasn’t supposed to be hitting. As dps, it’s way tougher. You see health bars go down but you know there are other people doing damage too.
I think it’s quite telling that one of the most popular WoW addons was the damage meter, so that dps players could compare how much damage each of them was doing. Again, fulfils a basic human need — to answer the question, “how did I do?” and to boast about it to other people.
But performance metrics are victims of their own success. People obsess on beating the metrics, because they’re so easy to show to other people. And sometimes that means you can end up encouraging all sorts of strange emergent behaviour which are nothing to do with the results that you originally wanted to measure.
I also think that metrics can be useful in teaching new players how to play in groups. One thing WoW is often criticised for is that you can very easily solo to max level, without ever learning to play your class in a group. In a game where grouping often means using a very different set of abilities to soloing, that’s really hampering how easily new player get into the endgame. Meters which let you compare your damage/healing/damage taking to other players of the same class can give people a baseline at least. And there’s no real substitute for watching how someone else plays your class to help a new player learn about using specific abilities.
I think Mythic have made a brave effort with the contribution metrics in WAR and it’s because we like the feedback that people are going to obsess about it over the whole lifetime of the game. This issue will never die. And no one will ever be totally happy with the metrics.
What you can’t measure
You can’t measure anything intangible. You can’t measure how hard people were trying. You can’t measure who was good for morale. You can’t measure individual leadership skill. You can’t measure redundancy — eg. people who were prepared to switch role to be emergency tanks or healers but weren’t needed to do that. And it’s very hard to measure flexibility and efficiency. Human beings could measure those things but they’re also subject to extra bias.
The metrics are particularly bad at measuring people doing dull but tactically important tasks like defending an objective that never gets attacked. Or holding some AP back (ie. not spam HoTing everything in sight) in case an emergency heal is needed. To the metric, that’s the same as being afk or idle. And players are terrifically sensitive to the idea that afk players (or leechers) might get rewarded for doing nothing. This is mostly because it’s behaviour that none of us want to encourage. Scenarios are fun because they are fair. They’d get a lot less fair if we regularly had half the group afk at the start.
So what contribution measurements do now is very tough and ready. Raw damage. Raw healing. And a lot of crucial contribution is not measured and possibly never can be. However, those crucial contributions have a much bigger part to play in the eventual outcome than the meters would imply.
I’d like to see the reward for actually winning a scenario being more heavily weighted. After all, this is the actual goal of the scenario. It’s unproductive to see members of the losing team earn more renown/xp than the winning one because they scored more kills. This is a war, it’s all about winning. I’d increase the randomness on PQ rewards too, assuming a minimum level of contribution. It’s practically impossible for a healer to ever top the meter on a keep take at the moment, for example. They simply require more damage than healing, but that doesn’t mean people who play those classes should always be cut out of the rewards.
I think measurements of individual contribution are worthwhile and people love them. But we need better ways to measure contributions for different classes. Sure, we all know Bright Wizards can beat the damage meters. So maybe we should take that into account when measuring the contribution of a Shadow Warrior. No one class should be singled out for lower contributions across the board; or if they are, it should be a sign that they need a tweak. Add some contribution for buffs and debuffs.
And finally, we need good ways to dissuade people from afking or logging out of scenarios. That’s not an issue to be resolved purely by contribution meters. I don’t think it’s ever going to be easy to find a way to reward people for defending flags but the scenarios which depend on this as a strategy can be weighted to reward winning more highly and random killing away from the flag much less highly.
But at the end of the day, there’s a limit to how smart a contribution meter can be. Some classes will always do better than others because they’re just better designed at whatever the meter was trying to measure. Some people will drift to those classes because they obsess about meters. But lets not forget that its actually the result that we want to reward. Winning the scenario, playing well in teams and in realms, teaching new players how to group, rewarding good group play.