The Skill Paradox

A little while ago I came across a comment on a blog which claimed that “as we know, most MMO players are bad”. I did – and do – find it curious, because this indeed does get passed around as a universal truth: most people we meet online are terrible players. Naturally and curiously except for those we’re having these discussions with, we’re all just fine. This, to hit where it hurts right from the start, is basically the same reasoning that constitutes racism: “all black people are savages, except for Joe, who’s a member in my Country Club”. We can even admit the popular argument that the blogging community in a way self-selects the better, because obviously more invested, players – and who doesn’t like to attribute themselves to a self-proclaimed elite; us bloggers, we’re awesome, aren’t we – and it still plays on the same lawn as racism, after all, the Country Club is also obviously a selective environment, but those other black people, the ones outside, they’re obviously savages. It is the hypocrisy that makes every representative of a group one knows by name and face an exception, while at the same time maintaining blanket judgements about the large anonymous group as a whole. That’s not what this post is about though. Instead, I’d like to consider how we arrive at the perception.

First of all, the claim itself that “most players are terrible” is inevitably wrong. As I casually remarked recently, we all live in the Gaussian Curve, and what it really means is that most players are average, as this is how most natural averages form, with only few cases when the data points are crowded near the extremes and the average is a mostly virtual value. An example of the latter could be the life expectation during the European Middle Ages, which was mathematically around 35, but if you look at the numbers closer, you realise that it doesn’t mean that “most people died around age 35”, but that the horrifyingly high infant mortality skewed the statistic and what it rather means is “if you survived past the age of 1, you were probably going to run for the 70”. But this is unnatural, an anomaly. Usually distributions follow the Gaussian Curve, which is why it’s called the Normal Distribution.

There are of course special situations, like, say, Wrath of the Lich King creating a particular expectation about content difficulty and the subsequent Cataclysm taking things in a somewhat different direction. But the aforementioned sentiments gets stated sweepingly, not in some differentiated “in early 2011, large parts of the WoW player base were not prepared for the content difficulty presented to them”, and gets accepted sweepingly. In fact, I can be reasonably sure that someone will actually comment on this very post that “no-no, really, everyone is terrible”. Well, maybe not after I wrote this, kind of falsifying the experiment. Or maybe not, because there are, like, 5 people commenting on my posts, so this isn’t exactly a relevant sample size of anything.

Still, the whole WotLK/Cata thing also highlights that, of course, skill is not an abstract concept, it is relative to difficulty of the task in question. So, yes, if we’re talking about reciting randomly selected works of Shakespeare while running a marathon in Antarctica, most people would be pretty terrible at it. Shopping for groceries on the other hand is something most would find pretty easy, catering to the casuals, almost. Here’s the thing though: in the context of a game, i.e. voluntary entertainment activity, people gravitate towards and away from activities such that in the long term, you are left with your normal distribution again – those for whom it’s too hard or too easy mostly leave. Even now, I think the Cata-difficulty issue is pretty much settling itself (Nils has some great recent posts on why having just one difficulty for everyone is bad design), because the averages are adjusting themselves, people are settling in their new relative positions and the content is self-nerfing.

Then there is always the possible explanation that there are, at any time, many new and inexperienced players. But frankly, this is not what the statement quoted at the outset states. “Most players are new” is not the same as “most players are bad”, even though it’s similarly unlikely. More importantly, catering to new players is by far not the same as catering to bad players. In fact, catering to new players is very important, as the expectation we have is that new players are going to learn, while bad players are not. Conversely, if you don’t let your players learn (cater to new players), you raise incompetent players (well hello, trivialised levelling game).

Last and least, because it’s mostly an exercise in being a smartass, the statement doesn’t even make sense on a logical-linguistic level. If “most players are terrible”, then terrible is the average, thus most players are average. If you’re the brilliant exception, then it’s you who is in the wrong place and probably should go back to the South Pole Stadium-slash-Theatre.

So far, we have a brief outline of why “most players are terrible” should be considered to be wrong. Yet there are lots of people who will tell you that and be genuinely surprised that anyone should think otherwise (remember the bit with the potential comment on this post? Wait for it, still may happen!). Why is this? After thinking about it for a bit, I realised that the solution is The Blub Paradox. It’s a fairly long and technical essay, rich on self-praise, but well worth reading if one is interested in the matter. I will proceed to adopt the part relevant to describe The Skill Paradox.

Imagine the Skill Continuum. No, it’s not the point in space-time a gamer occupies at 13:37. It’s the line on which all skill levels are laid out in increasing order, from a hopeless failure to the greatest winner. Now imagine a player A, whose skill places somewhere on the continuum, a safe distance from both extremes. As long as A is looking down the skill continuum, he recognises that he’s looking down. He can see all the things these players are doing wrong, all the mistakes they’re making, all the potentials for optimisation. If he’s looking up, however, he doesn’t realise that he’s looking up. All he can see are players who are basically just as good as him, only some of them tend to do things in weirdly esoteric ways. A may even consider some of them his inferiors, because, seriously, what sensible person would ever do those esoteric things. This is the Skill Paradox, meaning that we possess very pronounced abilities in recognising all the ways in which other players are worse than us, but are comparatively bad in perceiving the ways in which other players are better than us. The direct consequence is that we tend to think of ourselves as “very close to the top” regardless of where we actually place in the skill continuum, which in turn, especially if you’re not necessarily the self-scrutinising type, lends itself to the “most people are terrible” fallacy.

At this point I originally intended to write down personally experienced cases of when despite the apparently obvious presence of empirical evidence I wasn’t as good as I thought I were, and only got better after embracing details I previously had dismissed as irrelevant, ranging from my most newbish online gaming beginnings in the year 2000 to more recent experiences. In fact I even wrote down those examples, read through them, and they felt so full of unintended self-adulation that they made me sick. And, you know, as much as there is to say about how awesome I am (for example: how awesome I am), I ultimately decided that it would rather distract from than support the point.

The point being: as soon as you acquire even marginal competence in a field (note how we gently drifted away from being gaming-specific), you will find it easy to recognise the shortcomings of those below your level of competence – that’s why you’re better than them, you moved past making their mistakes. Your observations will objectively suggest that you’re “one of the best”, because everyone else you observe is “bad”, with the exception of those whose superiority you cannot fully comprehend as such, for the very same reason they are superior – they realised potentials you have yet to discover.

Shall you live in eternal self-doubt? No, not at all. If you feel you’re good, you probably are. No one is born a master. Just know there’s always a “better” as much as there’s a “worse”. This is also why I don’t like evaluating my own performance based on doing something others (supposedly) can’t do – it’s much more intriguing to consider what can be done than what can’t. Otherwise you’re going for that old trap where “anyone who has accomplished more than you has no life, and anyone who has accomplished less is a noob“.

There is another consequence to all of this, one that won’t sound particularly surprising: your ability to look up the Skill Continuum and realise that you are looking up is directly related to your ability to become better yourself. This is what teachers, trainers and guides do, they explain to their students how those esoteric things can make them better. The good ones do, the bad ones simply state what to do – and sadly there’s a huge demand just for that, seen in the MMORPG world where a lot more people blindly follow the cookie-cutter spec and rotation than actually understand why and under which conditions it is superior. Whenever you convince yourself that “this is as good as it gets”, you stop improving. A truism, really.

And just as I was about to finally post this (it’s the third rework or so), xkcd came out almost on cue: Our brains have just one scale, and we resize our experiences to fit.

11 thoughts on “The Skill Paradox

  1. Nils

    About that Gauss curve .. I doubt player skill is a gauss curve. I know everybody says it, but I’ve never actually seen proof. The IQ argument doesn’t hold, because the IQ is gaussian by definition.

    Somewhere in here Raph Koster mentions that in UO PvP most players lost all the time. And just a few won all the time. That’s the difference between “modus” and “mean”. It’s a big problem!

    However, a wonderful post. Didn’t know that paradox before! Thanks.

    Reply
    1. Rem Post author

      Interesting points!

      It is true, we don’t really know how player skill is distributed. We don’t even have a definition for player skill, much less an actual way of measuring it objectively (to what extent can skill even be equated with performance?), so as long as we can’t hook up sensors to all players’ brains and figure out what to look for, it’ll always be a bit of a mystery. I agree IQ is not an argument either. The reason to expect a Gaussian or close-to-Gaussian distribution is mainly because “that’s how things tend to pan out” for large numbers (important detail, which I have no doubts you’re fully aware of given your professional background, but not everyone may be: the qualitative difference between observations concerning 10-100 or hundreds of thousands of data points).

      The UO PvP example brings two thoughts to my mind.

      1. If most players lost and a few players won all the time, it directly leads to the conclusion that nearly all confrontations involved one of the “few” and almost no confrontations took place between the “most” players. This in itself is not a contradiction: we know the statistic about most EVE players never leaving safe space. It does raise the question though whether it is actually skill that is being observed or willingness. Some players (PKs) organised, prepared and trained with the specific purpose of killing other players, while “most” players just wanted to be left alone (by PKs). Cue Trammel.

      2. Outliers. Of course there are people, in any field, who truly excel in that field. I didn’t mean to dispute that. In the area of gaming, I have been (and in a way still am) acquainted with a number of them. For a while, I was reasonably close to being one myself. Brilliance exists, but it doesn’t mean that everyone who isn’t brilliant is rubbish (the number of people claiming everyone else to be rubbish alone would cause issues here). What I particularly dispute is the ability of a single actor observing from the inside – rather than having an outside (and at the same time highly competent, good luck with finding someone like that) perspective – to objectively evaluate their own standing, especially because their evaluation of abilities is limited by their own mastery of the field in question.

      For 5 years from 2000 to 2004, Michael Schumacher won the majority of the races (and all championships) in Formula 1. He’d always be counter-balanced by some sponsor-dubbing-as-driver whom everyone perceived as a rolling obstacle. The respective runner-ups would be mirrored by guys who’d always be happy to simply see the finish line. And in-between there would be a wide field of drivers and teams of gradually declining skill/capability who’d be fighting it out for the places 3-15 over the course of the season. Just because the best driver sat in the best car and kept winning it all, it didn’t mean everyone else was incompetent. There was always a broad “mid-field” of roughly comparable skill. And every single one of those went into every season with a “we should rank better this year” resolution.

      Of course the sample size of this example is at direct odds with my earlier reference to “large numbers”, but what I meant was to illustrate that “one guy winning all the time” doesn’t necessarily imply that everyone else is (equally) bad. He might just be the best.

  2. Max

    Well “average” =terrible in my book. That is not to say video game skill is something worthy of praise or appreciation, but I think just like everything – 90% of stuff is crap. Average, below average and slightly above average are just filler. They follow the rote (badly) and do not comprehend their domain at the level of being able to make positive contributions to it

    I mean of course games are for fun so its ultimately does not really matter how good you are, as long as its not a competition. It is like asking -are you good movie watcher or good reader. Most people are not but it doesnt matter.

    Now there are domains where it matters -such as questions of public policy , environmental impact etc. Not surprisingly people there are just as bad as in games. But those things have a lot more serious consequences than a raid wipe

    Reply
    1. Rem Post author

      First, let’s agree not to apologise for talking about video game skill. Let’s tautologically agree that it is important to the extent to which it is important and our reasoning is transferable to other areas of life to the extent to which it is transferable to other areas of life. So, you don’t need to quantify your every paragraph, I know what you mean 😉

      Here’s the thing: regardless of how you personally view the quality of a performance, the average task needs to be manageable through an average effort by an average person. Proof? Anthropological argument: we’d be extinct otherwise.

      Average isn’t great – and I didn’t mean to claim that. In fact, “great” is pretty much “significantly above average”. But equally, average isn’t terrible, because “terrible” is “significantly below average”. Average is “okay”. That for you 90% of stuff is crap, and I won’t even dispute that statement, means that you have high standards. That’s fine. But the average person has to make do with an average .. uh .. thing. And those average things are okay, even if you, or me, or someone else may want to use a better thing. Adjusted for the locality factor of course, let’s not take this to comparisons of living standards across the world.

      The average car is not some broken down shit, but a pretty decent ride. Mandatory regular inspections pull out the broken down shits, so the average is in fact better than reliable, safe and economically justifiable.

      I think I’m a better-than-average driver. But most average drivers get from A to B and back safely every day. The average trip of an average driver is an entirely sufficient performance. And maybe they all think they’re better-than-average, just as I do.

      The average living accommodations are just fine (even though I want to live better). The average pay is a respectable sum (even though I want to earn more). The average PC is more than capable of performing the tasks set before it by the average user (even though I would never buy it for myself). It’s slipping (significantly) below the average when things get bleak. The average is “okay”. And okay is okay.

      Don’t get me wrong, I’m not suggesting to strive for the average (unless one is below average, obviously). My entire post was about striving up, up, always up. It’s just that I don’t think decrying average as terrible is correct. You’re free to do it anyway, of course. Just ask yourself, who may be looking down on you and considering you or your efforts as “just average crap”, thinking they are in position to judge it all.

      To pull out the grandeur, the drive to be better than average and in the process to push the average upwards is the very factor that has been driving humanity forward ever since the average living accommodation was a cave.

  3. Kring

    As Max mentioned, average just means it’s the average. It doesn’t specify if it’s good or bad.

    > If you feel you’re good, you probably are.

    I would change that to:
    If you question yourself if you are good, you probably are.

    Another problem is that if you mix people with different goal everyone might be bad from everyone else point of view. Because if you only care about your task, you probably suck at the task that’s important to the other person.

    Reply
    1. Alq

      I’m with Kring on this.
      “If you question yourself if you are good, you probably are.”

      Dunning-Kruger anyone?!

      (And yes, I KNOW I go with my own self-doubt before you say anything!)

    2. Rem Post author

      Absolutely. As commented above/below, my entire post has been pretty much about the necessity to question oneself.

      The part in brackets is actually pretty much the reason I wrote the sentence in question to begin with 😉
      The line between self-question and self-doubt is as thin as it’s important. I think it’s important to look for answers, not just for questions. What’s the right amount of self-questioning, when it is too little, when is it too much? I guess the answer can only be 42!

    3. Rem Post author

      Partly disagreed about the first point. Good or bad are defined by average. They’re significant deviations from average. Without the average, good or bad as concepts do not exist.

      Example: The Millennium Falcon made the Kessel run in less than twelve parsecs (whatever the reason for a unit of length being used as a unit of time). The information is in fact entirely worthless, because we don’t know anything about what a typical – i.e. average – travel time/distance for the Kessel run is.

      Definitely agreed on the second point – however, my entire post has been pretty much about the importance of questioning yourself 😉

      Definitely agreed on the third point.

  4. Falding

    I have an issue with your “least” point. If “most players are terrible”, terrible is not the new average – at least not per se. Such reasoning is obviously valid for real-life scenarios which are behond intended creation. (I hear them shouting *Blasphemy*.) A perfectly good example being the IQ. However, playing a specific game means subjecting to its laws and measures, thus giving your performance a scale. Therefore, the individual performance is measured against this scale and “terrible” is just that … “terrible”. The game scale is created and designed (by Blizzard, Turbine, CCP …).

    To put it more succinctly: All the people in the world running the hundred meters is without scale, probably leading to a normal distribution of all performances. But all the people in the world running the hundred meters at the olympic games involves an a priori scale. Most would just be terrible at it.

    I have no idea what this means in the grand scheme of all things MMORPG, but these games DO have scales that a consciously designed by the game developers. Turning back to your post, I believe it may be valid to classify players as terrible at the specific game even though they fall into the average category when looking at all the performances at that game. For some games, “average” just doesn’t cut it. This type of criticism would not be of the “u n00b” sort, but of the “he is underperforming” sort. Scietific reasoning instead of social bashing.

    Reply
    1. Rem Post author

      This is true, and you know I prefer to be forthright about underperformance, and indeed scientific reasoning above social bashing was one of the things I meant to suggest.

      However, I don’t quite agree with your implication that the performance scale, or shall I say the terribleness scale, is universal. People don’t just randomly wind up running the hundred meters at Olympic games, to get there, you have to qualify by displaying out standing performances prior. This is what sets the performance expectation for Olympic results so high, not the other way round.

      Equally, you don’t just randomly find yourself fighting the hardest raid boss in a MMORPG, even though he/she is intentionally designed to be hard. I don’t think a MMORPG needs to be designed such that the hardest raid boss can be defeated by everyone, consequently I don’t think everyone has to defeat the hardest raid boss, consequently I don’t think the inability to defeat the hardest raid boss constitutes a “terrible” failure. Yes, a higher percentage of players should (be able to) beat the hardest boss than that of runners is allowed to participate in the Olympics, but let’s not fight over specific numbers.

      Of course there are truly bad players, just as there are brilliantly good players, but I don’t think judging everyone by their ability to perform the hardest possible task is fair. The hardest task should be for the best. I refer to Nils (the linked posts and many more) for elaborate reasoning on why one task difficulty is bad design.

  5. Lujanera

    There are many distributions other than the Gaussian distribution; assuming all phenomena will fit one distribution is incorrect.

    Also, formulating “terrible” in terms of whether another player is better or worse than the observer doesn’t seem like a very useful way to treat the topic of player skill. Most of the situations in which we talk about player skill involve some challenge (ie, a boss) that must be overcome. Players that are able to perform at a level sufficient to overcome a challenge have proven themselves to be better. Players that cannot overcome anything but the simplest of challenges can rightly be considered terrible.

    Reply

Leave a reply to Kring Cancel reply