Slow Thinking
by Steve
I’ve always had a hunch that proof is better than assumption. Which is a pretty silly hunch to have, if you think about it. Or a rather sensible one, I don’t know. Maybe I should test it or something.
Anyway, for some time I’ve kind of flown the evidence flag at work and elsewhere. I remember having battles of a sort with marketing teams who were convinced they knew what a particular demographic wanted, despite having no stats to back it up or any anecdotal or qualitative evidence from that group. As far as I can tell, they’d never even met this particular group of people, and had no inclination to do so. They just knew. They were experts. They had qualifications, experience, a feel for such things.
I was much happier looking for some evidence and letting that inform what I did. Maybe I just wasn’t as confident as they were in my own abilities, or in my gut instinct. But I think I just figured that actually seeing what happens was more robust than working on a set of assumptions or basing any decision on how I think, or what I’d like.
Working around evidence and stats and suchlike has been a whole lot easier because I’ve worked mainly in web-based roles, so there is already a lot of analytics-y stuff at my fingertips. Plus I’ve found quoting concrete facts like “95% of people leave your page in 10 seconds” holds a lot more weight in meetings than “I’m not really feeling your page”. Taking all that personal preference stuff out of the equation makes it harder to argue. Objectivity over subjectivity, or something.
That’s not to say I’m some sort of number-crunching robot. I never really liked maths. But I’ve found stats a really useful, and reasonably foolproof, way of getting to the heart of what people need and want. Somehow it feels more human and empathetic than relying on gut instinct or experience or whatever. Working in the charity sector I almost feel like there is less room for error. It matters that I get my job right. And looking at stats, analysing behaviour and asking people what they want and all that other stuff is much more reliable than just deciding I think we should do it this way because I know best…
All of this seemed to me self-evident (take “evidence-based medicine” as a concept, for example – I mean what other sort of medicine is there, or should there be?), but as I said, it was a bit of a hunch. So, it was a relief of sorts to read Moneyball and see that this philosophy was already pretty well accepted and proved in some quarters.
For those you who have no knowledge of Moneyball or the principles it promoted, or just want to read me flail around trying to explain it, the book chronicles the shift in thinking in baseball from gut instinct and experience to looking for proof. Essentially, how the Oakland Athletics and others moved from scouts to stats. A scout might see a player and think he looks great, but that might not turn out to be the case. Alternatively, a player might not look the part, but stats of his performance might show him to be more effective than appearances suggest. And that gave smaller clubs an advantage – they could spot the players who were good, but not obviously so.
Anyway, Moneyball principles have been applied to all sorts of areas and it seems like evidence-based work is more popular than ever. Knowledge really is power, it seems, to fall back on a comfy cliché.
And I was pleased to read yet more evidence for this kind of stuff, this time from a psychological perspective, in the New York Review of Books review of Daniel Kahneman’s Thinking, Fast and Slow. The book, and the review, are about ‘cognitive illusions’, ” false belief that we intuitively accept as true”. Kahneman argues that we have two modes of organising and accessing knowledge, System One and System Two.
System One is the intuitive, instinctive mode – where we can instantly come to a decision. This obviously had evolutionary benefits way back when big beasts were out to eat us and seems to happen with very little effort. System Two is more reflective and conscious and critical. This is the mode that finds evidence to help make a decision. Yet System Two involves effort and time, and we’re lazy. Hence System One so often wins out and we’re left with an “illusion of validity”.
Well, that’s my summary of a book review of a book I haven’t yet read, from an author I hadn’t even heard from this morning, psychology not really being my strongpoint. But it appeared to back up what I’ve found and read so far, and certainly gave me a lot more to think about. We all fall back on System One thinking. I know I do all the time, despite painting myself as Mr Evidence or whatever. But I think I feel even more likely to push myself to be less lazy, and think I now have a tinier idea why I should too.
And perhaps it explains some of our behaviour, and trends in behaviour, too – although I’m aware I’m at risk of taking something of an ill-educated leap here. It certainly made me think about this interesting, and dare I say inspirational, post on the future of blogging. Blogging may be less popular, less cool and less widespread than it previously was, but it is no less vital. It feels very much like the thoughtful, considered System Two of social media to the more immediate and frankly easier System One of Twitter, Facebook, Tumblr et al. The latter can be maintained with a few words, a few clicks and we can fall back on instant thought or curation of others’ work rather than our own creativity. Blogging, however, takes much more effort – but is probably more worthwhile and rewarding for it.
I’m quietly hopeful that Moneyball/System Two/whatever-you-want-to-call-it thinking has a future, be it in the workplace, the sports field or in social media. But I do think there is still work in convincing people that this philosophy is just as human, and just as meaningful, as any gut reaction or thought based on experience. Illusions and assumptions are there to be challenged. In being the most effective means of working, we hopefully reach and help more people, and in a better way. And used right, that can’t be a bad thing.
I’m somewhat with you on statistics/data, but I’ve had qualms too. I worked in education for quite a long time, and we always had a big push for data-driven decisions, but I’m philosophically torn on metrics in education: they sound great in theory, but I’d argue that it’s impossible to create a metric to measure what I’d argue are the most important skills: critical thinking (important for the general education population) and social interactions (important for all but especially our students with autism/Asperger’s). On a baseball level, I’m torn as a fan: Moneyball has made teams more efficient and given smaller market teams a chance to compete (up to a point; since everyone’s looking at the same metrics now, larger market teams can afford to overpay for players with good WAR, BAPIP, DIPS, etc.) but the game’s best moments (the last night of the 2011 regular season comes to mind, as does Game 6 of this year’s World Series) are so magical that you have to throw sabermetrics out the window altogether and just marvel.
That said, I read a different profile about Daniel Kahneman just this week. Fascinating guy. I’m looking forward to the NYRB article. Here’s the one I read.
LikeLike
Statistics/data certainly have their limitations. I guess the point I was flailing around wildly trying to make was that it is better to make a considered, reflective decision than to just rely on either a first impression or an assumption based on a hunch or prior experience. I guess those things that can’t be measured well by stats can perhaps be measured or recorded in a more qualitative way.
I guess the challenge then, if you’re the one recording, is to be as objective as possible and avoid any prejudices/preconceptions, which isn’t always easy.
I think qualitative gathering such as focus groups or surveys can be really useful and offers a different sort of insight to just raw data. I guess the issue there is that you don’t always get the full picture – people might focus on the negative, or tell you what you want to hear, or say they act in one way when in fact they act in quite another.
As for sport – I’m all for the magic. I guess stats and stuff come into their own when the game isn’t on. So much nonsense gets spouted about sport, and stats are a great way of dispelling myths and countering bullshit. But I’m a long way off watching a game with my Bill James Almanac and a calculator.
Also, some sports (baseball, cricket) lend themselves much more to stats than others (soccer, basketball) as the games are made up of discrete elements, and at least initially one-on-one encounters. There is a growing trend in sabermetric-flavoured stuff going on in soccer, but for such a fluid game with so many variables i’m not sure how useful stats can be. There’s a lot of work still to be done on what stats are meaningful and what ones aren’t.
Thanks for the Kahneman link – and interesting the article is from the Moneyball guy. And really interesting to read your persective on education. My wife works in special education within mainstream schoolling, and it seems like there’s a lot of fascinating work going on in recording and analysing development, but certainly a lot of limitations and potential pitfalls too.
And thanks for such a thoughful reply – and apologies that my one has gone on a bit!
LikeLike
Three cheers for special educators! And for some sports-based statistics (fantasy baseball! hip hip hooray!)
LikeLike
While we were discussing metrics in education, there was another discussion going on at the national level: Nicholas Kristof published an op-ed in the NYT praising the virtues of value-added educators, a stateside movement which attempts to justify hiring/firing teachers based solely on their students’ standardized test scores. There’ve been many rebuttals, but I particularly like this one. I tend to agree more with the dissenters here, but I thought I’d pass along the articles for further reading.
LikeLike
[…] My most-viewed post of 2012 was a silly, but apparently SEO-friendly post written in 2011. The two posts actually written in 2012 that received the most views were written right at the start of the […]
LikeLike
There’s definately a great deal to learn about this topic. I like all of the points you made.
LikeLike