Skip to main content

Measure f-ing everything, and assume f-ing nothing!! - Or how mentoring ruined lives :-(

I've been really enjoying the Freakonomics podcast of late. This episode and the lesson we should take a away from it, was a stark reminder of one of the most important things we should be doing - but often don't - in building products or making any decisions: measuring the impact of absolutely everything we do, including the things that seem obviously good.

I recommend listening to the podcast if you have the time, but here's the summary. Stephen Dubner describes the Cambridge Sommerville Youth Study. The impact of social intervention programs in general is hard to measure and so they seldom are. This was the first attempt at measuring the impact over a long period of time.

It's a great story and there are a few good take-aways, but here's the main one: troubled or at-risk youth that received mentoring (good mentoring!) had worse life outcomes across every dimension than the kids that were left alone. Despite the recipients saying that the mentoring was incredibly valuable and helped them, they were actually worse off across a bunch of measured dimension.

Everyone's assumption: mentoring troubled youth will help them in life was wrong - at least the kind of mentoring that they were doing then. They were able to actually measure and determine this only because computers were just starting to become available and the person leading the research was determined enough to figure out how to use them.

This isn't unusual. A lot of things can seem obviously good especially to people working on them, but most don't really do the work to understand (or measure) the long-term impact of the changes they're making. Sometimes it's because they don't feel the need to - a lesson learned in this Steven Seagal classic. Other times it's because it's really, really hard to measure these things.

This is a great reminder that the impact can be very, very different from what you expect or what seems "obvious". It can even ruin people's lives.

So, find a way to measure everything. Always.





Comments

Popular posts from this blog

Yup - humans still lack humanity

Every once in a while, I'm reminded that humans can be completely lacking in humanity.

My wife had the following experience yesterday on her ride back home. She got on the train and found a seat. The train was unusually crowded and it looked a lot of people had to stand for a long ride. An elderly Asian gentleman carrying a few things in both hands, was looking for spot, started to complain smilingly about the train being so full and stood in the aisle at the back of the carriage some seats away from her.

She expected someone closer to gentleman in the aisle (lots of younger people on the train) to give him their seat.

No one did.

The train started, and it was clear the man was having a lot of trouble standing up. Then at the next stop there was actually an announcement saying the train was full so please give up your seats to people who needed them.

Still nobody moved.

My wife got up walked to the end of the train and asked the gentleman to go over to her seat. She still couldn&#…

Whimsy when I changed my profile picture...

I changed by profile picture at work.



Later in the day, two people on my team had changed their profile pictures to these.. :-)



It made my day!

I changed my profile pic again today. Let's see how fast anyone catches on this time. :-)