Skip to main content

Why search is (still) hard.

Techmeme led me to this write up on ReadWriteWeb.
The write-up is a summary of a talk titled "Search is a Hard Problem" given by Udi Manber, a VP of Engineering at Google.

The article confirms that the devil is still in the details. Two little things from the article struck me:

  • The query pre-processing being attempted even before the search is run:
I knew Google did little things like stripping out articles from search strings, playing with participles/prefixes/suffixed and tenses to deliver better results, but I didn't realize they did stuff like this:

"For example, Google understands that 'GM' stands for 'General Motors', while 'GM foods' is actually 'genetically modified.' If you search for 'B&B AB', Google knows that is 'bed and breakfast in Alberta', while 'Ramstein AB' is 'Ramstein Airbase'."

  • The localization that is already part of the search engine.
That is typing in "government" in the US, takes you the the US government homepage, whereas typing it in, say, Australia take you to the Australian government's page.

Effective search results are really about getting at the intent of the user from what he/she has typed in and matching it to the attributes of the data that you've chosen to store. And given, that according to him, 20-25% of the queries they see each day they haven't seen before (and most of the world is still getting online) its going to be an ever-present challenge.

Comments

Popular posts from this blog

Whimsy when I changed my profile picture...

I changed by profile picture at work.



Later in the day, two people on my team had changed their profile pictures to these.. :-)



It made my day!

I changed my profile pic again today. Let's see how fast anyone catches on this time. :-)

Yup - humans still lack humanity

Every once in a while, I'm reminded that humans can be completely lacking in humanity.

My wife had the following experience yesterday on her ride back home. She got on the train and found a seat. The train was unusually crowded and it looked a lot of people had to stand for a long ride. An elderly Asian gentleman carrying a few things in both hands, was looking for spot, started to complain smilingly about the train being so full and stood in the aisle at the back of the carriage some seats away from her.

She expected someone closer to gentleman in the aisle (lots of younger people on the train) to give him their seat.

No one did.

The train started, and it was clear the man was having a lot of trouble standing up. Then at the next stop there was actually an announcement saying the train was full so please give up your seats to people who needed them.

Still nobody moved.

My wife got up walked to the end of the train and asked the gentleman to go over to her seat. She still couldn&#…

Measure f-ing everything, and assume f-ing nothing!! - Or how mentoring ruined lives :-(

I've been really enjoying the Freakonomics podcast of late. This episode and the lesson we should take a away from it, was a stark reminder of one of the most important things we should be doing - but often don't - in building products or making any decisions: measuring the impact of absolutely everything we do, including the things that seem obviously good.

I recommend listening to the podcast if you have the time, but here's the summary. Stephen Dubner describes the Cambridge Sommerville Youth Study. The impact of social intervention programs in general is hard to measure and so they seldom are. This was the first attempt at measuring the impact over a long period of time.

It's a great story and there are a few good take-aways, but here's the main one: troubled or at-risk youth that received mentoring (good mentoring!) had worse life outcomes across every dimension than the kids that were left alone. Despite the recipients saying that the mentoring was incredibl…