05 April 2007

Selection Effect

You are sitting on a chair with a notebook. You're kinda tired, so you keep dozing off. A noise wakes you, and you look up to see a woman rummaging through her purse. You make a note in your notebook. A while later she leaves. You doze off again. You hear women's voices outside in the hallway, and open your eyes as two walk in. Two more notes in your notebook. A woman jangling her keys is noted a few minutes later, another woman tossing something in the trash, and so on.

At the end of the day you have seen 212 women, and you yourself are the only man. Do you conclude that...

A) Women make more noise so when the men entered you didn't hear them and didn't mark them down? (A selection effect or bias.) -OR-
b) The room you were in has some reason for there to be more women - you're in the common room at a woman's dorm, you're in a woman's bathroom, you're in the lobby to an operahouse, you're in a nail salon, etc.? (There's a real effect.)

At what point do we decide that an observation is due to a detection bias, and at what point do we realize that despite the detection bias we're observing a real effect?

3 comments: