How sampling bias influences your HR decisions
When the sample you selected says nothing about the population at large.
What is it?
A sample is said to be biased when it was collected in such a way that some members of the intended population have a lower sampling probability than others: all individuals were not equally likely to have been selected. Thus the results are not reliable as the sample is not representative of the targeted population.
Sampling bias is a big problem for polling organisations and market research institutes. In 1936, before polling became a national sport in the US, the magazine Literary Digest called 2 million phone numbers to ask people who they intended to vote for president. The forecast made by the magazine turned out to be completely wrong (even though 2 million is a huge sample!) because the people who owned a phone in 1936 were not representative of the population. They were richer and more urban than the rest of the population. They were a biased sample.
Roughly around the same time, a man named George Gallup, predicted the result of the US presidential election with astounding accuracy, based on a sample of only 50,000 people. The inventor of the Gallup poll had come up with a successful statistical method of survey sampling for measuring public opinion. His samples were significantly more reliable.
Today, statisticians and sociologists are very familiar with sampling bias. Yet surveys and studies are consistently exposed to it. Although (almost) all individuals now have a phone, phone surveys remain unreliable because the people who answer phone surveys tend to be older, less active and more available than the population at large.
In fact all the polls that rely on the goodwill of respondents fall prey to the bias. Answering a series of questions is in itself a filter. Those who respond may be quite different from those who do not. In that case there is a self-selection bias, or non-response bias (when the group of people responding has different responses than the group of people not responding).
What does it mean for human resources?
HR departments often carry out surveys whose reliability depends on the quality of the selected samples. Only reliable results can emplower HR people to draw relevant conclusions and make decisions that may have an impact on every employee, whether it pertains to work spaces, engagement or training.
Unfortunately the samples are often biased. HR people give too much weight to groups of individuals that are more in the limelight, for example, that are more talked about by the media. The media tend to prefer what’s sensational to what’s representative. One could argue that the level of attention given millennials is an illustration of the phenomenon
Last but not least, HR departments are concerned by the subject of union representativeness, as they are responsible for social dialogue. Whether or not unions are actually representative of all employees is a critical question. As fewer and fewer people are affiliated to a union, the whole system of collective negotiation is in crisis today.
How can it be overcome?
To take sampling bias into account, HR people must learn to think like statisticians and “correct” their samples to make them more representative. Overrepresented profiles must sometimes be taken out and underrepresented profiles counted multiple times. But if some groups (women, for example) are completely absent, no correction can be made.
The more we rely on big data and quantitative analyses to make corporate decisions, the more it becomes vital to be familiar with the dangers of sampling bias and the techniques used by statisticians to remedy the bias.
Tags
- Add to favorites
- Share on Twitter
- Share on Facebook
- Share on LinkedIn
We are all biased!
Short and insightful articles to help you identify your cognitive biases, understand how they affect HR, and what to do to address them.
Ever heard of the "McNamara Fallacy"? Be careful what you measure
Don’t presume that what can’t be measured isn’t really important.
Want your team to be more productive? Why the Hawthorne Effect matters
Psychologists know that people do better when you pay attention to them. Learn how to be aware of this phenomenon and leverage all its positive.
Five compliments to overcome one criticism: discover the negativity bias
This bias has huge impact for companies: one single toxic manager, one harassment case, one employee burnout can durably damage your employer brand.