How sampling bias influences your HR decisions
Mar 31, 2020
When the sample you selected says nothing about the population at large.
What is it?
A sample is said to be biased when it was collected in such a way that some members of the intended population have a lower sampling probability than others: all individuals were not equally likely to have been selected. Thus the results are not reliable as the sample is not representative of the targeted population.
Sampling bias is a big problem for polling organizations and market research institutes. In 1936, before polling became a national sport in the US, the magazine Literary Digest called 2 million phone numbers to ask people who they intended to vote for president. The forecast made by the magazine turned out to be completely wrong (even though 2 million is a huge sample!) because the people who owned a phone in 1936 were not representative of the population. They were richer and more urban than the rest of the population. They were a biased sample.
Roughly around the same time, a man named George Gallup, predicted the result of the US presidential election with astounding accuracy, based on a sample of only 50,000 people. The inventor of the Gallup poll had come up with a successful statistical method of survey sampling for measuring public opinion. His samples were significantly more reliable.
Today, statisticians and sociologists are very familiar with sampling bias. Yet surveys and studies are consistently exposed to it. Although (almost) all individuals now have a phone, phone surveys remain unreliable because the people who answer phone surveys tend to be older, less active, and more available than the population at large.
In fact, all the polls that rely on the goodwill of respondents fall prey to the bias. Answering a series of questions is in itself a filter. Those who respond may be quite different from those who do not. In that case, there is a self-selection bias, or non-response bias (when the group of people responding has different responses than the group of people not responding).
What does it mean for human resources?
HR departments often carry out surveys whose reliability depends on the quality of the selected samples. Only reliable results can empower HR people to draw relevant conclusions and make decisions that may have an impact on every employee, whether it pertains to work spaces, engagement, or training.
Unfortunately the samples are often biased. HR people give too much weight to groups of individuals that are more in the limelight, for example, that are more talked about by the media. The media tend to prefer what’s sensational to what’s representative. One could argue that the level of attention given to millennials is an illustration of the phenomenon
Last but not least, HR departments are concerned by the subject of union representativeness, as they are responsible for social dialogue. Whether or not unions are actually representative of all employees is a critical question. As fewer and fewer people are affiliated with a union, the whole system of collective negotiation is in crisis today.
How can it be overcome?
To take sampling bias into account, HR people must learn to think like statisticians and “correct” their samples to make them more representative. Overrepresented profiles must sometimes be taken out and underrepresented profiles counted multiple times. But if some groups (women, for example) are completely absent, no correction can be made.
The more we rely on big data and quantitative analyses to make corporate decisions, the more it becomes vital to be familiar with the dangers of sampling bias and the techniques used by statisticians to remedy the bias.
Photo: Welcome to the Jungle
Follow Welcome to the Jungle on Facebook, LinkedIn, and Instagram, and subscribe to our newsletter to get our latest articles every day!
More inspiration: Laetitia Vitaud
Future of work author and speaker
Why “bring me solutions, not problems” is a terrible corporate phrase
Future of work expert Laetitia Vitaud takes a swing at a classic (and toxic) part of the managerial dictionary
Feb 23, 2023
What a dash of laziness can do for you
It would never occur to us to criticize an animal for needing sleep, so why do we judge each other so harshly?
Jan 05, 2023
Top Gun: An anti-ageist movie? Not exactly…
The blockbuster sequel shows the cognitive dissonance between what we say about valuing experience and the reality of our workplaces
Oct 06, 2022
Here’s why you don’t need to be irreplaceable at work
Do we want to be considered irreplaceable at work? Sure, it might give us a confidence boost in the moment, but is it worth it in the long run?
Sep 07, 2022
Is period leave really a good idea?
The debate over menstrual leave is growing louder in the US, but is it really a progressive policy?
Jul 27, 2022
Inside the jungle: The HR newsletter
Studies, events, expert analysis, and solutions... Every 2 weeks in your inbox
Are you proud of your corporate culture?
Give it the visibility it deserves.Our solutions