AI and its Shortcomings: What Happened at Amazon?

How did an inherent bias against women creep in?

Add bookmark

Arran Stewart

Artificial Intelligence

Artificial Intelligence in Recruiting

Artificial Intelligence is only as good as the person that programmed it, it’s as simple as that.

When considered the recent case with Amazon scraping its AI recruitment software due to discovering it had an inherent bias against women, it got everyone asking, How could this happen?

Sadly, bias still exists even when we consider ourselves unprejudiced. It especially exists unconsciously, in the way we write and speak. Take for example masculine and feminine language. In the case of Amazon’s AI as well as lots of AI that are written for the recruitment space, men and their inherent use of masculine words code this tech and language through the programming of these platforms, resulting in bias.

You would think that a computer system would be completely objective about its matching process, however when the taxonomy, rules, and touch points of a system’s machine are dictated by one demographic of person (in this case, men) its obvious that there may be biased traits that come out as ghosts of the machine.

There are three key areas we have to worry about right now on the back of this news:

1. How many other platforms are unknowingly operating with potential bias?
We would be naïve to think that this particular incident is exclusive to Amazon's tech and programmers. I feel that a storm is coming towards the world of online recruitment platforms and they may be subject to tests to ensure impartiality.

2. How do we test other platforms to see if they do have a bias?
What should be the parameters for identifying whether a platform performs unbiased or not?

3. How do we rectify these platforms and make the online recruitment world a level playing field for all those using it?

What Amazon has identified is a much larger and deeper AI issue than we all anticipate. These technologies have been popularly used in the recruitment industry for over a decade. How many millions of people have already been affected by biased technology on their career journey?

To conclude, there is a wide-open opportunity for an organization or body to now form that can independently check technology to make sure it performs its tasks objectively. Amazon has blown the whistle and all of us in the industry have a duty to our users to make sure our technology is being built to give everyone the most and fair opportunity to get the job they deserve.