Vai al ContenutoVai al Footer

Not For Her: the AI job interview that exposes gender discrimination

An interview with Nicola Gatti, Full Professor at the Politecnico di Milano, coordinator of FAIR’s Spoke 4 and one of the installation’s curators.

“I have to pay you less than someone who doesn’t have children because I can’t be sure of your commitment.” This is one of the remarks Sofia may hear during a simulated job interview. Anyone entering the darkened room of the Not For Her installation takes on Sofia’s role. The interview is conducted by an artificial intelligence system that adapts the dialogue to each participant’s responses and can lead to seven different scenarios involving gender discrimination.

Presented in 2025 at Triennale Milano as part of the Inequalities International Exhibition, the installation was conceived and developed by the Politecnico di Milano. Fondazione FAIR co-funded its development through Spoke 4.

Fondazione FAIR is Italy’s national hub for artificial intelligence research. It supports research into the foundations of AI and aims to serve as a trusted, long-term partner for institutions, public administrations, the scientific community, businesses and the media.

The project also led to the book Not For Her. AI Revealing the Unseen, which will be presented on 24 November at BookCity Milano, on the eve of the International Day for the Elimination of Violence against Women.

How do you design an AI system that can adapt to a person’s responses while steering the conversation towards a specific scenario? And how could this kind of simulation be used to train people who conduct job interviews? We discuss these questions with Professor Gatti.

What did participants know before starting the experience?

“They knew very little: that they would be interacting with an artificial intelligence installation and taking part in a job interview. They didn’t know they would experience discrimination. You enter a completely dark room and are asked to play a woman called Sofia, even if you are a man.”

“The system uses the information you give it to shape the rest of the conversation. For example, if you say whether you are in a relationship or have children, it uses that information to steer the dialogue towards a form of discrimination. It may end up saying: ‘Let me take you out to dinner, so we can take things further.’ Or, if you are a mother: ‘I have to pay you less than someone who doesn’t have children because I can’t be sure of your commitment.’ These are two possibilities: there are seven specific discrimination scenarios the conversation can lead to.”

So it starts with subtle forms of discrimination and moves towards more openly misogynistic behaviour?

“The forms of discrimination were developed with people who work on these issues, engage with the human resources sector and have experience analysing the impact of technology on society. These are not situations we made up: they have been analysed and validated. The system interrupts you if you talk too much; if you don’t speak, it starts reprimanding you. The interview becomes very natural and interactive, as though you were talking to real people.”

“I would add an observation here, separate from the explanation of how it works: the first time you try the experience, you feel extremely uncomfortable even before the discrimination begins. The discomfort is immediate. The system also rushes you to answer: it is an unexpected situation that takes you out of your comfort zone, and you struggle to interact with the machine as well as you could. Everyone comes out unsettled by the experience, which is dystopian and disturbing.”

“The difference between men and women also concerns how they perceive discrimination. Some men don’t recognise it in a statement such as: ‘I’ll pay you less because you have children.’ Their reaction is: ‘Of course, that seems perfectly normal to me.’ I’m exaggerating, but that is broadly what happens. Women, on the other hand, are much more sensitive to these forms of discrimination.”

“But the installation was not conceived as a tool for studying society. It was created to let people experience a dystopian situation involving discrimination: reading about it is one thing, watching it in a video is another, and experiencing it yourself is something else entirely. The impact of the message is completely different. What is interesting, though, is that by collecting data and observing how people behave, you can also investigate whether people across our society share the same sensitivity to discrimination.”

Were there any reactions that surprised you, or did the experience confirm your expectations?

“You have to bear in mind that those who worked on the project saw it evolve gradually. The people who tested it over several months, not just the technical team, tried it repeatedly: perhaps it didn’t work very well at first, but then it improved. So when you try the fully functioning system, you are not surprised, because you have watched it develop. And as you repeat the experience, it almost becomes a game: you feel more comfortable answering, you already have an idea of what you might say, and you no longer feel that sense of unease.”

“It is also worth adding that during testing, we weren’t trying it in an enclosed, completely dark room in front of a large screen. We were testing it in the office, on a small monitor. In that respect too, our exposure was gradual. The final result is very different from what someone who hadn’t seen the intermediate stages might imagine. But if you followed the entire development process, the result wouldn’t surprise you all that much.”

How does this project fit into FAIR’s research objectives?

“Within FAIR, I would describe it primarily as a technological exercise. FAIR was a project devoted to foundational research into artificial intelligence technologies. The question was: how can we build a system that can interact with a person and guide the conversation? The point is that when you use ChatGPT in the usual way, the system cannot steer you towards the communicative goal you want to achieve. It responds based on what you have written, whereas here the aim is to steer the interview towards a form of discrimination that specifically depends on the information you have provided.”

“Then there is everything else: recognising the face, facial expressions and tone of voice, and integrating all these elements.

To clarify: ChatGPT’s voice mode is impressive and responds to what you say. In this case, however, there is a third party involved. On one side, there are the artificial intelligence system and the human user; on the other, there is the person programming the system to follow a particular conversational path and arrive at a form of discrimination.”

“So you need a separate system to guide it: ChatGPT cannot automatically perform that function as well. This is the distinctive aspect of the development work: it was a technical and technological challenge, and from that perspective, it was entirely successful. The humanities component falls only partly within FAIR’s scope, because the project focuses primarily on technology.”

How could this kind of research benefit businesses? Job interviews, for example, come to mind.

“That is the most immediate application. But everything we have done can be replicated in other contexts. Initially, I also considered other situations involving discrimination: for example, discrimination against gay people, or against women in the workplace outside the interview setting, in the relationship between a manager and an assistant. We have a system that allows one or more AI avatars to engage in an interactive conversation with a person. The content needs to be programmed and developed, but in principle there could be many applications.”

“The current system, which conducts a job interview, could be developed into a commercial product that allows companies to carry out automated interviews. Imagine a company that receives 10,000 CVs and, after an initial screening, retains around 1,000, rejecting 9,000. These are illustrative figures. At present, it cannot interview all those candidates, so it has to rule out many more and meet only a very small number.”

“With this system, you could introduce an intermediate stage: an automated preliminary interview with 1,000, 2,000 or 3,000 candidates, which would then inform the decision about whom to meet in person. The impact would be enormous.”

Could it also be used to raise awareness and train interviewers, helping them recognise discriminatory behaviour they may not even be aware of?

“Yes, absolutely. More broadly, it could be used for training in many activities that involve talking and interacting with someone. Teaching comes to mind too: I could deliver a lesson in front of the machine to test my skills. Why not? The principle applies to any activity in which you have to speak and interact with another person.”

Does the book also refer to previous studies of discriminatory behaviour learned by artificial intelligence?

“Yes, there is a chapter presenting concrete examples of discrimination. I contributed to that chapter too.”

What are the next steps for the project?

“The big question is whether we can turn this system into a commercial product that businesses can use. We are exploring that possibility. It would also be interesting to build a visually realistic virtual environment around it. At present, the avatars are three figures made up of point clouds, which move and gesture. Everything is synchronised with what happens in the dialogue: they move their mouths and eyes, and their animation responds to what is happening.”

“For some applications, a realistic virtual environment would be very interesting. One of the possibilities we discussed with the Rector was to offer an experience that could also support educational and career guidance. People choosing a university course often have little idea of the work they might do after graduating, let alone the activities they will undertake during their studies.”

“Take this example: someone says, ‘I’m going to an arts high school because I like drawing,’ then finds themselves working on bas-reliefs, plaster casts or painting and thinks, ‘But I don’t want to do these things; I’m not good at them.’ Reading the word ‘bas-relief’ in a syllabus is not the same as actually trying it. The point is that, in many cases, we don’t understand something unless we experience it.”

“In the not-too-distant future, however dystopian it may seem today, this kind of preparation is very likely to become standard practice for job interviews. Imagine a scenario in which, for ten minutes, you can take on a role within a data science team and discover what working as a data scientist actually involves. This is an application we would very much like to develop. It would help people understand what they can and cannot do, and how closely that work matches their interests.”

“Students may view many professions negatively because of outdated stereotypes. Experiences that spark their interest in those professions can help them build a better future.”

Last news

Archives