AI ethics // EP15

Ep15: Why is AI biased against women?

Jaimee AI founder Sreyna Rath explains

Why would an AI tell a qualified software engineer to mention her baking and multitasking skills in a job interview? That actually happened to Sreyna Rath, and it led her to build Jaimee, an AI companion designed for women. In this episode she and Kim Fernandez unpack why AI so often reflects bias against women, and what that means for any business relying on it.

Based on AI Your Business Podcast2026-07-20

The short version

Sreyna's core idea is deceptively simple: AI is a mirror, not a crystal ball. It doesn't predict the future, it reflects the data, and the assumptions, it was trained on. When ChatGPT nudged a qualified software engineer toward "baking and multitasking", it wasn't malfunctioning, it was echoing patterns in its training data. For any business using AI to hire, market or make decisions, that reflection matters.

AI is a mirror, not a crystal ball. It reflects the biases in its training data straight back at users.
This isn't hypothetical. Amazon scrapped a hiring algorithm that penalised the word "women", and most AI companions are still built for male users.
With women about 20% of Australia's technical workforce, who builds and checks AI shapes whose bias gets baked in.

Why this matters

If AI quietly carries bias, so does any business decision you hand to it, screening candidates, targeting customers, generating content. The risk isn't only ethical, it's practical and legal: biased outputs can shut you out of talent, alienate customers and expose you to discrimination claims.

  • AI-assisted hiring can filter out great candidates on biased patterns
  • Marketing and content can quietly reinforce stereotypes
  • Biased outputs can create real legal and reputational exposure
  • Teams trust confident AI answers without checking the assumptions

What to do with this insight

Treat bias as something to actively check for, not assume away. Put a human in the loop wherever decisions affect people, and make diverse review part of how you build and buy AI.

  • Question AI outputs that involve people, hiring or fairness
  • Keep human review on any decision that affects someone's opportunities
  • Test tools for biased patterns before you rely on them
  • Bring diverse perspectives into how you select and check AI

How AI Collab thinks about it

Sreyna's "mirror, not crystal ball" line is one we'll be borrowing. AI is only as fair as the data behind it and the people checking it. We help businesses adopt AI with the governance and human review that catch this, so the tools build trust instead of quietly encoding old bias.

Frequently asked questions

Why is AI biased against women?
AI learns from historical data that reflects real-world bias, so it can repeat and amplify stereotypes about women unless it is carefully checked and corrected.
Can AI bias affect my business?
Yes. Biased AI can skew hiring, marketing and decisions, creating ethical, reputational and legal risk. Human review and testing help catch it.
What is Jaimee AI?
Jaimee is an AI companion designed specifically for women, created by Sreyna Rath after she experienced gender bias in mainstream AI tools.

Want to turn this into a practical AI plan?

AI Collab helps established Australian businesses move from interest to implementation: strategy, team capability, governance and custom builds that fit the way the business actually works.

Talk to AI CollabListen to the episode