Changing AI Language Changes Its Personality: A Risky HR Blind Spot

According to research from the University of Science and Technology of China published in Royal Society Open Science, changing the prompt language alters how chatbots behave on standard psychological evaluations. Testing five models—including ChatGPT-4o, Gemini 2.0 Flash, and DeepSeek-V3—in English and Mandarin, researchers found that linguistic shifts directly impact traits measured by the Big Five Inventory-2.

How much do the results actually shift?

The variations are large enough to break automated evaluations. For instance, ChatGPT-4o scored 4.74 in mental openness when tested in English, but that score dropped to 4.03 in Mandarin. DeepSeek-V3 showed the exact opposite pattern, scoring 4.81 in Mandarin and 4.38 in English. Evaluated on a scale from 0 to 5 across 60 questions over 30 repeated runs, the models also showed regional patterns: US and French models leaned toward higher agreeability in English, while Chinese models stood out in knowledge and curiosity metrics.

What does this mean for HR and screening?

If your organization uses AI models to simulate user behavior, process feedback, or assist in recruitment filters, language choice is a hidden variable that warps the output. A prompt written in Portuguese or English will yield a fundamentally different behavioral profile than the same prompt in Mandarin. Furthermore, the study noted that chatbot responses diverge significantly from real human data, proving that models cannot reliably stand in for human test subjects.

How to protect your workflows

If you rely on language models for candidate assessments or sentiment analysis, standardization is mandatory. Never switch prompt languages mid-project expecting consistent psychological baselines. Audit your workflows to ensure that evaluation benchmarks are locked into a single primary language, and treat multi-language deployment as introducing an entirely different model version.

Verdict: who should adjust their processes?

HR teams and product developers building persona-driven agents must account for this linguistic drift immediately. While these tools do not possess actual feelings, their training data ties linguistic structures to behavioral weightings. Ignoring this dynamic risks building automated filters that reject or promote candidates simply because of how a prompt was translated.