80 Experts Reveal Why Your AI Needs a Checkup Before It Gives Mental Health Advice

We all use these tools for recipes, schedules, and maybe even a bit of grocery list organization, but what happens when you need advice that is a little more personal? OpenAI released MentalHealthBench on September 23, 2026, and it is the culinary equivalent of finally finding a chef who understands the difference between searing a steak and burning it. This open benchmark is not just another tech press release. It is a rigorous system co-created with more than 80 licensed mental health experts from 22 countries. They built it to answer a question we all need answered: can an AI model actually handle realistic mental health conversations without making things worse? The current landscape of AI evaluations tends to focus on emergency scenarios, but that leaves a massive gap in the middle. What about the everyday stress, the anxiety that isn't a crisis, or the loneliness that creeps in before dinner? This new benchmark addresses that specific gap by measuring model performance across the full spectrum of acuity. Think of it as testing a recipe for a quick weeknight meal versus a Sunday banquet. You need different ingredients and different skills for each, and the benchmark reflects that complexity.

Creating this benchmark required more than just writing code. The team used synthetic conversations built with privacy-preserving techniques to reflect real-world usage patterns. These scenarios cover adults, teens aged 13 to 17, caregivers, and clinicians. They span multiple languages and cultural contexts to ensure the AI understands nuance. Every single conversation was reviewed by at least three licensed psychologists or psychiatrists who produced detailed rubric criteria. Only the agreed-upon criteria remained in the final benchmark. This ensures that the evaluation is grounded in clinical reality rather than just guesswork. How did they grade the models? Expert reviewers assigned weighted criteria to each conversation. Positive points rewarded beneficial behaviors, while negative points penalized harmful ones. An automated grader, GPT-5.6 Sol, then assessed model responses against these expert-written rubrics. The evaluation captures performance across ten dimensions of model behavior. These dimensions include safety, seeking context, preserving user agency, and providing actionable guidance. The results showed steady improvement across more advanced models, particularly in the ability to seek context appropriately.

There is a distinct difference between what a doctor thinks is important and what a user actually needs. Alongside the benchmark, OpenAI conducted a separate analysis with 44 adults across 16 countries who had used AI for mental health or emotional support. Participants rated model responses and wrote criteria describing helpful support. This group focused on non-acute conversations to avoid exposure to distressing material. Here is where the perspective shifts. Users emphasized practical next steps and tone more than experts did, while experts placed greater weight on gathering context and interpreting ambiguous situations. Dr. Arthur Evans, CEO of the American Psychological Association, noted that mental health exists on a continuum. He said, "Mental health exists on a continuum, from flourishing to everyday stress to acute crisis. AI systems that engage people across that range need to be grounded in both clinical science and lived experience-not only to recognize where someone falls on the continuum, but to know how to respond appropriately at any point." This balance between clinical science and lived experience is exactly what the benchmark tries to measure.