The company has launched MentalHealthBench, an open source evaluation tool designed to measure how artificial intelligence handles psychiatric interactions ranging from routine support to acute crisis scenarios. In initial tests using the benchmark, OpenAI's GPT-6 Astra achieved a score of 57.3 on realistic mental health conversations, significantly higher than the 32.1 recorded by GPT-4o.

More than 80 licensed psychiatrists and psychologists from 22 countries and 19 languages helped develop the benchmark's expert rubrics, which assess behaviors such as preserving user agency and seeking context. Each synthetic conversation case was reviewed by at least 3 experts, requiring a consensus of 2 for a criterion to be included. Final scores are generated by GPT-5.6 Sol, which acts as an LLM judge to grade model responses against the human-written criteria.