No separate organization called "OpenAI Lab" turned up in the sources reviewed for this article. OpenAI is a company, and the best-documented outside check on its work is government testing: on August 29, 2024, the U.S. AI Safety Institute signed a memorandum of understanding with OpenAI that gives the Institute access to major new models before and after public release. The Institute was re-established as the Center for AI Standards and Innovation (CAISI) in June 2025, and a September 2025 NIST post says CAISI worked with OpenAI and Anthropic to find security issues in their systems.
Key takeaways
- The August 29, 2024 agreements with OpenAI and Anthropic cover access to models before and after release, joint research on evaluation methods, and feedback on safety improvements.
- NIST's September 25, 2025 post says the work was done with the UK AI Security Institute and that OpenAI published its own account of security improvements.
- A published government evaluation of another company's model shows what such reports contain and how cautiously their authors describe them.
- Canada announced its own AI safety institute on November 12, 2024, but that announcement does not describe testing agreements with OpenAI.
What the 2024 agreement says
NIST announced on August 29, 2024 that the U.S. AI Safety Institute had signed separate memoranda of understanding with Anthropic and OpenAI. Each creates a framework for formal collaboration on AI safety research, testing, and evaluation. The Institute gets access to each company's major new models before and after public release, and the stated goals are research on how to evaluate capabilities and safety risks, and on how to reduce those risks. The Institute also planned to give both companies feedback on possible safety improvements, working with its UK counterpart. The NIST announcement is the primary source for these terms.
A memorandum of understanding sets out how two parties intend to cooperate. The announcement describes access and collaboration. It does not describe a legal duty to submit every model or a power to block a release.
What a government evaluation report looks like
The OpenAI agreement has not produced a public report that was found for this article, but the same institutes published a joint pre-deployment evaluation of another company's model, the upgraded Claude 3.5 Sonnet, on November 19, 2024. It is the clearest public example of the method, and it used OpenAI's o1-preview and GPT-4o as reference models. The NIST write-up reports four test areas: biological capabilities, cyber capabilities, software and AI development, and the effectiveness of built-in safeguards.
On a public suite of 40 cybersecurity challenges, the model solved 32.5 percent of tasks, against 35 percent for the best reference model. On a 47-challenge suite from the UK institute, it solved 36 percent of apprentice-level tasks against 29 percent for the best reference. The institutes also reported that public jailbreak methods got around the built-in safeguards in most cases. The authors called all findings preliminary, because testing ran for a limited time with finite resources, and warned that small score gaps may fall within the margin of error.
The September 2025 follow-up
A NIST post dated September 25, 2025 says CAISI worked with OpenAI and Anthropic to identify security issues in their advanced systems and to improve how AI security is measured, and that the evaluations were done in partnership with the UK AI Security Institute. The UK body was renamed from the AI Safety Institute on February 14, 2025. NIST adds that OpenAI published a blog post describing security improvements made as a result. The NIST page does not list the vulnerabilities, success rates, or fix times, and the OpenAI post itself could not be opened for this article, so those details are left out.
What is public and what is not
| Item | What a primary source states | What was not found |
|---|---|---|
| Agreement terms | NIST, August 29, 2024: access before and after release, joint research, safety feedback | Text of the memorandum, any list of models tested under it |
| Method | NIST, November 19, 2024: question sets, agent tasks, expert probing, red teaming, reference models | A comparable report for an OpenAI model |
| Security work with OpenAI | NIST, September 25, 2025: joint work with the UK institute, OpenAI blog post followed | Counts of flaws, severity, time to fix |
| Binding power | None described in the announcements | Any rule that forces a model to be withheld |
The Canadian side
The Government of Canada announced the Canadian Artificial Intelligence Safety Institute on November 12, 2024 with an initial budget of CAD 50 million over five years. It sits within Innovation, Science and Economic Development Canada. The Canadian Institute for Advanced Research funds investigator-led research, and the National Research Council of Canada leads government-directed projects. The announcement says the institute will join the International Network of AI Safety Institutes and engage research and business communities in projects to assess risks and test systems. It does not describe pre-release access agreements with any named developer.
Questions to ask about any claim of independent AI testing
- Who ran the test, and did the developer choose what was shared?
- Is the report public, or is only a summary of its conclusion public?
- Does it compare against named reference models and say how many tasks were used?
- Does it state its own limits, such as short testing time or small score gaps?
- Did the developer change the product afterward, and where is that described?
FAQ
Is there an OpenAI Lab?
Nothing under that name appeared in the primary sources reviewed. OpenAI's structure and ownership are covered elsewhere on this site.
Does the government approve OpenAI models before launch?
The announcements describe access to models and feedback on safety. They do not describe an approval step, and the sources reviewed identify no law that requires one.
What is CAISI?
It is the Center for AI Standards and Innovation, the successor to the U.S. AI Safety Institute, re-established in June 2025 according to NIST.
Does Canada test OpenAI models?
The Canadian institute's launch announcement does not describe that. Its stated work is research funding, government-directed projects, and international cooperation.
Related reading
- The Global Partnership on AI
- OpenAI's mission and structure
- Alternatives to OpenAI language models
- Companies working toward AGI
More in AI and Society.
Last verified: October 2026
Sources
- U.S. AI Safety Institute Signs Agreements Regarding AI Safety Research, Testing and Evaluation With Anthropic and OpenAI, NIST, August 29, 2024
- Pre-Deployment Evaluation of Anthropic's Upgraded Claude 3.5 Sonnet, NIST and UK AI Safety Institute, November 19, 2024
- CAISI Works with OpenAI and Anthropic to Promote Secure AI Innovation, NIST, September 25, 2025
- UK AI Safety Institute becomes UK AI Security Institute, UK government news release, February 14, 2025
- Announcement of the Canadian Artificial Intelligence Safety Institute, Innovation, Science and Economic Development Canada, November 12, 2024