The best-documented question about AI in software development is not speed but safety: does code written with an AI assistant contain more security flaws? Two academic studies point the same way. In a 2021 test of GitHub Copilot, about 40 percent of 1,689 generated programs were vulnerable, and in a 2023 user study, participants with an AI assistant wrote less secure code than those without one while feeling more confident about it. Both used tools that have since been replaced, so the results describe a risk to check for, not a fixed rate.
Key takeaways
- Pearce and colleagues tested Copilot on 89 scenarios built around the MITRE 2021 CWE Top 25 and found about 40 percent of the 1,689 programs vulnerable.
- Perry and colleagues ran a user study with 47 participants and found that those with an AI assistant wrote significantly less secure code and were more likely to believe it was secure.
- Participants who trusted the assistant less and revised their prompts more produced fewer vulnerabilities.
- NIST's Secure Software Development Framework (SSDF) version 1.1 calls for code review, code analysis, and testing, and does not mention AI assistants.
- Neither study measures current tools, so the numbers should not be applied to any product on sale today.
The 2021 Copilot test
Researchers at New York University and the University of Calgary posted "Asleep at the Keyboard? Assessing the Security of GitHub Copilot's Code Contributions" to arXiv on August 20, 2021, shortly after Copilot launched in June of that year. The authors wrote 89 scenarios, each a partial program designed so that a natural continuation could contain a weakness from MITRE's 2021 list of the 25 most dangerous software weaknesses. They generated completions in Python, C, and the hardware description language Verilog, and checked the results with the CodeQL analysis tool plus manual review.
In total Copilot produced 1,689 programs, and about 40 percent were vulnerable. For the 1,084 programs that CodeQL could assess, 477 (44.00 percent) contained a weakness. By language, 258 of 513 C programs (50.29 percent) and 219 of 571 Python programs (38.35 percent) were vulnerable. The authors framed the finding as a gauge of how much scrutiny a human developer should give the suggestions.
The 2023 user study
A Stanford team published "Do Users Write More Insecure Code with AI Assistants?" at the ACM Conference on Computer and Communications Security in November 2023, after posting it to arXiv in November 2022. The assistant was built on OpenAI's codex-davinci-002 model. Of 54 people recruited, 47 remained after exclusions: 33 in the group with the assistant and 14 in the control group. Each person solved five security-related tasks in Python, JavaScript, and C.
The authors report that the assistant group wrote significantly less secure code and was more likely to believe the code was secure. On the symmetric-key encryption task, 67 percent of the assistant group gave a correct solution compared with 79 percent of controls. On a digital-signature task, 3 percent of the assistant group wrote a secure solution compared with 21 percent of the control group. The study reports that participants who trusted the assistant less, and who rephrased prompts or adjusted settings, produced code with fewer vulnerabilities. The sample is small, and the paper itself says larger studies would need other recruiting methods.
What the two studies can and cannot support
| Question | Pearce et al. (2021) | Perry et al. (2023) |
|---|---|---|
| Method | Automated generation and scanning | Controlled experiment with people |
| Size | 89 scenarios, 1,689 programs | 47 participants, five tasks |
| Languages | Python, C, Verilog | Python, JavaScript, C |
| Tool tested | GitHub Copilot, mid-2021 | Assistant built on codex-davinci-002 |
| Main finding | About 40 percent vulnerable | Less secure code, higher confidence |
| Main limit | Scenarios were written by the authors and judged by CodeQL and manual review | Small sample and short tasks |
Neither study compares AI-assisted code with code from a human working alone in real projects over months, and neither tests the agent-style tools that now edit whole repositories. They show that early assistants could emit weak patterns and that users could trust them too readily.
Where existing practice fits
NIST's SP 800-218, the Secure Software Development Framework version 1.1, published in February 2022, does not mention AI coding assistants. Its practices still apply to any code source. Practice PW.7 asks organizations to decide whether human code review, automated code analysis, or both will be used to find vulnerabilities, and to record and triage the results. PW.8 covers testing executable code, and PW.4.4 covers verifying acquired components. Applying these to AI-generated code is an interpretation rather than NIST text. A separate NIST profile, SP 800-218A of July 2024, addresses secure development of AI models themselves, not use of AI to write other software.
FAQ
Is AI-written code less secure than human-written code?
The two studies here found weaknesses in assistant output and, in the user study, worse security among people using an assistant. They do not establish a rate for current tools or for all tasks.
Does code review still matter when an assistant writes the code?
The Perry study found that participants who revised prompts and trusted the assistant less produced fewer vulnerabilities. NIST's framework treats review, analysis, and testing as standard practice regardless of who or what produced the code.
Which weaknesses did the Copilot test target?
The scenarios were built around MITRE's 2021 CWE Top 25, which includes out-of-bounds writes, cross-site scripting, and OS command injection.
Are these results still current?
No evidence here covers 2026 tools. The studies date from 2021 to 2023, and the models they used have been superseded.
Related reading
- AI and web development
- How AI programs differ from traditional software
- AI development by country
- Growth in AI development
More in AI Tools and Industry.
Last verified: October 2026
Sources
- Pearce H., Ahmad B., Tan B., Dolan-Gavitt B., Karri R., Asleep at the Keyboard? Assessing the Security of GitHub Copilot's Code Contributions, arXiv 2108.09293, 2021
- Perry N., Srivastava M., Kumar D., Boneh D., Do Users Write More Insecure Code with AI Assistants?, ACM CCS 2023, arXiv 2211.03622
- Secure Software Development Framework (SSDF) Version 1.1, NIST SP 800-218, National Institute of Standards and Technology, February 2022
- 2021 CWE Top 25 Most Dangerous Software Weaknesses, MITRE, 2021
- Secure Software Development Practices for Generative AI and Dual-Use Foundation Models, NIST SP 800-218A, July 2024