A fair accounting of AI's pros and cons starts with one named system in one setting, because the same tool can help one group and harm another. The U.S. National Institute of Standards and Technology (NIST) AI Risk Management Framework asks organizations to document a system's potential benefits and, separately, its potential costs, including non-monetary ones. Canada's Algorithmic Impact Assessment turns a similar exercise into a score with four impact levels for federal automated decision systems.
Key takeaways
- NIST released AI RMF 1.0 on January 26, 2023 as a voluntary framework, and NIST's page notes that a revision is underway, linked to the White House AI Action Plan.
- The framework's mapping guidance treats benefits and costs as two separate items to be written down and checked, not one verdict.
- Canada's Algorithmic Impact Assessment poses 65 risk questions and 41 mitigation questions, then places a system at one of four impact levels.
- Neither tool says whether AI is good or bad in general. Both are ways to examine one deployment.
Why general pros-and-cons lists fall short
A list that says AI saves time and also spreads misinformation can be accurate in general and still say little about any particular deployment. NIST's guidance on mapping context makes a related point. It notes that highly accurate and optimized systems can still cause harm, and that tools deployed broadly may be reused or misused in ways their builders did not plan. It also asks organizations to list negative impacts to individuals, groups, communities, organizations, and society, not only to the customer who buys the product.
The practical consequence is that a useful evaluation names a purpose, a setting, and the people affected. "AI in hiring" becomes "a resume-ranking tool used by one employer for one job family," and the evidence can then be requested for that case.
What NIST asks organizations to write down
The mapping part of the NIST Playbook includes a short run of items on benefits, costs, scope, operators, and oversight. Paraphrased, they ask for the following.
- Benefits: the potential benefits of the intended functionality are examined and documented, including utility and positive impacts beyond benchmark scores, with input from end users and potentially affected communities.
- Costs: the costs that follow from expected or realized errors are examined and documented, including non-monetary costs. The guidance says negative impacts can run "from minor annoyance to serious injury, financial losses, or regulatory enforcement actions."
- Scope: the targeted application is specified, and the Playbook notes that narrower scopes make risks easier to map, measure, and manage.
- Operators: processes exist for the proficiency of the people who run and use the system, drawing on their domain expertise instead of replacing it.
- Oversight: human oversight procedures are defined, assessed, and documented, and for high-stakes systems they are evaluated before deployment.
How Canada's assessment turns the question into a score
The Algorithmic Impact Assessment (AIA) appears on the Government of Canada website as a required risk-assessment questionnaire backing the Treasury Board's Directive on Automated Decision-Making. It applies to departments and agencies covered by the directive, not to private companies. The questionnaire contains 65 risk questions in six areas and 41 mitigation questions in two areas, and the result is a percentage score mapped to four levels, numbered with Roman numerals: the lowest means little to no impact (0 to 25 percent), the second moderate (26 to 50), the third high (51 to 75), and the fourth very high (76 to 100).
Mitigation affects the score in a limited way. When the mitigation score reaches at least 80 percent of its maximum, 15 percent is deducted from the raw score, but mitigation measures do not themselves change the level; only a new assessment with updated project information can. The assessment is meant to be completed at the start of design and repeated before the system enters production, and the directive attaches stronger requirements, such as more demanding peer review and more human involvement in decisions, to higher levels.
A worksheet for one system
The table applies the NIST items to a hypothetical example, a city office that uses software to sort incoming service requests. It is an illustrative example. The entries are placeholders for what an evaluator would ask, not findings about any real product.
| Item | Question to answer | Evidence to request |
|---|---|---|
| Benefit | Which request types get routed faster, and for whom? | Processing times before and after, split by neighborhood and language |
| Cost | What happens to a request that is mislabeled? | Error rates by request type, and a record of appeals or re-routes |
| Scope | Does the tool only sort, or does it also decide priority? | A written statement of what staff may delegate to it |
| Operators | Do clerks know when to override it? | Training records and override instructions |
| Oversight | Who reviews errors, and how often? | Review schedule and a named owner |
What these tools cannot do
The NIST framework is voluntary, and its own pages say it is being revised, so the item wording may change. The Canadian assessment covers federal government systems, so a school board, a hospital, or a business falls outside it unless another rule applies. Both tools structure a judgment, and a completed worksheet can still rest on weak evidence. The strength of the evidence listed in the last column is what separates a documented benefit from an assumption.
This article gives general information about two public frameworks. It is not legal advice, and an organization deciding whether a requirement applies to it should consult the primary documents or a qualified professional.
FAQ
Does the NIST framework require anyone to do this?
No. NIST describes the AI RMF as intended for voluntary use. Other laws or contracts may require parts of it for particular organizations.
What does the top level mean in the Canadian assessment?
It is Level IV, a score of 76 to 100 percent, labeled very high impact. Higher levels carry stronger required measures under the directive.
Can a high accuracy score settle the benefit question?
No. NIST's guidance asks for benefits beyond benchmark performance and notes that highly accurate systems can cause harm, so the effect on the people involved still needs documenting.
Is there a generative AI version of the framework?
NIST released a Generative Artificial Intelligence Profile, NIST-AI-600-1, on July 26, 2024. It helps organizations identify risks specific to generative AI.
Related reading
- The AI debate across fields
- Benefits of AI that studies support
- AI's impact on modern society
- AI and human intelligence compared
More in AI Basics.
Last verified: October 2026
Sources
- NIST AI RMF Playbook, Map function (MAP 1.1, MAP 3.1 to MAP 3.5), National Institute of Standards and Technology
- AI Risk Management Framework, National Institute of Standards and Technology, released January 26, 2023
- Algorithmic Impact Assessment tool, Treasury Board of Canada Secretariat, page modified September 15, 2026