AI Crypto Backtests: Why a 2021 Track Record Proves Little

Why a 2021 ranking of AI cryptocurrencies cannot be checked: backtest overfitting research, a labeled simulation, and what regulators did after 2021.

Last verified: October 20268 min read

A ranked list of "best AI cryptocurrencies" written in 2021 cannot be checked today, because any AI or trading-model claim behind such a list rests on backtests, and a backtest says little unless it reports how many variations were tried. Research by Bailey, Borwein, López de Prado, and Zhu shows that with only five years of data, trying more than 45 independent strategy variations is almost guaranteed to produce a good-looking result that has no skill behind it. This page names, rates, and ranks no token.

Key takeaways

  • A backtest is a simulation on past data, so a high backtest score measures how well a rule fit history, not how it will do later.
  • The more variations a researcher tries on the same history, the more likely the best one is a fluke; the count of trials is rarely published.
  • In one published test on random data, 8,800 rule combinations produced an annualized Sharpe ratio of 1.27, and the authors' overfitting probability for it was 55 percent.
  • An illustrative simulation below shows the best of 100 skill-free rules scoring about 1.27 in sample and about 0.02 on fresh data.
  • A list dated 2021 cannot say which products still exist or were found misleading afterward; since then, U.S. regulators have warned about, and charged firms over, false AI trading claims.

Why an old ranking is the wrong kind of evidence

"AI cryptocurrency" has no legal or technical definition, so a list under that label compares unlike things: payment tokens, tokens for access to a computing service, and tokens whose only link to AI is the name. Any ordering of them in 2021 would have leaned on price history or on a model claim, and both are historical records. A record of what already happened does not become a method for choosing what comes next just because the word "AI" is attached to it.

The useful question for a reader today is narrower: how would anyone test whether an AI-based rule for trading digital assets has skill? The answer comes from statistics, and it applies to every list that quotes past performance, not only to crypto lists from one year.

What a backtest measures, and where it breaks

A backtest is a historical simulation of an algorithmic strategy. Its common score is the Sharpe ratio, which divides a strategy's average return above a risk-free rate by the variation in its returns, usually expressed per year. Higher is better-looking, and a ratio near zero means returns no better than noise.

The authors of the 2014 article "Pseudo-Mathematics and Financial Charlatanism," published in the Notices of the American Mathematical Society, separate in-sample results, which come from the data used to design the strategy, from out-of-sample results, which come from data held back. They state that it is relatively simple to overfit a strategy to any financial series so that it performs well in sample. Their quantitative example: if only five years of data are available, no more than 45 independent configurations should be tried, or the search will almost surely produce an in-sample Sharpe ratio of 1 with an expected out-of-sample ratio of zero. For a two-year backtest, seven configurations are enough to reach that point.

A follow-up paper, "The Probability of Backtest Overfitting" (Journal of Computational Finance, 2017), builds a method to estimate that risk. Its worked example draws 1,000 daily prices from a random walk, about four years, and searches a grid of 8,800 trading-rule combinations. The best rule reached an annualized Sharpe ratio of 1.27, yet the authors estimate the probability of backtest overfitting at 55 percent, and about 53 percent of the out-of-sample ratios were negative. The data contained no pattern at all.

An illustrative simulation of the effect

The table below is an illustrative example produced for this page, not a result from the paper. Each rule's daily returns are random draws with a mean of zero, so no rule has skill. For each row, the best rule by in-sample Sharpe ratio on 1,000 days was kept, then scored on a second, fresh 1,000 days. Figures are averages over 300 repeated runs with a fixed random seed.

Rules triedBest in-sample SharpeSame rule on fresh dataRuns where the best rule exceeded 1.0 in sample
10.010.003 percent
100.760.0120 percent
1001.270.0294 percent
1,0001.650.02100 percent

The pattern matters more than the exact decimals. Choosing the winner from more candidates raises the in-sample score and leaves the fresh-data score at zero. A marketing page that shows only the winner, without the number of candidates, shows the left column and hides the right one.

Questions to put to any AI trading claim

The checklist turns the research into questions a reader can ask of a list, a bot description, or a fund page. Not every source will answer them, and an unanswered question is itself information.

QuestionWhy it matters
How many rule variations were tried before this one was chosen?Without the count, the overfitting risk cannot be estimated.
How long is the history, and how much was held back untouched?Short histories allow few independent trials before results turn to noise.
Were results produced live with real money, or only simulated?Simulations leave out trading costs, delays, and the effect of the strategy's own orders.
Is the date of the claim stated?A claim from 2021 describes a market that has since had its own events and rule changes.
Who is registered to offer the product, and where?Securities and derivatives regulators publish registration lookups; a pitch that avoids the question warrants more checking.

What changed since 2021

The claim type behind such lists has drawn regulator attention since 2021. On January 25, 2024, the U.S. Commodity Futures Trading Commission (CFTC) issued a customer advisory stating that AI technology cannot predict the future or sudden market changes, and that claims of high or guaranteed returns are red flags of fraud. It advised checking the background of any trader or signal provider before handing over money, and treating promotions from strangers online with suspicion.

On March 18, 2024, the U.S. Securities and Exchange Commission (SEC) announced settled charges against two investment advisers, Delphia (USA) Inc. of Toronto and Global Predictions Inc. of San Francisco, over false and misleading statements about their use of artificial intelligence. The firms agreed to USD 400,000 in combined civil penalties without admitting or denying the findings. In February 2025 the SEC created a Cyber and Emerging Technologies Unit with a remit that includes misconduct involving AI and blockchain. In its fiscal 2025 enforcement summary, released April 7, 2026, the SEC listed a case against the founder of an AI company, Nate, Inc., alleging false statements about its AI use in raising more than USD 42 million from investors. That case concerned company stock, not a token, and it is an allegation, not a finding.

None of these actions rates or endorses any digital asset, and none changes the statistics above. The 2021 list remains a dated marketing snapshot, and the 2014 and 2017 research remains the reason to ask how many trials sit behind a track record. No newer edition of that research was found as of October 2026.

What stays out of this page

No token is rated here, and no statement is made about which digital asset will rise or fall. The statistics above explain why a model tuned on history inherits that history's accidents, and why attaching AI to a strategy does not remove market uncertainty. The page also does not say that AI cannot be useful around crypto markets. Fraud screening and transaction monitoring are separate uses with their own evidence, covered in the related articles listed below.

This article is general information about statistics and risk. It is not investment, legal, or tax advice, and it does not recommend buying, holding, or selling anything. Decisions about money belong with the reader and, where appropriate, a registered professional, and complaints or doubts about a specific offer belong with the provincial, territorial, or state securities regulator.

FAQ

Can a 2021 list of AI crypto projects still be used for research?

It can serve as a dated snapshot of what was being marketed in 2021. It cannot show which projects survived, which changed structure, or whether any model claim held up, so each item would need a fresh check against current primary documents.

What is overfitting in plain words?

Overfitting means a model has learned the accidents of one data set instead of a pattern that repeats. It scores well on the data it was built from and poorly on new data.

Does a good out-of-sample test solve the problem?

Not fully. If the same held-back data is reused for many choices, it stops being fresh. The research above argues that the number of trials has to be reported alongside any test.

Is a high Sharpe ratio a sign of skill?

Only when it comes from a small, disclosed number of trials and holds on data the design never touched. The random-data example above reached 1.27 with no skill at all.

Have regulators acted on AI trading claims since 2021?

Yes. The CFTC issued a customer advisory on AI trading scams on January 25, 2024, and the SEC settled charges with two advisers over false AI statements on March 18, 2024. Neither action ranks or approves any product.

Last verified: October 2026

Sources

  • Pseudo-Mathematics and Financial Charlatanism: The Effects of Backtest Overfitting on Out-of-Sample Performance, Bailey, Borwein, López de Prado, Zhu, Notices of the American Mathematical Society 61(5), 2014
  • The Probability of Backtest Overfitting, Bailey, Borwein, López de Prado, Zhu, Journal of Computational Finance 20(4), 2017 (author's working-paper copy, February 2015)
  • CFTC Customer Advisory Cautions the Public to Beware of Artificial Intelligence Scams, Commodity Futures Trading Commission, January 25, 2024
  • SEC Charges Two Investment Advisers with Making False and Misleading Statements About Their Use of Artificial Intelligence, Press Release 2024-36, U.S. Securities and Exchange Commission, March 18, 2024
  • SEC Announces Enforcement Results for Fiscal Year 2025, Press Release 2026-34, U.S. Securities and Exchange Commission, April 7, 2026
  • Illustrative simulation: random-signal strategies on simulated random series, 1,000 in-sample and 1,000 fresh days, 300 runs, fixed seed, run for this page, October 2026

New to artificial intelligence?

Start with the basics, then follow the guides to tools, rules, and effects on society.

Start with AI Basics