A support vector machine (SVM) is a supervised learning method that classifies data by finding the boundary with the widest margin between two groups, and it was introduced in its modern form by Corinna Cortes and Vladimir Vapnik in a 1995 paper in the journal Machine Learning. SVMs are still taught in machine learning courses and remain in the scikit-learn library, but the claim that they are "the future" of the field does not match the documentation, which lists real limits on data size and probability output.
Key takeaways
- An SVM picks the separating boundary with the largest gap to the nearest training points, and only those nearest points, the support vectors, define it.
- Cortes and Vapnik's 1995 paper extended the method from data that can be separated without errors to data that cannot, and compared it with classical algorithms on a character recognition benchmark.
- Kernel functions let an SVM draw curved boundaries. scikit-learn lists linear, polynomial, radial basis function, and sigmoid kernels.
- SVMs need scaled inputs, give no direct probabilities, and slow down quickly as the number of training examples grows.
The idea: the widest margin
Suppose a teacher wants to predict pass or fail from two numbers per student: hours of study and a practice quiz percentage. Plot each student as a point, and many straight lines can separate the passes from the fails. An SVM chooses the one whose distance to the closest point on each side is as large as possible. That distance is the margin. The closest points are the support vectors, and the rest of the training data could be deleted without moving the boundary. The scikit-learn guide cites this as a practical benefit: the decision function uses only a subset of training points, so the method is memory efficient.
The 1995 paper by Cortes and Vapnik, titled "Support-vector networks," appeared in Machine Learning, volume 20, pages 273 to 297. It describes mapping inputs into a very high-dimensional space, building a linear decision surface there, and handling training data that cannot be separated perfectly. The authors tested it against several classical methods on an optical character recognition benchmark.
A worked example with invented numbers
The table is an illustrative example with made-up students, not real data.
| Student | Hours studied | Quiz score (percent) | Outcome |
|---|---|---|---|
| A | 2 | 40 | Fail |
| B | 3 | 50 | Fail |
| C | 4 | 55 | Fail |
| D | 6 | 80 | Pass |
| E | 7 | 85 | Pass |
| F | 8 | 90 | Pass |
In this layout the two points closest across the gap are student C (4 hours, 55 percent) and student D (6 hours, 80 percent). A linear SVM would place its boundary midway between them, and those two students would be the support vectors. The example also shows why scaling matters. The gap between C and D is two hours but 25 quiz points, so in raw units the quiz score dominates the distance and hours barely count. The scikit-learn guide states that SVM algorithms are not scale invariant and recommends scaling each attribute, using the same scaling on test data.
Kernels: curved boundaries without new features
A straight line fails when classes are mixed in a ring or a spiral. A kernel function lets the SVM behave as if the data had been lifted into a space with many more dimensions, where a flat boundary can work, without computing that space explicitly. The scikit-learn user guide lists four built-in options: linear, polynomial, radial basis function (rbf), and sigmoid, and allows custom kernels. Choosing a kernel and its settings is part of the modeling work, and the guide warns that when the number of features greatly exceeds the number of samples, the kernel choice and the regularization term become critical to avoid overfitting.
Limits that the documentation states
The scikit-learn support vector machine guide is direct about costs. Fit time for the standard classifier scales between the square and the cube of the number of samples, depending on caching, and compute and storage needs increase rapidly with the number of training vectors. A linear variant can scale almost linearly to millions of samples. SVMs do not directly provide probability estimates; these are calculated with an expensive five-fold cross-validation, and the guide warns they may be inconsistent with the raw scores. For a school dashboard that must say "70 percent chance," that matters.
These points explain the position of SVMs today. They suit moderately sized datasets of measured features, and they are a clear way to teach margins and kernels. For very large datasets and for images, audio, and text, other methods dominate current practice.
Questions for a student project using an SVM
- Were the inputs scaled the same way for training and test data?
- Was the kernel chosen on a validation set, not on the final test set?
- Is the class balance reported, so that accuracy is not inflated by a majority class?
- Is a probability needed, and if so, how was it produced?
- Was a simple baseline such as logistic regression scored on the same split?
This article is general educational information about a method. It does not recommend a tool or a vendor, and it is not guidance for decisions about individual students.
FAQ
Is a support vector machine a type of neural network?
No. It is a separate method with its own optimization problem. The 1995 paper used the name "support-vector network," which can cause confusion. The training problem is a different one from fitting a layered network.
What is a support vector?
A training example that lies closest to the decision boundary. Only these points determine where the boundary sits.
Why does an SVM need scaled data?
The method works from distances between points, so a feature measured in large units can swamp the others. The scikit-learn guide recommends scaling each attribute.
Can an SVM report how confident it is?
Only indirectly. The scikit-learn guide says probabilities come from an added cross-validation step, and that it is often simpler to use the raw confidence scores when probabilities are not required.
Related reading
- The difference between AI and machine learning
- Non-neural alternatives in machine learning
- Limited memory AI and its examples
- Training a machine learning model
More in AI in education.
Last verified: October 2026
Sources
- Support-vector networks, Cortes C and Vapnik V, Machine Learning 20(3), 273-297, 1995
- Support Vector Machines, user guide, scikit-learn documentation, 2026