Skip to content
Snippets Groups Projects
Commit f0ce49fb authored by Fabian Pedregosa's avatar Fabian Pedregosa
Browse files

Fix spelling errors in svm doc.

Add a bit of doc.

From: Fabian Pedregosa <fabian.pedregosa@inria.fr>

git-svn-id: https://scikit-learn.svn.sourceforge.net/svnroot/scikit-learn/trunk@564 22fbfee3-77ab-4535-9bad-27d1bd3bc7d8
parent f7b9686a
Branches
No related tags found
No related merge requests found
......@@ -34,6 +34,13 @@ non-linear and the transformed space high dimensional; thus though the
classifier is a hyperplane in the high-dimensional feature space, it
may be non-linear in the original input space.
The decision function in this case will be:
.. math:: sgn(\sum_{i=1}^l \alpha_i K(x_i, x) + \rho)
where :math:`\alpha, \rho` can be accessed through fields support_ and
rho_ of the classifier instance, respectevely.
If the kernel used is a Gaussian radial basis function, the
corresponding feature space is a Hilbert space of infinite
dimension. Maximum margin classifiers are well regularized, so the
......@@ -41,10 +48,9 @@ infinite dimension does not spoil the results. Available kernels are,
* linear :math:`(1 + <x, x'>)`
* polynomial :math:`(1 + <x, x'>)^d`
* radial :math:`exp(-\gamma |x-x'|^2)`
* radial basis :math:`exp(-\gamma |x-x'|^2)`
* sigmoid :math:`tanh(x_i x_j + c)`
The exclusive-OR is the simplest problem that cannot be solved using a
linear kernel. In this problem, point (x, y) belongs has target 1 if
and only if x > 0 XOR y > 0. In the following example, we create a
......@@ -80,7 +86,7 @@ data close to the model prediction.
Distribution estimation
=======================
One-class SVM is used for outlayer detection, that is, given a set of
One-class SVM is used for out-layer detection, that is, given a set of
samples, it will detect the soft boundary of that set.
.. linteralinclude:: ../../examples/plot_svm_oneclass.py
......@@ -91,71 +97,28 @@ samples, it will detect the soft boundary of that set.
Scaling
=======
Support Vector Machine algorithms are not scale-invariant, so it is
highly recommended to standarize the input vector X to have mean 0 and
variance 1. Note that the *same* scaling must be applied to the test
vector to obtain meaningful results.
TODO
.. Mathematical formulation (Model selection)
.. ========================
.. C-support vector classification (C-SVC)
.. ---------------------------------------
.. Given training vectors :math:`x_i \in \mathbb{R}^n , i=1, ..., l` in
.. two classes, and a vector :math:`y \in \mathbb{R}^l` such that
.. :math:`y_i \in {1, -1}`, C-SVC solves the following primal problem:
.. .. math:: \min_{w, b, \xi} {1 \over 2} w^T w + C \sum_{i=1}^l \xi_i
.. .. math:: \textrm{subject to}\ y_i (w^T \phi(x_i) + b) \geq 1 - \xi_i
.. .. math:: \xi_i >= 0, i=1, .., l
.. Here training vectors :math:`x_i` are mapped into a higher (maybe
.. infinite) dimensional space by the function :math:`phi`. The decision
.. function is
.. .. math:: sgn(\sum_{i=0}^l y_i \alpha_i K(x_i, x) + b)
.. This is implemented in class SVC
See `The CookBook
<https://sourceforge.net/apps/trac/scikit-learn/wiki/CookBook>` for
some examples on scaling.
.. Nu-Support Vector Classification
.. --------------------------------
.. The nu-Support Vector Classification uses a new parameter :math:`\nu`
.. which controls the number of support vectors and trainign errors. The
.. parameter :math:`nu \in (0, 1]` is an upper bound on the fraction of
.. training errors and a lower bound of the fraction of support vectors.
.. Given training vectors :math:`x_i \in \mathbb{R}^n , i=1, ..., l` in
.. two classes, and a vector :math:`y \in \mathbb{R}^l` such that
.. :math:`y_i \in {1, -1}`, C-SVC solves the following primal problem:
.. .. math:: \min_{w, b, \xi} {1 \over 2} w^T w - \nu \rho + {1 \over 2} \sum_{i=1}^l \xi_i
.. \textrm{subject to}\ y_i (w^T \phi(x_i) + b) \geq \rho - \xi_i
.. \xi_i \geq 0, i=1, .., l, \rho \geq 0
.. The decision function is:
.. .. math:: sgn(\sum_{i=1}^l y_i \alpha_i K(x_i, x) + b
.. This is implemented in SVC(impl='nu-svc')
Low-level implementation
========================
Internally, we use libsvm[1] to handle all computations. Libsvm is binded
through some wrappers written in C and Cython.
Implementation details
======================
Internally, we use libsvm[1] to handle all computations. Libsvm is wrapped
using C and Cython.
.. [1] http://www.csie.ntu.edu.tw/~cjlin/libsvm/
References
==========
For a description of the implementation and details of the algorithms
used, please refer to
http://www.csie.ntu.edu.tw/~cjlin/papers/libsvm.pdf
http://en.wikipedia.org/wiki/Support_vector_machine
- http://www.csie.ntu.edu.tw/~cjlin/papers/libsvm.pdf
- http://en.wikipedia.org/wiki/Support_vector_machine
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment