From: Rahul Kavi Date: Fri, 4 Oct 2013 12:36:00 +0000 (-0400) Subject: updated documentation to reflect newer changes to LogisticRegression class X-Git-Tag: submit/tizen_ivi/20141117.190038~2^2~171^2~28 X-Git-Url: http://review.tizen.org/git/?a=commitdiff_plain;h=e4a90c19cc7187058572d662d27aa53a6f4f996b;p=profile%2Fivi%2Fopencv.git updated documentation to reflect newer changes to LogisticRegression class --- diff --git a/modules/ml/doc/logistic_regression.rst b/modules/ml/doc/logistic_regression.rst index 06f6956..09647e4 100644 --- a/modules/ml/doc/logistic_regression.rst +++ b/modules/ml/doc/logistic_regression.rst @@ -7,7 +7,7 @@ ML implements logistic regression, which is a probabilistic classification techn Like SVM, Logistic Regression can be extended to work on multi-class classification problems like digit recognition (i.e. recognizing digitis like 0,1 2, 3,... from the given images). This version of Logistic Regression supports both binary and multi-class classifications (for multi-class it creates a multiple 2-class classifiers). In order to train the logistic regression classifier, Batch Gradient Descent and Mini-Batch Gradient Descent algorithms are used (see [BatchDesWiki]_). -Logistic Regression is a discriminative classifier (see [LogRegTomMitch]_ for more details). Logistic Regression is implemented as a C++ class in ``CvLR``. +Logistic Regression is a discriminative classifier (see [LogRegTomMitch]_ for more details). Logistic Regression is implemented as a C++ class in ``LogisticRegression``. In Logistic Regression, we try to optimize the training paramater @@ -28,26 +28,26 @@ or class 0 if . In Logistic Regression, choosing the right parameters is of utmost importance for reducing the training error and ensuring high training accuracy. -``CvLR_TrainParams`` is the structure that defines parameters that are required to train a Logistic Regression classifier. -The learning rate is determined by ``CvLR_TrainParams.alpha``. It determines how faster we approach the solution. -It is a positive real number. Optimization algorithms like Batch Gradient Descent and Mini-Batch Gradient Descent are supported in ``CvLR``. +``LogisticRegressionParams`` is the structure that defines parameters that are required to train a Logistic Regression classifier. +The learning rate is determined by ``LogisticRegressionParams.alpha``. It determines how faster we approach the solution. +It is a positive real number. Optimization algorithms like Batch Gradient Descent and Mini-Batch Gradient Descent are supported in ``LogisticRegression``. It is important that we mention the number of iterations these optimization algorithms have to run. -The number of iterations are mentioned by ``CvLR_TrainParams.num_iters``. +The number of iterations are mentioned by ``LogisticRegressionParams.num_iters``. The number of iterations can be thought as number of steps taken and learning rate specifies if it is a long step or a short step. These two parameters define how fast we arrive at a possible solution. -In order to compensate for overfitting regularization is performed, which can be enabled by setting ``CvLR_TrainParams.regularized`` to a positive integer (greater than zero). -One can specify what kind of regularization has to be performed by setting ``CvLR_TrainParams.norm`` to ``CvLR::REG_L1`` or ``CvLR::REG_L2`` values. -``CvLR`` provides a choice of 2 training methods with Batch Gradient Descent or the Mini-Batch Gradient Descent. To specify this, set ``CvLR_TrainParams.train_method`` to either ``CvLR::BATCH`` or ``CvLR::MINI_BATCH``. -If ``CvLR_TrainParams`` is set to ``CvLR::MINI_BATCH``, the size of the mini batch has to be to a postive integer using ``CvLR_TrainParams.minibatchsize``. +In order to compensate for overfitting regularization is performed, which can be enabled by setting ``LogisticRegressionParams.regularized`` to a positive integer (greater than zero). +One can specify what kind of regularization has to be performed by setting ``LogisticRegressionParams.norm`` to ``LogisticRegression::REG_L1`` or ``LogisticRegression::REG_L2`` values. +``LogisticRegression`` provides a choice of 2 training methods with Batch Gradient Descent or the Mini-Batch Gradient Descent. To specify this, set ``LogisticRegressionParams.train_method`` to either ``LogisticRegression::BATCH`` or ``LogisticRegression::MINI_BATCH``. +If ``LogisticRegressionParams`` is set to ``LogisticRegression::MINI_BATCH``, the size of the mini batch has to be to a postive integer using ``LogisticRegressionParams.mini_batch_size``. A sample set of training parameters for the Logistic Regression classifier can be initialized as follows: :: - CvLR_TrainParams params; + LogisticRegressionParams params; params.alpha = 0.5; params.num_iters = 10000; - params.norm = CvLR::REG_L2; + params.norm = LogisticRegression::REG_L2; params.regularized = 1; - params.train_method = CvLR::MINI_BATCH; - params.minibatchsize = 10; + params.train_method = LogisticRegression::MINI_BATCH; + params.mini_batch_size = 10; .. [LogRegWiki] http://en.wikipedia.org/wiki/Logistic_regression. Wikipedia article about the Logistic Regression algorithm. @@ -56,9 +56,9 @@ A sample set of training parameters for the Logistic Regression classifier can b .. [LogRegTomMitch] http://www.cs.cmu.edu/~tom/NewChapters.html. "Generative and Discriminative Classifiers: Naive Bayes and Logistic Regression" in Machine Learning, Tom Mitchell. .. [BatchDesWiki] http://en.wikipedia.org/wiki/Gradient_descent_optimization. Wikipedia article about Gradient Descent based optimization. -CvLR_TrainParams ----------------- -.. ocv:struct:: CvLR_TrainParams +LogisticRegressionParams +------------------------ +.. ocv:struct:: LogisticRegressionParams Parameters of the Logistic Regression training algorithm. You can initialize the structure using a constructor or declaring the variable and initializing the the individual parameters. @@ -74,7 +74,7 @@ CvLR_TrainParams .. ocv:member:: int norm - The type of normalization applied. It takes value ``CvLR::L1`` or ``CvLR::L2``. + The type of normalization applied. It takes value ``LogisticRegression::L1`` or ``LogisticRegression::L2``. .. ocv:member:: int regularized @@ -82,89 +82,95 @@ CvLR_TrainParams .. ocv:member:: int train_method - The kind of training method used to train the classifier. It should be set to either ``CvLR::BATCH`` or ``CvLR::MINI_BATCH``. + The kind of training method used to train the classifier. It should be set to either ``LogisticRegression::BATCH`` or ``LogisticRegression::MINI_BATCH``. - .. ocv:member:: int minibatchsize + .. ocv:member:: int mini_batch_size - If the training method is set to CvLR::MINI_BATCH, it has to be set to positive integer. It can range from 1 to number of training samples. + If the training method is set to LogisticRegression::MINI_BATCH, it has to be set to positive integer. It can range from 1 to number of training samples. -CvLR_TrainParams::CvLR_TrainParams ----------------------------------- +LogisticRegressionParams::LogisticRegressionParams +-------------------------------------------------- The constructors. -.. ocv:function:: CvLR_TrainParams::CvLR_TrainParams() +.. ocv:function:: LogisticRegressionParams::LogisticRegressionParams() -.. ocv:function:: CvLR_TrainParams::CvLR_TrainParams(double alpha, int num_iters, int norm, int regularized, int train_method, int minbatchsize) +.. ocv:function:: LogisticRegressionParams::LogisticRegressionParams(double alpha, int num_iters, int norm, int regularized, int train_method, int minbatchsize) :param alpha: Specifies the learning rate. :param num_iters: Specifies the number of iterations. - :param norm: Specifies the kind of regularization to be applied. ``CvLR::REG_L1`` or ``CvLR::REG_L2``. To use this, set ``CvLR_TrainParams.regularized`` to a integer greater than zero. + :param norm: Specifies the kind of regularization to be applied. ``LogisticRegression::REG_L1`` or ``LogisticRegression::REG_L2``. To use this, set ``LogisticRegressionParams.regularized`` to a integer greater than zero. :param: regularized: To enable or disable regularization. Set to positive integer (greater than zero) to enable and to 0 to disable. - :param: train_method: Specifies the kind of training method used. It should be set to either ``CvLR::BATCH`` or ``CvLR::MINI_BATCH``. If using ``CvLR::MINI_BATCH``, set ``CvLR_TrainParams.minibatchsize`` to a positive integer. + :param: train_method: Specifies the kind of training method used. It should be set to either ``LogisticRegression::BATCH`` or ``LogisticRegression::MINI_BATCH``. If using ``LogisticRegression::MINI_BATCH``, set ``LogisticRegressionParams.mini_batch_size`` to a positive integer. - :param: minibatchsize: Specifies the number of training samples taken in each step of Mini-Batch Gradient Descent. + :param: mini_batch_size: Specifies the number of training samples taken in each step of Mini-Batch Gradient Descent. By initializing this structure, one can set all the parameters required for Logistic Regression classifier. -CvLR ----- -.. ocv:class:: CvLR : public CvStatModel +LogisticRegression +------------------ +.. ocv:class:: LogisticRegression : public CvStatModel Implements Logistic Regression classifier. -CvLR::CvLR ----------- +LogisticRegression::LogisticRegression +-------------------------------------- The constructors. -.. ocv:function:: CvLR::CvLR() +.. ocv:function:: LogisticRegression::LogisticRegression() -.. ocv:function:: CvLR::CvLR(const cv::Mat& data, const cv::Mat& labels, const CvLR_TrainParams& params) +.. ocv:function:: LogisticRegression::LogisticRegression(cv::InputArray data_ip, cv::InputArray labels_ip, const LogisticRegressionParams& params); :param data: The data variable of type ``CV_32F``. Each data instance has to be arranged per across different rows. - :param labels: The data variable of type ``CV_32F``. Each label instance has to be arranged across differnet rows. + :param labels_ip: The data variable of type ``CV_32F``. Each label instance has to be arranged across different rows. - :param params: The training parameters for the classifier of type ``CVLR_TrainParams``. + :param params: The training parameters for the classifier of type ``LogisticRegressionParams``. The constructor with parameters allows to create a Logistic Regression object intialized with given data and trains it. -CvLR::train ------------ +LogisticRegression::train +------------------------- Trains the Logistic Regression classifier and returns true if successful. -.. ocv:function:: bool CvLR::train(const cv::Mat& data, const cv::Mat& labels) +.. ocv:function:: bool LogisticRegression::train(cv::InputArray data_ip, cv::InputArray label_ip) - :param data: The data variable of type ``CV_32F``. Each data instance has to be arranged per across different rows. + :param data_ip: An InputArray variable of type ``CV_32F``. Each data instance has to be arranged per across different rows. - :param labels: The data variable of type ``CV_32F``. Each label instance has to be arranged across differnet rows. + :param labels_ip: An InputArray variable of type ``CV_32F``. Each label instance has to be arranged across differnet rows. -CvLR::predict -------------- +LogisticRegression::predict +--------------------------- Predicts responses for input samples and returns a float type. -.. ocv:function:: float CvLR::predict(const Mat& data) - - :param data: The data variable should be a row matrix and of type ``CV_32F``. - -.. ocv:function:: float CvLR::predict( const Mat& data, Mat& predicted_labels ) +.. ocv:function:: void LogisticRegression::predict( cv::InputArray data, cv::OutputArray predicted_labels ) const; :param data: The input data for the prediction algorithm. The ``data`` variable should be of type ``CV_32F``. :param predicted_labels: Predicted labels as a column matrix and of type ``CV_32S``. -The function ``CvLR::predict(const Mat& data)`` returns the label of single data variable. It should be used if data contains only 1 row. - -CvLR::get_learnt_mat() ----------------------- +LogisticRegression::get_learnt_thetas +--------------------------------------- This function returns the trained paramters arranged across rows. For a two class classifcation problem, it returns a row matrix. -.. ocv:function:: cv::Mat CvLR::get_learnt_mat() +.. ocv:function:: cv::Mat LogisticRegression::get_learnt_thetas() It returns learnt paramters of the Logistic Regression as a matrix of type ``CV_32F``. + +LogisticRegression::save +------------------------ +This function saves the trained LogisticRegression clasifier to disk. + +.. ocv:function:: void LogisticRegression::save(string filepath) const + +LogisticRegression::load +------------------------ +This function loads the trained LogisticRegression clasifier from disk. + +.. ocv:function:: void LogisticRegression::load(const string filepath)