.
In Logistic Regression, choosing the right parameters is of utmost importance for reducing the training error and ensuring high training accuracy.
-``LogisticRegressionParams`` is the structure that defines parameters that are required to train a Logistic Regression classifier.
-The learning rate is determined by ``LogisticRegressionParams.alpha``. It determines how faster we approach the solution.
+``LogisticRegression::Params`` is the structure that defines parameters that are required to train a Logistic Regression classifier.
+The learning rate is determined by ``LogisticRegression::Params.alpha``. It determines how faster we approach the solution.
It is a positive real number. Optimization algorithms like Batch Gradient Descent and Mini-Batch Gradient Descent are supported in ``LogisticRegression``.
It is important that we mention the number of iterations these optimization algorithms have to run.
-The number of iterations are mentioned by ``LogisticRegressionParams.num_iters``.
+The number of iterations are mentioned by ``LogisticRegression::Params.num_iters``.
The number of iterations can be thought as number of steps taken and learning rate specifies if it is a long step or a short step. These two parameters define how fast we arrive at a possible solution.
-In order to compensate for overfitting regularization is performed, which can be enabled by setting ``LogisticRegressionParams.regularized`` to a positive integer (greater than zero).
-One can specify what kind of regularization has to be performed by setting ``LogisticRegressionParams.norm`` to ``LogisticRegression::REG_L1`` or ``LogisticRegression::REG_L2`` values.
-``LogisticRegression`` provides a choice of 2 training methods with Batch Gradient Descent or the Mini-Batch Gradient Descent. To specify this, set ``LogisticRegressionParams.train_method`` to either ``LogisticRegression::BATCH`` or ``LogisticRegression::MINI_BATCH``.
-If ``LogisticRegressionParams`` is set to ``LogisticRegression::MINI_BATCH``, the size of the mini batch has to be to a postive integer using ``LogisticRegressionParams.mini_batch_size``.
+In order to compensate for overfitting regularization is performed, which can be enabled by setting ``LogisticRegression::Params.regularized`` to a positive integer (greater than zero).
+One can specify what kind of regularization has to be performed by setting ``LogisticRegression::Params.norm`` to ``LogisticRegression::REG_L1`` or ``LogisticRegression::REG_L2`` values.
+``LogisticRegression`` provides a choice of 2 training methods with Batch Gradient Descent or the Mini-Batch Gradient Descent. To specify this, set ``LogisticRegression::Params.train_method`` to either ``LogisticRegression::BATCH`` or ``LogisticRegression::MINI_BATCH``.
+If ``LogisticRegression::Params`` is set to ``LogisticRegression::MINI_BATCH``, the size of the mini batch has to be to a postive integer using ``LogisticRegression::Params.mini_batch_size``.
A sample set of training parameters for the Logistic Regression classifier can be initialized as follows:
+
::
- LogisticRegressionParams params;
+
+ LogisticRegression::Params params;
params.alpha = 0.5;
params.num_iters = 10000;
params.norm = LogisticRegression::REG_L2;
params.train_method = LogisticRegression::MINI_BATCH;
params.mini_batch_size = 10;
+**References:**
+
.. [LogRegWiki] http://en.wikipedia.org/wiki/Logistic_regression. Wikipedia article about the Logistic Regression algorithm.
.. [RenMalik2003] Learning a Classification Model for Segmentation. Proc. CVPR, Nice, France (2003).
.. [LogRegTomMitch] http://www.cs.cmu.edu/~tom/NewChapters.html. "Generative and Discriminative Classifiers: Naive Bayes and Logistic Regression" in Machine Learning, Tom Mitchell.
+
.. [BatchDesWiki] http://en.wikipedia.org/wiki/Gradient_descent_optimization. Wikipedia article about Gradient Descent based optimization.
-LogisticRegressionParams
-------------------------
-.. ocv:struct:: LogisticRegressionParams
+LogisticRegression::Params
+--------------------------
+.. ocv:struct:: LogisticRegression::Params
Parameters of the Logistic Regression training algorithm. You can initialize the structure using a constructor or declaring the variable and initializing the the individual parameters.
Sets termination criteria for training algorithm.
-LogisticRegressionParams::LogisticRegressionParams
---------------------------------------------------
-The constructors.
+LogisticRegression::Params::Params
+----------------------------------
+The constructors
-.. ocv:function:: LogisticRegressionParams::LogisticRegressionParams()
-
-.. ocv:function:: LogisticRegressionParams::LogisticRegressionParams(double learning_rate, int iters, int train_method, int normlization, int reg, int mini_batch_size)
+.. ocv:function:: LogisticRegression::Params::Params(double learning_rate = 0.001, int iters = 1000, int method = LogisticRegression::BATCH, int normlization = LogisticRegression::REG_L2, int reg = 1, int batch_size = 1)
:param learning_rate: Specifies the learning rate.
:param iters: Specifies the number of iterations.
- :param: train_method: Specifies the kind of training method used. It should be set to either ``LogisticRegression::BATCH`` or ``LogisticRegression::MINI_BATCH``. If using ``LogisticRegression::MINI_BATCH``, set ``LogisticRegressionParams.mini_batch_size`` to a positive integer.
-
- :param normalization: Specifies the kind of regularization to be applied. ``LogisticRegression::REG_L1`` or ``LogisticRegression::REG_L2`` (L1 norm or L2 norm). To use this, set ``LogisticRegressionParams.regularized`` to a integer greater than zero.
+ :param train_method: Specifies the kind of training method used. It should be set to either ``LogisticRegression::BATCH`` or ``LogisticRegression::MINI_BATCH``. If using ``LogisticRegression::MINI_BATCH``, set ``LogisticRegression::Params.mini_batch_size`` to a positive integer.
- :param: reg: To enable or disable regularization. Set to positive integer (greater than zero) to enable and to 0 to disable.
+ :param normalization: Specifies the kind of regularization to be applied. ``LogisticRegression::REG_L1`` or ``LogisticRegression::REG_L2`` (L1 norm or L2 norm). To use this, set ``LogisticRegression::Params.regularized`` to a integer greater than zero.
- :param: mini_batch_size: Specifies the number of training samples taken in each step of Mini-Batch Gradient Descent. Will only be used if using ``LogisticRegression::MINI_BATCH`` training algorithm. It has to take values less than the total number of training samples.
+ :param reg: To enable or disable regularization. Set to positive integer (greater than zero) to enable and to 0 to disable.
-The full constructor initializes corresponding members. The default constructor creates an object with dummy parameters.
-
-::
-
- LogisticRegressionParams::LogisticRegressionParams()
- {
- term_crit = cv::TermCriteria(TermCriteria::COUNT + TermCriteria::EPS, 1000, 0.001);
- alpha = 0.001;
- num_iters = 1000;
- norm = LogisticRegression::REG_L2;
- regularized = 1;
- train_method = LogisticRegression::BATCH;
- mini_batch_size = 1;
- }
+ :param mini_batch_size: Specifies the number of training samples taken in each step of Mini-Batch Gradient Descent. Will only be used if using ``LogisticRegression::MINI_BATCH`` training algorithm. It has to take values less than the total number of training samples.
By initializing this structure, one can set all the parameters required for Logistic Regression classifier.
LogisticRegression
------------------
-.. ocv:class:: LogisticRegression
-
-Implements Logistic Regression classifier.
-
-LogisticRegression::LogisticRegression
---------------------------------------
-The constructors.
-.. ocv:function:: LogisticRegression::LogisticRegression( const LogisticRegressionParams& params = LogisticRegressionParams())
+.. ocv:class:: LogisticRegression : public StatModel
- :param params: The training parameters for the classifier of type ``LogisticRegressionParams``.
-
-.. ocv:function:: LogisticRegression::LogisticRegression(cv::InputArray data_ip, cv::InputArray labels_ip, const LogisticRegressionParams& params)
+Implements Logistic Regression classifier.
- :param data: The data variable of type ``CV_32F``. Each data instance has to be arranged per across different rows.
+LogisticRegression::create
+--------------------------
+Creates empty model.
- :param labels_ip: The data variable of type ``CV_32F``. Each label instance has to be arranged across different rows.
+.. ocv:function:: Ptr<LogisticRegression> LogisticRegression::create( const Params& params = Params() )
- :param params: The training parameters for the classifier of type ``LogisticRegressionParams``.
+ :param params: The training parameters for the classifier of type ``LogisticRegression::Params``.
-The constructor with parameters allows to create a Logistic Regression object intialized with given data and trains it.
+Creates Logistic Regression model with parameters given.
LogisticRegression::train
-------------------------
Trains the Logistic Regression classifier and returns true if successful.
-.. ocv:function:: bool LogisticRegression::train(cv::InputArray data_ip, cv::InputArray label_ip)
+.. ocv:function:: bool LogisticRegression::train( const Ptr<TrainData>& trainData, int flags=0 )
- :param data_ip: An InputArray variable of type ``CV_32F``. Each data instance has to be arranged per across different rows.
-
- :param labels_ip: An InputArray variable of type ``CV_32F``. Each label instance has to be arranged across differnet rows.
+ :param trainData: Instance of ml::TrainData class holding learning data.
+ :param flags: Not used.
LogisticRegression::predict
---------------------------
Predicts responses for input samples and returns a float type.
-.. ocv:function:: void LogisticRegression::predict( cv::InputArray data, cv::OutputArray predicted_labels ) const
+.. ocv:function:: void LogisticRegression::predict( InputArray samples, OutputArray results=noArray(), int flags=0 ) const
+
+ :param samples: The input data for the prediction algorithm. Matrix [m x n], where each row contains variables (features) of one object being classified. Should have data type ``CV_32F``.
- :param data: The input data for the prediction algorithm. The ``data`` variable should be of type ``CV_32F``.
+ :param results: Predicted labels as a column matrix of type ``CV_32S``.
- :param predicted_labels: Predicted labels as a column matrix and of type ``CV_32S``.
+ :param flags: Not used.
LogisticRegression::get_learnt_thetas
-------------------------------------
This function returns the trained paramters arranged across rows. For a two class classifcation problem, it returns a row matrix.
-.. ocv:function:: const cv::Mat LogisticRegression::get_learnt_thetas() const
+.. ocv:function:: Mat LogisticRegression::get_learnt_thetas() const
It returns learnt paramters of the Logistic Regression as a matrix of type ``CV_32F``.
regularized = reg;
train_method = method;
mini_batch_size = batch_size;
- term_crit = cv::TermCriteria(TermCriteria::COUNT + TermCriteria::EPS, num_iters, alpha);
+ term_crit = TermCriteria(TermCriteria::COUNT + TermCriteria::EPS, num_iters, alpha);
}
class LogisticRegressionImpl : public LogisticRegression
virtual void clear();
virtual void write(FileStorage& fs) const;
virtual void read(const FileNode& fn);
- virtual cv::Mat get_learnt_thetas() const;
+ virtual Mat get_learnt_thetas() const;
virtual int getVarCount() const { return learnt_thetas.cols; }
virtual bool isTrained() const { return !learnt_thetas.empty(); }
virtual bool isClassifier() const { return true; }
virtual String getDefaultModelName() const { return "opencv_ml_lr"; }
protected:
- cv::Mat calc_sigmoid(const cv::Mat& data) const;
- double compute_cost(const cv::Mat& _data, const cv::Mat& _labels, const cv::Mat& _init_theta);
- cv::Mat compute_batch_gradient(const cv::Mat& _data, const cv::Mat& _labels, const cv::Mat& _init_theta);
- cv::Mat compute_mini_batch_gradient(const cv::Mat& _data, const cv::Mat& _labels, const cv::Mat& _init_theta);
- bool set_label_map(const cv::Mat& _labels_i);
- cv::Mat remap_labels(const cv::Mat& _labels_i, const map<int, int>& lmap) const;
+ Mat calc_sigmoid(const Mat& data) const;
+ double compute_cost(const Mat& _data, const Mat& _labels, const Mat& _init_theta);
+ Mat compute_batch_gradient(const Mat& _data, const Mat& _labels, const Mat& _init_theta);
+ Mat compute_mini_batch_gradient(const Mat& _data, const Mat& _labels, const Mat& _init_theta);
+ bool set_label_map(const Mat& _labels_i);
+ Mat remap_labels(const Mat& _labels_i, const map<int, int>& lmap) const;
protected:
Params params;
- cv::Mat learnt_thetas;
+ Mat learnt_thetas;
map<int, int> forward_mapper;
map<int, int> reverse_mapper;
- cv::Mat labels_o;
- cv::Mat labels_n;
+ Mat labels_o;
+ Mat labels_n;
};
Ptr<LogisticRegression> LogisticRegression::create(const Params& params)
bool LogisticRegressionImpl::train(const Ptr<TrainData>& trainData, int)
{
clear();
- cv::Mat _data_i = trainData->getSamples();
- cv::Mat _labels_i = trainData->getResponses();
+ Mat _data_i = trainData->getSamples();
+ Mat _labels_i = trainData->getResponses();
CV_Assert( !_labels_i.empty() && !_data_i.empty());
bool ok = false;
- cv::Mat labels;
+ Mat labels;
set_label_map(_labels_i);
int num_classes = (int) this->forward_mapper.size();
// add a column of ones
- cv::Mat data_t = cv::Mat::zeros(_data_i.rows, _data_i.cols+1, CV_32F);
- vconcat(cv::Mat(_data_i.rows, 1, _data_i.type(), Scalar::all(1.0)), data_t.col(0));
+ Mat data_t = Mat::zeros(_data_i.rows, _data_i.cols+1, CV_32F);
+ vconcat(Mat(_data_i.rows, 1, _data_i.type(), Scalar::all(1.0)), data_t.col(0));
for (int i=1;i<data_t.cols;i++)
{
}
- cv::Mat thetas = cv::Mat::zeros(num_classes, data_t.cols, CV_32F);
- cv::Mat init_theta = cv::Mat::zeros(data_t.cols, 1, CV_32F);
+ Mat thetas = Mat::zeros(num_classes, data_t.cols, CV_32F);
+ Mat init_theta = Mat::zeros(data_t.cols, 1, CV_32F);
- cv::Mat labels_l = remap_labels(_labels_i, this->forward_mapper);
- cv::Mat new_local_labels;
+ Mat labels_l = remap_labels(_labels_i, this->forward_mapper);
+ Mat new_local_labels;
int ii=0;
- cv::Mat new_theta;
+ Mat new_theta;
if(num_classes == 2)
{
}
this->learnt_thetas = thetas.clone();
- if( cvIsNaN( (double)cv::sum(this->learnt_thetas)[0] ) )
+ if( cvIsNaN( (double)sum(this->learnt_thetas)[0] ) )
{
CV_Error( CV_StsBadArg, "check training parameters. Invalid training classifier" );
}
{
/* returns a class of the predicted class
class names can be 1,2,3,4, .... etc */
- cv::Mat thetas, data, pred_labs;
+ Mat thetas, data, pred_labs;
data = samples.getMat();
// check if learnt_mats array is populated
}
// add a column of ones
- cv::Mat data_t = cv::Mat::zeros(data.rows, data.cols+1, CV_32F);
+ Mat data_t = Mat::zeros(data.rows, data.cols+1, CV_32F);
for (int i=0;i<data_t.cols;i++)
{
if(i==0)
{
- vconcat(cv::Mat(data.rows, 1, data.type(), Scalar::all(1.0)), data_t.col(i));
+ vconcat(Mat(data.rows, 1, data.type(), Scalar::all(1.0)), data_t.col(i));
continue;
}
vconcat(data.col(i-1), data_t.col(i));
Point min_loc;
Point max_loc;
- cv::Mat labels;
- cv::Mat labels_c;
- cv::Mat temp_pred;
- cv::Mat pred_m = cv::Mat::zeros(data_t.rows, thetas.rows, data.type());
+ Mat labels;
+ Mat labels_c;
+ Mat temp_pred;
+ Mat pred_m = Mat::zeros(data_t.rows, thetas.rows, data.type());
if(thetas.rows == 1)
{
for(int i = 0;i<thetas.rows;i++)
{
temp_pred = calc_sigmoid(data_t * thetas.row(i).t());
- cv::vconcat(temp_pred, pred_m.col(i));
+ vconcat(temp_pred, pred_m.col(i));
}
for(int i = 0;i<pred_m.rows;i++)
{
return 0;
}
-cv::Mat LogisticRegressionImpl::calc_sigmoid(const cv::Mat& data) const
+Mat LogisticRegressionImpl::calc_sigmoid(const Mat& data) const
{
- cv::Mat dest;
- cv::exp(-data, dest);
+ Mat dest;
+ exp(-data, dest);
return 1.0/(1.0+dest);
}
-double LogisticRegressionImpl::compute_cost(const cv::Mat& _data, const cv::Mat& _labels, const cv::Mat& _init_theta)
+double LogisticRegressionImpl::compute_cost(const Mat& _data, const Mat& _labels, const Mat& _init_theta)
{
int llambda = 0;
int m;
int n;
double cost = 0;
double rparameter = 0;
- cv::Mat gradient;
- cv::Mat theta_b;
- cv::Mat theta_c;
- cv::Mat d_a;
- cv::Mat d_b;
+ Mat theta_b;
+ Mat theta_c;
+ Mat d_a;
+ Mat d_b;
m = _data.rows;
n = _data.cols;
- gradient = cv::Mat::zeros( _init_theta.rows, _init_theta.cols, _init_theta.type());
theta_b = _init_theta(Range(1, n), Range::all());
- cv::multiply(theta_b, theta_b, theta_c, 1);
+ multiply(theta_b, theta_b, theta_c, 1);
if(this->params.regularized > 0)
{
if(this->params.norm == LogisticRegression::REG_L1)
{
- rparameter = (llambda/(2*m)) * cv::sum(theta_b)[0];
+ rparameter = (llambda/(2*m)) * sum(theta_b)[0];
}
else
{
// assuming it to be L2 by default
- rparameter = (llambda/(2*m)) * cv::sum(theta_c)[0];
+ rparameter = (llambda/(2*m)) * sum(theta_c)[0];
}
d_a = calc_sigmoid(_data* _init_theta);
- cv::log(d_a, d_a);
- cv::multiply(d_a, _labels, d_a);
+ log(d_a, d_a);
+ multiply(d_a, _labels, d_a);
d_b = 1 - calc_sigmoid(_data * _init_theta);
- cv::log(d_b, d_b);
- cv::multiply(d_b, 1-_labels, d_b);
+ log(d_b, d_b);
+ multiply(d_b, 1-_labels, d_b);
- cost = (-1.0/m) * (cv::sum(d_a)[0] + cv::sum(d_b)[0]);
+ cost = (-1.0/m) * (sum(d_a)[0] + sum(d_b)[0]);
cost = cost + rparameter;
return cost;
}
-cv::Mat LogisticRegressionImpl::compute_batch_gradient(const cv::Mat& _data, const cv::Mat& _labels, const cv::Mat& _init_theta)
+Mat LogisticRegressionImpl::compute_batch_gradient(const Mat& _data, const Mat& _labels, const Mat& _init_theta)
{
// implements batch gradient descent
if(this->params.alpha<=0)
int llambda = 0;
double ccost;
int m, n;
- cv::Mat pcal_a;
- cv::Mat pcal_b;
- cv::Mat pcal_ab;
- cv::Mat gradient;
- cv::Mat theta_p = _init_theta.clone();
+ Mat pcal_a;
+ Mat pcal_b;
+ Mat pcal_ab;
+ Mat gradient;
+ Mat theta_p = _init_theta.clone();
m = _data.rows;
n = _data.cols;
pcal_b = _data(Range::all(), Range(0,1));
- cv::multiply(pcal_a, pcal_b, pcal_ab, 1);
+ multiply(pcal_a, pcal_b, pcal_ab, 1);
gradient.row(0) = ((float)1/m) * sum(pcal_ab)[0];
{
pcal_b = _data(Range::all(), Range(ii,ii+1));
- cv::multiply(pcal_a, pcal_b, pcal_ab, 1);
+ multiply(pcal_a, pcal_b, pcal_ab, 1);
- gradient.row(ii) = (1.0/m)*cv::sum(pcal_ab)[0] + (llambda/m) * theta_p.row(ii);
+ gradient.row(ii) = (1.0/m)*sum(pcal_ab)[0] + (llambda/m) * theta_p.row(ii);
}
theta_p = theta_p - ( static_cast<double>(this->params.alpha)/m)*gradient;
return theta_p;
}
-cv::Mat LogisticRegressionImpl::compute_mini_batch_gradient(const cv::Mat& _data, const cv::Mat& _labels, const cv::Mat& _init_theta)
+Mat LogisticRegressionImpl::compute_mini_batch_gradient(const Mat& _data, const Mat& _labels, const Mat& _init_theta)
{
// implements batch gradient descent
int lambda_l = 0;
CV_Error( CV_StsBadArg, "number of iterations cannot be zero or a negative number" );
}
- cv::Mat pcal_a;
- cv::Mat pcal_b;
- cv::Mat pcal_ab;
- cv::Mat gradient;
- cv::Mat theta_p = _init_theta.clone();
- cv::Mat data_d;
- cv::Mat labels_l;
+ Mat pcal_a;
+ Mat pcal_b;
+ Mat pcal_ab;
+ Mat gradient;
+ Mat theta_p = _init_theta.clone();
+ Mat data_d;
+ Mat labels_l;
if(this->params.regularized > 0)
{
pcal_b = data_d(Range::all(), Range(0,1));
- cv::multiply(pcal_a, pcal_b, pcal_ab, 1);
+ multiply(pcal_a, pcal_b, pcal_ab, 1);
gradient.row(0) = ((float)1/m) * sum(pcal_ab)[0];
for(int k = 1;k<gradient.rows;k++)
{
pcal_b = data_d(Range::all(), Range(k,k+1));
- cv::multiply(pcal_a, pcal_b, pcal_ab, 1);
- gradient.row(k) = (1.0/m)*cv::sum(pcal_ab)[0] + (lambda_l/m) * theta_p.row(k);
+ multiply(pcal_a, pcal_b, pcal_ab, 1);
+ gradient.row(k) = (1.0/m)*sum(pcal_ab)[0] + (lambda_l/m) * theta_p.row(k);
}
theta_p = theta_p - ( static_cast<double>(this->params.alpha)/m)*gradient;
return theta_p;
}
-bool LogisticRegressionImpl::set_label_map(const cv::Mat &_labels_i)
+bool LogisticRegressionImpl::set_label_map(const Mat &_labels_i)
{
// this function creates two maps to map user defined labels to program friendly labels two ways.
int ii = 0;
- cv::Mat labels;
- bool ok = false;
+ Mat labels;
- this->labels_o = cv::Mat(0,1, CV_8U);
- this->labels_n = cv::Mat(0,1, CV_8U);
+ this->labels_o = Mat(0,1, CV_8U);
+ this->labels_n = Mat(0,1, CV_8U);
_labels_i.convertTo(labels, CV_32S);
{
this->reverse_mapper[it->second] = it->first;
}
- ok = true;
- return ok;
+ return true;
}
-cv::Mat LogisticRegressionImpl::remap_labels(const cv::Mat& _labels_i, const map<int, int>& lmap) const
+Mat LogisticRegressionImpl::remap_labels(const Mat& _labels_i, const map<int, int>& lmap) const
{
- cv::Mat labels;
+ Mat labels;
_labels_i.convertTo(labels, CV_32S);
- cv::Mat new_labels = cv::Mat::zeros(labels.rows, labels.cols, labels.type());
+ Mat new_labels = Mat::zeros(labels.rows, labels.cols, labels.type());
CV_Assert( lmap.size() > 0 );
}
}
-cv::Mat LogisticRegressionImpl::get_learnt_thetas() const
+Mat LogisticRegressionImpl::get_learnt_thetas() const
{
return this->learnt_thetas;
}
projects / profile / ivi / opencv.git / commitdiff