+++ /dev/null
-Scene Text Detection
-====================
-
-.. highlight:: cpp
-
-Class-specific Extremal Regions for Scene Text Detection
---------------------------------------------------------
-
-The scene text detection algorithm described below has been initially proposed by Lukás Neumann & Jiri Matas [Neumann12]. The main idea behind Class-specific Extremal Regions is similar to the MSER in that suitable Extremal Regions (ERs) are selected from the whole component tree of the image. However, this technique differs from MSER in that selection of suitable ERs is done by a sequential classifier trained for character detection, i.e. dropping the stability requirement of MSERs and selecting class-specific (not necessarily stable) regions.
-
-The component tree of an image is constructed by thresholding by an increasing value step-by-step from 0 to 255 and then linking the obtained connected components from successive levels in a hierarchy by their inclusion relation:
-
-.. image:: pics/component_tree.png
- :width: 100%
-
-The component tree may conatain a huge number of regions even for a very simple image as shown in the previous image. This number can easily reach the order of 1 x 10^6 regions for an average 1 Megapixel image. In order to efficiently select suitable regions among all the ERs the algorithm make use of a sequential classifier with two differentiated stages.
-
-In the first stage incrementally computable descriptors (area, perimeter, bounding box, and euler number) are computed (in O(1)) for each region r and used as features for a classifier which estimates the class-conditional probability p(r|character). Only the ERs which correspond to local maximum of the probability p(r|character) are selected (if their probability is above a global limit p_min and the difference between local maximum and local minimum is greater than a \delta_min value).
-
-In the second stage, the ERs that passed the first stage are classified into character and non-character classes using more informative but also more computationally expensive features. (Hole area ratio, convex hull ratio, and the number of outer boundary inflexion points).
-
-This ER filtering process is done in different single-channel projections of the input image in order to increase the character localization recall.
-
-After the ER filtering is done on each input channel, character candidates must be grouped in high-level text blocks (i.e. words, text lines, paragraphs, ...). The grouping algorithm used in this implementation has been proposed by Lluis Gomez and Dimosthenis Karatzas in [Gomez13] and basically consist in finding meaningful groups of regions using a perceptual organization based clustering analisys (see :ocv:func:`erGrouping`).
-
-
-To see the text detector at work, have a look at the textdetection demo: https://github.com/Itseez/opencv/blob/master/samples/cpp/textdetection.cpp
-
-
-.. [Neumann12] Neumann L., Matas J.: Real-Time Scene Text Localization and Recognition, CVPR 2012. The paper is available online at http://cmp.felk.cvut.cz/~neumalu1/neumann-cvpr2012.pdf
-
-.. [Gomez13] Gomez L. and Karatzas D.: Multi-script Text Extraction from Natural Scenes, ICDAR 2013. The paper is available online at http://158.109.8.37/files/GoK2013.pdf
-
-
-ERStat
-------
-.. ocv:struct:: ERStat
-
-The ERStat structure represents a class-specific Extremal Region (ER).
-
-An ER is a 4-connected set of pixels with all its grey-level values smaller than the values in its outer boundary. A class-specific ER is selected (using a classifier) from all the ER's in the component tree of the image. ::
-
- struct CV_EXPORTS ERStat
- {
- public:
- //! Constructor
- explicit ERStat(int level = 256, int pixel = 0, int x = 0, int y = 0);
- //! Destructor
- ~ERStat() { }
-
- //! seed point and threshold (max grey-level value)
- int pixel;
- int level;
-
- //! incrementally computable features
- int area;
- int perimeter;
- int euler; //!< euler number
- Rect rect; //!< bounding box
- double raw_moments[2]; //!< order 1 raw moments to derive the centroid
- double central_moments[3]; //!< order 2 central moments to construct the covariance matrix
- std::deque<int> *crossings;//!< horizontal crossings
- float med_crossings; //!< median of the crossings at three different height levels
-
- //! 2nd stage features
- float hole_area_ratio;
- float convex_hull_ratio;
- float num_inflexion_points;
-
- //! probability that the ER belongs to the class we are looking for
- double probability;
-
- //! pointers preserving the tree structure of the component tree
- ERStat* parent;
- ERStat* child;
- ERStat* next;
- ERStat* prev;
- };
-
-computeNMChannels
------------------
-Compute the different channels to be processed independently in the N&M algorithm [Neumann12].
-
-.. ocv:function:: void computeNMChannels(InputArray _src, OutputArrayOfArrays _channels, int _mode = ERFILTER_NM_RGBLGrad)
-
- :param _src: Source image. Must be RGB ``CV_8UC3``.
- :param _channels: Output vector<Mat> where computed channels are stored.
- :param _mode: Mode of operation. Currently the only available options are: **ERFILTER_NM_RGBLGrad** (used by default) and **ERFILTER_NM_IHSGrad**.
-
-In N&M algorithm, the combination of intensity (I), hue (H), saturation (S), and gradient magnitude channels (Grad) are used in order to obtain high localization recall. This implementation also provides an alternative combination of red (R), green (G), blue (B), lightness (L), and gradient magnitude (Grad).
-
-
-ERFilter
---------
-.. ocv:class:: ERFilter : public Algorithm
-
-Base class for 1st and 2nd stages of Neumann and Matas scene text detection algorithm [Neumann12]. ::
-
- class CV_EXPORTS ERFilter : public Algorithm
- {
- public:
-
- //! callback with the classifier is made a class.
- //! By doing it we hide SVM, Boost etc. Developers can provide their own classifiers
- class CV_EXPORTS Callback
- {
- public:
- virtual ~Callback() { }
- //! The classifier must return probability measure for the region.
- virtual double eval(const ERStat& stat) = 0;
- };
-
- /*!
- the key method. Takes image on input and returns the selected regions in a vector of ERStat
- only distinctive ERs which correspond to characters are selected by a sequential classifier
- */
- virtual void run( InputArray image, std::vector<ERStat>& regions ) = 0;
-
- (...)
-
- };
-
-
-
-ERFilter::Callback
-------------------
-Callback with the classifier is made a class. By doing it we hide SVM, Boost etc. Developers can provide their own classifiers to the ERFilter algorithm.
-
-.. ocv:class:: ERFilter::Callback
-
-ERFilter::Callback::eval
-------------------------
-The classifier must return probability measure for the region.
-
-.. ocv:function:: double ERFilter::Callback::eval(const ERStat& stat)
-
- :param stat: The region to be classified
-
-ERFilter::run
--------------
-The key method of ERFilter algorithm. Takes image on input and returns the selected regions in a vector of ERStat only distinctive ERs which correspond to characters are selected by a sequential classifier
-
-.. ocv:function:: void ERFilter::run( InputArray image, std::vector<ERStat>& regions )
-
- :param image: Sinle channel image ``CV_8UC1``
- :param regions: Output for the 1st stage and Input/Output for the 2nd. The selected Extremal Regions are stored here.
-
-Extracts the component tree (if needed) and filter the extremal regions (ER's) by using a given classifier.
-
-createERFilterNM1
------------------
-Create an Extremal Region Filter for the 1st stage classifier of N&M algorithm [Neumann12].
-
-.. ocv:function:: Ptr<ERFilter> createERFilterNM1( const Ptr<ERFilter::Callback>& cb, int thresholdDelta = 1, float minArea = 0.00025, float maxArea = 0.13, float minProbability = 0.4, bool nonMaxSuppression = true, float minProbabilityDiff = 0.1 )
-
- :param cb: Callback with the classifier. Default classifier can be implicitly load with function :ocv:func:`loadClassifierNM1`, e.g. from file in samples/cpp/trained_classifierNM1.xml
- :param thresholdDelta: Threshold step in subsequent thresholds when extracting the component tree
- :param minArea: The minimum area (% of image size) allowed for retreived ER's
- :param minArea: The maximum area (% of image size) allowed for retreived ER's
- :param minProbability: The minimum probability P(er|character) allowed for retreived ER's
- :param nonMaxSuppression: Whenever non-maximum suppression is done over the branch probabilities
- :param minProbability: The minimum probability difference between local maxima and local minima ERs
-
-The component tree of the image is extracted by a threshold increased step by step from 0 to 255, incrementally computable descriptors (aspect_ratio, compactness, number of holes, and number of horizontal crossings) are computed for each ER and used as features for a classifier which estimates the class-conditional probability P(er|character). The value of P(er|character) is tracked using the inclusion relation of ER across all thresholds and only the ERs which correspond to local maximum of the probability P(er|character) are selected (if the local maximum of the probability is above a global limit pmin and the difference between local maximum and local minimum is greater than minProbabilityDiff).
-
-createERFilterNM2
------------------
-Create an Extremal Region Filter for the 2nd stage classifier of N&M algorithm [Neumann12].
-
-.. ocv:function:: Ptr<ERFilter> createERFilterNM2( const Ptr<ERFilter::Callback>& cb, float minProbability = 0.3 )
-
- :param cb: Callback with the classifier. Default classifier can be implicitly load with function :ocv:func:`loadClassifierNM2`, e.g. from file in samples/cpp/trained_classifierNM2.xml
- :param minProbability: The minimum probability P(er|character) allowed for retreived ER's
-
-In the second stage, the ERs that passed the first stage are classified into character and non-character classes using more informative but also more computationally expensive features. The classifier uses all the features calculated in the first stage and the following additional features: hole area ratio, convex hull ratio, and number of outer inflexion points.
-
-loadClassifierNM1
------------------
-Allow to implicitly load the default classifier when creating an ERFilter object.
-
-.. ocv:function:: Ptr<ERFilter::Callback> loadClassifierNM1(const std::string& filename)
-
- :param filename: The XML or YAML file with the classifier model (e.g. trained_classifierNM1.xml)
-
-returns a pointer to ERFilter::Callback.
-
-loadClassifierNM2
------------------
-Allow to implicitly load the default classifier when creating an ERFilter object.
-
-.. ocv:function:: Ptr<ERFilter::Callback> loadClassifierNM2(const std::string& filename)
-
- :param filename: The XML or YAML file with the classifier model (e.g. trained_classifierNM2.xml)
-
-returns a pointer to ERFilter::Callback.
-
-erGrouping
-----------
-Find groups of Extremal Regions that are organized as text blocks.
-
-.. ocv:function:: void erGrouping( InputArrayOfArrays src, std::vector<std::vector<ERStat> > ®ions, const std::string& filename, float minProbablity, std::vector<Rect > &groups)
-
- :param src: Vector of sinle channel images CV_8UC1 from wich the regions were extracted
- :param regions: Vector of ER's retreived from the ERFilter algorithm from each channel
- :param filename: The XML or YAML file with the classifier model (e.g. trained_classifier_erGrouping.xml)
- :param minProbability: The minimum probability for accepting a group
- :param groups: The output of the algorithm are stored in this parameter as list of rectangles.
-
-This function implements the grouping algorithm described in [Gomez13]. Notice that this implementation constrains the results to horizontally-aligned text and latin script (since ERFilter classifiers are trained only for latin script detection).
-
-The algorithm combines two different clustering techniques in a single parameter-free procedure to detect groups of regions organized as text. The maximally meaningful groups are fist detected in several feature spaces, where each feature space is a combination of proximity information (x,y coordinates) and a similarity measure (intensity, color, size, gradient magnitude, etc.), thus providing a set of hypotheses of text groups. Evidence Accumulation framework is used to combine all these hypotheses to get the final estimate. Each of the resulting groups are finally validated using a classifier in order to assess if they form a valid horizontally-aligned text block.
+++ /dev/null
-Latent SVM
-===============================================================
-
-Discriminatively Trained Part Based Models for Object Detection
----------------------------------------------------------------
-
-The object detector described below has been initially proposed by
-P.F. Felzenszwalb in [Felzenszwalb2010]_. It is based on a
-Dalal-Triggs detector that uses a single filter on histogram of
-oriented gradients (HOG) features to represent an object category.
-This detector uses a sliding window approach, where a filter is
-applied at all positions and scales of an image. The first
-innovation is enriching the Dalal-Triggs model using a
-star-structured part-based model defined by a "root" filter
-(analogous to the Dalal-Triggs filter) plus a set of parts filters
-and associated deformation models. The score of one of star models
-at a particular position and scale within an image is the score of
-the root filter at the given location plus the sum over parts of the
-maximum, over placements of that part, of the part filter score on
-its location minus a deformation cost easuring the deviation of the
-part from its ideal location relative to the root. Both root and
-part filter scores are defined by the dot product between a filter
-(a set of weights) and a subwindow of a feature pyramid computed
-from the input image. Another improvement is a representation of the
-class of models by a mixture of star models. The score of a mixture
-model at a particular position and scale is the maximum over
-components, of the score of that component model at the given
-location.
-
-In OpenCV there are C implementation of Latent SVM and C++ wrapper of it.
-C version is the structure :ocv:struct:`CvObjectDetection` and a set of functions
-working with this structure (see :ocv:func:`cvLoadLatentSvmDetector`,
-:ocv:func:`cvReleaseLatentSvmDetector`, :ocv:func:`cvLatentSvmDetectObjects`).
-C++ version is the class :ocv:class:`LatentSvmDetector` and has slightly different
-functionality in contrast with C version - it supports loading and detection
-of several models.
-
-There are two examples of Latent SVM usage: ``samples/c/latentsvmdetect.cpp``
-and ``samples/cpp/latentsvm_multidetect.cpp``.
-
-.. highlight:: c
-
-
-CvLSVMFilterPosition
---------------------
-.. ocv:struct:: CvLSVMFilterPosition
-
- Structure describes the position of the filter in the feature pyramid.
-
- .. ocv:member:: unsigned int l
-
- level in the feature pyramid
-
- .. ocv:member:: unsigned int x
-
- x-coordinate in level l
-
- .. ocv:member:: unsigned int y
-
- y-coordinate in level l
-
-
-CvLSVMFilterObject
-------------------
-.. ocv:struct:: CvLSVMFilterObject
-
- Description of the filter, which corresponds to the part of the object.
-
- .. ocv:member:: CvLSVMFilterPosition V
-
- ideal (penalty = 0) position of the partial filter
- from the root filter position (V_i in the paper)
-
- .. ocv:member:: float fineFunction[4]
-
- vector describes penalty function (d_i in the paper)
- pf[0] * x + pf[1] * y + pf[2] * x^2 + pf[3] * y^2
-
- .. ocv:member:: int sizeX
- .. ocv:member:: int sizeY
-
- Rectangular map (sizeX x sizeY),
- every cell stores feature vector (dimension = p)
-
- .. ocv:member:: int numFeatures
-
- number of features
-
- .. ocv:member:: float *H
-
- matrix of feature vectors to set and get
- feature vectors (i,j) used formula H[(j * sizeX + i) * p + k],
- where k - component of feature vector in cell (i, j)
-
-CvLatentSvmDetector
--------------------
-.. ocv:struct:: CvLatentSvmDetector
-
- Structure contains internal representation of trained Latent SVM detector.
-
- .. ocv:member:: int num_filters
-
- total number of filters (root plus part) in model
-
- .. ocv:member:: int num_components
-
- number of components in model
-
- .. ocv:member:: int* num_part_filters
-
- array containing number of part filters for each component
-
- .. ocv:member:: CvLSVMFilterObject** filters
-
- root and part filters for all model components
-
- .. ocv:member:: float* b
-
- biases for all model components
-
- .. ocv:member:: float score_threshold
-
- confidence level threshold
-
-
-CvObjectDetection
------------------
-.. ocv:struct:: CvObjectDetection
-
- Structure contains the bounding box and confidence level for detected object.
-
- .. ocv:member:: CvRect rect
-
- bounding box for a detected object
-
- .. ocv:member:: float score
-
- confidence level
-
-
-cvLoadLatentSvmDetector
------------------------
-Loads trained detector from a file.
-
-.. ocv:function:: CvLatentSvmDetector* cvLoadLatentSvmDetector(const char* filename)
-
- :param filename: Name of the file containing the description of a trained detector
-
-
-cvReleaseLatentSvmDetector
---------------------------
-Release memory allocated for CvLatentSvmDetector structure.
-
-.. ocv:function:: void cvReleaseLatentSvmDetector(CvLatentSvmDetector** detector)
-
- :param detector: CvLatentSvmDetector structure to be released
-
-
-cvLatentSvmDetectObjects
-------------------------
-Find rectangular regions in the given image that are likely to contain objects
-and corresponding confidence levels.
-
-.. ocv:function:: CvSeq* cvLatentSvmDetectObjects( IplImage* image, CvLatentSvmDetector* detector, CvMemStorage* storage, float overlap_threshold=0.5f, int numThreads=-1 )
-
- :param image: image
- :param detector: LatentSVM detector in internal representation
- :param storage: Memory storage to store the resultant sequence of the object candidate rectangles
- :param overlap_threshold: Threshold for the non-maximum suppression algorithm
- :param numThreads: Number of threads used in parallel version of the algorithm
-
-.. highlight:: cpp
-
-LatentSvmDetector
------------------
-.. ocv:class:: LatentSvmDetector
-
-This is a C++ wrapping class of Latent SVM. It contains internal representation of several
-trained Latent SVM detectors (models) and a set of methods to load the detectors and detect objects
-using them.
-
-LatentSvmDetector::ObjectDetection
-----------------------------------
-.. ocv:struct:: LatentSvmDetector::ObjectDetection
-
- Structure contains the detection information.
-
- .. ocv:member:: Rect rect
-
- bounding box for a detected object
-
- .. ocv:member:: float score
-
- confidence level
-
- .. ocv:member:: int classID
-
- class (model or detector) ID that detect an object
-
-
-LatentSvmDetector::LatentSvmDetector
-------------------------------------
-Two types of constructors.
-
-.. ocv:function:: LatentSvmDetector::LatentSvmDetector()
-
-.. ocv:function:: LatentSvmDetector::LatentSvmDetector(const vector<String>& filenames, const vector<String>& classNames=vector<String>())
-
-
-
- :param filenames: A set of filenames storing the trained detectors (models). Each file contains one model. See examples of such files here /opencv_extra/testdata/cv/latentsvmdetector/models_VOC2007/.
-
- :param classNames: A set of trained models names. If it's empty then the name of each model will be constructed from the name of file containing the model. E.g. the model stored in "/home/user/cat.xml" will get the name "cat".
-
-LatentSvmDetector::~LatentSvmDetector
--------------------------------------
-Destructor.
-
-.. ocv:function:: LatentSvmDetector::~LatentSvmDetector()
-
-LatentSvmDetector::~clear
--------------------------
-Clear all trained models and their names stored in an class object.
-
-.. ocv:function:: void LatentSvmDetector::clear()
-
-LatentSvmDetector::load
------------------------
-Load the trained models from given ``.xml`` files and return ``true`` if at least one model was loaded.
-
-.. ocv:function:: bool LatentSvmDetector::load( const vector<String>& filenames, const vector<String>& classNames=vector<String>() )
-
- :param filenames: A set of filenames storing the trained detectors (models). Each file contains one model. See examples of such files here /opencv_extra/testdata/cv/latentsvmdetector/models_VOC2007/.
-
- :param classNames: A set of trained models names. If it's empty then the name of each model will be constructed from the name of file containing the model. E.g. the model stored in "/home/user/cat.xml" will get the name "cat".
-
-LatentSvmDetector::detect
--------------------------
-Find rectangular regions in the given image that are likely to contain objects of loaded classes (models)
-and corresponding confidence levels.
-
-.. ocv:function:: void LatentSvmDetector::detect( const Mat& image, vector<ObjectDetection>& objectDetections, float overlapThreshold=0.5f, int numThreads=-1 )
-
- :param image: An image.
- :param objectDetections: The detections: rectangulars, scores and class IDs.
- :param overlapThreshold: Threshold for the non-maximum suppression algorithm.
- :param numThreads: Number of threads used in parallel version of the algorithm.
-
-LatentSvmDetector::getClassNames
---------------------------------
-Return the class (model) names that were passed in constructor or method ``load`` or extracted from models filenames in those methods.
-
-.. ocv:function:: const vector<String>& LatentSvmDetector::getClassNames() const
-
-LatentSvmDetector::getClassCount
---------------------------------
-Return a count of loaded models (classes).
-
-.. ocv:function:: size_t LatentSvmDetector::getClassCount() const
-
-
-.. [Felzenszwalb2010] Felzenszwalb, P. F. and Girshick, R. B. and McAllester, D. and Ramanan, D. *Object Detection with Discriminatively Trained Part Based Models*. PAMI, vol. 32, no. 9, pp. 1627-1645, September 2010
:maxdepth: 2
cascade_classification
- latent_svm
erfilter
}
-#include "opencv2/objdetect/linemod.hpp"
-#include "opencv2/objdetect/erfilter.hpp"
#include "opencv2/objdetect/detection_based_tracker.hpp"
#endif
+++ /dev/null
-/*M///////////////////////////////////////////////////////////////////////////////////////
-//
-// IMPORTANT: READ BEFORE DOWNLOADING, COPYING, INSTALLING OR USING.
-//
-// By downloading, copying, installing or using the software you agree to this license.
-// If you do not agree to this license, do not download, install,
-// copy or use the software.
-//
-//
-// License Agreement
-// For Open Source Computer Vision Library
-//
-// Copyright (C) 2000-2008, Intel Corporation, all rights reserved.
-// Copyright (C) 2009, Willow Garage Inc., all rights reserved.
-// Copyright (C) 2013, OpenCV Foundation, all rights reserved.
-// Third party copyrights are property of their respective owners.
-//
-// Redistribution and use in source and binary forms, with or without modification,
-// are permitted provided that the following conditions are met:
-//
-// * Redistribution's of source code must retain the above copyright notice,
-// this list of conditions and the following disclaimer.
-//
-// * Redistribution's in binary form must reproduce the above copyright notice,
-// this list of conditions and the following disclaimer in the documentation
-// and/or other materials provided with the distribution.
-//
-// * The name of the copyright holders may not be used to endorse or promote products
-// derived from this software without specific prior written permission.
-//
-// This software is provided by the copyright holders and contributors "as is" and
-// any express or implied warranties, including, but not limited to, the implied
-// warranties of merchantability and fitness for a particular purpose are disclaimed.
-// In no event shall the Intel Corporation or contributors be liable for any direct,
-// indirect, incidental, special, exemplary, or consequential damages
-// (including, but not limited to, procurement of substitute goods or services;
-// loss of use, data, or profits; or business interruption) however caused
-// and on any theory of liability, whether in contract, strict liability,
-// or tort (including negligence or otherwise) arising in any way out of
-// the use of this software, even if advised of the possibility of such damage.
-//
-//M*/
-
-#ifndef __OPENCV_OBJDETECT_ERFILTER_HPP__
-#define __OPENCV_OBJDETECT_ERFILTER_HPP__
-
-#include "opencv2/core.hpp"
-#include <vector>
-#include <deque>
-#include <string>
-
-namespace cv
-{
-
-/*!
- Extremal Region Stat structure
-
- The ERStat structure represents a class-specific Extremal Region (ER).
-
- An ER is a 4-connected set of pixels with all its grey-level values smaller than the values
- in its outer boundary. A class-specific ER is selected (using a classifier) from all the ER's
- in the component tree of the image.
-*/
-struct CV_EXPORTS ERStat
-{
-public:
- //! Constructor
- explicit ERStat(int level = 256, int pixel = 0, int x = 0, int y = 0);
- //! Destructor
- ~ERStat() { }
-
- //! seed point and the threshold (max grey-level value)
- int pixel;
- int level;
-
- //! incrementally computable features
- int area;
- int perimeter;
- int euler; //!< euler number
- Rect rect;
- double raw_moments[2]; //!< order 1 raw moments to derive the centroid
- double central_moments[3]; //!< order 2 central moments to construct the covariance matrix
- std::deque<int> *crossings;//!< horizontal crossings
- float med_crossings; //!< median of the crossings at three different height levels
-
- //! 2nd stage features
- float hole_area_ratio;
- float convex_hull_ratio;
- float num_inflexion_points;
-
- // TODO Other features can be added (average color, standard deviation, and such)
-
-
- // TODO shall we include the pixel list whenever available (i.e. after 2nd stage) ?
- std::vector<int> *pixels;
-
- //! probability that the ER belongs to the class we are looking for
- double probability;
-
- //! pointers preserving the tree structure of the component tree
- ERStat* parent;
- ERStat* child;
- ERStat* next;
- ERStat* prev;
-
- //! wenever the regions is a local maxima of the probability
- bool local_maxima;
- ERStat* max_probability_ancestor;
- ERStat* min_probability_ancestor;
-};
-
-/*!
- Base class for 1st and 2nd stages of Neumann and Matas scene text detection algorithms
- Neumann L., Matas J.: Real-Time Scene Text Localization and Recognition, CVPR 2012
-
- Extracts the component tree (if needed) and filter the extremal regions (ER's) by using a given classifier.
-*/
-class CV_EXPORTS ERFilter : public Algorithm
-{
-public:
-
- //! callback with the classifier is made a class. By doing it we hide SVM, Boost etc.
- class CV_EXPORTS Callback
- {
- public:
- virtual ~Callback() { }
- //! The classifier must return probability measure for the region.
- virtual double eval(const ERStat& stat) = 0; //const = 0; //TODO why cannot use const = 0 here?
- };
-
- /*!
- the key method. Takes image on input and returns the selected regions in a vector of ERStat
- only distinctive ERs which correspond to characters are selected by a sequential classifier
- \param image is the input image
- \param regions is output for the first stage, input/output for the second one.
- */
- virtual void run( InputArray image, std::vector<ERStat>& regions ) = 0;
-
-
- //! set/get methods to set the algorithm properties,
- virtual void setCallback(const Ptr<ERFilter::Callback>& cb) = 0;
- virtual void setThresholdDelta(int thresholdDelta) = 0;
- virtual void setMinArea(float minArea) = 0;
- virtual void setMaxArea(float maxArea) = 0;
- virtual void setMinProbability(float minProbability) = 0;
- virtual void setMinProbabilityDiff(float minProbabilityDiff) = 0;
- virtual void setNonMaxSuppression(bool nonMaxSuppression) = 0;
- virtual int getNumRejected() = 0;
-};
-
-
-/*!
- Create an Extremal Region Filter for the 1st stage classifier of N&M algorithm
- Neumann L., Matas J.: Real-Time Scene Text Localization and Recognition, CVPR 2012
-
- The component tree of the image is extracted by a threshold increased step by step
- from 0 to 255, incrementally computable descriptors (aspect_ratio, compactness,
- number of holes, and number of horizontal crossings) are computed for each ER
- and used as features for a classifier which estimates the class-conditional
- probability P(er|character). The value of P(er|character) is tracked using the inclusion
- relation of ER across all thresholds and only the ERs which correspond to local maximum
- of the probability P(er|character) are selected (if the local maximum of the
- probability is above a global limit pmin and the difference between local maximum and
- local minimum is greater than minProbabilityDiff).
-
- \param cb Callback with the classifier.
- default classifier can be implicitly load with function loadClassifierNM1()
- from file in samples/cpp/trained_classifierNM1.xml
- \param thresholdDelta Threshold step in subsequent thresholds when extracting the component tree
- \param minArea The minimum area (% of image size) allowed for retreived ER's
- \param minArea The maximum area (% of image size) allowed for retreived ER's
- \param minProbability The minimum probability P(er|character) allowed for retreived ER's
- \param nonMaxSuppression Whenever non-maximum suppression is done over the branch probabilities
- \param minProbability The minimum probability difference between local maxima and local minima ERs
-*/
-CV_EXPORTS Ptr<ERFilter> createERFilterNM1(const Ptr<ERFilter::Callback>& cb,
- int thresholdDelta = 1, float minArea = 0.00025,
- float maxArea = 0.13, float minProbability = 0.4,
- bool nonMaxSuppression = true,
- float minProbabilityDiff = 0.1);
-
-/*!
- Create an Extremal Region Filter for the 2nd stage classifier of N&M algorithm
- Neumann L., Matas J.: Real-Time Scene Text Localization and Recognition, CVPR 2012
-
- In the second stage, the ERs that passed the first stage are classified into character
- and non-character classes using more informative but also more computationally expensive
- features. The classifier uses all the features calculated in the first stage and the following
- additional features: hole area ratio, convex hull ratio, and number of outer inflexion points.
-
- \param cb Callback with the classifier
- default classifier can be implicitly load with function loadClassifierNM2()
- from file in samples/cpp/trained_classifierNM2.xml
- \param minProbability The minimum probability P(er|character) allowed for retreived ER's
-*/
-CV_EXPORTS Ptr<ERFilter> createERFilterNM2(const Ptr<ERFilter::Callback>& cb,
- float minProbability = 0.3);
-
-
-/*!
- Allow to implicitly load the default classifier when creating an ERFilter object.
- The function takes as parameter the XML or YAML file with the classifier model
- (e.g. trained_classifierNM1.xml) returns a pointer to ERFilter::Callback.
-*/
-
-CV_EXPORTS Ptr<ERFilter::Callback> loadClassifierNM1(const std::string& filename);
-
-/*!
- Allow to implicitly load the default classifier when creating an ERFilter object.
- The function takes as parameter the XML or YAML file with the classifier model
- (e.g. trained_classifierNM1.xml) returns a pointer to ERFilter::Callback.
-*/
-
-CV_EXPORTS Ptr<ERFilter::Callback> loadClassifierNM2(const std::string& filename);
-
-
-// computeNMChannels operation modes
-enum { ERFILTER_NM_RGBLGrad = 0,
- ERFILTER_NM_IHSGrad = 1
- };
-
-/*!
- Compute the different channels to be processed independently in the N&M algorithm
- Neumann L., Matas J.: Real-Time Scene Text Localization and Recognition, CVPR 2012
-
- In N&M algorithm, the combination of intensity (I), hue (H), saturation (S), and gradient
- magnitude channels (Grad) are used in order to obtain high localization recall.
- This implementation also provides an alternative combination of red (R), green (G), blue (B),
- lightness (L), and gradient magnitude (Grad).
-
- \param _src Source image. Must be RGB CV_8UC3.
- \param _channels Output vector<Mat> where computed channels are stored.
- \param _mode Mode of operation. Currently the only available options are
- ERFILTER_NM_RGBLGrad (by default) and ERFILTER_NM_IHSGrad.
-
-*/
-CV_EXPORTS void computeNMChannels(InputArray _src, OutputArrayOfArrays _channels, int _mode = ERFILTER_NM_RGBLGrad);
-
-
-/*!
- Find groups of Extremal Regions that are organized as text blocks. This function implements
- the grouping algorithm described in:
- Gomez L. and Karatzas D.: Multi-script Text Extraction from Natural Scenes, ICDAR 2013.
- Notice that this implementation constrains the results to horizontally-aligned text and
- latin script (since ERFilter classifiers are trained only for latin script detection).
-
- The algorithm combines two different clustering techniques in a single parameter-free procedure
- to detect groups of regions organized as text. The maximally meaningful groups are fist detected
- in several feature spaces, where each feature space is a combination of proximity information
- (x,y coordinates) and a similarity measure (intensity, color, size, gradient magnitude, etc.),
- thus providing a set of hypotheses of text groups. Evidence Accumulation framework is used to
- combine all these hypotheses to get the final estimate. Each of the resulting groups are finally
- validated using a classifier in order to assest if they form a valid horizontally-aligned text block.
-
- \param src Vector of sinle channel images CV_8UC1 from wich the regions were extracted.
- \param regions Vector of ER's retreived from the ERFilter algorithm from each channel
- \param filename The XML or YAML file with the classifier model (e.g. trained_classifier_erGrouping.xml)
- \param minProbability The minimum probability for accepting a group
- \param groups The output of the algorithm are stored in this parameter as list of rectangles.
-*/
-CV_EXPORTS void erGrouping(InputArrayOfArrays src, std::vector<std::vector<ERStat> > ®ions,
- const std::string& filename, float minProbablity,
- std::vector<Rect > &groups);
-
-}
-#endif // _OPENCV_ERFILTER_HPP_
+++ /dev/null
-/*M///////////////////////////////////////////////////////////////////////////////////////
-//
-// IMPORTANT: READ BEFORE DOWNLOADING, COPYING, INSTALLING OR USING.
-//
-// By downloading, copying, installing or using the software you agree to this license.
-// If you do not agree to this license, do not download, install,
-// copy or use the software.
-//
-//
-// License Agreement
-// For Open Source Computer Vision Library
-//
-// Copyright (C) 2000-2008, Intel Corporation, all rights reserved.
-// Copyright (C) 2009, Willow Garage Inc., all rights reserved.
-// Copyright (C) 2013, OpenCV Foundation, all rights reserved.
-// Third party copyrights are property of their respective owners.
-//
-// Redistribution and use in source and binary forms, with or without modification,
-// are permitted provided that the following conditions are met:
-//
-// * Redistribution's of source code must retain the above copyright notice,
-// this list of conditions and the following disclaimer.
-//
-// * Redistribution's in binary form must reproduce the above copyright notice,
-// this list of conditions and the following disclaimer in the documentation
-// and/or other materials provided with the distribution.
-//
-// * The name of the copyright holders may not be used to endorse or promote products
-// derived from this software without specific prior written permission.
-//
-// This software is provided by the copyright holders and contributors "as is" and
-// any express or implied warranties, including, but not limited to, the implied
-// warranties of merchantability and fitness for a particular purpose are disclaimed.
-// In no event shall the Intel Corporation or contributors be liable for any direct,
-// indirect, incidental, special, exemplary, or consequential damages
-// (including, but not limited to, procurement of substitute goods or services;
-// loss of use, data, or profits; or business interruption) however caused
-// and on any theory of liability, whether in contract, strict liability,
-// or tort (including negligence or otherwise) arising in any way out of
-// the use of this software, even if advised of the possibility of such damage.
-//
-//M*/
-
-#ifndef __OPENCV_OBJDETECT_LINEMOD_HPP__
-#define __OPENCV_OBJDETECT_LINEMOD_HPP__
-
-#include "opencv2/core.hpp"
-#include <map>
-
-/****************************************************************************************\
-* LINE-MOD *
-\****************************************************************************************/
-
-namespace cv {
-namespace linemod {
-
-/// @todo Convert doxy comments to rst
-
-/**
- * \brief Discriminant feature described by its location and label.
- */
-struct CV_EXPORTS Feature
-{
- int x; ///< x offset
- int y; ///< y offset
- int label; ///< Quantization
-
- Feature() : x(0), y(0), label(0) {}
- Feature(int x, int y, int label);
-
- void read(const FileNode& fn);
- void write(FileStorage& fs) const;
-};
-
-inline Feature::Feature(int _x, int _y, int _label) : x(_x), y(_y), label(_label) {}
-
-struct CV_EXPORTS Template
-{
- int width;
- int height;
- int pyramid_level;
- std::vector<Feature> features;
-
- void read(const FileNode& fn);
- void write(FileStorage& fs) const;
-};
-
-/**
- * \brief Represents a modality operating over an image pyramid.
- */
-class QuantizedPyramid
-{
-public:
- // Virtual destructor
- virtual ~QuantizedPyramid() {}
-
- /**
- * \brief Compute quantized image at current pyramid level for online detection.
- *
- * \param[out] dst The destination 8-bit image. For each pixel at most one bit is set,
- * representing its classification.
- */
- virtual void quantize(Mat& dst) const =0;
-
- /**
- * \brief Extract most discriminant features at current pyramid level to form a new template.
- *
- * \param[out] templ The new template.
- */
- virtual bool extractTemplate(Template& templ) const =0;
-
- /**
- * \brief Go to the next pyramid level.
- *
- * \todo Allow pyramid scale factor other than 2
- */
- virtual void pyrDown() =0;
-
-protected:
- /// Candidate feature with a score
- struct Candidate
- {
- Candidate(int x, int y, int label, float score);
-
- /// Sort candidates with high score to the front
- bool operator<(const Candidate& rhs) const
- {
- return score > rhs.score;
- }
-
- Feature f;
- float score;
- };
-
- /**
- * \brief Choose candidate features so that they are not bunched together.
- *
- * \param[in] candidates Candidate features sorted by score.
- * \param[out] features Destination vector of selected features.
- * \param[in] num_features Number of candidates to select.
- * \param[in] distance Hint for desired distance between features.
- */
- static void selectScatteredFeatures(const std::vector<Candidate>& candidates,
- std::vector<Feature>& features,
- size_t num_features, float distance);
-};
-
-inline QuantizedPyramid::Candidate::Candidate(int x, int y, int label, float _score) : f(x, y, label), score(_score) {}
-
-/**
- * \brief Interface for modalities that plug into the LINE template matching representation.
- *
- * \todo Max response, to allow optimization of summing (255/MAX) features as uint8
- */
-class CV_EXPORTS Modality
-{
-public:
- // Virtual destructor
- virtual ~Modality() {}
-
- /**
- * \brief Form a quantized image pyramid from a source image.
- *
- * \param[in] src The source image. Type depends on the modality.
- * \param[in] mask Optional mask. If not empty, unmasked pixels are set to zero
- * in quantized image and cannot be extracted as features.
- */
- Ptr<QuantizedPyramid> process(const Mat& src,
- const Mat& mask = Mat()) const
- {
- return processImpl(src, mask);
- }
-
- virtual String name() const =0;
-
- virtual void read(const FileNode& fn) =0;
- virtual void write(FileStorage& fs) const =0;
-
- /**
- * \brief Create modality by name.
- *
- * The following modality types are supported:
- * - "ColorGradient"
- * - "DepthNormal"
- */
- static Ptr<Modality> create(const String& modality_type);
-
- /**
- * \brief Load a modality from file.
- */
- static Ptr<Modality> create(const FileNode& fn);
-
-protected:
- // Indirection is because process() has a default parameter.
- virtual Ptr<QuantizedPyramid> processImpl(const Mat& src,
- const Mat& mask) const =0;
-};
-
-/**
- * \brief Modality that computes quantized gradient orientations from a color image.
- */
-class CV_EXPORTS ColorGradient : public Modality
-{
-public:
- /**
- * \brief Default constructor. Uses reasonable default parameter values.
- */
- ColorGradient();
-
- /**
- * \brief Constructor.
- *
- * \param weak_threshold When quantizing, discard gradients with magnitude less than this.
- * \param num_features How many features a template must contain.
- * \param strong_threshold Consider as candidate features only gradients whose norms are
- * larger than this.
- */
- ColorGradient(float weak_threshold, size_t num_features, float strong_threshold);
-
- virtual String name() const;
-
- virtual void read(const FileNode& fn);
- virtual void write(FileStorage& fs) const;
-
- float weak_threshold;
- size_t num_features;
- float strong_threshold;
-
-protected:
- virtual Ptr<QuantizedPyramid> processImpl(const Mat& src,
- const Mat& mask) const;
-};
-
-/**
- * \brief Modality that computes quantized surface normals from a dense depth map.
- */
-class CV_EXPORTS DepthNormal : public Modality
-{
-public:
- /**
- * \brief Default constructor. Uses reasonable default parameter values.
- */
- DepthNormal();
-
- /**
- * \brief Constructor.
- *
- * \param distance_threshold Ignore pixels beyond this distance.
- * \param difference_threshold When computing normals, ignore contributions of pixels whose
- * depth difference with the central pixel is above this threshold.
- * \param num_features How many features a template must contain.
- * \param extract_threshold Consider as candidate feature only if there are no differing
- * orientations within a distance of extract_threshold.
- */
- DepthNormal(int distance_threshold, int difference_threshold, size_t num_features,
- int extract_threshold);
-
- virtual String name() const;
-
- virtual void read(const FileNode& fn);
- virtual void write(FileStorage& fs) const;
-
- int distance_threshold;
- int difference_threshold;
- size_t num_features;
- int extract_threshold;
-
-protected:
- virtual Ptr<QuantizedPyramid> processImpl(const Mat& src,
- const Mat& mask) const;
-};
-
-/**
- * \brief Debug function to colormap a quantized image for viewing.
- */
-void colormap(const Mat& quantized, Mat& dst);
-
-/**
- * \brief Represents a successful template match.
- */
-struct CV_EXPORTS Match
-{
- Match()
- {
- }
-
- Match(int x, int y, float similarity, const String& class_id, int template_id);
-
- /// Sort matches with high similarity to the front
- bool operator<(const Match& rhs) const
- {
- // Secondarily sort on template_id for the sake of duplicate removal
- if (similarity != rhs.similarity)
- return similarity > rhs.similarity;
- else
- return template_id < rhs.template_id;
- }
-
- bool operator==(const Match& rhs) const
- {
- return x == rhs.x && y == rhs.y && similarity == rhs.similarity && class_id == rhs.class_id;
- }
-
- int x;
- int y;
- float similarity;
- String class_id;
- int template_id;
-};
-
-inline
-Match::Match(int _x, int _y, float _similarity, const String& _class_id, int _template_id)
- : x(_x), y(_y), similarity(_similarity), class_id(_class_id), template_id(_template_id)
-{}
-
-/**
- * \brief Object detector using the LINE template matching algorithm with any set of
- * modalities.
- */
-class CV_EXPORTS Detector
-{
-public:
- /**
- * \brief Empty constructor, initialize with read().
- */
- Detector();
-
- /**
- * \brief Constructor.
- *
- * \param modalities Modalities to use (color gradients, depth normals, ...).
- * \param T_pyramid Value of the sampling step T at each pyramid level. The
- * number of pyramid levels is T_pyramid.size().
- */
- Detector(const std::vector< Ptr<Modality> >& modalities, const std::vector<int>& T_pyramid);
-
- /**
- * \brief Detect objects by template matching.
- *
- * Matches globally at the lowest pyramid level, then refines locally stepping up the pyramid.
- *
- * \param sources Source images, one for each modality.
- * \param threshold Similarity threshold, a percentage between 0 and 100.
- * \param[out] matches Template matches, sorted by similarity score.
- * \param class_ids If non-empty, only search for the desired object classes.
- * \param[out] quantized_images Optionally return vector<Mat> of quantized images.
- * \param masks The masks for consideration during matching. The masks should be CV_8UC1
- * where 255 represents a valid pixel. If non-empty, the vector must be
- * the same size as sources. Each element must be
- * empty or the same size as its corresponding source.
- */
- void match(const std::vector<Mat>& sources, float threshold, std::vector<Match>& matches,
- const std::vector<String>& class_ids = std::vector<String>(),
- OutputArrayOfArrays quantized_images = noArray(),
- const std::vector<Mat>& masks = std::vector<Mat>()) const;
-
- /**
- * \brief Add new object template.
- *
- * \param sources Source images, one for each modality.
- * \param class_id Object class ID.
- * \param object_mask Mask separating object from background.
- * \param[out] bounding_box Optionally return bounding box of the extracted features.
- *
- * \return Template ID, or -1 if failed to extract a valid template.
- */
- int addTemplate(const std::vector<Mat>& sources, const String& class_id,
- const Mat& object_mask, Rect* bounding_box = NULL);
-
- /**
- * \brief Add a new object template computed by external means.
- */
- int addSyntheticTemplate(const std::vector<Template>& templates, const String& class_id);
-
- /**
- * \brief Get the modalities used by this detector.
- *
- * You are not permitted to add/remove modalities, but you may dynamic_cast them to
- * tweak parameters.
- */
- const std::vector< Ptr<Modality> >& getModalities() const { return modalities; }
-
- /**
- * \brief Get sampling step T at pyramid_level.
- */
- int getT(int pyramid_level) const { return T_at_level[pyramid_level]; }
-
- /**
- * \brief Get number of pyramid levels used by this detector.
- */
- int pyramidLevels() const { return pyramid_levels; }
-
- /**
- * \brief Get the template pyramid identified by template_id.
- *
- * For example, with 2 modalities (Gradient, Normal) and two pyramid levels
- * (L0, L1), the order is (GradientL0, NormalL0, GradientL1, NormalL1).
- */
- const std::vector<Template>& getTemplates(const String& class_id, int template_id) const;
-
- int numTemplates() const;
- int numTemplates(const String& class_id) const;
- int numClasses() const { return static_cast<int>(class_templates.size()); }
-
- std::vector<String> classIds() const;
-
- void read(const FileNode& fn);
- void write(FileStorage& fs) const;
-
- String readClass(const FileNode& fn, const String &class_id_override = "");
- void writeClass(const String& class_id, FileStorage& fs) const;
-
- void readClasses(const std::vector<String>& class_ids,
- const String& format = "templates_%s.yml.gz");
- void writeClasses(const String& format = "templates_%s.yml.gz") const;
-
-protected:
- std::vector< Ptr<Modality> > modalities;
- int pyramid_levels;
- std::vector<int> T_at_level;
-
- typedef std::vector<Template> TemplatePyramid;
- typedef std::map<String, std::vector<TemplatePyramid> > TemplatesMap;
- TemplatesMap class_templates;
-
- typedef std::vector<Mat> LinearMemories;
- // Indexed as [pyramid level][modality][quantized label]
- typedef std::vector< std::vector<LinearMemories> > LinearMemoryPyramid;
-
- void matchClass(const LinearMemoryPyramid& lm_pyramid,
- const std::vector<Size>& sizes,
- float threshold, std::vector<Match>& matches,
- const String& class_id,
- const std::vector<TemplatePyramid>& template_pyramids) const;
-};
-
-/**
- * \brief Factory function for detector using LINE algorithm with color gradients.
- *
- * Default parameter settings suitable for VGA images.
- */
-CV_EXPORTS Ptr<Detector> getDefaultLINE();
-
-/**
- * \brief Factory function for detector using LINE-MOD algorithm with color gradients
- * and depth normals.
- *
- * Default parameter settings suitable for VGA images.
- */
-CV_EXPORTS Ptr<Detector> getDefaultLINEMOD();
-
-} // namespace linemod
-} // namespace cv
-
-#endif // __OPENCV_OBJDETECT_LINEMOD_HPP__
+++ /dev/null
-/*M///////////////////////////////////////////////////////////////////////////////////////
-//
-// IMPORTANT: READ BEFORE DOWNLOADING, COPYING, INSTALLING OR USING.
-//
-// By downloading, copying, installing or using the software you agree to this license.
-// If you do not agree to this license, do not download, install,
-// copy or use the software.
-//
-//
-// License Agreement
-// For Open Source Computer Vision Library
-//
-// Copyright (C) 2000-2008, Intel Corporation, all rights reserved.
-// Copyright (C) 2009, Willow Garage Inc., all rights reserved.
-// Third party copyrights are property of their respective owners.
-//
-// Redistribution and use in source and binary forms, with or without modification,
-// are permitted provided that the following conditions are met:
-//
-// * Redistribution's of source code must retain the above copyright notice,
-// this list of conditions and the following disclaimer.
-//
-// * Redistribution's in binary form must reproduce the above copyright notice,
-// this list of conditions and the following disclaimer in the documentation
-// and/or other materials provided with the distribution.
-//
-// * The name of the copyright holders may not be used to endorse or promote products
-// derived from this software without specific prior written permission.
-//
-// This software is provided by the copyright holders and contributors "as is" and
-// any express or implied warranties, including, but not limited to, the implied
-// warranties of merchantability and fitness for a particular purpose are disclaimed.
-// In no event shall the Intel Corporation or contributors be liable for any direct,
-// indirect, incidental, special, exemplary, or consequential damages
-// (including, but not limited to, procurement of substitute goods or services;
-// loss of use, data, or profits; or business interruption) however caused
-// and on any theory of liability, whether in contract, strict liability,
-// or tort (including negligence or otherwise) arising in any way out of
-// the use of this software, even if advised of the possibility of such damage.
-//
-//M*/
-
-#include "precomp.hpp"
-#include <fstream>
-#include <queue>
-
-#if defined _MSC_VER && _MSC_VER == 1500
- typedef int int_fast32_t;
-#else
- #ifndef INT32_MAX
- #define __STDC_LIMIT_MACROS
- #include <stdint.h>
- #endif
-#endif
-
-using namespace std;
-
-namespace cv
-{
-
-// Deletes a tree of ERStat regions starting at root. Used only
-// internally to this implementation.
-static void deleteERStatTree(ERStat* root) {
- queue<ERStat*> to_delete;
- to_delete.push(root);
- while (!to_delete.empty()) {
- ERStat* n = to_delete.front();
- to_delete.pop();
- ERStat* c = n->child;
- if (c != NULL) {
- to_delete.push(c);
- ERStat* sibling = c->next;
- while (sibling != NULL) {
- to_delete.push(sibling);
- sibling = sibling->next;
- }
- }
- delete n;
- }
-}
-
-ERStat::ERStat(int init_level, int init_pixel, int init_x, int init_y) : pixel(init_pixel),
- level(init_level), area(0), perimeter(0), euler(0), probability(1.0),
- parent(0), child(0), next(0), prev(0), local_maxima(0),
- max_probability_ancestor(0), min_probability_ancestor(0)
-{
- rect = Rect(init_x,init_y,1,1);
- raw_moments[0] = 0.0;
- raw_moments[1] = 0.0;
- central_moments[0] = 0.0;
- central_moments[1] = 0.0;
- central_moments[2] = 0.0;
- crossings = new std::deque<int>();
- crossings->push_back(0);
-}
-
-
-// derivative classes
-
-
-// the classe implementing the interface for the 1st and 2nd stages of Neumann and Matas algorithm
-class CV_EXPORTS ERFilterNM : public ERFilter
-{
-public:
- //Constructor
- ERFilterNM();
- //Destructor
- ~ERFilterNM() {}
-
- float minProbability;
- bool nonMaxSuppression;
- float minProbabilityDiff;
-
- // the key method. Takes image on input, vector of ERStat is output for the first stage,
- // input/output - for the second one.
- void run( InputArray image, std::vector<ERStat>& regions );
-
-protected:
- int thresholdDelta;
- float maxArea;
- float minArea;
-
- Ptr<ERFilter::Callback> classifier;
-
- // count of the rejected/accepted regions
- int num_rejected_regions;
- int num_accepted_regions;
-
-public:
-
- // set/get methods to set the algorithm properties,
- void setCallback(const Ptr<ERFilter::Callback>& cb);
- void setThresholdDelta(int thresholdDelta);
- void setMinArea(float minArea);
- void setMaxArea(float maxArea);
- void setMinProbability(float minProbability);
- void setMinProbabilityDiff(float minProbabilityDiff);
- void setNonMaxSuppression(bool nonMaxSuppression);
- int getNumRejected();
-
-private:
- // pointer to the input/output regions vector
- std::vector<ERStat> *regions;
- // image mask used for feature calculations
- Mat region_mask;
-
- // extract the component tree and store all the ER regions
- void er_tree_extract( InputArray image );
- // accumulate a pixel into an ER
- void er_add_pixel( ERStat *parent, int x, int y, int non_boundary_neighbours,
- int non_boundary_neighbours_horiz,
- int d_C1, int d_C2, int d_C3 );
- // merge an ER with its nested parent
- void er_merge( ERStat *parent, ERStat *child );
- // copy extracted regions into the output vector
- ERStat* er_save( ERStat *er, ERStat *parent, ERStat *prev );
- // recursively walk the tree and filter (remove) regions using the callback classifier
- ERStat* er_tree_filter( InputArray image, ERStat *stat, ERStat *parent, ERStat *prev );
- // recursively walk the tree selecting only regions with local maxima probability
- ERStat* er_tree_nonmax_suppression( ERStat *er, ERStat *parent, ERStat *prev );
-};
-
-
-// default 1st stage classifier
-class CV_EXPORTS ERClassifierNM1 : public ERFilter::Callback
-{
-public:
- //Constructor
- ERClassifierNM1(const std::string& filename);
- // Destructor
- ~ERClassifierNM1() {}
-
- // The classifier must return probability measure for the region.
- double eval(const ERStat& stat);
-
-private:
- CvBoost boost;
-};
-
-// default 2nd stage classifier
-class CV_EXPORTS ERClassifierNM2 : public ERFilter::Callback
-{
-public:
- //constructor
- ERClassifierNM2(const std::string& filename);
- // Destructor
- ~ERClassifierNM2() {}
-
- // The classifier must return probability measure for the region.
- double eval(const ERStat& stat);
-
-private:
- CvBoost boost;
-};
-
-
-
-
-
-// default constructor
-ERFilterNM::ERFilterNM()
-{
- thresholdDelta = 1;
- minArea = 0.;
- maxArea = 1.;
- minProbability = 0.;
- nonMaxSuppression = false;
- minProbabilityDiff = 1.;
- num_accepted_regions = 0;
- num_rejected_regions = 0;
-}
-
-// the key method. Takes image on input, vector of ERStat is output for the first stage,
-// input/output for the second one.
-void ERFilterNM::run( InputArray image, std::vector<ERStat>& _regions )
-{
-
- // assert correct image type
- CV_Assert( image.getMat().type() == CV_8UC1 );
-
- regions = &_regions;
- region_mask = Mat::zeros(image.getMat().rows+2, image.getMat().cols+2, CV_8UC1);
-
- // if regions vector is empty we must extract the entire component tree
- if ( regions->size() == 0 )
- {
- er_tree_extract( image );
- if (nonMaxSuppression)
- {
- vector<ERStat> aux_regions;
- regions->swap(aux_regions);
- regions->reserve(aux_regions.size());
- er_tree_nonmax_suppression( &aux_regions.front(), NULL, NULL );
- aux_regions.clear();
- }
- }
- else // if regions vector is already filled we'll just filter the current regions
- {
- // the tree root must have no parent
- CV_Assert( regions->front().parent == NULL );
-
- vector<ERStat> aux_regions;
- regions->swap(aux_regions);
- regions->reserve(aux_regions.size());
- er_tree_filter( image, &aux_regions.front(), NULL, NULL );
- aux_regions.clear();
- }
-}
-
-// extract the component tree and store all the ER regions
-// uses the algorithm described in
-// Linear time maximally stable extremal regions, D Nistér, H Stewénius – ECCV 2008
-void ERFilterNM::er_tree_extract( InputArray image )
-{
-
- Mat src = image.getMat();
- // assert correct image type
- CV_Assert( src.type() == CV_8UC1 );
-
- if (thresholdDelta > 1)
- {
- src = (src / thresholdDelta) -1;
- }
-
- const unsigned char * image_data = src.data;
- int width = src.cols, height = src.rows;
-
- // the component stack
- vector<ERStat*> er_stack;
-
- //the quads for euler number calculation
- unsigned char quads[3][4];
- quads[0][0] = 1 << 3;
- quads[0][1] = 1 << 2;
- quads[0][2] = 1 << 1;
- quads[0][3] = 1;
- quads[1][0] = (1<<2)|(1<<1)|(1);
- quads[1][1] = (1<<3)|(1<<1)|(1);
- quads[1][2] = (1<<3)|(1<<2)|(1);
- quads[1][3] = (1<<3)|(1<<2)|(1<<1);
- quads[2][0] = (1<<2)|(1<<1);
- quads[2][1] = (1<<3)|(1);
- quads[2][3] = 255;
-
-
- // masks to know if a pixel is accessible and if it has been already added to some region
- vector<bool> accessible_pixel_mask(width * height);
- vector<bool> accumulated_pixel_mask(width * height);
-
- // heap of boundary pixels
- vector<int> boundary_pixes[256];
- vector<int> boundary_edges[256];
-
- // add a dummy-component before start
- er_stack.push_back(new ERStat);
-
- // we'll look initially for all pixels with grey-level lower than a grey-level higher than any allowed in the image
- int threshold_level = (255/thresholdDelta)+1;
-
- // starting from the first pixel (0,0)
- int current_pixel = 0;
- int current_edge = 0;
- int current_level = image_data[0];
- accessible_pixel_mask[0] = true;
-
- bool push_new_component = true;
-
- for (;;) {
-
- int x = current_pixel % width;
- int y = current_pixel / width;
-
- // push a component with current level in the component stack
- if (push_new_component)
- er_stack.push_back(new ERStat(current_level, current_pixel, x, y));
- push_new_component = false;
-
- // explore the (remaining) edges to the neighbors to the current pixel
- for ( ; current_edge < 4; current_edge++)
- {
-
- int neighbour_pixel = current_pixel;
-
- switch (current_edge)
- {
- case 0: if (x < width - 1) neighbour_pixel = current_pixel + 1; break;
- case 1: if (y < height - 1) neighbour_pixel = current_pixel + width; break;
- case 2: if (x > 0) neighbour_pixel = current_pixel - 1; break;
- default: if (y > 0) neighbour_pixel = current_pixel - width; break;
- }
-
- // if neighbour is not accessible, mark it accessible and retreive its grey-level value
- if ( !accessible_pixel_mask[neighbour_pixel] && (neighbour_pixel != current_pixel) )
- {
-
- int neighbour_level = image_data[neighbour_pixel];
- accessible_pixel_mask[neighbour_pixel] = true;
-
- // if neighbour level is not lower than current level add neighbour to the boundary heap
- if (neighbour_level >= current_level)
- {
-
- boundary_pixes[neighbour_level].push_back(neighbour_pixel);
- boundary_edges[neighbour_level].push_back(0);
-
- // if neighbour level is lower than our threshold_level set threshold_level to neighbour level
- if (neighbour_level < threshold_level)
- threshold_level = neighbour_level;
-
- }
- else // if neighbour level is lower than current add current_pixel (and next edge)
- // to the boundary heap for later processing
- {
-
- boundary_pixes[current_level].push_back(current_pixel);
- boundary_edges[current_level].push_back(current_edge + 1);
-
- // if neighbour level is lower than threshold_level set threshold_level to neighbour level
- if (current_level < threshold_level)
- threshold_level = current_level;
-
- // consider the new pixel and its grey-level as current pixel
- current_pixel = neighbour_pixel;
- current_edge = 0;
- current_level = neighbour_level;
-
- // and push a new component
- push_new_component = true;
- break;
- }
- }
-
- } // else neigbor was already accessible
-
- if (push_new_component) continue;
-
-
- // once here we can add the current pixel to the component at the top of the stack
- // but first we find how many of its neighbours are part of the region boundary (needed for
- // perimeter and crossings calc.) and the increment in quads counts for euler number calc.
- int non_boundary_neighbours = 0;
- int non_boundary_neighbours_horiz = 0;
-
- unsigned char quad_before[4] = {0,0,0,0};
- unsigned char quad_after[4] = {0,0,0,0};
- quad_after[0] = 1<<1;
- quad_after[1] = 1<<3;
- quad_after[2] = 1<<2;
- quad_after[3] = 1;
-
- for (int edge = 0; edge < 8; edge++)
- {
- int neighbour4 = -1;
- int neighbour8 = -1;
- int cell = 0;
- switch (edge)
- {
- case 0: if (x < width - 1) { neighbour4 = neighbour8 = current_pixel + 1;} cell = 5; break;
- case 1: if ((x < width - 1)&&(y < height - 1)) { neighbour8 = current_pixel + 1 + width;} cell = 8; break;
- case 2: if (y < height - 1) { neighbour4 = neighbour8 = current_pixel + width;} cell = 7; break;
- case 3: if ((x > 0)&&(y < height - 1)) { neighbour8 = current_pixel - 1 + width;} cell = 6; break;
- case 4: if (x > 0) { neighbour4 = neighbour8 = current_pixel - 1;} cell = 3; break;
- case 5: if ((x > 0)&&(y > 0)) { neighbour8 = current_pixel - 1 - width;} cell = 0; break;
- case 6: if (y > 0) { neighbour4 = neighbour8 = current_pixel - width;} cell = 1; break;
- default: if ((x < width - 1)&&(y > 0)) { neighbour8 = current_pixel + 1 - width;} cell = 2; break;
- }
- if ((neighbour4 != -1)&&(accumulated_pixel_mask[neighbour4])&&(image_data[neighbour4]<=image_data[current_pixel]))
- {
- non_boundary_neighbours++;
- if ((edge == 0) || (edge == 4))
- non_boundary_neighbours_horiz++;
- }
-
- int pix_value = image_data[current_pixel] + 1;
- if (neighbour8 != -1)
- {
- if (accumulated_pixel_mask[neighbour8])
- pix_value = image_data[neighbour8];
- }
-
- if (pix_value<=image_data[current_pixel])
- {
- switch(cell)
- {
- case 0:
- quad_before[3] = quad_before[3] | (1<<3);
- quad_after[3] = quad_after[3] | (1<<3);
- break;
- case 1:
- quad_before[3] = quad_before[3] | (1<<2);
- quad_after[3] = quad_after[3] | (1<<2);
- quad_before[0] = quad_before[0] | (1<<3);
- quad_after[0] = quad_after[0] | (1<<3);
- break;
- case 2:
- quad_before[0] = quad_before[0] | (1<<2);
- quad_after[0] = quad_after[0] | (1<<2);
- break;
- case 3:
- quad_before[3] = quad_before[3] | (1<<1);
- quad_after[3] = quad_after[3] | (1<<1);
- quad_before[2] = quad_before[2] | (1<<3);
- quad_after[2] = quad_after[2] | (1<<3);
- break;
- case 5:
- quad_before[0] = quad_before[0] | (1);
- quad_after[0] = quad_after[0] | (1);
- quad_before[1] = quad_before[1] | (1<<2);
- quad_after[1] = quad_after[1] | (1<<2);
- break;
- case 6:
- quad_before[2] = quad_before[2] | (1<<1);
- quad_after[2] = quad_after[2] | (1<<1);
- break;
- case 7:
- quad_before[2] = quad_before[2] | (1);
- quad_after[2] = quad_after[2] | (1);
- quad_before[1] = quad_before[1] | (1<<1);
- quad_after[1] = quad_after[1] | (1<<1);
- break;
- default:
- quad_before[1] = quad_before[1] | (1);
- quad_after[1] = quad_after[1] | (1);
- break;
- }
- }
-
- }
-
- int C_before[3] = {0, 0, 0};
- int C_after[3] = {0, 0, 0};
-
- for (int p=0; p<3; p++)
- {
- for (int q=0; q<4; q++)
- {
- if ( (quad_before[0] == quads[p][q]) && ((p<2)||(q<2)) )
- C_before[p]++;
- if ( (quad_before[1] == quads[p][q]) && ((p<2)||(q<2)) )
- C_before[p]++;
- if ( (quad_before[2] == quads[p][q]) && ((p<2)||(q<2)) )
- C_before[p]++;
- if ( (quad_before[3] == quads[p][q]) && ((p<2)||(q<2)) )
- C_before[p]++;
-
- if ( (quad_after[0] == quads[p][q]) && ((p<2)||(q<2)) )
- C_after[p]++;
- if ( (quad_after[1] == quads[p][q]) && ((p<2)||(q<2)) )
- C_after[p]++;
- if ( (quad_after[2] == quads[p][q]) && ((p<2)||(q<2)) )
- C_after[p]++;
- if ( (quad_after[3] == quads[p][q]) && ((p<2)||(q<2)) )
- C_after[p]++;
- }
- }
-
- int d_C1 = C_after[0]-C_before[0];
- int d_C2 = C_after[1]-C_before[1];
- int d_C3 = C_after[2]-C_before[2];
-
- er_add_pixel(er_stack.back(), x, y, non_boundary_neighbours, non_boundary_neighbours_horiz, d_C1, d_C2, d_C3);
- accumulated_pixel_mask[current_pixel] = true;
-
- // if we have processed all the possible threshold levels (the hea is empty) we are done!
- if (threshold_level == (255/thresholdDelta)+1)
- {
-
- // save the extracted regions into the output vector
- regions->reserve(num_accepted_regions+1);
- er_save(er_stack.back(), NULL, NULL);
-
- // clean memory
- for (size_t r=0; r<er_stack.size(); r++)
- {
- ERStat *stat = er_stack.at(r);
- if (stat->crossings)
- {
- stat->crossings->clear();
- delete(stat->crossings);
- stat->crossings = NULL;
- }
- deleteERStatTree(stat);
- }
- er_stack.clear();
-
- return;
- }
-
-
- // pop the heap of boundary pixels
- current_pixel = boundary_pixes[threshold_level].back();
- boundary_pixes[threshold_level].erase(boundary_pixes[threshold_level].end()-1);
- current_edge = boundary_edges[threshold_level].back();
- boundary_edges[threshold_level].erase(boundary_edges[threshold_level].end()-1);
-
- while (boundary_pixes[threshold_level].empty() && (threshold_level < (255/thresholdDelta)+1))
- threshold_level++;
-
-
- int new_level = image_data[current_pixel];
-
- // if the new pixel has higher grey value than the current one
- if (new_level != current_level) {
-
- current_level = new_level;
-
- // process components on the top of the stack until we reach the higher grey-level
- while (er_stack.back()->level < new_level)
- {
- ERStat* er = er_stack.back();
- er_stack.erase(er_stack.end()-1);
-
- if (new_level < er_stack.back()->level)
- {
- er_stack.push_back(new ERStat(new_level, current_pixel, current_pixel%width, current_pixel/width));
- er_merge(er_stack.back(), er);
- break;
- }
-
- er_merge(er_stack.back(), er);
- }
-
- }
-
- }
-}
-
-// accumulate a pixel into an ER
-void ERFilterNM::er_add_pixel(ERStat *parent, int x, int y, int non_border_neighbours,
- int non_border_neighbours_horiz,
- int d_C1, int d_C2, int d_C3)
-{
- parent->area++;
- parent->perimeter += 4 - 2*non_border_neighbours;
-
- if (parent->crossings->size()>0)
- {
- if (y<parent->rect.y) parent->crossings->push_front(2);
- else if (y>parent->rect.br().y-1) parent->crossings->push_back(2);
- else {
- parent->crossings->at(y - parent->rect.y) += 2-2*non_border_neighbours_horiz;
- }
- } else {
- parent->crossings->push_back(2);
- }
-
- parent->euler += (d_C1 - d_C2 + 2*d_C3) / 4;
-
- int new_x1 = min(parent->rect.x,x);
- int new_y1 = min(parent->rect.y,y);
- int new_x2 = max(parent->rect.br().x-1,x);
- int new_y2 = max(parent->rect.br().y-1,y);
- parent->rect.x = new_x1;
- parent->rect.y = new_y1;
- parent->rect.width = new_x2-new_x1+1;
- parent->rect.height = new_y2-new_y1+1;
-
- parent->raw_moments[0] += x;
- parent->raw_moments[1] += y;
-
- parent->central_moments[0] += x * x;
- parent->central_moments[1] += x * y;
- parent->central_moments[2] += y * y;
-}
-
-// merge an ER with its nested parent
-void ERFilterNM::er_merge(ERStat *parent, ERStat *child)
-{
-
- parent->area += child->area;
-
- parent->perimeter += child->perimeter;
-
-
- for (int i=parent->rect.y; i<=min(parent->rect.br().y-1,child->rect.br().y-1); i++)
- if (i-child->rect.y >= 0)
- parent->crossings->at(i-parent->rect.y) += child->crossings->at(i-child->rect.y);
-
- for (int i=parent->rect.y-1; i>=child->rect.y; i--)
- if (i-child->rect.y < (int)child->crossings->size())
- parent->crossings->push_front(child->crossings->at(i-child->rect.y));
- else
- parent->crossings->push_front(0);
-
- for (int i=parent->rect.br().y; i<child->rect.y; i++)
- parent->crossings->push_back(0);
-
- for (int i=max(parent->rect.br().y,child->rect.y); i<=child->rect.br().y-1; i++)
- parent->crossings->push_back(child->crossings->at(i-child->rect.y));
-
- parent->euler += child->euler;
-
- int new_x1 = min(parent->rect.x,child->rect.x);
- int new_y1 = min(parent->rect.y,child->rect.y);
- int new_x2 = max(parent->rect.br().x-1,child->rect.br().x-1);
- int new_y2 = max(parent->rect.br().y-1,child->rect.br().y-1);
- parent->rect.x = new_x1;
- parent->rect.y = new_y1;
- parent->rect.width = new_x2-new_x1+1;
- parent->rect.height = new_y2-new_y1+1;
-
- parent->raw_moments[0] += child->raw_moments[0];
- parent->raw_moments[1] += child->raw_moments[1];
-
- parent->central_moments[0] += child->central_moments[0];
- parent->central_moments[1] += child->central_moments[1];
- parent->central_moments[2] += child->central_moments[2];
-
- vector<int> m_crossings;
- m_crossings.push_back(child->crossings->at((int)(child->rect.height)/6));
- m_crossings.push_back(child->crossings->at((int)3*(child->rect.height)/6));
- m_crossings.push_back(child->crossings->at((int)5*(child->rect.height)/6));
- std::sort(m_crossings.begin(), m_crossings.end());
- child->med_crossings = (float)m_crossings.at(1);
-
- // free unnecessary mem
- child->crossings->clear();
- delete(child->crossings);
- child->crossings = NULL;
-
- // recover the original grey-level
- child->level = child->level*thresholdDelta;
-
- // before saving calculate P(child|character) and filter if possible
- if (classifier != NULL)
- {
- child->probability = classifier->eval(*child);
- }
-
- if ( (((classifier!=NULL)?(child->probability >= minProbability):true)||(nonMaxSuppression)) &&
- ((child->area >= (minArea*region_mask.rows*region_mask.cols)) &&
- (child->area <= (maxArea*region_mask.rows*region_mask.cols)) &&
- (child->rect.width > 2) && (child->rect.height > 2)) )
- {
-
- num_accepted_regions++;
-
- child->next = parent->child;
- if (parent->child)
- parent->child->prev = child;
- parent->child = child;
- child->parent = parent;
-
- } else {
-
- num_rejected_regions++;
-
- if (child->prev !=NULL)
- child->prev->next = child->next;
-
- ERStat *new_child = child->child;
- if (new_child != NULL)
- {
- while (new_child->next != NULL)
- new_child = new_child->next;
- new_child->next = parent->child;
- if (parent->child)
- parent->child->prev = new_child;
- parent->child = child->child;
- child->child->parent = parent;
- }
-
- // free mem
- if(child->crossings)
- {
- child->crossings->clear();
- delete(child->crossings);
- child->crossings = NULL;
- }
- delete(child);
- }
-
-}
-
-// copy extracted regions into the output vector
-ERStat* ERFilterNM::er_save( ERStat *er, ERStat *parent, ERStat *prev )
-{
-
- regions->push_back(*er);
-
- regions->back().parent = parent;
- if (prev != NULL)
- {
- prev->next = &(regions->back());
- }
- else if (parent != NULL)
- parent->child = &(regions->back());
-
- ERStat *old_prev = NULL;
- ERStat *this_er = ®ions->back();
-
- if (this_er->parent == NULL)
- {
- this_er->probability = 0;
- }
-
- if (nonMaxSuppression)
- {
- if (this_er->parent == NULL)
- {
- this_er->max_probability_ancestor = this_er;
- this_er->min_probability_ancestor = this_er;
- }
- else
- {
- this_er->max_probability_ancestor = (this_er->probability > parent->max_probability_ancestor->probability)? this_er : parent->max_probability_ancestor;
-
- this_er->min_probability_ancestor = (this_er->probability < parent->min_probability_ancestor->probability)? this_er : parent->min_probability_ancestor;
-
- if ( (this_er->max_probability_ancestor->probability > minProbability) && (this_er->max_probability_ancestor->probability - this_er->min_probability_ancestor->probability > minProbabilityDiff))
- {
- this_er->max_probability_ancestor->local_maxima = true;
- if ((this_er->max_probability_ancestor == this_er) && (this_er->parent->local_maxima))
- {
- this_er->parent->local_maxima = false;
- }
- }
- else if (this_er->probability < this_er->parent->probability)
- {
- this_er->min_probability_ancestor = this_er;
- }
- else if (this_er->probability > this_er->parent->probability)
- {
- this_er->max_probability_ancestor = this_er;
- }
-
-
- }
- }
-
- for (ERStat * child = er->child; child; child = child->next)
- {
- old_prev = er_save(child, this_er, old_prev);
- }
-
- return this_er;
-}
-
-// recursively walk the tree and filter (remove) regions using the callback classifier
-ERStat* ERFilterNM::er_tree_filter ( InputArray image, ERStat * stat, ERStat *parent, ERStat *prev )
-{
- Mat src = image.getMat();
- // assert correct image type
- CV_Assert( src.type() == CV_8UC1 );
-
- //Fill the region and calculate 2nd stage features
- Mat region = region_mask(Rect(Point(stat->rect.x,stat->rect.y),Point(stat->rect.br().x+2,stat->rect.br().y+2)));
- region = Scalar(0);
- int newMaskVal = 255;
- int flags = 4 + (newMaskVal << 8) + FLOODFILL_FIXED_RANGE + FLOODFILL_MASK_ONLY;
- Rect rect;
-
- floodFill( src(Rect(Point(stat->rect.x,stat->rect.y),Point(stat->rect.br().x,stat->rect.br().y))),
- region, Point(stat->pixel%src.cols - stat->rect.x, stat->pixel/src.cols - stat->rect.y),
- Scalar(255), &rect, Scalar(stat->level), Scalar(0), flags );
- rect.width += 2;
- rect.height += 2;
- region = region(rect);
-
- vector<vector<Point> > contours;
- vector<Point> contour_poly;
- vector<Vec4i> hierarchy;
- findContours( region, contours, hierarchy, RETR_TREE, CHAIN_APPROX_NONE, Point(0, 0) );
- //TODO check epsilon parameter of approxPolyDP (set empirically) : we want more precission
- // if the region is very small because otherwise we'll loose all the convexities
- approxPolyDP( Mat(contours[0]), contour_poly, (float)min(rect.width,rect.height)/17, true );
-
- bool was_convex = false;
- int num_inflexion_points = 0;
-
- for (int p = 0 ; p<(int)contour_poly.size(); p++)
- {
- int p_prev = p-1;
- int p_next = p+1;
- if (p_prev == -1)
- p_prev = (int)contour_poly.size()-1;
- if (p_next == (int)contour_poly.size())
- p_next = 0;
-
- double angle_next = atan2((double)(contour_poly[p_next].y-contour_poly[p].y),
- (double)(contour_poly[p_next].x-contour_poly[p].x));
- double angle_prev = atan2((double)(contour_poly[p_prev].y-contour_poly[p].y),
- (double)(contour_poly[p_prev].x-contour_poly[p].x));
- if ( angle_next < 0 )
- angle_next = 2.*CV_PI + angle_next;
-
- double angle = (angle_next - angle_prev);
- if (angle > 2.*CV_PI)
- angle = angle - 2.*CV_PI;
- else if (angle < 0)
- angle = 2.*CV_PI + std::abs(angle);
-
- if (p>0)
- {
- if ( ((angle > CV_PI)&&(!was_convex)) || ((angle < CV_PI)&&(was_convex)) )
- num_inflexion_points++;
- }
- was_convex = (angle > CV_PI);
-
- }
-
- floodFill(region, Point(0,0), Scalar(255), 0);
- int holes_area = region.cols*region.rows-countNonZero(region);
-
- int hull_area = 0;
-
- {
-
- vector<Point> hull;
- convexHull(contours[0], hull, false);
- hull_area = (int)contourArea(hull);
- }
-
-
- stat->hole_area_ratio = (float)holes_area / stat->area;
- stat->convex_hull_ratio = (float)hull_area / (float)contourArea(contours[0]);
- stat->num_inflexion_points = (float)num_inflexion_points;
-
-
- // calculate P(child|character) and filter if possible
- if ( (classifier != NULL) && (stat->parent != NULL) )
- {
- stat->probability = classifier->eval(*stat);
- }
-
- if ( ( ((classifier != NULL)?(stat->probability >= minProbability):true) &&
- ((stat->area >= minArea*region_mask.rows*region_mask.cols) &&
- (stat->area <= maxArea*region_mask.rows*region_mask.cols)) ) ||
- (stat->parent == NULL) )
- {
-
- num_accepted_regions++;
- regions->push_back(*stat);
-
- regions->back().parent = parent;
- regions->back().next = NULL;
- regions->back().child = NULL;
-
- if (prev != NULL)
- prev->next = &(regions->back());
- else if (parent != NULL)
- parent->child = &(regions->back());
-
- ERStat *old_prev = NULL;
- ERStat *this_er = ®ions->back();
-
- for (ERStat * child = stat->child; child; child = child->next)
- {
- old_prev = er_tree_filter(image, child, this_er, old_prev);
- }
-
- return this_er;
-
- } else {
-
- num_rejected_regions++;
-
- ERStat *old_prev = prev;
-
- for (ERStat * child = stat->child; child; child = child->next)
- {
- old_prev = er_tree_filter(image, child, parent, old_prev);
- }
-
- return old_prev;
- }
-
-}
-
-// recursively walk the tree selecting only regions with local maxima probability
-ERStat* ERFilterNM::er_tree_nonmax_suppression ( ERStat * stat, ERStat *parent, ERStat *prev )
-{
-
- if ( ( stat->local_maxima ) || ( stat->parent == NULL ) )
- {
-
- regions->push_back(*stat);
-
- regions->back().parent = parent;
- regions->back().next = NULL;
- regions->back().child = NULL;
-
- if (prev != NULL)
- prev->next = &(regions->back());
- else if (parent != NULL)
- parent->child = &(regions->back());
-
- ERStat *old_prev = NULL;
- ERStat *this_er = ®ions->back();
-
- for (ERStat * child = stat->child; child; child = child->next)
- {
- old_prev = er_tree_nonmax_suppression( child, this_er, old_prev );
- }
-
- return this_er;
-
- } else {
-
- num_rejected_regions++;
- num_accepted_regions--;
-
- ERStat *old_prev = prev;
-
- for (ERStat * child = stat->child; child; child = child->next)
- {
- old_prev = er_tree_nonmax_suppression( child, parent, old_prev );
- }
-
- return old_prev;
- }
-
-}
-
-void ERFilterNM::setCallback(const Ptr<ERFilter::Callback>& cb)
-{
- classifier = cb;
-}
-
-void ERFilterNM::setMinArea(float _minArea)
-{
- CV_Assert( (_minArea >= 0) && (_minArea < maxArea) );
- minArea = _minArea;
- return;
-}
-
-void ERFilterNM::setMaxArea(float _maxArea)
-{
- CV_Assert(_maxArea <= 1);
- CV_Assert(minArea < _maxArea);
- maxArea = _maxArea;
- return;
-}
-
-void ERFilterNM::setThresholdDelta(int _thresholdDelta)
-{
- CV_Assert( (_thresholdDelta > 0) && (_thresholdDelta <= 128) );
- thresholdDelta = _thresholdDelta;
- return;
-}
-
-void ERFilterNM::setMinProbability(float _minProbability)
-{
- CV_Assert( (_minProbability >= 0.0) && (_minProbability <= 1.0) );
- minProbability = _minProbability;
- return;
-}
-
-void ERFilterNM::setMinProbabilityDiff(float _minProbabilityDiff)
-{
- CV_Assert( (_minProbabilityDiff >= 0.0) && (_minProbabilityDiff <= 1.0) );
- minProbabilityDiff = _minProbabilityDiff;
- return;
-}
-
-void ERFilterNM::setNonMaxSuppression(bool _nonMaxSuppression)
-{
- nonMaxSuppression = _nonMaxSuppression;
- return;
-}
-
-int ERFilterNM::getNumRejected()
-{
- return num_rejected_regions;
-}
-
-
-
-
-// load default 1st stage classifier if found
-ERClassifierNM1::ERClassifierNM1(const std::string& filename)
-{
-
- if (ifstream(filename.c_str()))
- boost.load( filename.c_str(), "boost" );
- else
- CV_Error(CV_StsBadArg, "Default classifier file not found!");
-}
-
-double ERClassifierNM1::eval(const ERStat& stat)
-{
- //Classify
- float arr[] = {0,(float)(stat.rect.width)/(stat.rect.height), // aspect ratio
- sqrt((float)(stat.area))/stat.perimeter, // compactness
- (float)(1-stat.euler), //number of holes
- stat.med_crossings};
-
- vector<float> sample (arr, arr + sizeof(arr) / sizeof(arr[0]) );
-
- float votes = boost.predict( Mat(sample), Mat(), Range::all(), false, true );
-
- // Logistic Correction returns a probability value (in the range(0,1))
- return (double)1-(double)1/(1+exp(-2*votes));
-}
-
-
-// load default 2nd stage classifier if found
-ERClassifierNM2::ERClassifierNM2(const std::string& filename)
-{
- if (ifstream(filename.c_str()))
- boost.load( filename.c_str(), "boost" );
- else
- CV_Error(CV_StsBadArg, "Default classifier file not found!");
-}
-
-double ERClassifierNM2::eval(const ERStat& stat)
-{
- //Classify
- float arr[] = {0,(float)(stat.rect.width)/(stat.rect.height), // aspect ratio
- sqrt((float)(stat.area))/stat.perimeter, // compactness
- (float)(1-stat.euler), //number of holes
- stat.med_crossings, stat.hole_area_ratio,
- stat.convex_hull_ratio, stat.num_inflexion_points};
-
- vector<float> sample (arr, arr + sizeof(arr) / sizeof(arr[0]) );
-
- float votes = boost.predict( Mat(sample), Mat(), Range::all(), false, true );
-
- // Logistic Correction returns a probability value (in the range(0,1))
- return (double)1-(double)1/(1+exp(-2*votes));
-}
-
-
-/*!
- Create an Extremal Region Filter for the 1st stage classifier of N&M algorithm
- Neumann L., Matas J.: Real-Time Scene Text Localization and Recognition, CVPR 2012
-
- The component tree of the image is extracted by a threshold increased step by step
- from 0 to 255, incrementally computable descriptors (aspect_ratio, compactness,
- number of holes, and number of horizontal crossings) are computed for each ER
- and used as features for a classifier which estimates the class-conditional
- probability P(er|character). The value of P(er|character) is tracked using the inclusion
- relation of ER across all thresholds and only the ERs which correspond to local maximum
- of the probability P(er|character) are selected (if the local maximum of the
- probability is above a global limit pmin and the difference between local maximum and
- local minimum is greater than minProbabilityDiff).
-
- \param cb Callback with the classifier.
- default classifier can be implicitly load with function loadClassifierNM1()
- from file in samples/cpp/trained_classifierNM1.xml
- \param thresholdDelta Threshold step in subsequent thresholds when extracting the component tree
- \param minArea The minimum area (% of image size) allowed for retreived ER's
- \param minArea The maximum area (% of image size) allowed for retreived ER's
- \param minProbability The minimum probability P(er|character) allowed for retreived ER's
- \param nonMaxSuppression Whenever non-maximum suppression is done over the branch probabilities
- \param minProbability The minimum probability difference between local maxima and local minima ERs
-*/
-Ptr<ERFilter> createERFilterNM1(const Ptr<ERFilter::Callback>& cb, int thresholdDelta,
- float minArea, float maxArea, float minProbability,
- bool nonMaxSuppression, float minProbabilityDiff)
-{
-
- CV_Assert( (minProbability >= 0.) && (minProbability <= 1.) );
- CV_Assert( (minArea < maxArea) && (minArea >=0.) && (maxArea <= 1.) );
- CV_Assert( (thresholdDelta >= 0) && (thresholdDelta <= 128) );
- CV_Assert( (minProbabilityDiff >= 0.) && (minProbabilityDiff <= 1.) );
-
- Ptr<ERFilterNM> filter = makePtr<ERFilterNM>();
-
- filter->setCallback(cb);
-
- filter->setThresholdDelta(thresholdDelta);
- filter->setMinArea(minArea);
- filter->setMaxArea(maxArea);
- filter->setMinProbability(minProbability);
- filter->setNonMaxSuppression(nonMaxSuppression);
- filter->setMinProbabilityDiff(minProbabilityDiff);
- return (Ptr<ERFilter>)filter;
-}
-
-/*!
- Create an Extremal Region Filter for the 2nd stage classifier of N&M algorithm
- Neumann L., Matas J.: Real-Time Scene Text Localization and Recognition, CVPR 2012
-
- In the second stage, the ERs that passed the first stage are classified into character
- and non-character classes using more informative but also more computationally expensive
- features. The classifier uses all the features calculated in the first stage and the following
- additional features: hole area ratio, convex hull ratio, and number of outer inflexion points.
-
- \param cb Callback with the classifier
- default classifier can be implicitly load with function loadClassifierNM1()
- from file in samples/cpp/trained_classifierNM2.xml
- \param minProbability The minimum probability P(er|character) allowed for retreived ER's
-*/
-Ptr<ERFilter> createERFilterNM2(const Ptr<ERFilter::Callback>& cb, float minProbability)
-{
-
- CV_Assert( (minProbability >= 0.) && (minProbability <= 1.) );
-
- Ptr<ERFilterNM> filter = makePtr<ERFilterNM>();
-
- filter->setCallback(cb);
-
- filter->setMinProbability(minProbability);
- return (Ptr<ERFilter>)filter;
-}
-
-/*!
- Allow to implicitly load the default classifier when creating an ERFilter object.
- The function takes as parameter the XML or YAML file with the classifier model
- (e.g. trained_classifierNM1.xml) returns a pointer to ERFilter::Callback.
-*/
-Ptr<ERFilter::Callback> loadClassifierNM1(const std::string& filename)
-
-{
- return makePtr<ERClassifierNM1>(filename);
-}
-
-/*!
- Allow to implicitly load the default classifier when creating an ERFilter object.
- The function takes as parameter the XML or YAML file with the classifier model
- (e.g. trained_classifierNM2.xml) returns a pointer to ERFilter::Callback.
-*/
-Ptr<ERFilter::Callback> loadClassifierNM2(const std::string& filename)
-{
- return makePtr<ERClassifierNM2>(filename);
-}
-
-
-/* ------------------------------------------------------------------------------------*/
-/* -------------------------------- Compute Channels NM -------------------------------*/
-/* ------------------------------------------------------------------------------------*/
-
-
-void get_gradient_magnitude(Mat& _grey_img, Mat& _gradient_magnitude);
-
-void get_gradient_magnitude(Mat& _grey_img, Mat& _gradient_magnitude)
-{
- Mat C = Mat_<float>(_grey_img);
-
- Mat kernel = (Mat_<float>(1,3) << -1,0,1);
- Mat grad_x;
- filter2D(C, grad_x, -1, kernel, Point(-1,-1), 0, BORDER_DEFAULT);
-
- Mat kernel2 = (Mat_<float>(3,1) << -1,0,1);
- Mat grad_y;
- filter2D(C, grad_y, -1, kernel2, Point(-1,-1), 0, BORDER_DEFAULT);
-
- magnitude( grad_x, grad_y, _gradient_magnitude);
-}
-
-
-/*!
- Compute the diferent channels to be processed independently in the N&M algorithm
- Neumann L., Matas J.: Real-Time Scene Text Localization and Recognition, CVPR 2012
-
- In N&M algorithm, the combination of intensity (I), hue (H), saturation (S), and gradient
- magnitude channels (Grad) are used in order to obatin high localization recall.
- This implementation also the alternative combination of red (R), grren (G), blue (B),
- lightness (L), and gradient magnitude (Grad).
-
- \param _src Source image. Must be RGB CV_8UC3.
- \param _channels Output vector<Mat> where computed channels are stored.
- \param _mode Mode of operation. Currently the only available options are
- ERFILTER_NM_RGBLGrad and ERFILTER_NM_IHSGrad.
-
-*/
-void computeNMChannels(InputArray _src, OutputArrayOfArrays _channels, int _mode)
-{
-
- CV_Assert( ( _mode == ERFILTER_NM_RGBLGrad ) || ( _mode == ERFILTER_NM_IHSGrad ) );
-
- Mat src = _src.getMat();
- if( src.empty() )
- {
- _channels.release();
- return;
- }
-
- // assert RGB image
- CV_Assert(src.type() == CV_8UC3);
-
- if (_mode == ERFILTER_NM_IHSGrad)
- {
- _channels.create( 4, 1, src.depth());
-
- Mat hsv;
- cvtColor(src, hsv, COLOR_RGB2HSV);
- vector<Mat> channelsHSV;
- split(hsv, channelsHSV);
-
- for (int i = 0; i < src.channels(); i++)
- {
- _channels.create(src.rows, src.cols, CV_8UC1, i);
- Mat channel = _channels.getMat(i);
- channelsHSV.at(i).copyTo(channel);
- }
-
- Mat grey;
- cvtColor(src, grey, COLOR_RGB2GRAY);
- Mat gradient_magnitude = Mat_<float>(grey.size());
- get_gradient_magnitude( grey, gradient_magnitude);
- gradient_magnitude.convertTo(gradient_magnitude, CV_8UC1);
-
- _channels.create(src.rows, src.cols, CV_8UC1, 3);
- Mat channelGrad = _channels.getMat(3);
- gradient_magnitude.copyTo(channelGrad);
-
- } else if (_mode == ERFILTER_NM_RGBLGrad) {
-
- _channels.create( 5, 1, src.depth());
-
- vector<Mat> channelsRGB;
- split(src, channelsRGB);
- for (int i = 0; i < src.channels(); i++)
- {
- _channels.create(src.rows, src.cols, CV_8UC1, i);
- Mat channel = _channels.getMat(i);
- channelsRGB.at(i).copyTo(channel);
- }
-
- Mat hls;
- cvtColor(src, hls, COLOR_RGB2HLS);
- vector<Mat> channelsHLS;
- split(hls, channelsHLS);
-
- _channels.create(src.rows, src.cols, CV_8UC1, 3);
- Mat channelL = _channels.getMat(3);
- channelsHLS.at(1).copyTo(channelL);
-
- Mat grey;
- cvtColor(src, grey, COLOR_RGB2GRAY);
- Mat gradient_magnitude = Mat_<float>(grey.size());
- get_gradient_magnitude( grey, gradient_magnitude);
- gradient_magnitude.convertTo(gradient_magnitude, CV_8UC1);
-
- _channels.create(src.rows, src.cols, CV_8UC1, 4);
- Mat channelGrad = _channels.getMat(4);
- gradient_magnitude.copyTo(channelGrad);
- }
-}
-
-
-
-/* ------------------------------------------------------------------------------------*/
-/* -------------------------------- ER Grouping Algorithm -----------------------------*/
-/* ------------------------------------------------------------------------------------*/
-
-
-/* NFA approximation functions */
-
-// ln(10)
-#ifndef M_LN10
-#define M_LN10 2.30258509299404568401799145468436421
-#endif
-// Doubles relative error factor
-#define RELATIVE_ERROR_FACTOR 100.0
-
-// Compare doubles by relative error.
-static int double_equal(double a, double b)
-{
- double abs_diff,aa,bb,abs_max;
-
- /* trivial case */
- if( a == b ) return true;
-
- abs_diff = fabs(a-b);
- aa = fabs(a);
- bb = fabs(b);
- abs_max = aa > bb ? aa : bb;
-
- /* DBL_MIN is the smallest normalized number, thus, the smallest
- number whose relative error is bounded by DBL_EPSILON. For
- smaller numbers, the same quantization steps as for DBL_MIN
- are used. Then, for smaller numbers, a meaningful "relative"
- error should be computed by dividing the difference by DBL_MIN. */
- if( abs_max < DBL_MIN ) abs_max = DBL_MIN;
-
- /* equal if relative error <= factor x eps */
- return (abs_diff / abs_max) <= (RELATIVE_ERROR_FACTOR * DBL_EPSILON);
-}
-
-
-/*
- Computes the natural logarithm of the absolute value of
- the gamma function of x using the Lanczos approximation.
- See http://www.rskey.org/gamma.htm
-*/
-static double log_gamma_lanczos(double x)
-{
- static double q[7] = { 75122.6331530, 80916.6278952, 36308.2951477,
- 8687.24529705, 1168.92649479, 83.8676043424,
- 2.50662827511 };
- double a = (x+0.5) * log(x+5.5) - (x+5.5);
- double b = 0.0;
- int n;
-
- for(n=0;n<7;n++)
- {
- a -= log( x + (double) n );
- b += q[n] * pow( x, (double) n );
- }
- return a + log(b);
-}
-
-/*
- Computes the natural logarithm of the absolute value of
- the gamma function of x using Windschitl method.
- See http://www.rskey.org/gamma.htm
-*/
-static double log_gamma_windschitl(double x)
-{
- return 0.918938533204673 + (x-0.5)*log(x) - x
- + 0.5*x*log( x*sinh(1/x) + 1/(810.0*pow(x,6.0)) );
-}
-
-/*
- Computes the natural logarithm of the absolute value of
- the gamma function of x. When x>15 use log_gamma_windschitl(),
- otherwise use log_gamma_lanczos().
-*/
-#define log_gamma(x) ((x)>15.0?log_gamma_windschitl(x):log_gamma_lanczos(x))
-
-// Size of the table to store already computed inverse values.
-#define TABSIZE 100000
-
-/*
- Computes -log10(NFA).
- NFA stands for Number of False Alarms:
-*/
-static double NFA(int n, int k, double p, double logNT)
-{
- static double inv[TABSIZE]; /* table to keep computed inverse values */
- double tolerance = 0.1; /* an error of 10% in the result is accepted */
- double log1term,term,bin_term,mult_term,bin_tail,err,p_term;
- int i;
-
- if (p<=0)
- p=0.000000000000000000000000000001;
- if (p>=1)
- p=0.999999999999999999999999999999;
-
- /* check parameters */
- if( n<0 || k<0 || k>n || p<=0.0 || p>=1.0 )
- {
- CV_Error(CV_StsBadArg, "erGrouping wrong n, k or p values in NFA call!");
- }
-
- /* trivial cases */
- if( n==0 || k==0 ) return -logNT;
- if( n==k ) return -logNT - (double) n * log10(p);
-
- /* probability term */
- p_term = p / (1.0-p);
-
- /* compute the first term of the series */
- log1term = log_gamma( (double) n + 1.0 ) - log_gamma( (double) k + 1.0 )
- - log_gamma( (double) (n-k) + 1.0 )
- + (double) k * log(p) + (double) (n-k) * log(1.0-p);
- term = exp(log1term);
-
- /* in some cases no more computations are needed */
- if( double_equal(term,0.0) ) /* the first term is almost zero */
- {
- if( (double) k > (double) n * p ) /* at begin or end of the tail? */
- return -log1term / M_LN10 - logNT; /* end: use just the first term */
- else
- return -logNT; /* begin: the tail is roughly 1 */
- }
-
- /* compute more terms if needed */
- bin_tail = term;
- for(i=k+1;i<=n;i++)
- {
- bin_term = (double) (n-i+1) * ( i<TABSIZE ?
- ( inv[i]!=0.0 ? inv[i] : ( inv[i] = 1.0 / (double) i ) ) :
- 1.0 / (double) i );
-
- mult_term = bin_term * p_term;
- term *= mult_term;
- bin_tail += term;
- if(bin_term<1.0)
- {
- err = term * ( ( 1.0 - pow( mult_term, (double) (n-i+1) ) ) /
- (1.0-mult_term) - 1.0 );
- if( err < tolerance * fabs(-log10(bin_tail)-logNT) * bin_tail ) break;
- }
- }
- return -log10(bin_tail) - logNT;
-}
-
-
-// Minibox : smallest enclosing box of a set of n points in d dimensions
-
-class Minibox {
-private:
- vector<float> edge_begin;
- vector<float> edge_end;
- bool initialized;
-
-public:
- // creates an empty box
- Minibox();
-
- // copies p to the internal point set
- void check_in (vector<float> *p);
-
- // returns the volume of the box
- long double volume();
-};
-
-Minibox::Minibox()
-{
- initialized = false;
-}
-
-void Minibox::check_in (vector<float> *p)
-{
- if(!initialized) for (int i=0; i<(int)p->size(); i++)
- {
- edge_begin.push_back(p->at(i));
- edge_end.push_back(p->at(i)+0.00000000000000001f);
- initialized = true;
- }
- else for (int i=0; i<(int)p->size(); i++)
- {
- edge_begin.at(i) = min(p->at(i),edge_begin.at(i));
- edge_end.at(i) = max(p->at(i),edge_end.at(i));
- }
-}
-
-long double Minibox::volume ()
-{
- long double volume_ = 1;
- for (int i=0; i<(int)edge_begin.size(); i++)
- {
- volume_ = volume_ * (edge_end.at(i) - edge_begin.at(i));
- }
- return (volume_);
-}
-
-
-#define MAX_NUM_FEATURES 9
-
-
-/* Hierarchical Clustering classes and functions */
-
-
-// Hierarchical Clustering linkage variants
-enum method_codes
-{
- METHOD_METR_SINGLE = 0,
- METHOD_METR_AVERAGE = 1
-};
-
-#ifndef INT32_MAX
-#define MAX_INDEX 0x7fffffffL
-#else
-#define MAX_INDEX INT32_MAX
-#endif
-
-// A node in the hierarchical clustering algorithm
-struct node {
- int_fast32_t node1, node2;
- double dist;
-
- inline friend bool operator< (const node a, const node b)
- {
- // Numbers are always smaller than NaNs.
- return a.dist < b.dist || (a.dist==a.dist && b.dist!=b.dist);
- }
-};
-
-// self-destructing array pointer
-template <typename type>
-class auto_array_ptr {
-private:
- type * ptr;
-public:
- auto_array_ptr() { ptr = NULL; }
- template <typename index>
- auto_array_ptr(index const size) { init(size); }
- template <typename index, typename value>
- auto_array_ptr(index const size, value const val) { init(size, val); }
-
- ~auto_array_ptr()
- {
- delete [] ptr;
- }
- void free() {
- delete [] ptr;
- ptr = NULL;
- }
- template <typename index>
- void init(index const size)
- {
- ptr = new type [size];
- }
- template <typename index, typename value>
- void init(index const size, value const val)
- {
- init(size);
- for (index i=0; i<size; i++) ptr[i] = val;
- }
- inline operator type *() const { return ptr; }
-};
-
-// The result of the hierarchical clustering algorithm
-class cluster_result {
-private:
- auto_array_ptr<node> Z;
- int_fast32_t pos;
-
-public:
- cluster_result(const int_fast32_t size): Z(size)
- {
- pos = 0;
- }
-
- void append(const int_fast32_t node1, const int_fast32_t node2, const double dist)
- {
- Z[pos].node1 = node1;
- Z[pos].node2 = node2;
- Z[pos].dist = dist;
- pos++;
- }
-
- node * operator[] (const int_fast32_t idx) const { return Z + idx; }
-
- void sqrt() const
- {
- for (int_fast32_t i=0; i<pos; i++)
- Z[i].dist = ::sqrt(Z[i].dist);
- }
-
- void sqrt(const double) const // ignore the argument
- {
- sqrt();
- }
-};
-
-// Class for a doubly linked list
-class doubly_linked_list {
-public:
- int_fast32_t start;
- auto_array_ptr<int_fast32_t> succ;
-
-private:
- auto_array_ptr<int_fast32_t> pred;
-
-public:
- doubly_linked_list(const int_fast32_t size): succ(size+1), pred(size+1)
- {
- for (int_fast32_t i=0; i<size; i++)
- {
- pred[i+1] = i;
- succ[i] = i+1;
- }
- start = 0;
- }
-
- void remove(const int_fast32_t idx)
- {
- // Remove an index from the list.
- if (idx==start)
- {
- start = succ[idx];
- } else {
- succ[pred[idx]] = succ[idx];
- pred[succ[idx]] = pred[idx];
- }
- succ[idx] = 0; // Mark as inactive
- }
-
- bool is_inactive(int_fast32_t idx) const
- {
- return (succ[idx]==0);
- }
-};
-
-// Indexing functions
-// D is the upper triangular part of a symmetric (NxN)-matrix
-// We require r_ < c_ !
-#define D_(r_,c_) ( D[(static_cast<std::ptrdiff_t>(2*N-3-(r_))*(r_)>>1)+(c_)-1] )
-// Z is an ((N-1)x4)-array
-#define Z_(_r, _c) (Z[(_r)*4 + (_c)])
-
-/*
- Lookup function for a union-find data structure.
-
- The function finds the root of idx by going iteratively through all
- parent elements until a root is found. An element i is a root if
- nodes[i] is zero. To make subsequent searches faster, the entry for
- idx and all its parents is updated with the root element.
-*/
-class union_find {
-private:
- auto_array_ptr<int_fast32_t> parent;
- int_fast32_t nextparent;
-
-public:
- void init(const int_fast32_t size)
- {
- parent.init(2*size-1, 0);
- nextparent = size;
- }
-
- int_fast32_t Find (int_fast32_t idx) const
- {
- if (parent[idx] !=0 ) // a -> b
- {
- int_fast32_t p = idx;
- idx = parent[idx];
- if (parent[idx] !=0 ) // a -> b -> c
- {
- do
- {
- idx = parent[idx];
- } while (parent[idx] != 0);
- do
- {
- int_fast32_t tmp = parent[p];
- parent[p] = idx;
- p = tmp;
- } while (parent[p] != idx);
- }
- }
- return idx;
- }
-
- void Union (const int_fast32_t node1, const int_fast32_t node2)
- {
- parent[node1] = parent[node2] = nextparent++;
- }
-};
-
-
-/* Functions for the update of the dissimilarity array */
-
-inline static void f_single( double * const b, const double a )
-{
- if (*b > a) *b = a;
-}
-inline static void f_average( double * const b, const double a, const double s, const double t)
-{
- *b = s*a + t*(*b);
-}
-
-/*
- This is the NN-chain algorithm.
-
- N: integer
- D: condensed distance matrix N*(N-1)/2
- Z2: output data structure
-*/
-template <const unsigned char method, typename t_members>
-static void NN_chain_core(const int_fast32_t N, double * const D, t_members * const members, cluster_result & Z2)
-{
- int_fast32_t i;
-
- auto_array_ptr<int_fast32_t> NN_chain(N);
- int_fast32_t NN_chain_tip = 0;
-
- int_fast32_t idx1, idx2;
-
- double size1, size2;
- doubly_linked_list active_nodes(N);
-
- double min;
-
- for (int_fast32_t j=0; j<N-1; j++)
- {
- if (NN_chain_tip <= 3)
- {
- NN_chain[0] = idx1 = active_nodes.start;
- NN_chain_tip = 1;
-
- idx2 = active_nodes.succ[idx1];
- min = D_(idx1,idx2);
-
- for (i=active_nodes.succ[idx2]; i<N; i=active_nodes.succ[i])
- {
- if (D_(idx1,i) < min)
- {
- min = D_(idx1,i);
- idx2 = i;
- }
- }
- } // a: idx1 b: idx2
- else {
- NN_chain_tip -= 3;
- idx1 = NN_chain[NN_chain_tip-1];
- idx2 = NN_chain[NN_chain_tip];
- min = idx1<idx2 ? D_(idx1,idx2) : D_(idx2,idx1);
- } // a: idx1 b: idx2
-
- do {
- NN_chain[NN_chain_tip] = idx2;
-
- for (i=active_nodes.start; i<idx2; i=active_nodes.succ[i])
- {
- if (D_(i,idx2) < min)
- {
- min = D_(i,idx2);
- idx1 = i;
- }
- }
- for (i=active_nodes.succ[idx2]; i<N; i=active_nodes.succ[i])
- {
- if (D_(idx2,i) < min)
- {
- min = D_(idx2,i);
- idx1 = i;
- }
- }
-
- idx2 = idx1;
- idx1 = NN_chain[NN_chain_tip++];
-
- } while (idx2 != NN_chain[NN_chain_tip-2]);
-
- Z2.append(idx1, idx2, min);
-
- if (idx1>idx2)
- {
- int_fast32_t tmp = idx1;
- idx1 = idx2;
- idx2 = tmp;
- }
-
- //if ( method == METHOD_METR_AVERAGE )
- {
- size1 = static_cast<double>(members[idx1]);
- size2 = static_cast<double>(members[idx2]);
- members[idx2] += members[idx1];
- }
-
- // Remove the smaller index from the valid indices (active_nodes).
- active_nodes.remove(idx1);
-
- switch (method) {
- case METHOD_METR_SINGLE:
- /*
- Single linkage.
- */
- // Update the distance matrix in the range [start, idx1).
- for (i=active_nodes.start; i<idx1; i=active_nodes.succ[i])
- f_single(&D_(i, idx2), D_(i, idx1) );
- // Update the distance matrix in the range (idx1, idx2).
- for (; i<idx2; i=active_nodes.succ[i])
- f_single(&D_(i, idx2), D_(idx1, i) );
- // Update the distance matrix in the range (idx2, N).
- for (i=active_nodes.succ[idx2]; i<N; i=active_nodes.succ[i])
- f_single(&D_(idx2, i), D_(idx1, i) );
- break;
-
- case METHOD_METR_AVERAGE:
- {
- /*
- Average linkage.
- */
- // Update the distance matrix in the range [start, idx1).
- double s = size1/(size1+size2);
- double t = size2/(size1+size2);
- for (i=active_nodes.start; i<idx1; i=active_nodes.succ[i])
- f_average(&D_(i, idx2), D_(i, idx1), s, t );
- // Update the distance matrix in the range (idx1, idx2).
- for (; i<idx2; i=active_nodes.succ[i])
- f_average(&D_(i, idx2), D_(idx1, i), s, t );
- // Update the distance matrix in the range (idx2, N).
- for (i=active_nodes.succ[idx2]; i<N; i=active_nodes.succ[i])
- f_average(&D_(idx2, i), D_(idx1, i), s, t );
- break;
- }
- }
- }
-}
-
-
-/*
- Clustering methods for vector data
-*/
-
-template <typename t_dissimilarity>
-static void MST_linkage_core_vector(const int_fast32_t N,
- t_dissimilarity & dist,
- cluster_result & Z2) {
-/*
- Hierarchical clustering using the minimum spanning tree
-
- N: integer, number of data points
- dist: function pointer to the metric
- Z2: output data structure
-*/
- int_fast32_t i;
- int_fast32_t idx2;
- doubly_linked_list active_nodes(N);
- auto_array_ptr<double> d(N);
-
- int_fast32_t prev_node;
- double min;
-
- // first iteration
- idx2 = 1;
- min = d[1] = dist(0,1);
- for (i=2; min!=min && i<N; i++) // eliminate NaNs if possible
- {
- min = d[i] = dist(0,i);
- idx2 = i;
- }
-
- for ( ; i<N; i++)
- {
- d[i] = dist(0,i);
- if (d[i] < min)
- {
- min = d[i];
- idx2 = i;
- }
- }
-
- Z2.append(0, idx2, min);
-
- for (int_fast32_t j=1; j<N-1; j++)
- {
- prev_node = idx2;
- active_nodes.remove(prev_node);
-
- idx2 = active_nodes.succ[0];
- min = d[idx2];
-
- for (i=idx2; min!=min && i<N; i=active_nodes.succ[i]) // eliminate NaNs if possible
- {
- min = d[i] = dist(i, prev_node);
- idx2 = i;
- }
-
- for ( ; i<N; i=active_nodes.succ[i])
- {
- double tmp = dist(i, prev_node);
- if (d[i] > tmp)
- d[i] = tmp;
- if (d[i] < min)
- {
- min = d[i];
- idx2 = i;
- }
- }
- Z2.append(prev_node, idx2, min);
- }
-}
-
-class linkage_output {
-private:
- double * Z;
- int_fast32_t pos;
-
-public:
- linkage_output(double * const _Z)
- {
- this->Z = _Z;
- pos = 0;
- }
-
- void append(const int_fast32_t node1, const int_fast32_t node2, const double dist, const double size)
- {
- if (node1<node2)
- {
- Z[pos++] = static_cast<double>(node1);
- Z[pos++] = static_cast<double>(node2);
- } else {
- Z[pos++] = static_cast<double>(node2);
- Z[pos++] = static_cast<double>(node1);
- }
- Z[pos++] = dist;
- Z[pos++] = size;
- }
-};
-
-
-/*
- Generate the specific output format for a dendrogram from the
- clustering output.
-
- The list of merging steps can be sorted or unsorted.
-*/
-
-// The size of a node is either 1 (a single point) or is looked up from
-// one of the clusters.
-#define size_(r_) ( ((r_<N) ? 1 : Z_(r_-N,3)) )
-
-static void generate_dendrogram(double * const Z, cluster_result & Z2, const int_fast32_t N)
-{
- // The array "nodes" is a union-find data structure for the cluster
- // identites (only needed for unsorted cluster_result input).
- union_find nodes;
- std::stable_sort(Z2[0], Z2[N-1]);
- nodes.init(N);
-
- linkage_output output(Z);
- int_fast32_t node1, node2;
-
- for (int_fast32_t i=0; i<N-1; i++) {
- // Get two data points whose clusters are merged in step i.
- // Find the cluster identifiers for these points.
- node1 = nodes.Find(Z2[i]->node1);
- node2 = nodes.Find(Z2[i]->node2);
- // Merge the nodes in the union-find data structure by making them
- // children of a new node.
- nodes.Union(node1, node2);
- output.append(node1, node2, Z2[i]->dist, size_(node1)+size_(node2));
- }
-}
-
-/*
- Clustering on vector data
-*/
-
-enum {
- // metrics
- METRIC_EUCLIDEAN = 0,
- METRIC_CITYBLOCK = 1,
- METRIC_SEUCLIDEAN = 2,
- METRIC_SQEUCLIDEAN = 3
-};
-
-/*
- This class handles all the information about the dissimilarity
- computation.
-*/
-class dissimilarity {
-private:
- double * Xa;
- auto_array_ptr<double> Xnew;
- std::ptrdiff_t dim; // size_t saves many statis_cast<> in products
- int_fast32_t N;
- int_fast32_t * members;
- void (cluster_result::*postprocessfn) (const double) const;
- double postprocessarg;
-
- double (dissimilarity::*distfn) (const int_fast32_t, const int_fast32_t) const;
-
- auto_array_ptr<double> precomputed;
-
- double * V;
- const double * V_data;
-
-public:
- dissimilarity (double * const _Xa, int _Num, int _dim,
- int_fast32_t * const _members,
- const unsigned char method,
- const unsigned char metric,
- bool temp_point_array)
- : Xa(_Xa),
- dim(_dim),
- N(_Num),
- members(_members),
- postprocessfn(NULL),
- V(NULL)
- {
- switch (method) {
- case METHOD_METR_SINGLE: // only single linkage allowed here but others may come...
- default:
- postprocessfn = NULL; // default
- switch (metric)
- {
- case METRIC_EUCLIDEAN:
- set_euclidean();
- break;
- case METRIC_SEUCLIDEAN:
- case METRIC_SQEUCLIDEAN:
- distfn = &dissimilarity::sqeuclidean;
- break;
- case METRIC_CITYBLOCK:
- set_cityblock();
- break;
- }
- }
-
- if (temp_point_array)
- {
- Xnew.init((N-1)*dim);
- }
- }
-
- ~dissimilarity()
- {
- free(V);
- }
-
- inline double operator () (const int_fast32_t i, const int_fast32_t j) const
- {
- return (this->*distfn)(i,j);
- }
-
- inline double X (const int_fast32_t i, const int_fast32_t j) const
- {
- return Xa[i*dim+j];
- }
-
- inline bool Xb (const int_fast32_t i, const int_fast32_t j) const
- {
- return reinterpret_cast<bool *>(Xa)[i*dim+j];
- }
-
- inline double * Xptr(const int_fast32_t i, const int_fast32_t j) const
- {
- return Xa+i*dim+j;
- }
-
- void postprocess(cluster_result & Z2) const
- {
- if (postprocessfn!=NULL)
- {
- (Z2.*postprocessfn)(postprocessarg);
- }
- }
-
- double sqeuclidean(const int_fast32_t i, const int_fast32_t j) const
- {
- double sum = 0;
- double const * Pi = Xa+i*dim;
- double const * Pj = Xa+j*dim;
- for (int_fast32_t k=0; k<dim; k++)
- {
- double diff = Pi[k] - Pj[k];
- sum += diff*diff;
- }
- return sum;
- }
-
-private:
-
- void set_euclidean()
- {
- distfn = &dissimilarity::sqeuclidean;
- postprocessfn = &cluster_result::sqrt;
- }
-
- void set_cityblock()
- {
- distfn = &dissimilarity::cityblock;
- }
-
- double seuclidean(const int_fast32_t i, const int_fast32_t j) const
- {
- double sum = 0;
- for (int_fast32_t k=0; k<dim; k++)
- {
- double diff = X(i,k)-X(j,k);
- sum += diff*diff/V_data[k];
- }
- return sum;
- }
-
- double cityblock(const int_fast32_t i, const int_fast32_t j) const
- {
- double sum = 0;
- for (int_fast32_t k=0; k<dim; k++)
- {
- sum += fabs(X(i,k)-X(j,k));
- }
- return sum;
- }
-};
-
-/*Clustering for the "stored matrix approach": the input is the array of pairwise dissimilarities*/
-static int linkage(double *D, int N, double * Z)
-{
- CV_Assert(N >=1);
- CV_Assert(N <= MAX_INDEX/4);
-
- try
- {
-
- cluster_result Z2(N-1);
- auto_array_ptr<int_fast32_t> members;
- // The distance update formula needs the number of data points in a cluster.
- members.init(N, 1);
- NN_chain_core<METHOD_METR_AVERAGE, int_fast32_t>(N, D, members, Z2);
- generate_dendrogram(Z, Z2, N);
-
- } // try
- catch (const std::bad_alloc&)
- {
- CV_Error(CV_StsNoMem, "Not enough Memory for erGrouping hierarchical clustering structures!");
- }
- catch(const std::exception&)
- {
- CV_Error(CV_StsError, "Uncaught exception in erGrouping!");
- }
- catch(...)
- {
- CV_Error(CV_StsError, "C++ exception (unknown reason) in erGrouping!");
- }
- return 0;
-
-}
-
-/*Clustering for the "stored data approach": the input are points in a vector space.*/
-static int linkage_vector(double *X, int N, int dim, double * Z, unsigned char method, unsigned char metric)
-{
-
- CV_Assert(N >=1);
- CV_Assert(N <= MAX_INDEX/4);
- CV_Assert(dim >=1);
-
- try
- {
- cluster_result Z2(N-1);
- auto_array_ptr<int_fast32_t> members;
- dissimilarity dist(X, N, dim, members, method, metric, false);
- MST_linkage_core_vector(N, dist, Z2);
- dist.postprocess(Z2);
- generate_dendrogram(Z, Z2, N);
- } // try
- catch (const std::bad_alloc&)
- {
- CV_Error(CV_StsNoMem, "Not enough Memory for erGrouping hierarchical clustering structures!");
- }
- catch(const std::exception&)
- {
- CV_Error(CV_StsError, "Uncaught exception in erGrouping!");
- }
- catch(...)
- {
- CV_Error(CV_StsError, "C++ exception (unknown reason) in erGrouping!");
- }
- return 0;
-}
-
-
-/* Maximal Meaningful Clusters Detection */
-
-struct HCluster{
- int num_elem; // number of elements
- vector<int> elements; // elements (contour ID)
- int nfa; // the number of false alarms for this merge
- float dist; // distance of the merge
- float dist_ext; // distamce where this merge will merge with another
- long double volume; // volume of the bounding sphere (or bounding box)
- long double volume_ext; // volume of the sphere(or box) + envolvent empty space
- vector<vector<float> > points; // nD points in this cluster
- bool max_meaningful; // is this merge max meaningul ?
- vector<int> max_in_branch; // otherwise which merges are the max_meaningful in this branch
- int min_nfa_in_branch; // min nfa detected within the chilhood
- int node1;
- int node2;
-};
-
-class MaxMeaningfulClustering
-{
-public:
- unsigned char method_;
- unsigned char metric_;
-
- /// Constructor.
- MaxMeaningfulClustering(unsigned char method, unsigned char metric){ method_=method; metric_=metric; }
-
- void operator()(double *data, unsigned int num, int dim, unsigned char method,
- unsigned char metric, vector< vector<int> > *meaningful_clusters);
- void operator()(double *data, unsigned int num, unsigned char method,
- vector< vector<int> > *meaningful_clusters);
-
-private:
- /// Helper functions
- void build_merge_info(double *dendogram, double *data, int num, int dim, bool use_full_merge_rule,
- vector<HCluster> *merge_info, vector< vector<int> > *meaningful_clusters);
- void build_merge_info(double *dendogram, int num, vector<HCluster> *merge_info,
- vector< vector<int> > *meaningful_clusters);
-
- /// Number of False Alarms
- int nfa(float sigma, int k, int N);
-
-};
-
-void MaxMeaningfulClustering::operator()(double *data, unsigned int num, int dim, unsigned char method,
- unsigned char metric, vector< vector<int> > *meaningful_clusters)
-{
-
- double *Z = (double*)malloc(((num-1)*4) * sizeof(double)); // we need 4 floats foreach sample merge.
- if (Z == NULL)
- CV_Error(CV_StsNoMem, "Not enough Memory for erGrouping hierarchical clustering structures!");
-
- linkage_vector(data, (int)num, dim, Z, method, metric);
-
- vector<HCluster> merge_info;
- build_merge_info(Z, data, (int)num, dim, false, &merge_info, meaningful_clusters);
-
- free(Z);
- merge_info.clear();
-}
-
-void MaxMeaningfulClustering::operator()(double *data, unsigned int num, unsigned char method,
- vector< vector<int> > *meaningful_clusters)
-{
-
- CV_Assert(method == METHOD_METR_AVERAGE);
-
- double *Z = (double*)malloc(((num-1)*4) * sizeof(double)); // we need 4 floats foreach sample merge.
- if (Z == NULL)
- CV_Error(CV_StsNoMem, "Not enough Memory for erGrouping hierarchical clustering structures!");
-
- linkage(data, (int)num, Z);
-
- vector<HCluster> merge_info;
- build_merge_info(Z, (int)num, &merge_info, meaningful_clusters);
-
- free(Z);
- merge_info.clear();
-}
-
-void MaxMeaningfulClustering::build_merge_info(double *Z, double *X, int N, int dim,
- bool use_full_merge_rule,
- vector<HCluster> *merge_info,
- vector< vector<int> > *meaningful_clusters)
-{
-
- // walk the whole dendogram
- for (int i=0; i<(N-1)*4; i=i+4)
- {
- HCluster cluster;
- cluster.num_elem = (int)Z[i+3]; //number of elements
-
- int node1 = (int)Z[i];
- int node2 = (int)Z[i+1];
- float dist = (float)Z[i+2];
-
- if (node1<N)
- {
- vector<float> point;
- for (int n=0; n<dim; n++)
- point.push_back((float)X[node1*dim+n]);
- cluster.points.push_back(point);
- cluster.elements.push_back((int)node1);
- }
- else
- {
- for (int ii=0; ii<(int)merge_info->at(node1-N).points.size(); ii++)
- {
- cluster.points.push_back(merge_info->at(node1-N).points[ii]);
- cluster.elements.push_back(merge_info->at(node1-N).elements[ii]);
- }
- //update the extended volume of node1 using the dist where this cluster merge with another
- merge_info->at(node1-N).dist_ext = dist;
- }
- if (node2<N)
- {
- vector<float> point;
- for (int n=0; n<dim; n++)
- point.push_back((float)X[node2*dim+n]);
- cluster.points.push_back(point);
- cluster.elements.push_back((int)node2);
- }
- else
- {
- for (int ii=0; ii<(int)merge_info->at(node2-N).points.size(); ii++)
- {
- cluster.points.push_back(merge_info->at(node2-N).points[ii]);
- cluster.elements.push_back(merge_info->at(node2-N).elements[ii]);
- }
-
- //update the extended volume of node2 using the dist where this cluster merge with another
- merge_info->at(node2-N).dist_ext = dist;
- }
-
- Minibox mb;
- for (int ii=0; ii<(int)cluster.points.size(); ii++)
- {
- mb.check_in(&cluster.points.at(ii));
- }
-
- cluster.dist = dist;
- cluster.volume = mb.volume();
- if (cluster.volume >= 1)
- cluster.volume = 0.999999;
- if (cluster.volume == 0)
- cluster.volume = 0.001;
-
- cluster.volume_ext=1;
-
- if (node1>=N)
- {
- merge_info->at(node1-N).volume_ext = cluster.volume;
- }
- if (node2>=N)
- {
- merge_info->at(node2-N).volume_ext = cluster.volume;
- }
-
- cluster.node1 = node1;
- cluster.node2 = node2;
-
- merge_info->push_back(cluster);
-
- }
-
- for (int i=0; i<(int)merge_info->size(); i++)
- {
-
- merge_info->at(i).nfa = nfa((float)merge_info->at(i).volume,
- merge_info->at(i).num_elem, N);
- int node1 = merge_info->at(i).node1;
- int node2 = merge_info->at(i).node2;
-
- if ((node1<N)&&(node2<N))
- {
- // both nodes are individual samples (nfa=1) : each cluster is max.
- merge_info->at(i).max_meaningful = true;
- merge_info->at(i).max_in_branch.push_back(i);
- merge_info->at(i).min_nfa_in_branch = merge_info->at(i).nfa;
- } else {
- if ((node1>=N)&&(node2>=N))
- {
- // both nodes are "sets" : we must evaluate the merging condition
- if ( ( (use_full_merge_rule) &&
- ((merge_info->at(i).nfa < merge_info->at(node1-N).nfa + merge_info->at(node2-N).nfa) &&
- (merge_info->at(i).nfa < min(merge_info->at(node1-N).min_nfa_in_branch,
- merge_info->at(node2-N).min_nfa_in_branch))) ) ||
- ( (!use_full_merge_rule) &&
- ((merge_info->at(i).nfa < min(merge_info->at(node1-N).min_nfa_in_branch,
- merge_info->at(node2-N).min_nfa_in_branch))) ) )
- {
- merge_info->at(i).max_meaningful = true;
- merge_info->at(i).max_in_branch.push_back(i);
- merge_info->at(i).min_nfa_in_branch = merge_info->at(i).nfa;
- for (int k =0; k<(int)merge_info->at(node1-N).max_in_branch.size(); k++)
- merge_info->at(merge_info->at(node1-N).max_in_branch.at(k)).max_meaningful = false;
- for (int k =0; k<(int)merge_info->at(node2-N).max_in_branch.size(); k++)
- merge_info->at(merge_info->at(node2-N).max_in_branch.at(k)).max_meaningful = false;
- } else {
- merge_info->at(i).max_meaningful = false;
- merge_info->at(i).max_in_branch.insert(merge_info->at(i).max_in_branch.end(),
- merge_info->at(node1-N).max_in_branch.begin(),
- merge_info->at(node1-N).max_in_branch.end());
- merge_info->at(i).max_in_branch.insert(merge_info->at(i).max_in_branch.end(),
- merge_info->at(node2-N).max_in_branch.begin(),
- merge_info->at(node2-N).max_in_branch.end());
-
- if (merge_info->at(i).nfa < min(merge_info->at(node1-N).min_nfa_in_branch,
- merge_info->at(node2-N).min_nfa_in_branch))
-
- merge_info->at(i).min_nfa_in_branch = merge_info->at(i).nfa;
- else
- merge_info->at(i).min_nfa_in_branch = min(merge_info->at(node1-N).min_nfa_in_branch,
- merge_info->at(node2-N).min_nfa_in_branch);
- }
- } else {
-
- //one of the nodes is a "set" and the other is an individual sample : check merging condition
- if (node1>=N)
- {
- if ((merge_info->at(i).nfa < merge_info->at(node1-N).nfa + 1) &&
- (merge_info->at(i).nfa<merge_info->at(node1-N).min_nfa_in_branch))
- {
- merge_info->at(i).max_meaningful = true;
- merge_info->at(i).max_in_branch.push_back(i);
- merge_info->at(i).min_nfa_in_branch = merge_info->at(i).nfa;
- for (int k =0; k<(int)merge_info->at(node1-N).max_in_branch.size(); k++)
- merge_info->at(merge_info->at(node1-N).max_in_branch.at(k)).max_meaningful = false;
- } else {
- merge_info->at(i).max_meaningful = false;
- merge_info->at(i).max_in_branch.insert(merge_info->at(i).max_in_branch.end(),
- merge_info->at(node1-N).max_in_branch.begin(),
- merge_info->at(node1-N).max_in_branch.end());
- merge_info->at(i).min_nfa_in_branch = min(merge_info->at(i).nfa,
- merge_info->at(node1-N).min_nfa_in_branch);
- }
- } else {
- if ((merge_info->at(i).nfa < merge_info->at(node2-N).nfa + 1) &&
- (merge_info->at(i).nfa<merge_info->at(node2-N).min_nfa_in_branch))
- {
- merge_info->at(i).max_meaningful = true;
- merge_info->at(i).max_in_branch.push_back(i);
- merge_info->at(i).min_nfa_in_branch = merge_info->at(i).nfa;
- for (int k =0; k<(int)merge_info->at(node2-N).max_in_branch.size(); k++)
- merge_info->at(merge_info->at(node2-N).max_in_branch.at(k)).max_meaningful = false;
- } else {
- merge_info->at(i).max_meaningful = false;
- merge_info->at(i).max_in_branch.insert(merge_info->at(i).max_in_branch.end(),
- merge_info->at(node2-N).max_in_branch.begin(),
- merge_info->at(node2-N).max_in_branch.end());
- merge_info->at(i).min_nfa_in_branch = min(merge_info->at(i).nfa,
- merge_info->at(node2-N).min_nfa_in_branch);
- }
- }
- }
- }
- }
-
- for (int i=0; i<(int)merge_info->size(); i++)
- {
- if (merge_info->at(i).max_meaningful)
- {
- vector<int> cluster;
- for (int k=0; k<(int)merge_info->at(i).elements.size();k++)
- cluster.push_back(merge_info->at(i).elements.at(k));
- meaningful_clusters->push_back(cluster);
- }
- }
-
-}
-
-void MaxMeaningfulClustering::build_merge_info(double *Z, int N, vector<HCluster> *merge_info,
- vector< vector<int> > *meaningful_clusters)
-{
-
- // walk the whole dendogram
- for (int i=0; i<(N-1)*4; i=i+4)
- {
- HCluster cluster;
- cluster.num_elem = (int)Z[i+3]; //number of elements
-
- int node1 = (int)Z[i];
- int node2 = (int)Z[i+1];
- float dist = (float)Z[i+2];
- if (dist != dist) //this is to avoid NaN values
- dist=0;
-
- if (node1<N)
- {
- cluster.elements.push_back((int)node1);
- }
- else
- {
- for (int ii=0; ii<(int)merge_info->at(node1-N).elements.size(); ii++)
- {
- cluster.elements.push_back(merge_info->at(node1-N).elements[ii]);
- }
- }
- if (node2<N)
- {
- cluster.elements.push_back((int)node2);
- }
- else
- {
- for (int ii=0; ii<(int)merge_info->at(node2-N).elements.size(); ii++)
- {
- cluster.elements.push_back(merge_info->at(node2-N).elements[ii]);
- }
- }
-
- cluster.dist = dist;
- if (cluster.dist >= 1)
- cluster.dist = 0.999999f;
- if (cluster.dist == 0)
- cluster.dist = 1.e-25f;
-
- cluster.dist_ext = 1;
-
- if (node1>=N)
- {
- merge_info->at(node1-N).dist_ext = cluster.dist;
- }
- if (node2>=N)
- {
- merge_info->at(node2-N).dist_ext = cluster.dist;
- }
-
- cluster.node1 = node1;
- cluster.node2 = node2;
-
- merge_info->push_back(cluster);
- }
-
- for (int i=0; i<(int)merge_info->size(); i++)
- {
-
- merge_info->at(i).nfa = nfa(merge_info->at(i).dist,
- merge_info->at(i).num_elem, N);
- int node1 = merge_info->at(i).node1;
- int node2 = merge_info->at(i).node2;
-
- if ((node1<N)&&(node2<N))
- {
- // both nodes are individual samples (nfa=1) so this cluster is max.
- merge_info->at(i).max_meaningful = true;
- merge_info->at(i).max_in_branch.push_back(i);
- merge_info->at(i).min_nfa_in_branch = merge_info->at(i).nfa;
- } else {
- if ((node1>=N)&&(node2>=N))
- {
- // both nodes are "sets" so we must evaluate the merging condition
- if ((merge_info->at(i).nfa < merge_info->at(node1-N).nfa + merge_info->at(node2-N).nfa) &&
- (merge_info->at(i).nfa < min(merge_info->at(node1-N).min_nfa_in_branch,
- merge_info->at(node2-N).min_nfa_in_branch)))
- {
- merge_info->at(i).max_meaningful = true;
- merge_info->at(i).max_in_branch.push_back(i);
- merge_info->at(i).min_nfa_in_branch = merge_info->at(i).nfa;
- for (int k =0; k<(int)merge_info->at(node1-N).max_in_branch.size(); k++)
- merge_info->at(merge_info->at(node1-N).max_in_branch.at(k)).max_meaningful = false;
- for (int k =0; k<(int)merge_info->at(node2-N).max_in_branch.size(); k++)
- merge_info->at(merge_info->at(node2-N).max_in_branch.at(k)).max_meaningful = false;
- } else {
- merge_info->at(i).max_meaningful = false;
- merge_info->at(i).max_in_branch.insert(merge_info->at(i).max_in_branch.end(),
- merge_info->at(node1-N).max_in_branch.begin(),
- merge_info->at(node1-N).max_in_branch.end());
- merge_info->at(i).max_in_branch.insert(merge_info->at(i).max_in_branch.end(),
- merge_info->at(node2-N).max_in_branch.begin(),
- merge_info->at(node2-N).max_in_branch.end());
- if (merge_info->at(i).nfa < min(merge_info->at(node1-N).min_nfa_in_branch,
- merge_info->at(node2-N).min_nfa_in_branch))
- merge_info->at(i).min_nfa_in_branch = merge_info->at(i).nfa;
- else
- merge_info->at(i).min_nfa_in_branch = min(merge_info->at(node1-N).min_nfa_in_branch,
- merge_info->at(node2-N).min_nfa_in_branch);
- }
-
- } else {
-
- // one node is a "set" and the other is an indivisual sample: check merging condition
- if (node1>=N)
- {
- if ((merge_info->at(i).nfa < merge_info->at(node1-N).nfa + 1) &&
- (merge_info->at(i).nfa<merge_info->at(node1-N).min_nfa_in_branch))
- {
- merge_info->at(i).max_meaningful = true;
- merge_info->at(i).max_in_branch.push_back(i);
- merge_info->at(i).min_nfa_in_branch = merge_info->at(i).nfa;
-
- for (int k =0; k<(int)merge_info->at(node1-N).max_in_branch.size(); k++)
- merge_info->at(merge_info->at(node1-N).max_in_branch.at(k)).max_meaningful = false;
-
- } else {
- merge_info->at(i).max_meaningful = false;
- merge_info->at(i).max_in_branch.insert(merge_info->at(i).max_in_branch.end(),
- merge_info->at(node1-N).max_in_branch.begin(),
- merge_info->at(node1-N).max_in_branch.end());
- merge_info->at(i).min_nfa_in_branch = min(merge_info->at(i).nfa,
- merge_info->at(node1-N).min_nfa_in_branch);
- }
- } else {
- if ((merge_info->at(i).nfa < merge_info->at(node2-N).nfa + 1) &&
- (merge_info->at(i).nfa<merge_info->at(node2-N).min_nfa_in_branch))
- {
- merge_info->at(i).max_meaningful = true;
- merge_info->at(i).max_in_branch.push_back(i);
- merge_info->at(i).min_nfa_in_branch = merge_info->at(i).nfa;
- for (int k =0; k<(int)merge_info->at(node2-N).max_in_branch.size(); k++)
- merge_info->at(merge_info->at(node2-N).max_in_branch.at(k)).max_meaningful = false;
- } else {
- merge_info->at(i).max_meaningful = false;
- merge_info->at(i).max_in_branch.insert(merge_info->at(i).max_in_branch.end(),
- merge_info->at(node2-N).max_in_branch.begin(),
- merge_info->at(node2-N).max_in_branch.end());
- merge_info->at(i).min_nfa_in_branch = min(merge_info->at(i).nfa,
- merge_info->at(node2-N).min_nfa_in_branch);
- }
- }
- }
- }
- }
-
- for (int i=0; i<(int)merge_info->size(); i++)
- {
- if (merge_info->at(i).max_meaningful)
- {
- vector<int> cluster;
- for (int k=0; k<(int)merge_info->at(i).elements.size();k++)
- cluster.push_back(merge_info->at(i).elements.at(k));
- meaningful_clusters->push_back(cluster);
- }
- }
-
-}
-
-int MaxMeaningfulClustering::nfa(float sigma, int k, int N)
-{
- // use an approximation for the nfa calculations (faster)
- return -1*(int)NFA( N, k, (double) sigma, 0);
-}
-
-void accumulate_evidence(vector<vector<int> > *meaningful_clusters, int grow, Mat *co_occurrence);
-
-void accumulate_evidence(vector<vector<int> > *meaningful_clusters, int grow, Mat *co_occurrence)
-{
- for (int k=0; k<(int)meaningful_clusters->size(); k++)
- for (int i=0; i<(int)meaningful_clusters->at(k).size(); i++)
- for (int j=i; j<(int)meaningful_clusters->at(k).size(); j++)
- if (meaningful_clusters->at(k).at(i) != meaningful_clusters->at(k).at(j))
- {
- co_occurrence->at<double>(meaningful_clusters->at(k).at(i), meaningful_clusters->at(k).at(j)) += grow;
- co_occurrence->at<double>(meaningful_clusters->at(k).at(j), meaningful_clusters->at(k).at(i)) += grow;
- }
-}
-
-// ERFeatures structure stores additional features for a given ERStat instance
-struct ERFeatures
-{
- int area;
- Point center;
- Rect rect;
- float intensity_mean; ///< mean intensity of the whole region
- float intensity_std; ///< intensity standard deviation of the whole region
- float boundary_intensity_mean; ///< mean intensity of the boundary of the region
- float boundary_intensity_std; ///< intensity standard deviation of the boundary of the region
- double stroke_mean; ///< mean stroke width approximation of the whole region
- double stroke_std; ///< stroke standard deviation of the whole region
- double gradient_mean; ///< mean gradient magnitude of the whole region
- double gradient_std; ///< gradient magnitude standard deviation of the whole region
-};
-
-float extract_features(InputOutputArray src, vector<ERStat> ®ions, vector<ERFeatures> &features);
-void ergrouping(InputOutputArray src, vector<ERStat> ®ions);
-
-float extract_features(InputOutputArray src, vector<ERStat> ®ions, vector<ERFeatures> &features)
-{
- // assert correct image type
- CV_Assert( (src.type() == CV_8UC1) || (src.type() == CV_8UC3) );
-
- CV_Assert( !regions.empty() );
- CV_Assert( features.empty() );
-
- Mat grey = src.getMat();
-
- Mat gradient_magnitude = Mat_<float>(grey.size());
- get_gradient_magnitude( grey, gradient_magnitude);
-
- Mat region_mask = Mat::zeros(grey.rows+2, grey.cols+2, CV_8UC1);
-
- float max_stroke = 0;
-
- for (int r=0; r<(int)regions.size(); r++)
- {
- ERFeatures f;
- ERStat *stat = ®ions.at(r);
-
- f.area = stat->area;
- f.rect = stat->rect;
- f.center = Point(f.rect.x+(f.rect.width/2),f.rect.y+(f.rect.height/2));
-
- if (regions.at(r).parent != NULL)
- {
-
- //Fill the region and calculate features
- Mat region = region_mask(Rect(Point(stat->rect.x,stat->rect.y),
- Point(stat->rect.br().x+2,stat->rect.br().y+2)));
- region = Scalar(0);
- int newMaskVal = 255;
- int flags = 4 + (newMaskVal << 8) + FLOODFILL_FIXED_RANGE + FLOODFILL_MASK_ONLY;
- Rect rect;
-
- floodFill( grey(Rect(Point(stat->rect.x,stat->rect.y),Point(stat->rect.br().x,stat->rect.br().y))),
- region, Point(stat->pixel%grey.cols - stat->rect.x, stat->pixel/grey.cols - stat->rect.y),
- Scalar(255), &rect, Scalar(stat->level), Scalar(0), flags );
- rect.width += 2;
- rect.height += 2;
- Mat rect_mask = region_mask(Rect(stat->rect.x+1,stat->rect.y+1,stat->rect.width,stat->rect.height));
-
-
- Scalar mean,std;
- meanStdDev( grey(stat->rect), mean, std, rect_mask);
- f.intensity_mean = (float)mean[0];
- f.intensity_std = (float)std[0];
-
- Mat tmp;
- distanceTransform(rect_mask, tmp, DIST_L1,3);
-
- meanStdDev(tmp,mean,std,rect_mask);
- f.stroke_mean = mean[0];
- f.stroke_std = std[0];
-
- if (f.stroke_mean > max_stroke)
- max_stroke = (float)f.stroke_mean;
-
- Mat element = getStructuringElement( MORPH_RECT, Size(5, 5), Point(2, 2) );
- dilate(rect_mask, tmp, element);
- absdiff(tmp, rect_mask, tmp);
-
- meanStdDev( grey(stat->rect), mean, std, tmp);
- f.boundary_intensity_mean = (float)mean[0];
- f.boundary_intensity_std = (float)std[0];
-
- Mat tmp2;
- dilate(rect_mask, tmp, element);
- erode (rect_mask, tmp2, element);
- absdiff(tmp, tmp2, tmp);
-
- meanStdDev( gradient_magnitude(stat->rect), mean, std, tmp);
- f.gradient_mean = mean[0];
- f.gradient_std = std[0];
-
- rect_mask = Scalar(0);
-
- } else {
-
- f.intensity_mean = 0;
- f.intensity_std = 0;
-
- f.stroke_mean = 0;
- f.stroke_std = 0;
-
- f.boundary_intensity_mean = 0;
- f.boundary_intensity_std = 0;
-
- f.gradient_mean = 0;
- f.gradient_std = 0;
- }
-
- features.push_back(f);
- }
-
- return max_stroke;
-}
-
-static bool edge_comp (Vec4f i,Vec4f j)
-{
- Point a = Point(cvRound(i[0]), cvRound(i[1]));
- Point b = Point(cvRound(i[2]), cvRound(i[3]));
- double edist_i = cv::norm(a-b);
- a = Point(cvRound(j[0]), cvRound(j[1]));
- b = Point(cvRound(j[2]), cvRound(j[3]));
- double edist_j = cv::norm(a-b);
- return (edist_i<edist_j);
-}
-
-static bool find_vertex(vector<Point> &vertex, Point &p)
-{
- for (int i=0; i<(int)vertex.size(); i++)
- {
- if (vertex.at(i) == p)
- return true;
- }
- return false;
-}
-
-
-/*!
- Find groups of Extremal Regions that are organized as text blocks. This function implements
- the grouping algorithm described in:
- Gomez L. and Karatzas D.: Multi-script Text Extraction from Natural Scenes, ICDAR 2013.
- Notice that this implementation constrains the results to horizontally-aligned text and
- latin script (since ERFilter default classifiers are trained only for latin script detection).
-
- The algorithm combines two different clustering techniques in a single parameter-free procedure
- to detect groups of regions organized as text. The maximally meaningful groups are fist detected
- in several feature spaces, where each feature space is a combination of proximity information
- (x,y coordinates) and a similarity measure (intensity, color, size, gradient magnitude, etc.),
- thus providing a set of hypotheses of text groups. Evidence Accumulation framework is used to
- combine all these hypotheses to get the final estimate. Each of the resulting groups are finally
- validated by a classifier in order to assest if they form a valid horizontally-aligned text block.
-
- \param src Vector of sinle channel images CV_8UC1 from wich the regions were extracted.
- \param regions Vector of ER's retreived from the ERFilter algorithm from each channel
- \param filename The XML or YAML file with the classifier model (e.g. trained_classifier_erGrouping.xml)
- \param minProbability The minimum probability for accepting a group
- \param groups The output of the algorithm are stored in this parameter as list of rectangles.
-*/
-void erGrouping(InputArrayOfArrays _src, vector<vector<ERStat> > ®ions, const std::string& filename, float minProbability, std::vector<Rect > &text_boxes)
-{
-
- // TODO assert correct vector<Mat>
-
- CvBoost group_boost;
- if (ifstream(filename.c_str()))
- group_boost.load( filename.c_str(), "boost" );
- else
- CV_Error(CV_StsBadArg, "erGrouping: Default classifier file not found!");
-
- std::vector<Mat> src;
- _src.getMatVector(src);
-
- CV_Assert ( !src.empty() );
- CV_Assert ( src.size() == regions.size() );
-
- if (!text_boxes.empty())
- {
- text_boxes.clear();
- }
-
- for (int c=0; c<(int)src.size(); c++)
- {
- Mat img = src.at(c);
-
- // assert correct image type
- CV_Assert( img.type() == CV_8UC1 );
-
- CV_Assert( !regions.at(c).empty() );
-
- if ( regions.at(c).size() < 3 )
- continue;
-
-
- std::vector<vector<int> > meaningful_clusters;
- vector<ERFeatures> features;
- float max_stroke = extract_features(img,regions.at(c),features);
-
- MaxMeaningfulClustering mm_clustering(METHOD_METR_SINGLE, METRIC_SEUCLIDEAN);
-
- Mat co_occurrence_matrix = Mat::zeros((int)regions.at(c).size(), (int)regions.at(c).size(), CV_64F);
-
- int num_features = MAX_NUM_FEATURES;
-
- // Find the Max. Meaningful Clusters in each feature space independently
- int dims[MAX_NUM_FEATURES] = {3,3,3,3,3,3,3,3,3};
-
- for (int f=0; f<num_features; f++)
- {
- unsigned int N = (unsigned int)regions.at(c).size();
- if (N<3) break;
- int dim = dims[f];
- double *data = (double*)malloc(dim*N * sizeof(double));
- if (data == NULL)
- CV_Error(CV_StsNoMem, "Not enough Memory for erGrouping hierarchical clustering structures!");
- int count = 0;
- for (int i=0; i<(int)regions.at(c).size(); i++)
- {
- data[count] = (double)features.at(i).center.x/img.cols;
- data[count+1] = (double)features.at(i).center.y/img.rows;
- switch (f)
- {
- case 0:
- data[count+2] = (double)features.at(i).intensity_mean/255;
- break;
- case 1:
- data[count+2] = (double)features.at(i).boundary_intensity_mean/255;
- break;
- case 2:
- data[count+2] = (double)features.at(i).rect.y/img.rows;
- break;
- case 3:
- data[count+2] = (double)(features.at(i).rect.y+features.at(i).rect.height)/img.rows;
- break;
- case 4:
- data[count+2] = (double)max(features.at(i).rect.height,
- features.at(i).rect.width)/max(img.rows,img.cols);
- break;
- case 5:
- data[count+2] = (double)features.at(i).stroke_mean/max_stroke;
- break;
- case 6:
- data[count+2] = (double)features.at(i).area/(img.rows*img.cols);
- break;
- case 7:
- data[count+2] = (double)(features.at(i).rect.height*
- features.at(i).rect.width)/(img.rows*img.cols);
- break;
- case 8:
- data[count+2] = (double)features.at(i).gradient_mean/255;
- break;
- }
- count = count+dim;
- }
-
- mm_clustering(data, N, dim, METHOD_METR_SINGLE, METRIC_SEUCLIDEAN, &meaningful_clusters);
-
- // Accumulate evidence in the coocurrence matrix
- accumulate_evidence(&meaningful_clusters, 1, &co_occurrence_matrix);
-
- free(data);
- meaningful_clusters.clear();
- }
-
- double minVal;
- double maxVal;
- minMaxLoc(co_occurrence_matrix, &minVal, &maxVal);
-
- maxVal = num_features - 1;
- minVal=0;
-
- co_occurrence_matrix = maxVal - co_occurrence_matrix;
- co_occurrence_matrix = co_occurrence_matrix / maxVal;
-
- // we want a sparse matrix
- double *D = (double*)malloc((regions.at(c).size()*regions.at(c).size()) * sizeof(double));
- if (D == NULL)
- CV_Error(CV_StsNoMem, "Not enough Memory for erGrouping hierarchical clustering structures!");
-
- int pos = 0;
- for (int i = 0; i<co_occurrence_matrix.rows; i++)
- {
- for (int j = i+1; j<co_occurrence_matrix.cols; j++)
- {
- D[pos] = (double)co_occurrence_matrix.at<double>(i, j);
- pos++;
- }
- }
-
- // Find the Max. Meaningful Clusters in the co-occurrence matrix
- mm_clustering(D, (unsigned int)regions.at(c).size(), METHOD_METR_AVERAGE, &meaningful_clusters);
- free(D);
-
-
-
- /* --------------------------------- Groups Validation --------------------------------*/
- /* Examine each of the clusters in order to assest if they are valid text lines or not */
- /* ------------------------------------------------------------------------------------*/
-
- vector<vector<float> > data_arrays(meaningful_clusters.size());
- vector<Rect> groups_rects(meaningful_clusters.size());
-
- // Collect group level features and classify the group
- for (int i=(int)meaningful_clusters.size()-1; i>=0; i--)
- {
-
- Rect group_rect;
- float sumx=0, sumy=0, sumxy=0, sumx2=0;
-
- // linear regression slope helps discriminating horizontal aligned groups
- for (int j=0; j<(int)meaningful_clusters.at(i).size();j++)
- {
- if (j==0)
- {
- group_rect = regions.at(c).at(meaningful_clusters.at(i).at(j)).rect;
- } else {
- group_rect = group_rect | regions.at(c).at(meaningful_clusters.at(i).at(j)).rect;
- }
-
- sumx += regions.at(c).at(meaningful_clusters.at(i).at(j)).rect.x +
- regions.at(c).at(meaningful_clusters.at(i).at(j)).rect.width/2;
- sumy += regions.at(c).at(meaningful_clusters.at(i).at(j)).rect.y +
- regions.at(c).at(meaningful_clusters.at(i).at(j)).rect.height/2;
- sumxy += (regions.at(c).at(meaningful_clusters.at(i).at(j)).rect.x +
- regions.at(c).at(meaningful_clusters.at(i).at(j)).rect.width/2)*
- (regions.at(c).at(meaningful_clusters.at(i).at(j)).rect.y +
- regions.at(c).at(meaningful_clusters.at(i).at(j)).rect.height/2);
- sumx2 += (regions.at(c).at(meaningful_clusters.at(i).at(j)).rect.x +
- regions.at(c).at(meaningful_clusters.at(i).at(j)).rect.width/2)*
- (regions.at(c).at(meaningful_clusters.at(i).at(j)).rect.x +
- regions.at(c).at(meaningful_clusters.at(i).at(j)).rect.width/2);
- }
- // line coefficients
- //float a0=(sumy*sumx2-sumx*sumxy)/((int)meaningful_clusters.at(i).size()*sumx2-sumx*sumx);
- float a1=((int)meaningful_clusters.at(i).size()*sumxy-sumx*sumy) /
- ((int)meaningful_clusters.at(i).size()*sumx2-sumx*sumx);
-
- if (a1 != a1)
- data_arrays.at(i).push_back(1.f);
- else
- data_arrays.at(i).push_back(a1);
-
- groups_rects.at(i) = group_rect;
-
- // group probability mean
- double group_probability_mean = 0;
- // number of non-overlapping regions
- vector<Rect> individual_components;
-
- // The variance of several similarity features is also helpful
- vector<float> strokes;
- vector<float> grad_magnitudes;
- vector<float> intensities;
- vector<float> bg_intensities;
-
- // We'll try to remove groups with repetitive patterns using averaged SAD
- // SAD = Sum of Absolute Differences
- Mat grey = img;
- Mat sad = Mat::zeros(regions.at(c).at(meaningful_clusters.at(i).at(0)).rect.size() , CV_8UC1);
- Mat region_mask = Mat::zeros(grey.rows+2, grey.cols+2, CV_8UC1);
- float sad_value = 0;
- Mat ratios = Mat::zeros(1, (int)meaningful_clusters.at(i).size(), CV_32FC1);
- //Mat holes = Mat::zeros(1, (int)meaningful_clusters.at(i).size(), CV_32FC1);
-
- for (int j=0; j<(int)meaningful_clusters.at(i).size();j++)
- {
- ERStat *stat = ®ions.at(c).at(meaningful_clusters.at(i).at(j));
-
- //Fill the region
- Mat region = region_mask(Rect(Point(stat->rect.x,stat->rect.y),
- Point(stat->rect.br().x+2,stat->rect.br().y+2)));
- region = Scalar(0);
- int newMaskVal = 255;
- int flags = 4 + (newMaskVal << 8) + FLOODFILL_FIXED_RANGE + FLOODFILL_MASK_ONLY;
- Rect rect;
-
- floodFill( grey(Rect(Point(stat->rect.x,stat->rect.y),Point(stat->rect.br().x,stat->rect.br().y))),
- region, Point(stat->pixel%grey.cols - stat->rect.x, stat->pixel/grey.cols - stat->rect.y),
- Scalar(255), &rect, Scalar(stat->level), Scalar(0), flags );
-
- Mat mask = Mat::zeros(regions.at(c).at(meaningful_clusters.at(i).at(0)).rect.size() , CV_8UC1);
- resize(region, mask, mask.size());
- mask = mask - 254;
- if (j!=0)
- {
- // accumulate Sum of Absolute Differences
- absdiff(sad, mask, sad);
- Scalar s = sum(sad);
- sad_value += (float)s[0]/(sad.rows*sad.cols);
- }
- mask.copyTo(sad);
- ratios.at<float>(0,j) = (float)min(stat->rect.width, stat->rect.height) /
- max(stat->rect.width, stat->rect.height);
- //holes.at<float>(0,j) = (float)stat->hole_area_ratio;
-
- strokes.push_back((float)features.at(meaningful_clusters.at(i).at(j)).stroke_mean);
- grad_magnitudes.push_back((float)features.at(meaningful_clusters.at(i).at(j)).gradient_mean);
- intensities.push_back(features.at(meaningful_clusters.at(i).at(j)).intensity_mean);
- bg_intensities.push_back(features.at(meaningful_clusters.at(i).at(j)).boundary_intensity_mean);
- group_probability_mean += regions.at(c).at(meaningful_clusters.at(i).at(j)).probability;
-
- if (j==0)
- {
- group_rect = features.at(meaningful_clusters.at(i).at(j)).rect;
- individual_components.push_back(group_rect);
- } else {
- bool matched = false;
- for (int k=0; k<(int)individual_components.size(); k++)
- {
- Rect intersection = individual_components.at(k) &
- features.at(meaningful_clusters.at(i).at(j)).rect;
-
- if ((intersection == features.at(meaningful_clusters.at(i).at(j)).rect) ||
- (intersection == individual_components.at(k)))
- {
- individual_components.at(k) = individual_components.at(k) |
- features.at(meaningful_clusters.at(i).at(j)).rect;
- matched = true;
- }
- }
-
- if (!matched)
- individual_components.push_back(features.at(meaningful_clusters.at(i).at(j)).rect);
-
- group_rect = group_rect | features.at(meaningful_clusters.at(i).at(j)).rect;
- }
- }
- group_probability_mean = group_probability_mean / meaningful_clusters.at(i).size();
-
- data_arrays.at(i).insert(data_arrays.at(i).begin(),(float)individual_components.size());
-
- // variance of widths and heights help to discriminate groups with high height variability
- vector<int> widths;
- vector<int> heights;
- // the MST edge orientations histogram may be dominated by the horizontal axis orientation
- Subdiv2D subdiv(Rect(0,0,src.at(0).cols,src.at(0).rows));
-
- for (int r=0; r < (int)individual_components.size(); r++)
- {
- widths.push_back(individual_components.at(r).width);
- heights.push_back(individual_components.at(r).height);
-
- Point2f fp( (float)individual_components.at(r).x + individual_components.at(r).width/2,
- (float)individual_components.at(r).y + individual_components.at(r).height/2 );
- subdiv.insert(fp);
- }
-
- Scalar mean, std;
- meanStdDev(Mat(widths), mean, std);
- data_arrays.at(i).push_back((float)(std[0]/mean[0]));
- data_arrays.at(i).push_back((float)mean[0]);
- meanStdDev(Mat(heights), mean, std);
- data_arrays.at(i).push_back((float)(std[0]/mean[0]));
-
- vector<Vec4f> edgeList;
- subdiv.getEdgeList(edgeList);
- std::sort (edgeList.begin(), edgeList.end(), edge_comp);
- vector<Point> mst_vertices;
-
- int horiz_edges = 0, non_horiz_edges = 0;
- vector<float> edge_distances;
-
- for( size_t k = 0; k < edgeList.size(); k++ )
- {
- Vec4f e = edgeList[k];
- Point pt0 = Point(cvRound(e[0]), cvRound(e[1]));
- Point pt1 = Point(cvRound(e[2]), cvRound(e[3]));
- if (((pt0.x>0)&&(pt0.x<src.at(0).cols)&&(pt0.y>0)&&(pt0.y<src.at(0).rows) &&
- (pt1.x>0)&&(pt1.x<src.at(0).cols)&&(pt1.y>0)&&(pt1.y<src.at(0).rows)) &&
- ((!find_vertex(mst_vertices,pt0)) ||
- (!find_vertex(mst_vertices,pt1))))
- {
- double angle = atan2((double)(pt0.y-pt1.y),(double)(pt0.x-pt1.x));
- //if ( (abs(angle) < 0.35) || (abs(angle) > 5.93) || ((abs(angle) > 2.79)&&(abs(angle) < 3.49)) )
- if ( (abs(angle) < 0.25) || (abs(angle) > 6.03) || ((abs(angle) > 2.88)&&(abs(angle) < 3.4)) )
- {
- horiz_edges++;
- edge_distances.push_back((float)norm(pt0-pt1));
- }
- else
- non_horiz_edges++;
- mst_vertices.push_back(pt0);
- mst_vertices.push_back(pt1);
- }
- }
-
- if (horiz_edges == 0)
- data_arrays.at(i).push_back(0.f);
- else
- data_arrays.at(i).push_back((float)horiz_edges/(horiz_edges+non_horiz_edges));
-
- // remove groups where objects are not equidistant enough
- Scalar dist_mean, dist_std;
- meanStdDev(Mat(edge_distances),dist_mean, dist_std);
- if (dist_std[0] == 0)
- data_arrays.at(i).push_back(0.f);
- else
- data_arrays.at(i).push_back((float)(dist_std[0]/dist_mean[0]));
-
- if (dist_mean[0] == 0)
- data_arrays.at(i).push_back(0.f);
- else
- data_arrays.at(i).push_back((float)dist_mean[0]/data_arrays.at(i).at(3));
-
- //meanStdDev( holes, mean, std);
- //float holes_mean = (float)mean[0];
- meanStdDev( ratios, mean, std);
-
- data_arrays.at(i).push_back((float)sad_value / ((int)meaningful_clusters.at(i).size()-1));
- meanStdDev( Mat(strokes), mean, std);
- data_arrays.at(i).push_back((float)(std[0]/mean[0]));
- meanStdDev( Mat(grad_magnitudes), mean, std);
- data_arrays.at(i).push_back((float)(std[0]/mean[0]));
- meanStdDev( Mat(intensities), mean, std);
- data_arrays.at(i).push_back((float)std[0]);
- meanStdDev( Mat(bg_intensities), mean, std);
- data_arrays.at(i).push_back((float)std[0]);
-
- // Validate only groups with more than 2 non-overlapping regions
- if (data_arrays.at(i).at(0) > 2)
- {
- data_arrays.at(i).insert(data_arrays.at(i).begin(),0.f);
- float votes = group_boost.predict( Mat(data_arrays.at(i)), Mat(), Range::all(), false, true );
- // Logistic Correction returns a probability value (in the range(0,1))
- double probability = (double)1-(double)1/(1+exp(-2*votes));
-
- if (probability > minProbability)
- text_boxes.push_back(groups_rects.at(i));
- }
- }
-
- }
-
- // check for colinear groups that can be merged
- for (int i=0; i<(int)text_boxes.size(); i++)
- {
- int ay1 = text_boxes.at(i).y;
- int ay2 = text_boxes.at(i).y + text_boxes.at(i).height;
- int ax1 = text_boxes.at(i).x;
- int ax2 = text_boxes.at(i).x + text_boxes.at(i).width;
- for (int j=(int)text_boxes.size()-1; j>i; j--)
- {
- int by1 = text_boxes.at(j).y;
- int by2 = text_boxes.at(j).y + text_boxes.at(j).height;
- int bx1 = text_boxes.at(j).x;
- int bx2 = text_boxes.at(j).x + text_boxes.at(j).width;
-
- int y_intersection = min(ay2,by2) - max(ay1,by1);
-
- if (y_intersection > 0.6*(max(text_boxes.at(i).height,text_boxes.at(j).height)))
- {
- int xdist = min(abs(ax2-bx1),abs(bx2-ax1));
- Rect intersection = text_boxes.at(i) & text_boxes.at(j);
- if ( (xdist < 0.75*(max(text_boxes.at(i).height,text_boxes.at(j).height))) ||
- (intersection.width != 0))
- {
- text_boxes.at(i) = text_boxes.at(i) | text_boxes.at(j);
- text_boxes.erase(text_boxes.begin()+j);
- }
- }
-
- }
- }
-
-}
-}
+++ /dev/null
-/*M///////////////////////////////////////////////////////////////////////////////////////
-//
-// IMPORTANT: READ BEFORE DOWNLOADING, COPYING, INSTALLING OR USING.
-//
-// By downloading, copying, installing or using the software you agree to this license.
-// If you do not agree to this license, do not download, install,
-// copy or use the software.
-//
-//
-// License Agreement
-// For Open Source Computer Vision Library
-//
-// Copyright (C) 2000-2008, Intel Corporation, all rights reserved.
-// Copyright (C) 2009, Willow Garage Inc., all rights reserved.
-// Third party copyrights are property of their respective owners.
-//
-// Redistribution and use in source and binary forms, with or without modification,
-// are permitted provided that the following conditions are met:
-//
-// * Redistribution's of source code must retain the above copyright notice,
-// this list of conditions and the following disclaimer.
-//
-// * Redistribution's in binary form must reproduce the above copyright notice,
-// this list of conditions and the following disclaimer in the documentation
-// and/or other materials provided with the distribution.
-//
-// * The name of the copyright holders may not be used to endorse or promote products
-// derived from this software without specific prior written permission.
-//
-// This software is provided by the copyright holders and contributors "as is" and
-// any express or implied warranties, including, but not limited to, the implied
-// warranties of merchantability and fitness for a particular purpose are disclaimed.
-// In no event shall the Intel Corporation or contributors be liable for any direct,
-// indirect, incidental, special, exemplary, or consequential damages
-// (including, but not limited to, procurement of substitute goods or services;
-// loss of use, data, or profits; or business interruption) however caused
-// and on any theory of liability, whether in contract, strict liability,
-// or tort (including negligence or otherwise) arising in any way out of
-// the use of this software, even if advised of the possibility of such damage.
-//
-//M*/
-
-#include "precomp.hpp"
-#include <limits>
-
-namespace cv
-{
-namespace linemod
-{
-
-// struct Feature
-
-/**
- * \brief Get the label [0,8) of the single bit set in quantized.
- */
-static inline int getLabel(int quantized)
-{
- switch (quantized)
- {
- case 1: return 0;
- case 2: return 1;
- case 4: return 2;
- case 8: return 3;
- case 16: return 4;
- case 32: return 5;
- case 64: return 6;
- case 128: return 7;
- default:
- CV_Error(Error::StsBadArg, "Invalid value of quantized parameter");
- return -1; //avoid warning
- }
-}
-
-void Feature::read(const FileNode& fn)
-{
- FileNodeIterator fni = fn.begin();
- fni >> x >> y >> label;
-}
-
-void Feature::write(FileStorage& fs) const
-{
- fs << "[:" << x << y << label << "]";
-}
-
-// struct Template
-
-/**
- * \brief Crop a set of overlapping templates from different modalities.
- *
- * \param[in,out] templates Set of templates representing the same object view.
- *
- * \return The bounding box of all the templates in original image coordinates.
- */
-static Rect cropTemplates(std::vector<Template>& templates)
-{
- int min_x = std::numeric_limits<int>::max();
- int min_y = std::numeric_limits<int>::max();
- int max_x = std::numeric_limits<int>::min();
- int max_y = std::numeric_limits<int>::min();
-
- // First pass: find min/max feature x,y over all pyramid levels and modalities
- for (int i = 0; i < (int)templates.size(); ++i)
- {
- Template& templ = templates[i];
-
- for (int j = 0; j < (int)templ.features.size(); ++j)
- {
- int x = templ.features[j].x << templ.pyramid_level;
- int y = templ.features[j].y << templ.pyramid_level;
- min_x = std::min(min_x, x);
- min_y = std::min(min_y, y);
- max_x = std::max(max_x, x);
- max_y = std::max(max_y, y);
- }
- }
-
- /// @todo Why require even min_x, min_y?
- if (min_x % 2 == 1) --min_x;
- if (min_y % 2 == 1) --min_y;
-
- // Second pass: set width/height and shift all feature positions
- for (int i = 0; i < (int)templates.size(); ++i)
- {
- Template& templ = templates[i];
- templ.width = (max_x - min_x) >> templ.pyramid_level;
- templ.height = (max_y - min_y) >> templ.pyramid_level;
- int offset_x = min_x >> templ.pyramid_level;
- int offset_y = min_y >> templ.pyramid_level;
-
- for (int j = 0; j < (int)templ.features.size(); ++j)
- {
- templ.features[j].x -= offset_x;
- templ.features[j].y -= offset_y;
- }
- }
-
- return Rect(min_x, min_y, max_x - min_x, max_y - min_y);
-}
-
-void Template::read(const FileNode& fn)
-{
- width = fn["width"];
- height = fn["height"];
- pyramid_level = fn["pyramid_level"];
-
- FileNode features_fn = fn["features"];
- features.resize(features_fn.size());
- FileNodeIterator it = features_fn.begin(), it_end = features_fn.end();
- for (int i = 0; it != it_end; ++it, ++i)
- {
- features[i].read(*it);
- }
-}
-
-void Template::write(FileStorage& fs) const
-{
- fs << "width" << width;
- fs << "height" << height;
- fs << "pyramid_level" << pyramid_level;
-
- fs << "features" << "[";
- for (int i = 0; i < (int)features.size(); ++i)
- {
- features[i].write(fs);
- }
- fs << "]"; // features
-}
-
-/****************************************************************************************\
-* Modality interfaces *
-\****************************************************************************************/
-
-void QuantizedPyramid::selectScatteredFeatures(const std::vector<Candidate>& candidates,
- std::vector<Feature>& features,
- size_t num_features, float distance)
-{
- features.clear();
- float distance_sq = distance * distance;
- int i = 0;
- while (features.size() < num_features)
- {
- Candidate c = candidates[i];
-
- // Add if sufficient distance away from any previously chosen feature
- bool keep = true;
- for (int j = 0; (j < (int)features.size()) && keep; ++j)
- {
- Feature f = features[j];
- keep = (c.f.x - f.x)*(c.f.x - f.x) + (c.f.y - f.y)*(c.f.y - f.y) >= distance_sq;
- }
- if (keep)
- features.push_back(c.f);
-
- if (++i == (int)candidates.size())
- {
- // Start back at beginning, and relax required distance
- i = 0;
- distance -= 1.0f;
- distance_sq = distance * distance;
- }
- }
-}
-
-Ptr<Modality> Modality::create(const String& modality_type)
-{
- if (modality_type == "ColorGradient")
- return makePtr<ColorGradient>();
- else if (modality_type == "DepthNormal")
- return makePtr<DepthNormal>();
- else
- return Ptr<Modality>();
-}
-
-Ptr<Modality> Modality::create(const FileNode& fn)
-{
- String type = fn["type"];
- Ptr<Modality> modality = create(type);
- modality->read(fn);
- return modality;
-}
-
-void colormap(const Mat& quantized, Mat& dst)
-{
- std::vector<Vec3b> lut(8);
- lut[0] = Vec3b( 0, 0, 255);
- lut[1] = Vec3b( 0, 170, 255);
- lut[2] = Vec3b( 0, 255, 170);
- lut[3] = Vec3b( 0, 255, 0);
- lut[4] = Vec3b(170, 255, 0);
- lut[5] = Vec3b(255, 170, 0);
- lut[6] = Vec3b(255, 0, 0);
- lut[7] = Vec3b(255, 0, 170);
-
- dst = Mat::zeros(quantized.size(), CV_8UC3);
- for (int r = 0; r < dst.rows; ++r)
- {
- const uchar* quant_r = quantized.ptr(r);
- Vec3b* dst_r = dst.ptr<Vec3b>(r);
- for (int c = 0; c < dst.cols; ++c)
- {
- uchar q = quant_r[c];
- if (q)
- dst_r[c] = lut[getLabel(q)];
- }
- }
-}
-
-/****************************************************************************************\
-* Color gradient modality *
-\****************************************************************************************/
-
-// Forward declaration
-void hysteresisGradient(Mat& magnitude, Mat& angle,
- Mat& ap_tmp, float threshold);
-
-/**
- * \brief Compute quantized orientation image from color image.
- *
- * Implements section 2.2 "Computing the Gradient Orientations."
- *
- * \param[in] src The source 8-bit, 3-channel image.
- * \param[out] magnitude Destination floating-point array of squared magnitudes.
- * \param[out] angle Destination 8-bit array of orientations. Each bit
- * represents one bin of the orientation space.
- * \param threshold Magnitude threshold. Keep only gradients whose norms are
- * larger than this.
- */
-static void quantizedOrientations(const Mat& src, Mat& magnitude,
- Mat& angle, float threshold)
-{
- magnitude.create(src.size(), CV_32F);
-
- // Allocate temporary buffers
- Size size = src.size();
- Mat sobel_3dx; // per-channel horizontal derivative
- Mat sobel_3dy; // per-channel vertical derivative
- Mat sobel_dx(size, CV_32F); // maximum horizontal derivative
- Mat sobel_dy(size, CV_32F); // maximum vertical derivative
- Mat sobel_ag; // final gradient orientation (unquantized)
- Mat smoothed;
-
- // Compute horizontal and vertical image derivatives on all color channels separately
- static const int KERNEL_SIZE = 7;
- // For some reason cvSmooth/cv::GaussianBlur, cvSobel/cv::Sobel have different defaults for border handling...
- GaussianBlur(src, smoothed, Size(KERNEL_SIZE, KERNEL_SIZE), 0, 0, BORDER_REPLICATE);
- Sobel(smoothed, sobel_3dx, CV_16S, 1, 0, 3, 1.0, 0.0, BORDER_REPLICATE);
- Sobel(smoothed, sobel_3dy, CV_16S, 0, 1, 3, 1.0, 0.0, BORDER_REPLICATE);
-
- short * ptrx = (short *)sobel_3dx.data;
- short * ptry = (short *)sobel_3dy.data;
- float * ptr0x = (float *)sobel_dx.data;
- float * ptr0y = (float *)sobel_dy.data;
- float * ptrmg = (float *)magnitude.data;
-
- const int length1 = static_cast<const int>(sobel_3dx.step1());
- const int length2 = static_cast<const int>(sobel_3dy.step1());
- const int length3 = static_cast<const int>(sobel_dx.step1());
- const int length4 = static_cast<const int>(sobel_dy.step1());
- const int length5 = static_cast<const int>(magnitude.step1());
- const int length0 = sobel_3dy.cols * 3;
-
- for (int r = 0; r < sobel_3dy.rows; ++r)
- {
- int ind = 0;
-
- for (int i = 0; i < length0; i += 3)
- {
- // Use the gradient orientation of the channel whose magnitude is largest
- int mag1 = ptrx[i+0] * ptrx[i + 0] + ptry[i + 0] * ptry[i + 0];
- int mag2 = ptrx[i+1] * ptrx[i + 1] + ptry[i + 1] * ptry[i + 1];
- int mag3 = ptrx[i+2] * ptrx[i + 2] + ptry[i + 2] * ptry[i + 2];
-
- if (mag1 >= mag2 && mag1 >= mag3)
- {
- ptr0x[ind] = ptrx[i];
- ptr0y[ind] = ptry[i];
- ptrmg[ind] = (float)mag1;
- }
- else if (mag2 >= mag1 && mag2 >= mag3)
- {
- ptr0x[ind] = ptrx[i + 1];
- ptr0y[ind] = ptry[i + 1];
- ptrmg[ind] = (float)mag2;
- }
- else
- {
- ptr0x[ind] = ptrx[i + 2];
- ptr0y[ind] = ptry[i + 2];
- ptrmg[ind] = (float)mag3;
- }
- ++ind;
- }
- ptrx += length1;
- ptry += length2;
- ptr0x += length3;
- ptr0y += length4;
- ptrmg += length5;
- }
-
- // Calculate the final gradient orientations
- phase(sobel_dx, sobel_dy, sobel_ag, true);
- hysteresisGradient(magnitude, angle, sobel_ag, threshold * threshold);
-}
-
-void hysteresisGradient(Mat& magnitude, Mat& quantized_angle,
- Mat& angle, float threshold)
-{
- // Quantize 360 degree range of orientations into 16 buckets
- // Note that [0, 11.25), [348.75, 360) both get mapped in the end to label 0,
- // for stability of horizontal and vertical features.
- Mat_<unsigned char> quantized_unfiltered;
- angle.convertTo(quantized_unfiltered, CV_8U, 16.0 / 360.0);
-
- // Zero out top and bottom rows
- /// @todo is this necessary, or even correct?
- memset(quantized_unfiltered.ptr(), 0, quantized_unfiltered.cols);
- memset(quantized_unfiltered.ptr(quantized_unfiltered.rows - 1), 0, quantized_unfiltered.cols);
- // Zero out first and last columns
- for (int r = 0; r < quantized_unfiltered.rows; ++r)
- {
- quantized_unfiltered(r, 0) = 0;
- quantized_unfiltered(r, quantized_unfiltered.cols - 1) = 0;
- }
-
- // Mask 16 buckets into 8 quantized orientations
- for (int r = 1; r < angle.rows - 1; ++r)
- {
- uchar* quant_r = quantized_unfiltered.ptr<uchar>(r);
- for (int c = 1; c < angle.cols - 1; ++c)
- {
- quant_r[c] &= 7;
- }
- }
-
- // Filter the raw quantized image. Only accept pixels where the magnitude is above some
- // threshold, and there is local agreement on the quantization.
- quantized_angle = Mat::zeros(angle.size(), CV_8U);
- for (int r = 1; r < angle.rows - 1; ++r)
- {
- float* mag_r = magnitude.ptr<float>(r);
-
- for (int c = 1; c < angle.cols - 1; ++c)
- {
- if (mag_r[c] > threshold)
- {
- // Compute histogram of quantized bins in 3x3 patch around pixel
- int histogram[8] = {0, 0, 0, 0, 0, 0, 0, 0};
-
- uchar* patch3x3_row = &quantized_unfiltered(r-1, c-1);
- histogram[patch3x3_row[0]]++;
- histogram[patch3x3_row[1]]++;
- histogram[patch3x3_row[2]]++;
-
- patch3x3_row += quantized_unfiltered.step1();
- histogram[patch3x3_row[0]]++;
- histogram[patch3x3_row[1]]++;
- histogram[patch3x3_row[2]]++;
-
- patch3x3_row += quantized_unfiltered.step1();
- histogram[patch3x3_row[0]]++;
- histogram[patch3x3_row[1]]++;
- histogram[patch3x3_row[2]]++;
-
- // Find bin with the most votes from the patch
- int max_votes = 0;
- int index = -1;
- for (int i = 0; i < 8; ++i)
- {
- if (max_votes < histogram[i])
- {
- index = i;
- max_votes = histogram[i];
- }
- }
-
- // Only accept the quantization if majority of pixels in the patch agree
- static const int NEIGHBOR_THRESHOLD = 5;
- if (max_votes >= NEIGHBOR_THRESHOLD)
- quantized_angle.at<uchar>(r, c) = uchar(1 << index);
- }
- }
- }
-}
-
-class ColorGradientPyramid : public QuantizedPyramid
-{
-public:
- ColorGradientPyramid(const Mat& src, const Mat& mask,
- float weak_threshold, size_t num_features,
- float strong_threshold);
-
- virtual void quantize(Mat& dst) const;
-
- virtual bool extractTemplate(Template& templ) const;
-
- virtual void pyrDown();
-
-protected:
- /// Recalculate angle and magnitude images
- void update();
-
- Mat src;
- Mat mask;
-
- int pyramid_level;
- Mat angle;
- Mat magnitude;
-
- float weak_threshold;
- size_t num_features;
- float strong_threshold;
-};
-
-ColorGradientPyramid::ColorGradientPyramid(const Mat& _src, const Mat& _mask,
- float _weak_threshold, size_t _num_features,
- float _strong_threshold)
- : src(_src),
- mask(_mask),
- pyramid_level(0),
- weak_threshold(_weak_threshold),
- num_features(_num_features),
- strong_threshold(_strong_threshold)
-{
- update();
-}
-
-void ColorGradientPyramid::update()
-{
- quantizedOrientations(src, magnitude, angle, weak_threshold);
-}
-
-void ColorGradientPyramid::pyrDown()
-{
- // Some parameters need to be adjusted
- num_features /= 2; /// @todo Why not 4?
- ++pyramid_level;
-
- // Downsample the current inputs
- Size size(src.cols / 2, src.rows / 2);
- Mat next_src;
- cv::pyrDown(src, next_src, size);
- src = next_src;
- if (!mask.empty())
- {
- Mat next_mask;
- resize(mask, next_mask, size, 0.0, 0.0, INTER_NEAREST);
- mask = next_mask;
- }
-
- update();
-}
-
-void ColorGradientPyramid::quantize(Mat& dst) const
-{
- dst = Mat::zeros(angle.size(), CV_8U);
- angle.copyTo(dst, mask);
-}
-
-bool ColorGradientPyramid::extractTemplate(Template& templ) const
-{
- // Want features on the border to distinguish from background
- Mat local_mask;
- if (!mask.empty())
- {
- erode(mask, local_mask, Mat(), Point(-1,-1), 1, BORDER_REPLICATE);
- subtract(mask, local_mask, local_mask);
- }
-
- // Create sorted list of all pixels with magnitude greater than a threshold
- std::vector<Candidate> candidates;
- bool no_mask = local_mask.empty();
- float threshold_sq = strong_threshold*strong_threshold;
- for (int r = 0; r < magnitude.rows; ++r)
- {
- const uchar* angle_r = angle.ptr<uchar>(r);
- const float* magnitude_r = magnitude.ptr<float>(r);
- const uchar* mask_r = no_mask ? NULL : local_mask.ptr<uchar>(r);
-
- for (int c = 0; c < magnitude.cols; ++c)
- {
- if (no_mask || mask_r[c])
- {
- uchar quantized = angle_r[c];
- if (quantized > 0)
- {
- float score = magnitude_r[c];
- if (score > threshold_sq)
- {
- candidates.push_back(Candidate(c, r, getLabel(quantized), score));
- }
- }
- }
- }
- }
- // We require a certain number of features
- if (candidates.size() < num_features)
- return false;
- // NOTE: Stable sort to agree with old code, which used std::list::sort()
- std::stable_sort(candidates.begin(), candidates.end());
-
- // Use heuristic based on surplus of candidates in narrow outline for initial distance threshold
- float distance = static_cast<float>(candidates.size() / num_features + 1);
- selectScatteredFeatures(candidates, templ.features, num_features, distance);
-
- // Size determined externally, needs to match templates for other modalities
- templ.width = -1;
- templ.height = -1;
- templ.pyramid_level = pyramid_level;
-
- return true;
-}
-
-ColorGradient::ColorGradient()
- : weak_threshold(10.0f),
- num_features(63),
- strong_threshold(55.0f)
-{
-}
-
-ColorGradient::ColorGradient(float _weak_threshold, size_t _num_features, float _strong_threshold)
- : weak_threshold(_weak_threshold),
- num_features(_num_features),
- strong_threshold(_strong_threshold)
-{
-}
-
-static const char CG_NAME[] = "ColorGradient";
-
-String ColorGradient::name() const
-{
- return CG_NAME;
-}
-
-Ptr<QuantizedPyramid> ColorGradient::processImpl(const Mat& src,
- const Mat& mask) const
-{
- return makePtr<ColorGradientPyramid>(src, mask, weak_threshold, num_features, strong_threshold);
-}
-
-void ColorGradient::read(const FileNode& fn)
-{
- String type = fn["type"];
- CV_Assert(type == CG_NAME);
-
- weak_threshold = fn["weak_threshold"];
- num_features = int(fn["num_features"]);
- strong_threshold = fn["strong_threshold"];
-}
-
-void ColorGradient::write(FileStorage& fs) const
-{
- fs << "type" << CG_NAME;
- fs << "weak_threshold" << weak_threshold;
- fs << "num_features" << int(num_features);
- fs << "strong_threshold" << strong_threshold;
-}
-
-/****************************************************************************************\
-* Depth normal modality *
-\****************************************************************************************/
-
-// Contains GRANULARITY and NORMAL_LUT
-#include "normal_lut.i"
-
-static void accumBilateral(long delta, long i, long j, long * A, long * b, int threshold)
-{
- long f = std::abs(delta) < threshold ? 1 : 0;
-
- const long fi = f * i;
- const long fj = f * j;
-
- A[0] += fi * i;
- A[1] += fi * j;
- A[3] += fj * j;
- b[0] += fi * delta;
- b[1] += fj * delta;
-}
-
-/**
- * \brief Compute quantized normal image from depth image.
- *
- * Implements section 2.6 "Extension to Dense Depth Sensors."
- *
- * \param[in] src The source 16-bit depth image (in mm).
- * \param[out] dst The destination 8-bit image. Each bit represents one bin of
- * the view cone.
- * \param distance_threshold Ignore pixels beyond this distance.
- * \param difference_threshold When computing normals, ignore contributions of pixels whose
- * depth difference with the central pixel is above this threshold.
- *
- * \todo Should also need camera model, or at least focal lengths? Replace distance_threshold with mask?
- */
-static void quantizedNormals(const Mat& src, Mat& dst, int distance_threshold,
- int difference_threshold)
-{
- dst = Mat::zeros(src.size(), CV_8U);
-
- const unsigned short * lp_depth = src.ptr<ushort>();
- unsigned char * lp_normals = dst.ptr<uchar>();
-
- const int l_W = src.cols;
- const int l_H = src.rows;
-
- const int l_r = 5; // used to be 7
- const int l_offset0 = -l_r - l_r * l_W;
- const int l_offset1 = 0 - l_r * l_W;
- const int l_offset2 = +l_r - l_r * l_W;
- const int l_offset3 = -l_r;
- const int l_offset4 = +l_r;
- const int l_offset5 = -l_r + l_r * l_W;
- const int l_offset6 = 0 + l_r * l_W;
- const int l_offset7 = +l_r + l_r * l_W;
-
- const int l_offsetx = GRANULARITY / 2;
- const int l_offsety = GRANULARITY / 2;
-
- for (int l_y = l_r; l_y < l_H - l_r - 1; ++l_y)
- {
- const unsigned short * lp_line = lp_depth + (l_y * l_W + l_r);
- unsigned char * lp_norm = lp_normals + (l_y * l_W + l_r);
-
- for (int l_x = l_r; l_x < l_W - l_r - 1; ++l_x)
- {
- long l_d = lp_line[0];
-
- if (l_d < distance_threshold)
- {
- // accum
- long l_A[4]; l_A[0] = l_A[1] = l_A[2] = l_A[3] = 0;
- long l_b[2]; l_b[0] = l_b[1] = 0;
- accumBilateral(lp_line[l_offset0] - l_d, -l_r, -l_r, l_A, l_b, difference_threshold);
- accumBilateral(lp_line[l_offset1] - l_d, 0, -l_r, l_A, l_b, difference_threshold);
- accumBilateral(lp_line[l_offset2] - l_d, +l_r, -l_r, l_A, l_b, difference_threshold);
- accumBilateral(lp_line[l_offset3] - l_d, -l_r, 0, l_A, l_b, difference_threshold);
- accumBilateral(lp_line[l_offset4] - l_d, +l_r, 0, l_A, l_b, difference_threshold);
- accumBilateral(lp_line[l_offset5] - l_d, -l_r, +l_r, l_A, l_b, difference_threshold);
- accumBilateral(lp_line[l_offset6] - l_d, 0, +l_r, l_A, l_b, difference_threshold);
- accumBilateral(lp_line[l_offset7] - l_d, +l_r, +l_r, l_A, l_b, difference_threshold);
-
- // solve
- long l_det = l_A[0] * l_A[3] - l_A[1] * l_A[1];
- long l_ddx = l_A[3] * l_b[0] - l_A[1] * l_b[1];
- long l_ddy = -l_A[1] * l_b[0] + l_A[0] * l_b[1];
-
- /// @todo Magic number 1150 is focal length? This is something like
- /// f in SXGA mode, but in VGA is more like 530.
- float l_nx = static_cast<float>(1150 * l_ddx);
- float l_ny = static_cast<float>(1150 * l_ddy);
- float l_nz = static_cast<float>(-l_det * l_d);
-
- float l_sqrt = sqrtf(l_nx * l_nx + l_ny * l_ny + l_nz * l_nz);
-
- if (l_sqrt > 0)
- {
- float l_norminv = 1.0f / (l_sqrt);
-
- l_nx *= l_norminv;
- l_ny *= l_norminv;
- l_nz *= l_norminv;
-
- //*lp_norm = fabs(l_nz)*255;
-
- int l_val1 = static_cast<int>(l_nx * l_offsetx + l_offsetx);
- int l_val2 = static_cast<int>(l_ny * l_offsety + l_offsety);
- int l_val3 = static_cast<int>(l_nz * GRANULARITY + GRANULARITY);
-
- *lp_norm = NORMAL_LUT[l_val3][l_val2][l_val1];
- }
- else
- {
- *lp_norm = 0; // Discard shadows from depth sensor
- }
- }
- else
- {
- *lp_norm = 0; //out of depth
- }
- ++lp_line;
- ++lp_norm;
- }
- }
- medianBlur(dst, dst, 5);
-}
-
-class DepthNormalPyramid : public QuantizedPyramid
-{
-public:
- DepthNormalPyramid(const Mat& src, const Mat& mask,
- int distance_threshold, int difference_threshold, size_t num_features,
- int extract_threshold);
-
- virtual void quantize(Mat& dst) const;
-
- virtual bool extractTemplate(Template& templ) const;
-
- virtual void pyrDown();
-
-protected:
- Mat mask;
-
- int pyramid_level;
- Mat normal;
-
- size_t num_features;
- int extract_threshold;
-};
-
-DepthNormalPyramid::DepthNormalPyramid(const Mat& src, const Mat& _mask,
- int distance_threshold, int difference_threshold, size_t _num_features,
- int _extract_threshold)
- : mask(_mask),
- pyramid_level(0),
- num_features(_num_features),
- extract_threshold(_extract_threshold)
-{
- quantizedNormals(src, normal, distance_threshold, difference_threshold);
-}
-
-void DepthNormalPyramid::pyrDown()
-{
- // Some parameters need to be adjusted
- num_features /= 2; /// @todo Why not 4?
- extract_threshold /= 2;
- ++pyramid_level;
-
- // In this case, NN-downsample the quantized image
- Mat next_normal;
- Size size(normal.cols / 2, normal.rows / 2);
- resize(normal, next_normal, size, 0.0, 0.0, INTER_NEAREST);
- normal = next_normal;
- if (!mask.empty())
- {
- Mat next_mask;
- resize(mask, next_mask, size, 0.0, 0.0, INTER_NEAREST);
- mask = next_mask;
- }
-}
-
-void DepthNormalPyramid::quantize(Mat& dst) const
-{
- dst = Mat::zeros(normal.size(), CV_8U);
- normal.copyTo(dst, mask);
-}
-
-bool DepthNormalPyramid::extractTemplate(Template& templ) const
-{
- // Features right on the object border are unreliable
- Mat local_mask;
- if (!mask.empty())
- {
- erode(mask, local_mask, Mat(), Point(-1,-1), 2, BORDER_REPLICATE);
- }
-
- // Compute distance transform for each individual quantized orientation
- Mat temp = Mat::zeros(normal.size(), CV_8U);
- Mat distances[8];
- for (int i = 0; i < 8; ++i)
- {
- temp.setTo(1 << i, local_mask);
- bitwise_and(temp, normal, temp);
- // temp is now non-zero at pixels in the mask with quantized orientation i
- distanceTransform(temp, distances[i], DIST_C, 3);
- }
-
- // Count how many features taken for each label
- int label_counts[8] = {0, 0, 0, 0, 0, 0, 0, 0};
-
- // Create sorted list of candidate features
- std::vector<Candidate> candidates;
- bool no_mask = local_mask.empty();
- for (int r = 0; r < normal.rows; ++r)
- {
- const uchar* normal_r = normal.ptr<uchar>(r);
- const uchar* mask_r = no_mask ? NULL : local_mask.ptr<uchar>(r);
-
- for (int c = 0; c < normal.cols; ++c)
- {
- if (no_mask || mask_r[c])
- {
- uchar quantized = normal_r[c];
-
- if (quantized != 0 && quantized != 255) // background and shadow
- {
- int label = getLabel(quantized);
-
- // Accept if distance to a pixel belonging to a different label is greater than
- // some threshold. IOW, ideal feature is in the center of a large homogeneous
- // region.
- float score = distances[label].at<float>(r, c);
- if (score >= extract_threshold)
- {
- candidates.push_back( Candidate(c, r, label, score) );
- ++label_counts[label];
- }
- }
- }
- }
- }
- // We require a certain number of features
- if (candidates.size() < num_features)
- return false;
-
- // Prefer large distances, but also want to collect features over all 8 labels.
- // So penalize labels with lots of candidates.
- for (size_t i = 0; i < candidates.size(); ++i)
- {
- Candidate& c = candidates[i];
- c.score /= (float)label_counts[c.f.label];
- }
- std::stable_sort(candidates.begin(), candidates.end());
-
- // Use heuristic based on object area for initial distance threshold
- float area = no_mask ? (float)normal.total() : (float)countNonZero(local_mask);
- float distance = sqrtf(area) / sqrtf((float)num_features) + 1.5f;
- selectScatteredFeatures(candidates, templ.features, num_features, distance);
-
- // Size determined externally, needs to match templates for other modalities
- templ.width = -1;
- templ.height = -1;
- templ.pyramid_level = pyramid_level;
-
- return true;
-}
-
-DepthNormal::DepthNormal()
- : distance_threshold(2000),
- difference_threshold(50),
- num_features(63),
- extract_threshold(2)
-{
-}
-
-DepthNormal::DepthNormal(int _distance_threshold, int _difference_threshold, size_t _num_features,
- int _extract_threshold)
- : distance_threshold(_distance_threshold),
- difference_threshold(_difference_threshold),
- num_features(_num_features),
- extract_threshold(_extract_threshold)
-{
-}
-
-static const char DN_NAME[] = "DepthNormal";
-
-String DepthNormal::name() const
-{
- return DN_NAME;
-}
-
-Ptr<QuantizedPyramid> DepthNormal::processImpl(const Mat& src,
- const Mat& mask) const
-{
- return makePtr<DepthNormalPyramid>(src, mask, distance_threshold, difference_threshold,
- num_features, extract_threshold);
-}
-
-void DepthNormal::read(const FileNode& fn)
-{
- String type = fn["type"];
- CV_Assert(type == DN_NAME);
-
- distance_threshold = fn["distance_threshold"];
- difference_threshold = fn["difference_threshold"];
- num_features = int(fn["num_features"]);
- extract_threshold = fn["extract_threshold"];
-}
-
-void DepthNormal::write(FileStorage& fs) const
-{
- fs << "type" << DN_NAME;
- fs << "distance_threshold" << distance_threshold;
- fs << "difference_threshold" << difference_threshold;
- fs << "num_features" << int(num_features);
- fs << "extract_threshold" << extract_threshold;
-}
-
-/****************************************************************************************\
-* Response maps *
-\****************************************************************************************/
-
-static void orUnaligned8u(const uchar * src, const int src_stride,
- uchar * dst, const int dst_stride,
- const int width, const int height)
-{
-#if CV_SSE2
- volatile bool haveSSE2 = checkHardwareSupport(CV_CPU_SSE2);
-#if CV_SSE3
- volatile bool haveSSE3 = checkHardwareSupport(CV_CPU_SSE3);
-#endif
- bool src_aligned = reinterpret_cast<unsigned long long>(src) % 16 == 0;
-#endif
-
- for (int r = 0; r < height; ++r)
- {
- int c = 0;
-
-#if CV_SSE2
- // Use aligned loads if possible
- if (haveSSE2 && src_aligned)
- {
- for ( ; c < width - 15; c += 16)
- {
- const __m128i* src_ptr = reinterpret_cast<const __m128i*>(src + c);
- __m128i* dst_ptr = reinterpret_cast<__m128i*>(dst + c);
- *dst_ptr = _mm_or_si128(*dst_ptr, *src_ptr);
- }
- }
-#if CV_SSE3
- // Use LDDQU for fast unaligned load
- else if (haveSSE3)
- {
- for ( ; c < width - 15; c += 16)
- {
- __m128i val = _mm_lddqu_si128(reinterpret_cast<const __m128i*>(src + c));
- __m128i* dst_ptr = reinterpret_cast<__m128i*>(dst + c);
- *dst_ptr = _mm_or_si128(*dst_ptr, val);
- }
- }
-#endif
- // Fall back to MOVDQU
- else if (haveSSE2)
- {
- for ( ; c < width - 15; c += 16)
- {
- __m128i val = _mm_loadu_si128(reinterpret_cast<const __m128i*>(src + c));
- __m128i* dst_ptr = reinterpret_cast<__m128i*>(dst + c);
- *dst_ptr = _mm_or_si128(*dst_ptr, val);
- }
- }
-#endif
- for ( ; c < width; ++c)
- dst[c] |= src[c];
-
- // Advance to next row
- src += src_stride;
- dst += dst_stride;
- }
-}
-
-/**
- * \brief Spread binary labels in a quantized image.
- *
- * Implements section 2.3 "Spreading the Orientations."
- *
- * \param[in] src The source 8-bit quantized image.
- * \param[out] dst Destination 8-bit spread image.
- * \param T Sampling step. Spread labels T/2 pixels in each direction.
- */
-static void spread(const Mat& src, Mat& dst, int T)
-{
- // Allocate and zero-initialize spread (OR'ed) image
- dst = Mat::zeros(src.size(), CV_8U);
-
- // Fill in spread gradient image (section 2.3)
- for (int r = 0; r < T; ++r)
- {
- int height = src.rows - r;
- for (int c = 0; c < T; ++c)
- {
- orUnaligned8u(&src.at<unsigned char>(r, c), static_cast<const int>(src.step1()), dst.ptr(),
- static_cast<const int>(dst.step1()), src.cols - c, height);
- }
- }
-}
-
-// Auto-generated by create_similarity_lut.py
-CV_DECL_ALIGNED(16) static const unsigned char SIMILARITY_LUT[256] = {0, 4, 3, 4, 2, 4, 3, 4, 1, 4, 3, 4, 2, 4, 3, 4, 0, 0, 1, 1, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 3, 3, 0, 3, 4, 4, 3, 3, 4, 4, 2, 3, 4, 4, 3, 3, 4, 4, 0, 1, 0, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 0, 2, 3, 3, 4, 4, 4, 4, 3, 3, 3, 3, 4, 4, 4, 4, 0, 2, 1, 2, 0, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 0, 1, 2, 2, 3, 3, 3, 3, 4, 4, 4, 4, 4, 4, 4, 4, 0, 3, 2, 3, 1, 3, 2, 3, 0, 3, 2, 3, 1, 3, 2, 3, 0, 0, 1, 1, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 3, 3, 0, 4, 3, 4, 2, 4, 3, 4, 1, 4, 3, 4, 2, 4, 3, 4, 0, 1, 0, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 0, 3, 4, 4, 3, 3, 4, 4, 2, 3, 4, 4, 3, 3, 4, 4, 0, 2, 1, 2, 0, 2, 1, 2, 1, 2, 1, 2, 1, 2, 1, 2, 0, 2, 3, 3, 4, 4, 4, 4, 3, 3, 3, 3, 4, 4, 4, 4, 0, 3, 2, 3, 1, 3, 2, 3, 0, 3, 2, 3, 1, 3, 2, 3, 0, 1, 2, 2, 3, 3, 3, 3, 4, 4, 4, 4, 4, 4, 4, 4};
-
-/**
- * \brief Precompute response maps for a spread quantized image.
- *
- * Implements section 2.4 "Precomputing Response Maps."
- *
- * \param[in] src The source 8-bit spread quantized image.
- * \param[out] response_maps Vector of 8 response maps, one for each bit label.
- */
-static void computeResponseMaps(const Mat& src, std::vector<Mat>& response_maps)
-{
- CV_Assert((src.rows * src.cols) % 16 == 0);
-
- // Allocate response maps
- response_maps.resize(8);
- for (int i = 0; i < 8; ++i)
- response_maps[i].create(src.size(), CV_8U);
-
- Mat lsb4(src.size(), CV_8U);
- Mat msb4(src.size(), CV_8U);
-
- for (int r = 0; r < src.rows; ++r)
- {
- const uchar* src_r = src.ptr(r);
- uchar* lsb4_r = lsb4.ptr(r);
- uchar* msb4_r = msb4.ptr(r);
-
- for (int c = 0; c < src.cols; ++c)
- {
- // Least significant 4 bits of spread image pixel
- lsb4_r[c] = src_r[c] & 15;
- // Most significant 4 bits, right-shifted to be in [0, 16)
- msb4_r[c] = (src_r[c] & 240) >> 4;
- }
- }
-
-#if CV_SSSE3
- volatile bool haveSSSE3 = checkHardwareSupport(CV_CPU_SSSE3);
- if (haveSSSE3)
- {
- const __m128i* lut = reinterpret_cast<const __m128i*>(SIMILARITY_LUT);
- for (int ori = 0; ori < 8; ++ori)
- {
- __m128i* map_data = response_maps[ori].ptr<__m128i>();
- __m128i* lsb4_data = lsb4.ptr<__m128i>();
- __m128i* msb4_data = msb4.ptr<__m128i>();
-
- // Precompute the 2D response map S_i (section 2.4)
- for (int i = 0; i < (src.rows * src.cols) / 16; ++i)
- {
- // Using SSE shuffle for table lookup on 4 orientations at a time
- // The most/least significant 4 bits are used as the LUT index
- __m128i res1 = _mm_shuffle_epi8(lut[2*ori + 0], lsb4_data[i]);
- __m128i res2 = _mm_shuffle_epi8(lut[2*ori + 1], msb4_data[i]);
-
- // Combine the results into a single similarity score
- map_data[i] = _mm_max_epu8(res1, res2);
- }
- }
- }
- else
-#endif
- {
- // For each of the 8 quantized orientations...
- for (int ori = 0; ori < 8; ++ori)
- {
- uchar* map_data = response_maps[ori].ptr<uchar>();
- uchar* lsb4_data = lsb4.ptr<uchar>();
- uchar* msb4_data = msb4.ptr<uchar>();
- const uchar* lut_low = SIMILARITY_LUT + 32*ori;
- const uchar* lut_hi = lut_low + 16;
-
- for (int i = 0; i < src.rows * src.cols; ++i)
- {
- map_data[i] = std::max(lut_low[ lsb4_data[i] ], lut_hi[ msb4_data[i] ]);
- }
- }
- }
-}
-
-/**
- * \brief Convert a response map to fast linearized ordering.
- *
- * Implements section 2.5 "Linearizing the Memory for Parallelization."
- *
- * \param[in] response_map The 2D response map, an 8-bit image.
- * \param[out] linearized The response map in linearized order. It has T*T rows,
- * each of which is a linear memory of length (W/T)*(H/T).
- * \param T Sampling step.
- */
-static void linearize(const Mat& response_map, Mat& linearized, int T)
-{
- CV_Assert(response_map.rows % T == 0);
- CV_Assert(response_map.cols % T == 0);
-
- // linearized has T^2 rows, where each row is a linear memory
- int mem_width = response_map.cols / T;
- int mem_height = response_map.rows / T;
- linearized.create(T*T, mem_width * mem_height, CV_8U);
-
- // Outer two for loops iterate over top-left T^2 starting pixels
- int index = 0;
- for (int r_start = 0; r_start < T; ++r_start)
- {
- for (int c_start = 0; c_start < T; ++c_start)
- {
- uchar* memory = linearized.ptr(index);
- ++index;
-
- // Inner two loops copy every T-th pixel into the linear memory
- for (int r = r_start; r < response_map.rows; r += T)
- {
- const uchar* response_data = response_map.ptr(r);
- for (int c = c_start; c < response_map.cols; c += T)
- *memory++ = response_data[c];
- }
- }
- }
-}
-
-/****************************************************************************************\
-* Linearized similarities *
-\****************************************************************************************/
-
-static const unsigned char* accessLinearMemory(const std::vector<Mat>& linear_memories,
- const Feature& f, int T, int W)
-{
- // Retrieve the TxT grid of linear memories associated with the feature label
- const Mat& memory_grid = linear_memories[f.label];
- CV_DbgAssert(memory_grid.rows == T*T);
- CV_DbgAssert(f.x >= 0);
- CV_DbgAssert(f.y >= 0);
- // The LM we want is at (x%T, y%T) in the TxT grid (stored as the rows of memory_grid)
- int grid_x = f.x % T;
- int grid_y = f.y % T;
- int grid_index = grid_y * T + grid_x;
- CV_DbgAssert(grid_index >= 0);
- CV_DbgAssert(grid_index < memory_grid.rows);
- const unsigned char* memory = memory_grid.ptr(grid_index);
- // Within the LM, the feature is at (x/T, y/T). W is the "width" of the LM, the
- // input image width decimated by T.
- int lm_x = f.x / T;
- int lm_y = f.y / T;
- int lm_index = lm_y * W + lm_x;
- CV_DbgAssert(lm_index >= 0);
- CV_DbgAssert(lm_index < memory_grid.cols);
- return memory + lm_index;
-}
-
-/**
- * \brief Compute similarity measure for a given template at each sampled image location.
- *
- * Uses linear memories to compute the similarity measure as described in Fig. 7.
- *
- * \param[in] linear_memories Vector of 8 linear memories, one for each label.
- * \param[in] templ Template to match against.
- * \param[out] dst Destination 8-bit similarity image of size (W/T, H/T).
- * \param size Size (W, H) of the original input image.
- * \param T Sampling step.
- */
-static void similarity(const std::vector<Mat>& linear_memories, const Template& templ,
- Mat& dst, Size size, int T)
-{
- // 63 features or less is a special case because the max similarity per-feature is 4.
- // 255/4 = 63, so up to that many we can add up similarities in 8 bits without worrying
- // about overflow. Therefore here we use _mm_add_epi8 as the workhorse, whereas a more
- // general function would use _mm_add_epi16.
- CV_Assert(templ.features.size() <= 63);
- /// @todo Handle more than 255/MAX_RESPONSE features!!
-
- // Decimate input image size by factor of T
- int W = size.width / T;
- int H = size.height / T;
-
- // Feature dimensions, decimated by factor T and rounded up
- int wf = (templ.width - 1) / T + 1;
- int hf = (templ.height - 1) / T + 1;
-
- // Span is the range over which we can shift the template around the input image
- int span_x = W - wf;
- int span_y = H - hf;
-
- // Compute number of contiguous (in memory) pixels to check when sliding feature over
- // image. This allows template to wrap around left/right border incorrectly, so any
- // wrapped template matches must be filtered out!
- int template_positions = span_y * W + span_x + 1; // why add 1?
- //int template_positions = (span_y - 1) * W + span_x; // More correct?
-
- /// @todo In old code, dst is buffer of size m_U. Could make it something like
- /// (span_x)x(span_y) instead?
- dst = Mat::zeros(H, W, CV_8U);
- uchar* dst_ptr = dst.ptr<uchar>();
-
-#if CV_SSE2
- volatile bool haveSSE2 = checkHardwareSupport(CV_CPU_SSE2);
-#if CV_SSE3
- volatile bool haveSSE3 = checkHardwareSupport(CV_CPU_SSE3);
-#endif
-#endif
-
- // Compute the similarity measure for this template by accumulating the contribution of
- // each feature
- for (int i = 0; i < (int)templ.features.size(); ++i)
- {
- // Add the linear memory at the appropriate offset computed from the location of
- // the feature in the template
- Feature f = templ.features[i];
- // Discard feature if out of bounds
- /// @todo Shouldn't actually see x or y < 0 here?
- if (f.x < 0 || f.x >= size.width || f.y < 0 || f.y >= size.height)
- continue;
- const uchar* lm_ptr = accessLinearMemory(linear_memories, f, T, W);
-
- // Now we do an aligned/unaligned add of dst_ptr and lm_ptr with template_positions elements
- int j = 0;
- // Process responses 16 at a time if vectorization possible
-#if CV_SSE2
-#if CV_SSE3
- if (haveSSE3)
- {
- // LDDQU may be more efficient than MOVDQU for unaligned load of next 16 responses
- for ( ; j < template_positions - 15; j += 16)
- {
- __m128i responses = _mm_lddqu_si128(reinterpret_cast<const __m128i*>(lm_ptr + j));
- __m128i* dst_ptr_sse = reinterpret_cast<__m128i*>(dst_ptr + j);
- *dst_ptr_sse = _mm_add_epi8(*dst_ptr_sse, responses);
- }
- }
- else
-#endif
- if (haveSSE2)
- {
- // Fall back to MOVDQU
- for ( ; j < template_positions - 15; j += 16)
- {
- __m128i responses = _mm_loadu_si128(reinterpret_cast<const __m128i*>(lm_ptr + j));
- __m128i* dst_ptr_sse = reinterpret_cast<__m128i*>(dst_ptr + j);
- *dst_ptr_sse = _mm_add_epi8(*dst_ptr_sse, responses);
- }
- }
-#endif
- for ( ; j < template_positions; ++j)
- dst_ptr[j] = uchar(dst_ptr[j] + lm_ptr[j]);
- }
-}
-
-/**
- * \brief Compute similarity measure for a given template in a local region.
- *
- * \param[in] linear_memories Vector of 8 linear memories, one for each label.
- * \param[in] templ Template to match against.
- * \param[out] dst Destination 8-bit similarity image, 16x16.
- * \param size Size (W, H) of the original input image.
- * \param T Sampling step.
- * \param center Center of the local region.
- */
-static void similarityLocal(const std::vector<Mat>& linear_memories, const Template& templ,
- Mat& dst, Size size, int T, Point center)
-{
- // Similar to whole-image similarity() above. This version takes a position 'center'
- // and computes the energy in the 16x16 patch centered on it.
- CV_Assert(templ.features.size() <= 63);
-
- // Compute the similarity map in a 16x16 patch around center
- int W = size.width / T;
- dst = Mat::zeros(16, 16, CV_8U);
-
- // Offset each feature point by the requested center. Further adjust to (-8,-8) from the
- // center to get the top-left corner of the 16x16 patch.
- // NOTE: We make the offsets multiples of T to agree with results of the original code.
- int offset_x = (center.x / T - 8) * T;
- int offset_y = (center.y / T - 8) * T;
-
-#if CV_SSE2
- volatile bool haveSSE2 = checkHardwareSupport(CV_CPU_SSE2);
-#if CV_SSE3
- volatile bool haveSSE3 = checkHardwareSupport(CV_CPU_SSE3);
-#endif
- __m128i* dst_ptr_sse = dst.ptr<__m128i>();
-#endif
-
- for (int i = 0; i < (int)templ.features.size(); ++i)
- {
- Feature f = templ.features[i];
- f.x += offset_x;
- f.y += offset_y;
- // Discard feature if out of bounds, possibly due to applying the offset
- if (f.x < 0 || f.y < 0 || f.x >= size.width || f.y >= size.height)
- continue;
-
- const uchar* lm_ptr = accessLinearMemory(linear_memories, f, T, W);
-
- // Process whole row at a time if vectorization possible
-#if CV_SSE2
-#if CV_SSE3
- if (haveSSE3)
- {
- // LDDQU may be more efficient than MOVDQU for unaligned load of 16 responses from current row
- for (int row = 0; row < 16; ++row)
- {
- __m128i aligned = _mm_lddqu_si128(reinterpret_cast<const __m128i*>(lm_ptr));
- dst_ptr_sse[row] = _mm_add_epi8(dst_ptr_sse[row], aligned);
- lm_ptr += W; // Step to next row
- }
- }
- else
-#endif
- if (haveSSE2)
- {
- // Fall back to MOVDQU
- for (int row = 0; row < 16; ++row)
- {
- __m128i aligned = _mm_loadu_si128(reinterpret_cast<const __m128i*>(lm_ptr));
- dst_ptr_sse[row] = _mm_add_epi8(dst_ptr_sse[row], aligned);
- lm_ptr += W; // Step to next row
- }
- }
- else
-#endif
- {
- uchar* dst_ptr = dst.ptr<uchar>();
- for (int row = 0; row < 16; ++row)
- {
- for (int col = 0; col < 16; ++col)
- dst_ptr[col] = uchar(dst_ptr[col] + lm_ptr[col]);
- dst_ptr += 16;
- lm_ptr += W;
- }
- }
- }
-}
-
-static void addUnaligned8u16u(const uchar * src1, const uchar * src2, ushort * res, int length)
-{
- const uchar * end = src1 + length;
-
- while (src1 != end)
- {
- *res = *src1 + *src2;
-
- ++src1;
- ++src2;
- ++res;
- }
-}
-
-/**
- * \brief Accumulate one or more 8-bit similarity images.
- *
- * \param[in] similarities Source 8-bit similarity images.
- * \param[out] dst Destination 16-bit similarity image.
- */
-static void addSimilarities(const std::vector<Mat>& similarities, Mat& dst)
-{
- if (similarities.size() == 1)
- {
- similarities[0].convertTo(dst, CV_16U);
- }
- else
- {
- // NOTE: add() seems to be rather slow in the 8U + 8U -> 16U case
- dst.create(similarities[0].size(), CV_16U);
- addUnaligned8u16u(similarities[0].ptr(), similarities[1].ptr(), dst.ptr<ushort>(), static_cast<int>(dst.total()));
-
- /// @todo Optimize 16u + 8u -> 16u when more than 2 modalities
- for (size_t i = 2; i < similarities.size(); ++i)
- add(dst, similarities[i], dst, noArray(), CV_16U);
- }
-}
-
-/****************************************************************************************\
-* High-level Detector API *
-\****************************************************************************************/
-
-Detector::Detector()
-{
-}
-
-Detector::Detector(const std::vector< Ptr<Modality> >& _modalities,
- const std::vector<int>& T_pyramid)
- : modalities(_modalities),
- pyramid_levels(static_cast<int>(T_pyramid.size())),
- T_at_level(T_pyramid)
-{
-}
-
-void Detector::match(const std::vector<Mat>& sources, float threshold, std::vector<Match>& matches,
- const std::vector<String>& class_ids, OutputArrayOfArrays quantized_images,
- const std::vector<Mat>& masks) const
-{
- matches.clear();
- if (quantized_images.needed())
- quantized_images.create(1, static_cast<int>(pyramid_levels * modalities.size()), CV_8U);
-
- CV_Assert(sources.size() == modalities.size());
- // Initialize each modality with our sources
- std::vector< Ptr<QuantizedPyramid> > quantizers;
- for (int i = 0; i < (int)modalities.size(); ++i){
- Mat mask, source;
- source = sources[i];
- if(!masks.empty()){
- CV_Assert(masks.size() == modalities.size());
- mask = masks[i];
- }
- CV_Assert(mask.empty() || mask.size() == source.size());
- quantizers.push_back(modalities[i]->process(source, mask));
- }
- // pyramid level -> modality -> quantization
- LinearMemoryPyramid lm_pyramid(pyramid_levels,
- std::vector<LinearMemories>(modalities.size(), LinearMemories(8)));
-
- // For each pyramid level, precompute linear memories for each modality
- std::vector<Size> sizes;
- for (int l = 0; l < pyramid_levels; ++l)
- {
- int T = T_at_level[l];
- std::vector<LinearMemories>& lm_level = lm_pyramid[l];
-
- if (l > 0)
- {
- for (int i = 0; i < (int)quantizers.size(); ++i)
- quantizers[i]->pyrDown();
- }
-
- Mat quantized, spread_quantized;
- std::vector<Mat> response_maps;
- for (int i = 0; i < (int)quantizers.size(); ++i)
- {
- quantizers[i]->quantize(quantized);
- spread(quantized, spread_quantized, T);
- computeResponseMaps(spread_quantized, response_maps);
-
- LinearMemories& memories = lm_level[i];
- for (int j = 0; j < 8; ++j)
- linearize(response_maps[j], memories[j], T);
-
- if (quantized_images.needed()) //use copyTo here to side step reference semantics.
- quantized.copyTo(quantized_images.getMatRef(static_cast<int>(l*quantizers.size() + i)));
- }
-
- sizes.push_back(quantized.size());
- }
-
- if (class_ids.empty())
- {
- // Match all templates
- TemplatesMap::const_iterator it = class_templates.begin(), itend = class_templates.end();
- for ( ; it != itend; ++it)
- matchClass(lm_pyramid, sizes, threshold, matches, it->first, it->second);
- }
- else
- {
- // Match only templates for the requested class IDs
- for (int i = 0; i < (int)class_ids.size(); ++i)
- {
- TemplatesMap::const_iterator it = class_templates.find(class_ids[i]);
- if (it != class_templates.end())
- matchClass(lm_pyramid, sizes, threshold, matches, it->first, it->second);
- }
- }
-
- // Sort matches by similarity, and prune any duplicates introduced by pyramid refinement
- std::sort(matches.begin(), matches.end());
- std::vector<Match>::iterator new_end = std::unique(matches.begin(), matches.end());
- matches.erase(new_end, matches.end());
-}
-
-// Used to filter out weak matches
-struct MatchPredicate
-{
- MatchPredicate(float _threshold) : threshold(_threshold) {}
- bool operator() (const Match& m) { return m.similarity < threshold; }
- float threshold;
-};
-
-void Detector::matchClass(const LinearMemoryPyramid& lm_pyramid,
- const std::vector<Size>& sizes,
- float threshold, std::vector<Match>& matches,
- const String& class_id,
- const std::vector<TemplatePyramid>& template_pyramids) const
-{
- // For each template...
- for (size_t template_id = 0; template_id < template_pyramids.size(); ++template_id)
- {
- const TemplatePyramid& tp = template_pyramids[template_id];
-
- // First match over the whole image at the lowest pyramid level
- /// @todo Factor this out into separate function
- const std::vector<LinearMemories>& lowest_lm = lm_pyramid.back();
-
- // Compute similarity maps for each modality at lowest pyramid level
- std::vector<Mat> similarities(modalities.size());
- int lowest_start = static_cast<int>(tp.size() - modalities.size());
- int lowest_T = T_at_level.back();
- int num_features = 0;
- for (int i = 0; i < (int)modalities.size(); ++i)
- {
- const Template& templ = tp[lowest_start + i];
- num_features += static_cast<int>(templ.features.size());
- similarity(lowest_lm[i], templ, similarities[i], sizes.back(), lowest_T);
- }
-
- // Combine into overall similarity
- /// @todo Support weighting the modalities
- Mat total_similarity;
- addSimilarities(similarities, total_similarity);
-
- // Convert user-friendly percentage to raw similarity threshold. The percentage
- // threshold scales from half the max response (what you would expect from applying
- // the template to a completely random image) to the max response.
- // NOTE: This assumes max per-feature response is 4, so we scale between [2*nf, 4*nf].
- int raw_threshold = static_cast<int>(2*num_features + (threshold / 100.f) * (2*num_features) + 0.5f);
-
- // Find initial matches
- std::vector<Match> candidates;
- for (int r = 0; r < total_similarity.rows; ++r)
- {
- ushort* row = total_similarity.ptr<ushort>(r);
- for (int c = 0; c < total_similarity.cols; ++c)
- {
- int raw_score = row[c];
- if (raw_score > raw_threshold)
- {
- int offset = lowest_T / 2 + (lowest_T % 2 - 1);
- int x = c * lowest_T + offset;
- int y = r * lowest_T + offset;
- float score =(raw_score * 100.f) / (4 * num_features) + 0.5f;
- candidates.push_back(Match(x, y, score, class_id, static_cast<int>(template_id)));
- }
- }
- }
-
- // Locally refine each match by marching up the pyramid
- for (int l = pyramid_levels - 2; l >= 0; --l)
- {
- const std::vector<LinearMemories>& lms = lm_pyramid[l];
- int T = T_at_level[l];
- int start = static_cast<int>(l * modalities.size());
- Size size = sizes[l];
- int border = 8 * T;
- int offset = T / 2 + (T % 2 - 1);
- int max_x = size.width - tp[start].width - border;
- int max_y = size.height - tp[start].height - border;
-
- std::vector<Mat> similarities2(modalities.size());
- Mat total_similarity2;
- for (int m = 0; m < (int)candidates.size(); ++m)
- {
- Match& match2 = candidates[m];
- int x = match2.x * 2 + 1; /// @todo Support other pyramid distance
- int y = match2.y * 2 + 1;
-
- // Require 8 (reduced) row/cols to the up/left
- x = std::max(x, border);
- y = std::max(y, border);
-
- // Require 8 (reduced) row/cols to the down/left, plus the template size
- x = std::min(x, max_x);
- y = std::min(y, max_y);
-
- // Compute local similarity maps for each modality
- int numFeatures = 0;
- for (int i = 0; i < (int)modalities.size(); ++i)
- {
- const Template& templ = tp[start + i];
- numFeatures += static_cast<int>(templ.features.size());
- similarityLocal(lms[i], templ, similarities2[i], size, T, Point(x, y));
- }
- addSimilarities(similarities2, total_similarity2);
-
- // Find best local adjustment
- int best_score = 0;
- int best_r = -1, best_c = -1;
- for (int r = 0; r < total_similarity2.rows; ++r)
- {
- ushort* row = total_similarity2.ptr<ushort>(r);
- for (int c = 0; c < total_similarity2.cols; ++c)
- {
- int score = row[c];
- if (score > best_score)
- {
- best_score = score;
- best_r = r;
- best_c = c;
- }
- }
- }
- // Update current match
- match2.x = (x / T - 8 + best_c) * T + offset;
- match2.y = (y / T - 8 + best_r) * T + offset;
- match2.similarity = (best_score * 100.f) / (4 * numFeatures);
- }
-
- // Filter out any matches that drop below the similarity threshold
- std::vector<Match>::iterator new_end = std::remove_if(candidates.begin(), candidates.end(),
- MatchPredicate(threshold));
- candidates.erase(new_end, candidates.end());
- }
-
- matches.insert(matches.end(), candidates.begin(), candidates.end());
- }
-}
-
-int Detector::addTemplate(const std::vector<Mat>& sources, const String& class_id,
- const Mat& object_mask, Rect* bounding_box)
-{
- int num_modalities = static_cast<int>(modalities.size());
- std::vector<TemplatePyramid>& template_pyramids = class_templates[class_id];
- int template_id = static_cast<int>(template_pyramids.size());
-
- TemplatePyramid tp;
- tp.resize(num_modalities * pyramid_levels);
-
- // For each modality...
- for (int i = 0; i < num_modalities; ++i)
- {
- // Extract a template at each pyramid level
- Ptr<QuantizedPyramid> qp = modalities[i]->process(sources[i], object_mask);
- for (int l = 0; l < pyramid_levels; ++l)
- {
- /// @todo Could do mask subsampling here instead of in pyrDown()
- if (l > 0)
- qp->pyrDown();
-
- bool success = qp->extractTemplate(tp[l*num_modalities + i]);
- if (!success)
- return -1;
- }
- }
-
- Rect bb = cropTemplates(tp);
- if (bounding_box)
- *bounding_box = bb;
-
- /// @todo Can probably avoid a copy of tp here with swap
- template_pyramids.push_back(tp);
- return template_id;
-}
-
-int Detector::addSyntheticTemplate(const std::vector<Template>& templates, const String& class_id)
-{
- std::vector<TemplatePyramid>& template_pyramids = class_templates[class_id];
- int template_id = static_cast<int>(template_pyramids.size());
- template_pyramids.push_back(templates);
- return template_id;
-}
-
-const std::vector<Template>& Detector::getTemplates(const String& class_id, int template_id) const
-{
- TemplatesMap::const_iterator i = class_templates.find(class_id);
- CV_Assert(i != class_templates.end());
- CV_Assert(i->second.size() > size_t(template_id));
- return i->second[template_id];
-}
-
-int Detector::numTemplates() const
-{
- int ret = 0;
- TemplatesMap::const_iterator i = class_templates.begin(), iend = class_templates.end();
- for ( ; i != iend; ++i)
- ret += static_cast<int>(i->second.size());
- return ret;
-}
-
-int Detector::numTemplates(const String& class_id) const
-{
- TemplatesMap::const_iterator i = class_templates.find(class_id);
- if (i == class_templates.end())
- return 0;
- return static_cast<int>(i->second.size());
-}
-
-std::vector<String> Detector::classIds() const
-{
- std::vector<String> ids;
- TemplatesMap::const_iterator i = class_templates.begin(), iend = class_templates.end();
- for ( ; i != iend; ++i)
- {
- ids.push_back(i->first);
- }
-
- return ids;
-}
-
-void Detector::read(const FileNode& fn)
-{
- class_templates.clear();
- pyramid_levels = fn["pyramid_levels"];
- fn["T"] >> T_at_level;
-
- modalities.clear();
- FileNode modalities_fn = fn["modalities"];
- FileNodeIterator it = modalities_fn.begin(), it_end = modalities_fn.end();
- for ( ; it != it_end; ++it)
- {
- modalities.push_back(Modality::create(*it));
- }
-}
-
-void Detector::write(FileStorage& fs) const
-{
- fs << "pyramid_levels" << pyramid_levels;
- fs << "T" << T_at_level;
-
- fs << "modalities" << "[";
- for (int i = 0; i < (int)modalities.size(); ++i)
- {
- fs << "{";
- modalities[i]->write(fs);
- fs << "}";
- }
- fs << "]"; // modalities
-}
-
- String Detector::readClass(const FileNode& fn, const String &class_id_override)
- {
- // Verify compatible with Detector settings
- FileNode mod_fn = fn["modalities"];
- CV_Assert(mod_fn.size() == modalities.size());
- FileNodeIterator mod_it = mod_fn.begin(), mod_it_end = mod_fn.end();
- int i = 0;
- for ( ; mod_it != mod_it_end; ++mod_it, ++i)
- CV_Assert(modalities[i]->name() == (String)(*mod_it));
- CV_Assert((int)fn["pyramid_levels"] == pyramid_levels);
-
- // Detector should not already have this class
- String class_id;
- if (class_id_override.empty())
- {
- String class_id_tmp = fn["class_id"];
- CV_Assert(class_templates.find(class_id_tmp) == class_templates.end());
- class_id = class_id_tmp;
- }
- else
- {
- class_id = class_id_override;
- }
-
- TemplatesMap::value_type v(class_id, std::vector<TemplatePyramid>());
- std::vector<TemplatePyramid>& tps = v.second;
- int expected_id = 0;
-
- FileNode tps_fn = fn["template_pyramids"];
- tps.resize(tps_fn.size());
- FileNodeIterator tps_it = tps_fn.begin(), tps_it_end = tps_fn.end();
- for ( ; tps_it != tps_it_end; ++tps_it, ++expected_id)
- {
- int template_id = (*tps_it)["template_id"];
- CV_Assert(template_id == expected_id);
- FileNode templates_fn = (*tps_it)["templates"];
- tps[template_id].resize(templates_fn.size());
-
- FileNodeIterator templ_it = templates_fn.begin(), templ_it_end = templates_fn.end();
- int idx = 0;
- for ( ; templ_it != templ_it_end; ++templ_it)
- {
- tps[template_id][idx++].read(*templ_it);
- }
- }
-
- class_templates.insert(v);
- return class_id;
-}
-
-void Detector::writeClass(const String& class_id, FileStorage& fs) const
-{
- TemplatesMap::const_iterator it = class_templates.find(class_id);
- CV_Assert(it != class_templates.end());
- const std::vector<TemplatePyramid>& tps = it->second;
-
- fs << "class_id" << it->first;
- fs << "modalities" << "[:";
- for (size_t i = 0; i < modalities.size(); ++i)
- fs << modalities[i]->name();
- fs << "]"; // modalities
- fs << "pyramid_levels" << pyramid_levels;
- fs << "template_pyramids" << "[";
- for (size_t i = 0; i < tps.size(); ++i)
- {
- const TemplatePyramid& tp = tps[i];
- fs << "{";
- fs << "template_id" << int(i); //TODO is this cast correct? won't be good if rolls over...
- fs << "templates" << "[";
- for (size_t j = 0; j < tp.size(); ++j)
- {
- fs << "{";
- tp[j].write(fs);
- fs << "}"; // current template
- }
- fs << "]"; // templates
- fs << "}"; // current pyramid
- }
- fs << "]"; // pyramids
-}
-
-void Detector::readClasses(const std::vector<String>& class_ids,
- const String& format)
-{
- for (size_t i = 0; i < class_ids.size(); ++i)
- {
- const String& class_id = class_ids[i];
- String filename = cv::format(format.c_str(), class_id.c_str());
- FileStorage fs(filename, FileStorage::READ);
- readClass(fs.root());
- }
-}
-
-void Detector::writeClasses(const String& format) const
-{
- TemplatesMap::const_iterator it = class_templates.begin(), it_end = class_templates.end();
- for ( ; it != it_end; ++it)
- {
- const String& class_id = it->first;
- String filename = cv::format(format.c_str(), class_id.c_str());
- FileStorage fs(filename, FileStorage::WRITE);
- writeClass(class_id, fs);
- }
-}
-
-static const int T_DEFAULTS[] = {5, 8};
-
-Ptr<Detector> getDefaultLINE()
-{
- std::vector< Ptr<Modality> > modalities;
- modalities.push_back(makePtr<ColorGradient>());
- return makePtr<Detector>(modalities, std::vector<int>(T_DEFAULTS, T_DEFAULTS + 2));
-}
-
-Ptr<Detector> getDefaultLINEMOD()
-{
- std::vector< Ptr<Modality> > modalities;
- modalities.push_back(makePtr<ColorGradient>());
- modalities.push_back(makePtr<DepthNormal>());
- return makePtr<Detector>(modalities, std::vector<int>(T_DEFAULTS, T_DEFAULTS + 2));
-}
-
-} // namespace linemod
-} // namespace cv
+++ /dev/null
-// Auto-generated by scripts/create_depth_normal_lut.py 20
-static const int GRANULARITY = 20;
-
-static const unsigned char NORMAL_LUT[20][20][20] = {{{32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128, 128}, {16, 32, 32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 128, 128, 128, 128, 128, 128, 128, 128}, {16, 16, 16, 32, 32, 32, 32, 32, 32, 64, 64, 64, 128, 128, 128, 128, 128, 128, 1, 1}, {16, 16, 16, 16, 16, 16, 32, 32, 32, 32, 64, 128, 128, 128, 128, 1, 1, 1, 1, 1}, {16, 16, 16, 16, 16, 16, 16, 16, 32, 32, 64, 128, 128, 1, 1, 1, 1, 1, 1, 1}, {16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1}, {16, 16, 16, 16, 16, 16, 16, 16, 8, 8, 4, 2, 2, 1, 1, 1, 1, 1, 1, 1}, {16, 16, 16, 16, 16, 16, 8, 8, 8, 8, 4, 2, 2, 2, 2, 1, 1, 1, 1, 1}, {16, 16, 16, 8, 8, 8, 8, 8, 8, 4, 4, 4, 2, 2, 2, 2, 2, 2, 1, 1}, {16, 8, 8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 2, 2, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2}}, {{32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128, 128}, {16, 32, 32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 128, 128, 128, 128, 128, 128, 128, 128}, {16, 16, 16, 32, 32, 32, 32, 32, 32, 64, 64, 64, 128, 128, 128, 128, 128, 128, 1, 1}, {16, 16, 16, 16, 16, 16, 32, 32, 32, 32, 64, 128, 128, 128, 128, 1, 1, 1, 1, 1}, {16, 16, 16, 16, 16, 16, 16, 16, 32, 32, 64, 128, 128, 1, 1, 1, 1, 1, 1, 1}, {16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1}, {16, 16, 16, 16, 16, 16, 16, 16, 8, 8, 4, 2, 2, 1, 1, 1, 1, 1, 1, 1}, {16, 16, 16, 16, 16, 16, 8, 8, 8, 8, 4, 2, 2, 2, 2, 1, 1, 1, 1, 1}, {16, 16, 16, 8, 8, 8, 8, 8, 8, 4, 4, 4, 2, 2, 2, 2, 2, 2, 1, 1}, {16, 8, 8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 2, 2, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2}}, {{32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128, 128}, {16, 32, 32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 128, 128, 128, 128, 128, 128, 128, 128}, {16, 16, 16, 32, 32, 32, 32, 32, 32, 64, 64, 64, 128, 128, 128, 128, 128, 128, 1, 1}, {16, 16, 16, 16, 16, 16, 32, 32, 32, 32, 64, 128, 128, 128, 128, 1, 1, 1, 1, 1}, {16, 16, 16, 16, 16, 16, 16, 16, 32, 32, 64, 128, 128, 1, 1, 1, 1, 1, 1, 1}, {16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1}, {16, 16, 16, 16, 16, 16, 16, 16, 8, 8, 4, 2, 2, 1, 1, 1, 1, 1, 1, 1}, {16, 16, 16, 16, 16, 16, 8, 8, 8, 8, 4, 2, 2, 2, 2, 1, 1, 1, 1, 1}, {16, 16, 16, 8, 8, 8, 8, 8, 8, 4, 4, 4, 2, 2, 2, 2, 2, 2, 1, 1}, {16, 8, 8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 2, 2, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2}}, {{32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128, 128}, {16, 32, 32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 128, 128, 128, 128, 128, 128, 128, 128}, {16, 16, 16, 32, 32, 32, 32, 32, 32, 64, 64, 64, 128, 128, 128, 128, 128, 128, 1, 1}, {16, 16, 16, 16, 16, 16, 32, 32, 32, 32, 64, 128, 128, 128, 128, 1, 1, 1, 1, 1}, {16, 16, 16, 16, 16, 16, 16, 16, 32, 32, 64, 128, 128, 1, 1, 1, 1, 1, 1, 1}, {16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1}, {16, 16, 16, 16, 16, 16, 16, 16, 8, 8, 4, 2, 2, 1, 1, 1, 1, 1, 1, 1}, {16, 16, 16, 16, 16, 16, 8, 8, 8, 8, 4, 2, 2, 2, 2, 1, 1, 1, 1, 1}, {16, 16, 16, 8, 8, 8, 8, 8, 8, 4, 4, 4, 2, 2, 2, 2, 2, 2, 1, 1}, {16, 8, 8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 2, 2, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2}}, {{32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128, 128}, {16, 32, 32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 128, 128, 128, 128, 128, 128, 128, 128}, {16, 16, 16, 32, 32, 32, 32, 32, 32, 64, 64, 64, 128, 128, 128, 128, 128, 128, 1, 1}, {16, 16, 16, 16, 16, 16, 32, 32, 32, 32, 64, 128, 128, 128, 128, 1, 1, 1, 1, 1}, {16, 16, 16, 16, 16, 16, 16, 16, 32, 32, 64, 128, 128, 1, 1, 1, 1, 1, 1, 1}, {16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1}, {16, 16, 16, 16, 16, 16, 16, 16, 8, 8, 4, 2, 2, 1, 1, 1, 1, 1, 1, 1}, {16, 16, 16, 16, 16, 16, 8, 8, 8, 8, 4, 2, 2, 2, 2, 1, 1, 1, 1, 1}, {16, 16, 16, 8, 8, 8, 8, 8, 8, 4, 4, 4, 2, 2, 2, 2, 2, 2, 1, 1}, {16, 8, 8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 2, 2, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2}}, {{32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128, 128}, {16, 32, 32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 128, 128, 128, 128, 128, 128, 128, 128}, {16, 16, 16, 32, 32, 32, 32, 32, 32, 64, 64, 64, 128, 128, 128, 128, 128, 128, 1, 1}, {16, 16, 16, 16, 16, 16, 32, 32, 32, 32, 64, 128, 128, 128, 128, 1, 1, 1, 1, 1}, {16, 16, 16, 16, 16, 16, 16, 16, 32, 32, 64, 128, 128, 1, 1, 1, 1, 1, 1, 1}, {16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1}, {16, 16, 16, 16, 16, 16, 16, 16, 8, 8, 4, 2, 2, 1, 1, 1, 1, 1, 1, 1}, {16, 16, 16, 16, 16, 16, 8, 8, 8, 8, 4, 2, 2, 2, 2, 1, 1, 1, 1, 1}, {16, 16, 16, 8, 8, 8, 8, 8, 8, 4, 4, 4, 2, 2, 2, 2, 2, 2, 1, 1}, {16, 8, 8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 2, 2, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2}}, {{32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128, 128}, {16, 32, 32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 128, 128, 128, 128, 128, 128, 128, 128}, {16, 16, 16, 32, 32, 32, 32, 32, 32, 64, 64, 64, 128, 128, 128, 128, 128, 128, 1, 1}, {16, 16, 16, 16, 16, 16, 32, 32, 32, 32, 64, 128, 128, 128, 128, 1, 1, 1, 1, 1}, {16, 16, 16, 16, 16, 16, 16, 16, 32, 32, 64, 128, 128, 1, 1, 1, 1, 1, 1, 1}, {16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1}, {16, 16, 16, 16, 16, 16, 16, 16, 8, 8, 4, 2, 2, 1, 1, 1, 1, 1, 1, 1}, {16, 16, 16, 16, 16, 16, 8, 8, 8, 8, 4, 2, 2, 2, 2, 1, 1, 1, 1, 1}, {16, 16, 16, 8, 8, 8, 8, 8, 8, 4, 4, 4, 2, 2, 2, 2, 2, 2, 1, 1}, {16, 8, 8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 2, 2, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2}}, {{32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128, 128}, {16, 32, 32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 128, 128, 128, 128, 128, 128, 128, 128}, {16, 16, 16, 32, 32, 32, 32, 32, 32, 64, 64, 64, 128, 128, 128, 128, 128, 128, 1, 1}, {16, 16, 16, 16, 16, 16, 32, 32, 32, 32, 64, 128, 128, 128, 128, 1, 1, 1, 1, 1}, {16, 16, 16, 16, 16, 16, 16, 16, 32, 32, 64, 128, 128, 1, 1, 1, 1, 1, 1, 1}, {16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1}, {16, 16, 16, 16, 16, 16, 16, 16, 8, 8, 4, 2, 2, 1, 1, 1, 1, 1, 1, 1}, {16, 16, 16, 16, 16, 16, 8, 8, 8, 8, 4, 2, 2, 2, 2, 1, 1, 1, 1, 1}, {16, 16, 16, 8, 8, 8, 8, 8, 8, 4, 4, 4, 2, 2, 2, 2, 2, 2, 1, 1}, {16, 8, 8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 2, 2, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2}}, {{32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128, 128}, {16, 32, 32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 128, 128, 128, 128, 128, 128, 128, 128}, {16, 16, 16, 32, 32, 32, 32, 32, 32, 64, 64, 64, 128, 128, 128, 128, 128, 128, 1, 1}, {16, 16, 16, 16, 16, 16, 32, 32, 32, 32, 64, 128, 128, 128, 128, 1, 1, 1, 1, 1}, {16, 16, 16, 16, 16, 16, 16, 16, 32, 32, 64, 128, 128, 1, 1, 1, 1, 1, 1, 1}, {16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1}, {16, 16, 16, 16, 16, 16, 16, 16, 8, 8, 4, 2, 2, 1, 1, 1, 1, 1, 1, 1}, {16, 16, 16, 16, 16, 16, 8, 8, 8, 8, 4, 2, 2, 2, 2, 1, 1, 1, 1, 1}, {16, 16, 16, 8, 8, 8, 8, 8, 8, 4, 4, 4, 2, 2, 2, 2, 2, 2, 1, 1}, {16, 8, 8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 2, 2, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2}}, {{32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128, 128}, {16, 32, 32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 128, 128, 128, 128, 128, 128, 128, 128}, {16, 16, 16, 32, 32, 32, 32, 32, 32, 64, 64, 64, 128, 128, 128, 128, 128, 128, 1, 1}, {16, 16, 16, 16, 16, 16, 32, 32, 32, 32, 64, 128, 128, 128, 128, 1, 1, 1, 1, 1}, {16, 16, 16, 16, 16, 16, 16, 16, 32, 32, 64, 128, 128, 1, 1, 1, 1, 1, 1, 1}, {16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1}, {16, 16, 16, 16, 16, 16, 16, 16, 8, 8, 4, 2, 2, 1, 1, 1, 1, 1, 1, 1}, {16, 16, 16, 16, 16, 16, 8, 8, 8, 8, 4, 2, 2, 2, 2, 1, 1, 1, 1, 1}, {16, 16, 16, 8, 8, 8, 8, 8, 8, 4, 4, 4, 2, 2, 2, 2, 2, 2, 1, 1}, {16, 8, 8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 2, 2, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2}}, {{32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128, 128}, {16, 32, 32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 128, 128, 128, 128, 128, 128, 128, 128}, {16, 16, 16, 32, 32, 32, 32, 32, 32, 64, 64, 64, 128, 128, 128, 128, 128, 128, 1, 1}, {16, 16, 16, 16, 16, 16, 32, 32, 32, 32, 64, 128, 128, 128, 128, 1, 1, 1, 1, 1}, {16, 16, 16, 16, 16, 16, 16, 16, 32, 32, 64, 128, 128, 1, 1, 1, 1, 1, 1, 1}, {16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1}, {16, 16, 16, 16, 16, 16, 16, 16, 8, 8, 4, 2, 2, 1, 1, 1, 1, 1, 1, 1}, {16, 16, 16, 16, 16, 16, 8, 8, 8, 8, 4, 2, 2, 2, 2, 1, 1, 1, 1, 1}, {16, 16, 16, 8, 8, 8, 8, 8, 8, 4, 4, 4, 2, 2, 2, 2, 2, 2, 1, 1}, {16, 8, 8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 2, 2, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2}}, {{32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128, 128}, {16, 32, 32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 128, 128, 128, 128, 128, 128, 128, 128}, {16, 16, 16, 32, 32, 32, 32, 32, 32, 64, 64, 64, 128, 128, 128, 128, 128, 128, 1, 1}, {16, 16, 16, 16, 16, 16, 32, 32, 32, 32, 64, 128, 128, 128, 128, 1, 1, 1, 1, 1}, {16, 16, 16, 16, 16, 16, 16, 16, 32, 32, 64, 128, 128, 1, 1, 1, 1, 1, 1, 1}, {16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1}, {16, 16, 16, 16, 16, 16, 16, 16, 8, 8, 4, 2, 2, 1, 1, 1, 1, 1, 1, 1}, {16, 16, 16, 16, 16, 16, 8, 8, 8, 8, 4, 2, 2, 2, 2, 1, 1, 1, 1, 1}, {16, 16, 16, 8, 8, 8, 8, 8, 8, 4, 4, 4, 2, 2, 2, 2, 2, 2, 1, 1}, {16, 8, 8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 2, 2, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2}}, {{32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128, 128}, {16, 32, 32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 128, 128, 128, 128, 128, 128, 128, 128}, {16, 16, 16, 32, 32, 32, 32, 32, 32, 64, 64, 64, 128, 128, 128, 128, 128, 128, 1, 1}, {16, 16, 16, 16, 16, 16, 32, 32, 32, 32, 64, 128, 128, 128, 128, 1, 1, 1, 1, 1}, {16, 16, 16, 16, 16, 16, 16, 16, 32, 32, 64, 128, 128, 1, 1, 1, 1, 1, 1, 1}, {16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1}, {16, 16, 16, 16, 16, 16, 16, 16, 8, 8, 4, 2, 2, 1, 1, 1, 1, 1, 1, 1}, {16, 16, 16, 16, 16, 16, 8, 8, 8, 8, 4, 2, 2, 2, 2, 1, 1, 1, 1, 1}, {16, 16, 16, 8, 8, 8, 8, 8, 8, 4, 4, 4, 2, 2, 2, 2, 2, 2, 1, 1}, {16, 8, 8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 2, 2, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2}}, {{32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128, 128}, {16, 32, 32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 128, 128, 128, 128, 128, 128, 128, 128}, {16, 16, 16, 32, 32, 32, 32, 32, 32, 64, 64, 64, 128, 128, 128, 128, 128, 128, 1, 1}, {16, 16, 16, 16, 16, 16, 32, 32, 32, 32, 64, 128, 128, 128, 128, 1, 1, 1, 1, 1}, {16, 16, 16, 16, 16, 16, 16, 16, 32, 32, 64, 128, 128, 1, 1, 1, 1, 1, 1, 1}, {16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1}, {16, 16, 16, 16, 16, 16, 16, 16, 8, 8, 4, 2, 2, 1, 1, 1, 1, 1, 1, 1}, {16, 16, 16, 16, 16, 16, 8, 8, 8, 8, 4, 2, 2, 2, 2, 1, 1, 1, 1, 1}, {16, 16, 16, 8, 8, 8, 8, 8, 8, 4, 4, 4, 2, 2, 2, 2, 2, 2, 1, 1}, {16, 8, 8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 2, 2, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2}}, {{32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128, 128}, {16, 32, 32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 128, 128, 128, 128, 128, 128, 128, 128}, {16, 16, 16, 32, 32, 32, 32, 32, 32, 64, 64, 64, 128, 128, 128, 128, 128, 128, 1, 1}, {16, 16, 16, 16, 16, 16, 32, 32, 32, 32, 64, 128, 128, 128, 128, 1, 1, 1, 1, 1}, {16, 16, 16, 16, 16, 16, 16, 16, 32, 32, 64, 128, 128, 1, 1, 1, 1, 1, 1, 1}, {16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1}, {16, 16, 16, 16, 16, 16, 16, 16, 8, 8, 4, 2, 2, 1, 1, 1, 1, 1, 1, 1}, {16, 16, 16, 16, 16, 16, 8, 8, 8, 8, 4, 2, 2, 2, 2, 1, 1, 1, 1, 1}, {16, 16, 16, 8, 8, 8, 8, 8, 8, 4, 4, 4, 2, 2, 2, 2, 2, 2, 1, 1}, {16, 8, 8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 2, 2, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2}}, {{32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128, 128}, {16, 32, 32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 128, 128, 128, 128, 128, 128, 128, 128}, {16, 16, 16, 32, 32, 32, 32, 32, 32, 64, 64, 64, 128, 128, 128, 128, 128, 128, 1, 1}, {16, 16, 16, 16, 16, 16, 32, 32, 32, 32, 64, 128, 128, 128, 128, 1, 1, 1, 1, 1}, {16, 16, 16, 16, 16, 16, 16, 16, 32, 32, 64, 128, 128, 1, 1, 1, 1, 1, 1, 1}, {16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1}, {16, 16, 16, 16, 16, 16, 16, 16, 8, 8, 4, 2, 2, 1, 1, 1, 1, 1, 1, 1}, {16, 16, 16, 16, 16, 16, 8, 8, 8, 8, 4, 2, 2, 2, 2, 1, 1, 1, 1, 1}, {16, 16, 16, 8, 8, 8, 8, 8, 8, 4, 4, 4, 2, 2, 2, 2, 2, 2, 1, 1}, {16, 8, 8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 2, 2, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2}}, {{32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128, 128}, {16, 32, 32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 128, 128, 128, 128, 128, 128, 128, 128}, {16, 16, 16, 32, 32, 32, 32, 32, 32, 64, 64, 64, 128, 128, 128, 128, 128, 128, 1, 1}, {16, 16, 16, 16, 16, 16, 32, 32, 32, 32, 64, 128, 128, 128, 128, 1, 1, 1, 1, 1}, {16, 16, 16, 16, 16, 16, 16, 16, 32, 32, 64, 128, 128, 1, 1, 1, 1, 1, 1, 1}, {16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1}, {16, 16, 16, 16, 16, 16, 16, 16, 8, 8, 4, 2, 2, 1, 1, 1, 1, 1, 1, 1}, {16, 16, 16, 16, 16, 16, 8, 8, 8, 8, 4, 2, 2, 2, 2, 1, 1, 1, 1, 1}, {16, 16, 16, 8, 8, 8, 8, 8, 8, 4, 4, 4, 2, 2, 2, 2, 2, 2, 1, 1}, {16, 8, 8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 2, 2, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2}}, {{32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128, 128}, {16, 32, 32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 128, 128, 128, 128, 128, 128, 128, 128}, {16, 16, 16, 32, 32, 32, 32, 32, 32, 64, 64, 64, 128, 128, 128, 128, 128, 128, 1, 1}, {16, 16, 16, 16, 16, 16, 32, 32, 32, 32, 64, 128, 128, 128, 128, 1, 1, 1, 1, 1}, {16, 16, 16, 16, 16, 16, 16, 16, 32, 32, 64, 128, 128, 1, 1, 1, 1, 1, 1, 1}, {16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1}, {16, 16, 16, 16, 16, 16, 16, 16, 8, 8, 4, 2, 2, 1, 1, 1, 1, 1, 1, 1}, {16, 16, 16, 16, 16, 16, 8, 8, 8, 8, 4, 2, 2, 2, 2, 1, 1, 1, 1, 1}, {16, 16, 16, 8, 8, 8, 8, 8, 8, 4, 4, 4, 2, 2, 2, 2, 2, 2, 1, 1}, {16, 8, 8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 2, 2, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2}}, {{32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128, 128}, {16, 32, 32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 128, 128, 128, 128, 128, 128, 128, 128}, {16, 16, 16, 32, 32, 32, 32, 32, 32, 64, 64, 64, 128, 128, 128, 128, 128, 128, 1, 1}, {16, 16, 16, 16, 16, 16, 32, 32, 32, 32, 64, 128, 128, 128, 128, 1, 1, 1, 1, 1}, {16, 16, 16, 16, 16, 16, 16, 16, 32, 32, 64, 128, 128, 1, 1, 1, 1, 1, 1, 1}, {16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1}, {16, 16, 16, 16, 16, 16, 16, 16, 8, 8, 4, 2, 2, 1, 1, 1, 1, 1, 1, 1}, {16, 16, 16, 16, 16, 16, 8, 8, 8, 8, 4, 2, 2, 2, 2, 1, 1, 1, 1, 1}, {16, 16, 16, 8, 8, 8, 8, 8, 8, 4, 4, 4, 2, 2, 2, 2, 2, 2, 1, 1}, {16, 8, 8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 2, 2, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2}}, {{32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128, 128}, {32, 32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 64, 64, 128, 128, 128, 128, 128, 128, 128}, {16, 32, 32, 32, 32, 32, 32, 32, 32, 64, 64, 64, 128, 128, 128, 128, 128, 128, 128, 128}, {16, 16, 16, 32, 32, 32, 32, 32, 32, 64, 64, 64, 128, 128, 128, 128, 128, 128, 1, 1}, {16, 16, 16, 16, 16, 16, 32, 32, 32, 32, 64, 128, 128, 128, 128, 1, 1, 1, 1, 1}, {16, 16, 16, 16, 16, 16, 16, 16, 32, 32, 64, 128, 128, 1, 1, 1, 1, 1, 1, 1}, {16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1}, {16, 16, 16, 16, 16, 16, 16, 16, 8, 8, 4, 2, 2, 1, 1, 1, 1, 1, 1, 1}, {16, 16, 16, 16, 16, 16, 8, 8, 8, 8, 4, 2, 2, 2, 2, 1, 1, 1, 1, 1}, {16, 16, 16, 8, 8, 8, 8, 8, 8, 4, 4, 4, 2, 2, 2, 2, 2, 2, 1, 1}, {16, 8, 8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 2, 2, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2}, {8, 8, 8, 8, 8, 8, 8, 4, 4, 4, 4, 4, 4, 4, 2, 2, 2, 2, 2, 2}}};
+++ /dev/null
-<?xml version="1.0" encoding="UTF-8"?>
-<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
-<plist version="1.0">
-<dict>
- <key>CFBundleDevelopmentRegion</key>
- <string>English</string>
- <key>CFBundleExecutable</key>
- <string>${EXECUTABLE_NAME}</string>
- <key>CFBundleIdentifier</key>
- <string>de.rwth-aachen.ient.FaceTracker</string>
- <key>CFBundleInfoDictionaryVersion</key>
- <string>6.0</string>
- <key>CFBundlePackageType</key>
- <string>APPL</string>
- <key>CFBundleSignature</key>
- <string>????</string>
- <key>CFBundleVersion</key>
- <string>1.0</string>
-</dict>
-</plist>
+++ /dev/null
-
-#include <OpenCV/OpenCV.h>
-#include <cassert>
-#include <iostream>
-
-
-const char * WINDOW_NAME = "Face Tracker";
-const CFIndex CASCADE_NAME_LEN = 2048;
- char CASCADE_NAME[CASCADE_NAME_LEN] = "~/opencv/data/haarcascades/haarcascade_frontalface_alt2.xml";
-
-using namespace std;
-
-int main (int argc, char * const argv[])
-{
- const int scale = 2;
-
- // locate haar cascade from inside application bundle
- // (this is the mac way to package application resources)
- CFBundleRef mainBundle = CFBundleGetMainBundle ();
- assert (mainBundle);
- CFURLRef cascade_url = CFBundleCopyResourceURL (mainBundle, CFSTR("haarcascade_frontalface_alt2"), CFSTR("xml"), NULL);
- assert (cascade_url);
- Boolean got_it = CFURLGetFileSystemRepresentation (cascade_url, true,
- reinterpret_cast<UInt8 *>(CASCADE_NAME), CASCADE_NAME_LEN);
- if (! got_it)
- abort ();
-
- // create all necessary instances
- cvNamedWindow (WINDOW_NAME, CV_WINDOW_AUTOSIZE);
- CvCapture * camera = cvCreateCameraCapture (CV_CAP_ANY);
- CvHaarClassifierCascade* cascade = (CvHaarClassifierCascade*) cvLoad (CASCADE_NAME, 0, 0, 0);
- CvMemStorage* storage = cvCreateMemStorage(0);
- assert (storage);
-
- // you do own an iSight, don't you ?!?
- if (! camera)
- abort ();
-
- // did we load the cascade?!?
- if (! cascade)
- abort ();
-
- // get an initial frame and duplicate it for later work
- IplImage * current_frame = cvQueryFrame (camera);
- IplImage * draw_image = cvCreateImage(cvSize (current_frame->width, current_frame->height), IPL_DEPTH_8U, 3);
- IplImage * gray_image = cvCreateImage(cvSize (current_frame->width, current_frame->height), IPL_DEPTH_8U, 1);
- IplImage * small_image = cvCreateImage(cvSize (current_frame->width / scale, current_frame->height / scale), IPL_DEPTH_8U, 1);
- assert (current_frame && gray_image && draw_image);
-
- // as long as there are images ...
- while (current_frame = cvQueryFrame (camera))
- {
- // convert to gray and downsize
- cvCvtColor (current_frame, gray_image, CV_BGR2GRAY);
- cvResize (gray_image, small_image, CV_INTER_LINEAR);
-
- // detect faces
- CvSeq* faces = cvHaarDetectObjects (small_image, cascade, storage,
- 1.1, 2, CV_HAAR_DO_CANNY_PRUNING,
- cvSize (30, 30));
-
- // draw faces
- cvFlip (current_frame, draw_image, 1);
- for (int i = 0; i < (faces ? faces->total : 0); i++)
- {
- CvRect* r = (CvRect*) cvGetSeqElem (faces, i);
- CvPoint center;
- int radius;
- center.x = cvRound((small_image->width - r->width*0.5 - r->x) *scale);
- center.y = cvRound((r->y + r->height*0.5)*scale);
- radius = cvRound((r->width + r->height)*0.25*scale);
- cvCircle (draw_image, center, radius, CV_RGB(0,255,0), 3, 8, 0 );
- }
-
- // just show the image
- cvShowImage (WINDOW_NAME, draw_image);
-
- // wait a tenth of a second for keypress and window drawing
- int key = cvWaitKey (100);
- if (key == 'q' || key == 'Q')
- break;
- }
-
- // be nice and return no error
- return 0;
-}
+++ /dev/null
-// !$*UTF8*$!
-{
- archiveVersion = 1;
- classes = {
- };
- objectVersion = 42;
- objects = {
-
-/* Begin PBXBuildFile section */
- 4D7DBE8E0C04A90C00D8835D /* FaceTracker.cpp in Sources */ = {isa = PBXBuildFile; fileRef = 08FB7796FE84155DC02AAC07 /* FaceTracker.cpp */; };
- 4D95C9BE0C0577B200983E4D /* OpenCV.framework in Frameworks */ = {isa = PBXBuildFile; fileRef = 4D06E1E00C039982004AF23F /* OpenCV.framework */; };
- 4D95C9D80C0577BD00983E4D /* OpenCV.framework in CopyFiles */ = {isa = PBXBuildFile; fileRef = 4D06E1E00C039982004AF23F /* OpenCV.framework */; };
- 4DBF87310C05731500880673 /* haarcascade_frontalface_alt2.xml in Resources */ = {isa = PBXBuildFile; fileRef = 4DBF87300C05731500880673 /* haarcascade_frontalface_alt2.xml */; };
-/* End PBXBuildFile section */
-
-/* Begin PBXCopyFilesBuildPhase section */
- 4D7DBE8F0C04A93300D8835D /* CopyFiles */ = {
- isa = PBXCopyFilesBuildPhase;
- buildActionMask = 2147483647;
- dstPath = "";
- dstSubfolderSpec = 10;
- files = (
- 4D95C9D80C0577BD00983E4D /* OpenCV.framework in CopyFiles */,
- );
- runOnlyForDeploymentPostprocessing = 0;
- };
-/* End PBXCopyFilesBuildPhase section */
-
-/* Begin PBXFileReference section */
- 08FB7796FE84155DC02AAC07 /* FaceTracker.cpp */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = sourcecode.cpp.cpp; path = FaceTracker.cpp; sourceTree = "<group>"; };
- 4D06E1E00C039982004AF23F /* OpenCV.framework */ = {isa = PBXFileReference; lastKnownFileType = wrapper.framework; name = OpenCV.framework; path = ../../../OpenCV.framework; sourceTree = SOURCE_ROOT; };
- 4D4CDBCC0C0630060001A8A2 /* README.txt */ = {isa = PBXFileReference; fileEncoding = 4; lastKnownFileType = text; path = README.txt; sourceTree = "<group>"; };
- 4D7DBE570C04A8FF00D8835D /* FaceTracker.app */ = {isa = PBXFileReference; explicitFileType = wrapper.application; includeInIndex = 0; path = FaceTracker.app; sourceTree = BUILT_PRODUCTS_DIR; };
- 4D7DBE590C04A8FF00D8835D /* FaceTracker-Info.plist */ = {isa = PBXFileReference; lastKnownFileType = text.xml; path = "FaceTracker-Info.plist"; sourceTree = "<group>"; };
- 4DBF87300C05731500880673 /* haarcascade_frontalface_alt2.xml */ = {isa = PBXFileReference; fileEncoding = 5; lastKnownFileType = text.xml; name = haarcascade_frontalface_alt2.xml; path = ../../../data/haarcascades/haarcascade_frontalface_alt2.xml; sourceTree = SOURCE_ROOT; };
-/* End PBXFileReference section */
-
-/* Begin PBXFrameworksBuildPhase section */
- 4D7DBE550C04A8FF00D8835D /* Frameworks */ = {
- isa = PBXFrameworksBuildPhase;
- buildActionMask = 2147483647;
- files = (
- 4D95C9BE0C0577B200983E4D /* OpenCV.framework in Frameworks */,
- );
- runOnlyForDeploymentPostprocessing = 0;
- };
-/* End PBXFrameworksBuildPhase section */
-
-/* Begin PBXGroup section */
- 08FB7794FE84155DC02AAC07 /* FrameworkTest */ = {
- isa = PBXGroup;
- children = (
- 4D4CDBCC0C0630060001A8A2 /* README.txt */,
- 08FB7795FE84155DC02AAC07 /* Source */,
- 4DBF872C0C0572BC00880673 /* Resources */,
- 4D9D40B00C04AC1600EEFFD0 /* Frameworks */,
- 1AB674ADFE9D54B511CA2CBB /* Products */,
- );
- name = FrameworkTest;
- sourceTree = "<group>";
- };
- 08FB7795FE84155DC02AAC07 /* Source */ = {
- isa = PBXGroup;
- children = (
- 08FB7796FE84155DC02AAC07 /* FaceTracker.cpp */,
- );
- name = Source;
- sourceTree = "<group>";
- };
- 1AB674ADFE9D54B511CA2CBB /* Products */ = {
- isa = PBXGroup;
- children = (
- 4D7DBE570C04A8FF00D8835D /* FaceTracker.app */,
- );
- name = Products;
- sourceTree = "<group>";
- };
- 4D9D40B00C04AC1600EEFFD0 /* Frameworks */ = {
- isa = PBXGroup;
- children = (
- 4D06E1E00C039982004AF23F /* OpenCV.framework */,
- );
- name = Frameworks;
- sourceTree = "<group>";
- };
- 4DBF872C0C0572BC00880673 /* Resources */ = {
- isa = PBXGroup;
- children = (
- 4DBF87300C05731500880673 /* haarcascade_frontalface_alt2.xml */,
- 4D7DBE590C04A8FF00D8835D /* FaceTracker-Info.plist */,
- );
- name = Resources;
- sourceTree = "<group>";
- };
-/* End PBXGroup section */
-
-/* Begin PBXNativeTarget section */
- 4D7DBE560C04A8FF00D8835D /* FaceTracker */ = {
- isa = PBXNativeTarget;
- buildConfigurationList = 4D7DBE5A0C04A8FF00D8835D /* Build configuration list for PBXNativeTarget "FaceTracker" */;
- buildPhases = (
- 4D7DBE530C04A8FF00D8835D /* Resources */,
- 4D7DBE540C04A8FF00D8835D /* Sources */,
- 4D7DBE550C04A8FF00D8835D /* Frameworks */,
- 4D7DBE8F0C04A93300D8835D /* CopyFiles */,
- );
- buildRules = (
- );
- dependencies = (
- );
- name = FaceTracker;
- productName = FaceTracker;
- productReference = 4D7DBE570C04A8FF00D8835D /* FaceTracker.app */;
- productType = "com.apple.product-type.application";
- };
-/* End PBXNativeTarget section */
-
-/* Begin PBXProject section */
- 08FB7793FE84155DC02AAC07 /* Project object */ = {
- isa = PBXProject;
- buildConfigurationList = 1DEB923508733DC60010E9CD /* Build configuration list for PBXProject "FaceTracker" */;
- hasScannedForEncodings = 1;
- mainGroup = 08FB7794FE84155DC02AAC07 /* FrameworkTest */;
- projectDirPath = "";
- targets = (
- 4D7DBE560C04A8FF00D8835D /* FaceTracker */,
- );
- };
-/* End PBXProject section */
-
-/* Begin PBXResourcesBuildPhase section */
- 4D7DBE530C04A8FF00D8835D /* Resources */ = {
- isa = PBXResourcesBuildPhase;
- buildActionMask = 2147483647;
- files = (
- 4DBF87310C05731500880673 /* haarcascade_frontalface_alt2.xml in Resources */,
- );
- runOnlyForDeploymentPostprocessing = 0;
- };
-/* End PBXResourcesBuildPhase section */
-
-/* Begin PBXSourcesBuildPhase section */
- 4D7DBE540C04A8FF00D8835D /* Sources */ = {
- isa = PBXSourcesBuildPhase;
- buildActionMask = 2147483647;
- files = (
- 4D7DBE8E0C04A90C00D8835D /* FaceTracker.cpp in Sources */,
- );
- runOnlyForDeploymentPostprocessing = 0;
- };
-/* End PBXSourcesBuildPhase section */
-
-/* Begin XCBuildConfiguration section */
- 1DEB923608733DC60010E9CD /* Debug */ = {
- isa = XCBuildConfiguration;
- buildSettings = {
- GCC_WARN_ABOUT_RETURN_TYPE = YES;
- GCC_WARN_UNUSED_VARIABLE = YES;
- PREBINDING = NO;
- SDKROOT = /Developer/SDKs/MacOSX10.4u.sdk;
- };
- name = Debug;
- };
- 1DEB923708733DC60010E9CD /* Release */ = {
- isa = XCBuildConfiguration;
- buildSettings = {
- ARCHS = (
- ppc,
- i386,
- );
- GCC_WARN_ABOUT_RETURN_TYPE = YES;
- GCC_WARN_UNUSED_VARIABLE = YES;
- PREBINDING = NO;
- SDKROOT = /Developer/SDKs/MacOSX10.4u.sdk;
- };
- name = Release;
- };
- 4D7DBE5B0C04A8FF00D8835D /* Debug */ = {
- isa = XCBuildConfiguration;
- buildSettings = {
- COPY_PHASE_STRIP = NO;
- FRAMEWORK_SEARCH_PATHS = (
- "$(inherited)",
- "$(FRAMEWORK_SEARCH_PATHS_QUOTED_1)",
- "$(FRAMEWORK_SEARCH_PATHS_QUOTED_2)",
- );
- FRAMEWORK_SEARCH_PATHS_QUOTED_1 = "\"$(SRCROOT)/../opencv\"";
- FRAMEWORK_SEARCH_PATHS_QUOTED_2 = "\"$(SRCROOT)/../../..\"";
- GCC_DYNAMIC_NO_PIC = NO;
- GCC_ENABLE_FIX_AND_CONTINUE = YES;
- GCC_GENERATE_DEBUGGING_SYMBOLS = YES;
- GCC_MODEL_TUNING = G5;
- GCC_OPTIMIZATION_LEVEL = 0;
- GCC_PRECOMPILE_PREFIX_HEADER = YES;
- GCC_PREFIX_HEADER = "$(SYSTEM_LIBRARY_DIR)/Frameworks/Carbon.framework/Headers/Carbon.h";
- INFOPLIST_FILE = "FaceTracker-Info.plist";
- INSTALL_PATH = "$(HOME)/Applications";
- OTHER_LDFLAGS = (
- "-framework",
- Carbon,
- );
- PREBINDING = NO;
- PRODUCT_NAME = FaceTracker;
- WRAPPER_EXTENSION = app;
- ZERO_LINK = YES;
- };
- name = Debug;
- };
- 4D7DBE5C0C04A8FF00D8835D /* Release */ = {
- isa = XCBuildConfiguration;
- buildSettings = {
- COPY_PHASE_STRIP = YES;
- FRAMEWORK_SEARCH_PATHS = (
- "$(inherited)",
- "$(FRAMEWORK_SEARCH_PATHS_QUOTED_1)",
- "$(FRAMEWORK_SEARCH_PATHS_QUOTED_2)",
- );
- FRAMEWORK_SEARCH_PATHS_QUOTED_1 = "\"$(SRCROOT)/../opencv\"";
- FRAMEWORK_SEARCH_PATHS_QUOTED_2 = "\"$(SRCROOT)/../../..\"";
- GCC_ENABLE_FIX_AND_CONTINUE = NO;
- GCC_GENERATE_DEBUGGING_SYMBOLS = NO;
- GCC_MODEL_TUNING = G5;
- GCC_PRECOMPILE_PREFIX_HEADER = YES;
- GCC_PREFIX_HEADER = "$(SYSTEM_LIBRARY_DIR)/Frameworks/Carbon.framework/Headers/Carbon.h";
- INFOPLIST_FILE = "FaceTracker-Info.plist";
- INSTALL_PATH = "$(HOME)/Applications";
- OTHER_LDFLAGS = (
- "-framework",
- Carbon,
- );
- PREBINDING = NO;
- PRODUCT_NAME = FaceTracker;
- WRAPPER_EXTENSION = app;
- ZERO_LINK = NO;
- };
- name = Release;
- };
-/* End XCBuildConfiguration section */
-
-/* Begin XCConfigurationList section */
- 1DEB923508733DC60010E9CD /* Build configuration list for PBXProject "FaceTracker" */ = {
- isa = XCConfigurationList;
- buildConfigurations = (
- 1DEB923608733DC60010E9CD /* Debug */,
- 1DEB923708733DC60010E9CD /* Release */,
- );
- defaultConfigurationIsVisible = 0;
- defaultConfigurationName = Release;
- };
- 4D7DBE5A0C04A8FF00D8835D /* Build configuration list for PBXNativeTarget "FaceTracker" */ = {
- isa = XCConfigurationList;
- buildConfigurations = (
- 4D7DBE5B0C04A8FF00D8835D /* Debug */,
- 4D7DBE5C0C04A8FF00D8835D /* Release */,
- );
- defaultConfigurationIsVisible = 0;
- defaultConfigurationName = Release;
- };
-/* End XCConfigurationList section */
- };
- rootObject = 08FB7793FE84155DC02AAC07 /* Project object */;
-}
+++ /dev/null
-FaceTracker/REAME.txt
-2007-05-24, Mark Asbach <asbach@ient.rwth-aachen.de>
-
-Objective:
-This document is intended to get you up and running with an OpenCV Framework on Mac OS X
-
-Building the OpenCV.framework:
-In the main directory of the opencv distribution, you will find a shell script called
-'make_frameworks.sh' that does all of the typical unixy './configure && make' stuff required
-to build a universal binary framework. Invoke this script from Terminal.app, wait some minutes
-and you are done.
-
-OpenCV is a Private Framework:
-On Mac OS X the concept of Framework bundles is meant to simplify distribution of shared libraries,
-accompanying headers and documentation. There are however to subtly different 'flavours' of
-Frameworks: public and private ones. The public frameworks get installed into the Frameworks
-diretories in /Library, /System/Library or ~/Library and are meant to be shared amongst
-applications. The private frameworks are only distributed as parts of an Application Bundle.
-This makes it easier to deploy applications because they bring their own framework invisibly to
-the user. No installation of the framework is necessary and different applications can bring
-different versions of the same framework without any conflict.
-Since OpenCV is still a moving target, it seems best to avoid any installation and versioning issues
-for an end user. The OpenCV framework that currently comes with this demo application therefore
-is a Private Framework.
-
-Use it for targets that result in an Application Bundle:
-Since it is a Private Framework, it must be copied to the Frameworks/ directory of an Application
-Bundle, which means, it is useless for plain unix console applications. You should create a Carbon
-or a Cocoa application target in XCode for your projects. Then add the OpenCV.framework just like
-in this demo and add a Copy Files build phase to your target. Let that phase copy to the Framework
-directory and drop the OpenCV.framework on the build phase (again just like in this demo code).
-
-The resulting application bundle will be self contained and if you set compiler option correctly
-(in the "Build" tab of the "Project Info" window you should find 'i386 ppc' for the architectures),
-your application can just be copied to any OS 10.4 Mac and used without further installation.
+++ /dev/null
-#include <opencv2/core.hpp>
-#include <opencv2/core/utility.hpp>
-#include <opencv2/imgproc/imgproc_c.h> // cvFindContours
-#include <opencv2/imgproc.hpp>
-#include <opencv2/objdetect.hpp>
-#include <opencv2/videoio.hpp>
-#include <opencv2/highgui.hpp>
-#include <iterator>
-#include <set>
-#include <cstdio>
-#include <iostream>
-
-// Function prototypes
-void subtractPlane(const cv::Mat& depth, cv::Mat& mask, std::vector<CvPoint>& chain, double f);
-
-std::vector<CvPoint> maskFromTemplate(const std::vector<cv::linemod::Template>& templates,
- int num_modalities, cv::Point offset, cv::Size size,
- cv::Mat& mask, cv::Mat& dst);
-
-void templateConvexHull(const std::vector<cv::linemod::Template>& templates,
- int num_modalities, cv::Point offset, cv::Size size,
- cv::Mat& dst);
-
-void drawResponse(const std::vector<cv::linemod::Template>& templates,
- int num_modalities, cv::Mat& dst, cv::Point offset, int T);
-
-cv::Mat displayQuantized(const cv::Mat& quantized);
-
-// Copy of cv_mouse from cv_utilities
-class Mouse
-{
-public:
- static void start(const std::string& a_img_name)
- {
- cv::setMouseCallback(a_img_name.c_str(), Mouse::cv_on_mouse, 0);
- }
- static int event(void)
- {
- int l_event = m_event;
- m_event = -1;
- return l_event;
- }
- static int x(void)
- {
- return m_x;
- }
- static int y(void)
- {
- return m_y;
- }
-
-private:
- static void cv_on_mouse(int a_event, int a_x, int a_y, int, void *)
- {
- m_event = a_event;
- m_x = a_x;
- m_y = a_y;
- }
-
- static int m_event;
- static int m_x;
- static int m_y;
-};
-int Mouse::m_event;
-int Mouse::m_x;
-int Mouse::m_y;
-
-static void help()
-{
- printf("Usage: openni_demo [templates.yml]\n\n"
- "Place your object on a planar, featureless surface. With the mouse,\n"
- "frame it in the 'color' window and right click to learn a first template.\n"
- "Then press 'l' to enter online learning mode, and move the camera around.\n"
- "When the match score falls between 90-95%% the demo will add a new template.\n\n"
- "Keys:\n"
- "\t h -- This help page\n"
- "\t l -- Toggle online learning\n"
- "\t m -- Toggle printing match result\n"
- "\t t -- Toggle printing timings\n"
- "\t w -- Write learned templates to disk\n"
- "\t [ ] -- Adjust matching threshold: '[' down, ']' up\n"
- "\t q -- Quit\n\n");
-}
-
-// Adapted from cv_timer in cv_utilities
-class Timer
-{
-public:
- Timer() : start_(0), time_(0) {}
-
- void start()
- {
- start_ = cv::getTickCount();
- }
-
- void stop()
- {
- CV_Assert(start_ != 0);
- int64 end = cv::getTickCount();
- time_ += end - start_;
- start_ = 0;
- }
-
- double time()
- {
- double ret = time_ / cv::getTickFrequency();
- time_ = 0;
- return ret;
- }
-
-private:
- int64 start_, time_;
-};
-
-// Functions to store detector and templates in single XML/YAML file
-static cv::Ptr<cv::linemod::Detector> readLinemod(const std::string& filename)
-{
- cv::Ptr<cv::linemod::Detector> detector = cv::makePtr<cv::linemod::Detector>();
- cv::FileStorage fs(filename, cv::FileStorage::READ);
- detector->read(fs.root());
-
- cv::FileNode fn = fs["classes"];
- for (cv::FileNodeIterator i = fn.begin(), iend = fn.end(); i != iend; ++i)
- detector->readClass(*i);
-
- return detector;
-}
-
-static void writeLinemod(const cv::Ptr<cv::linemod::Detector>& detector, const std::string& filename)
-{
- cv::FileStorage fs(filename, cv::FileStorage::WRITE);
- detector->write(fs);
-
- std::vector<cv::String> ids = detector->classIds();
- fs << "classes" << "[";
- for (int i = 0; i < (int)ids.size(); ++i)
- {
- fs << "{";
- detector->writeClass(ids[i], fs);
- fs << "}"; // current class
- }
- fs << "]"; // classes
-}
-
-
-int main(int argc, char * argv[])
-{
- // Various settings and flags
- bool show_match_result = true;
- bool show_timings = false;
- bool learn_online = false;
- int num_classes = 0;
- int matching_threshold = 80;
- /// @todo Keys for changing these?
- cv::Size roi_size(200, 200);
- int learning_lower_bound = 90;
- int learning_upper_bound = 95;
-
- // Timers
- Timer extract_timer;
- Timer match_timer;
-
- // Initialize HighGUI
- help();
- cv::namedWindow("color");
- cv::namedWindow("normals");
- Mouse::start("color");
-
- // Initialize LINEMOD data structures
- cv::Ptr<cv::linemod::Detector> detector;
- std::string filename;
- if (argc == 1)
- {
- filename = "linemod_templates.yml";
- detector = cv::linemod::getDefaultLINEMOD();
- }
- else
- {
- detector = readLinemod(argv[1]);
-
- std::vector<cv::String> ids = detector->classIds();
- num_classes = detector->numClasses();
- printf("Loaded %s with %d classes and %d templates\n",
- argv[1], num_classes, detector->numTemplates());
- if (!ids.empty())
- {
- printf("Class ids:\n");
- std::copy(ids.begin(), ids.end(), std::ostream_iterator<std::string>(std::cout, "\n"));
- }
- }
- int num_modalities = (int)detector->getModalities().size();
-
- // Open Kinect sensor
- cv::VideoCapture capture( cv::CAP_OPENNI );
- if (!capture.isOpened())
- {
- printf("Could not open OpenNI-capable sensor\n");
- return -1;
- }
- capture.set(cv::CAP_PROP_OPENNI_REGISTRATION, 1);
- double focal_length = capture.get(cv::CAP_OPENNI_DEPTH_GENERATOR_FOCAL_LENGTH);
- //printf("Focal length = %f\n", focal_length);
-
- // Main loop
- cv::Mat color, depth;
- for(;;)
- {
- // Capture next color/depth pair
- capture.grab();
- capture.retrieve(depth, cv::CAP_OPENNI_DEPTH_MAP);
- capture.retrieve(color, cv::CAP_OPENNI_BGR_IMAGE);
-
- std::vector<cv::Mat> sources;
- sources.push_back(color);
- sources.push_back(depth);
- cv::Mat display = color.clone();
-
- if (!learn_online)
- {
- cv::Point mouse(Mouse::x(), Mouse::y());
- int event = Mouse::event();
-
- // Compute ROI centered on current mouse location
- cv::Point roi_offset(roi_size.width / 2, roi_size.height / 2);
- cv::Point pt1 = mouse - roi_offset; // top left
- cv::Point pt2 = mouse + roi_offset; // bottom right
-
- if (event == cv::EVENT_RBUTTONDOWN)
- {
- // Compute object mask by subtracting the plane within the ROI
- std::vector<CvPoint> chain(4);
- chain[0] = pt1;
- chain[1] = cv::Point(pt2.x, pt1.y);
- chain[2] = pt2;
- chain[3] = cv::Point(pt1.x, pt2.y);
- cv::Mat mask;
- subtractPlane(depth, mask, chain, focal_length);
-
- cv::imshow("mask", mask);
-
- // Extract template
- std::string class_id = cv::format("class%d", num_classes);
- cv::Rect bb;
- extract_timer.start();
- int template_id = detector->addTemplate(sources, class_id, mask, &bb);
- extract_timer.stop();
- if (template_id != -1)
- {
- printf("*** Added template (id %d) for new object class %d***\n",
- template_id, num_classes);
- //printf("Extracted at (%d, %d) size %dx%d\n", bb.x, bb.y, bb.width, bb.height);
- }
-
- ++num_classes;
- }
-
- // Draw ROI for display
- cv::rectangle(display, pt1, pt2, CV_RGB(0,0,0), 3);
- cv::rectangle(display, pt1, pt2, CV_RGB(255,255,0), 1);
- }
-
- // Perform matching
- std::vector<cv::linemod::Match> matches;
- std::vector<cv::String> class_ids;
- std::vector<cv::Mat> quantized_images;
- match_timer.start();
- detector->match(sources, (float)matching_threshold, matches, class_ids, quantized_images);
- match_timer.stop();
-
- int classes_visited = 0;
- std::set<std::string> visited;
-
- for (int i = 0; (i < (int)matches.size()) && (classes_visited < num_classes); ++i)
- {
- cv::linemod::Match m = matches[i];
-
- if (visited.insert(m.class_id).second)
- {
- ++classes_visited;
-
- if (show_match_result)
- {
- printf("Similarity: %5.1f%%; x: %3d; y: %3d; class: %s; template: %3d\n",
- m.similarity, m.x, m.y, m.class_id.c_str(), m.template_id);
- }
-
- // Draw matching template
- const std::vector<cv::linemod::Template>& templates = detector->getTemplates(m.class_id, m.template_id);
- drawResponse(templates, num_modalities, display, cv::Point(m.x, m.y), detector->getT(0));
-
- if (learn_online == true)
- {
- /// @todo Online learning possibly broken by new gradient feature extraction,
- /// which assumes an accurate object outline.
-
- // Compute masks based on convex hull of matched template
- cv::Mat color_mask, depth_mask;
- std::vector<CvPoint> chain = maskFromTemplate(templates, num_modalities,
- cv::Point(m.x, m.y), color.size(),
- color_mask, display);
- subtractPlane(depth, depth_mask, chain, focal_length);
-
- cv::imshow("mask", depth_mask);
-
- // If pretty sure (but not TOO sure), add new template
- if (learning_lower_bound < m.similarity && m.similarity < learning_upper_bound)
- {
- extract_timer.start();
- int template_id = detector->addTemplate(sources, m.class_id, depth_mask);
- extract_timer.stop();
- if (template_id != -1)
- {
- printf("*** Added template (id %d) for existing object class %s***\n",
- template_id, m.class_id.c_str());
- }
- }
- }
- }
- }
-
- if (show_match_result && matches.empty())
- printf("No matches found...\n");
- if (show_timings)
- {
- printf("Training: %.2fs\n", extract_timer.time());
- printf("Matching: %.2fs\n", match_timer.time());
- }
- if (show_match_result || show_timings)
- printf("------------------------------------------------------------\n");
-
- cv::imshow("color", display);
- cv::imshow("normals", quantized_images[1]);
-
- cv::FileStorage fs;
- char key = (char)cv::waitKey(10);
- if( key == 'q' )
- break;
-
- switch (key)
- {
- case 'h':
- help();
- break;
- case 'm':
- // toggle printing match result
- show_match_result = !show_match_result;
- printf("Show match result %s\n", show_match_result ? "ON" : "OFF");
- break;
- case 't':
- // toggle printing timings
- show_timings = !show_timings;
- printf("Show timings %s\n", show_timings ? "ON" : "OFF");
- break;
- case 'l':
- // toggle online learning
- learn_online = !learn_online;
- printf("Online learning %s\n", learn_online ? "ON" : "OFF");
- break;
- case '[':
- // decrement threshold
- matching_threshold = std::max(matching_threshold - 1, -100);
- printf("New threshold: %d\n", matching_threshold);
- break;
- case ']':
- // increment threshold
- matching_threshold = std::min(matching_threshold + 1, +100);
- printf("New threshold: %d\n", matching_threshold);
- break;
- case 'w':
- // write model to disk
- writeLinemod(detector, filename);
- printf("Wrote detector and templates to %s\n", filename.c_str());
- break;
- default:
- ;
- }
- }
- return 0;
-}
-
-static void reprojectPoints(const std::vector<cv::Point3d>& proj, std::vector<cv::Point3d>& real, double f)
-{
- real.resize(proj.size());
- double f_inv = 1.0 / f;
-
- for (int i = 0; i < (int)proj.size(); ++i)
- {
- double Z = proj[i].z;
- real[i].x = (proj[i].x - 320.) * (f_inv * Z);
- real[i].y = (proj[i].y - 240.) * (f_inv * Z);
- real[i].z = Z;
- }
-}
-
-static void filterPlane(IplImage * ap_depth, std::vector<IplImage *> & a_masks, std::vector<CvPoint> & a_chain, double f)
-{
- const int l_num_cost_pts = 200;
-
- float l_thres = 4;
-
- IplImage * lp_mask = cvCreateImage(cvGetSize(ap_depth), IPL_DEPTH_8U, 1);
- cvSet(lp_mask, cvRealScalar(0));
-
- std::vector<CvPoint> l_chain_vector;
-
- float l_chain_length = 0;
- float * lp_seg_length = new float[a_chain.size()];
-
- for (int l_i = 0; l_i < (int)a_chain.size(); ++l_i)
- {
- float x_diff = (float)(a_chain[(l_i + 1) % a_chain.size()].x - a_chain[l_i].x);
- float y_diff = (float)(a_chain[(l_i + 1) % a_chain.size()].y - a_chain[l_i].y);
- lp_seg_length[l_i] = sqrt(x_diff*x_diff + y_diff*y_diff);
- l_chain_length += lp_seg_length[l_i];
- }
- for (int l_i = 0; l_i < (int)a_chain.size(); ++l_i)
- {
- if (lp_seg_length[l_i] > 0)
- {
- int l_cur_num = cvRound(l_num_cost_pts * lp_seg_length[l_i] / l_chain_length);
- float l_cur_len = lp_seg_length[l_i] / l_cur_num;
-
- for (int l_j = 0; l_j < l_cur_num; ++l_j)
- {
- float l_ratio = (l_cur_len * l_j / lp_seg_length[l_i]);
-
- CvPoint l_pts;
-
- l_pts.x = cvRound(l_ratio * (a_chain[(l_i + 1) % a_chain.size()].x - a_chain[l_i].x) + a_chain[l_i].x);
- l_pts.y = cvRound(l_ratio * (a_chain[(l_i + 1) % a_chain.size()].y - a_chain[l_i].y) + a_chain[l_i].y);
-
- l_chain_vector.push_back(l_pts);
- }
- }
- }
- std::vector<cv::Point3d> lp_src_3Dpts(l_chain_vector.size());
-
- for (int l_i = 0; l_i < (int)l_chain_vector.size(); ++l_i)
- {
- lp_src_3Dpts[l_i].x = l_chain_vector[l_i].x;
- lp_src_3Dpts[l_i].y = l_chain_vector[l_i].y;
- lp_src_3Dpts[l_i].z = CV_IMAGE_ELEM(ap_depth, unsigned short, cvRound(lp_src_3Dpts[l_i].y), cvRound(lp_src_3Dpts[l_i].x));
- //CV_IMAGE_ELEM(lp_mask,unsigned char,(int)lp_src_3Dpts[l_i].Y,(int)lp_src_3Dpts[l_i].X)=255;
- }
- //cv_show_image(lp_mask,"hallo2");
-
- reprojectPoints(lp_src_3Dpts, lp_src_3Dpts, f);
-
- CvMat * lp_pts = cvCreateMat((int)l_chain_vector.size(), 4, CV_32F);
- CvMat * lp_v = cvCreateMat(4, 4, CV_32F);
- CvMat * lp_w = cvCreateMat(4, 1, CV_32F);
-
- for (int l_i = 0; l_i < (int)l_chain_vector.size(); ++l_i)
- {
- CV_MAT_ELEM(*lp_pts, float, l_i, 0) = (float)lp_src_3Dpts[l_i].x;
- CV_MAT_ELEM(*lp_pts, float, l_i, 1) = (float)lp_src_3Dpts[l_i].y;
- CV_MAT_ELEM(*lp_pts, float, l_i, 2) = (float)lp_src_3Dpts[l_i].z;
- CV_MAT_ELEM(*lp_pts, float, l_i, 3) = 1.0f;
- }
- cvSVD(lp_pts, lp_w, 0, lp_v);
-
- float l_n[4] = {CV_MAT_ELEM(*lp_v, float, 0, 3),
- CV_MAT_ELEM(*lp_v, float, 1, 3),
- CV_MAT_ELEM(*lp_v, float, 2, 3),
- CV_MAT_ELEM(*lp_v, float, 3, 3)};
-
- float l_norm = sqrt(l_n[0] * l_n[0] + l_n[1] * l_n[1] + l_n[2] * l_n[2]);
-
- l_n[0] /= l_norm;
- l_n[1] /= l_norm;
- l_n[2] /= l_norm;
- l_n[3] /= l_norm;
-
- float l_max_dist = 0;
-
- for (int l_i = 0; l_i < (int)l_chain_vector.size(); ++l_i)
- {
- float l_dist = l_n[0] * CV_MAT_ELEM(*lp_pts, float, l_i, 0) +
- l_n[1] * CV_MAT_ELEM(*lp_pts, float, l_i, 1) +
- l_n[2] * CV_MAT_ELEM(*lp_pts, float, l_i, 2) +
- l_n[3] * CV_MAT_ELEM(*lp_pts, float, l_i, 3);
-
- if (fabs(l_dist) > l_max_dist)
- l_max_dist = l_dist;
- }
- //std::cerr << "plane: " << l_n[0] << ";" << l_n[1] << ";" << l_n[2] << ";" << l_n[3] << " maxdist: " << l_max_dist << " end" << std::endl;
- int l_minx = ap_depth->width;
- int l_miny = ap_depth->height;
- int l_maxx = 0;
- int l_maxy = 0;
-
- for (int l_i = 0; l_i < (int)a_chain.size(); ++l_i)
- {
- l_minx = std::min(l_minx, a_chain[l_i].x);
- l_miny = std::min(l_miny, a_chain[l_i].y);
- l_maxx = std::max(l_maxx, a_chain[l_i].x);
- l_maxy = std::max(l_maxy, a_chain[l_i].y);
- }
- int l_w = l_maxx - l_minx + 1;
- int l_h = l_maxy - l_miny + 1;
- int l_nn = (int)a_chain.size();
-
- CvPoint * lp_chain = new CvPoint[l_nn];
-
- for (int l_i = 0; l_i < l_nn; ++l_i)
- lp_chain[l_i] = a_chain[l_i];
-
- cvFillPoly(lp_mask, &lp_chain, &l_nn, 1, cvScalar(255, 255, 255));
-
- delete[] lp_chain;
-
- //cv_show_image(lp_mask,"hallo1");
-
- std::vector<cv::Point3d> lp_dst_3Dpts(l_h * l_w);
-
- int l_ind = 0;
-
- for (int l_r = 0; l_r < l_h; ++l_r)
- {
- for (int l_c = 0; l_c < l_w; ++l_c)
- {
- lp_dst_3Dpts[l_ind].x = l_c + l_minx;
- lp_dst_3Dpts[l_ind].y = l_r + l_miny;
- lp_dst_3Dpts[l_ind].z = CV_IMAGE_ELEM(ap_depth, unsigned short, l_r + l_miny, l_c + l_minx);
- ++l_ind;
- }
- }
- reprojectPoints(lp_dst_3Dpts, lp_dst_3Dpts, f);
-
- l_ind = 0;
-
- for (int l_r = 0; l_r < l_h; ++l_r)
- {
- for (int l_c = 0; l_c < l_w; ++l_c)
- {
- float l_dist = (float)(l_n[0] * lp_dst_3Dpts[l_ind].x + l_n[1] * lp_dst_3Dpts[l_ind].y + lp_dst_3Dpts[l_ind].z * l_n[2] + l_n[3]);
-
- ++l_ind;
-
- if (CV_IMAGE_ELEM(lp_mask, unsigned char, l_r + l_miny, l_c + l_minx) != 0)
- {
- if (fabs(l_dist) < std::max(l_thres, (l_max_dist * 2.0f)))
- {
- for (int l_p = 0; l_p < (int)a_masks.size(); ++l_p)
- {
- int l_col = cvRound((l_c + l_minx) / (l_p + 1.0));
- int l_row = cvRound((l_r + l_miny) / (l_p + 1.0));
-
- CV_IMAGE_ELEM(a_masks[l_p], unsigned char, l_row, l_col) = 0;
- }
- }
- else
- {
- for (int l_p = 0; l_p < (int)a_masks.size(); ++l_p)
- {
- int l_col = cvRound((l_c + l_minx) / (l_p + 1.0));
- int l_row = cvRound((l_r + l_miny) / (l_p + 1.0));
-
- CV_IMAGE_ELEM(a_masks[l_p], unsigned char, l_row, l_col) = 255;
- }
- }
- }
- }
- }
- cvReleaseImage(&lp_mask);
- cvReleaseMat(&lp_pts);
- cvReleaseMat(&lp_w);
- cvReleaseMat(&lp_v);
-}
-
-void subtractPlane(const cv::Mat& depth, cv::Mat& mask, std::vector<CvPoint>& chain, double f)
-{
- mask = cv::Mat::zeros(depth.size(), CV_8U);
- std::vector<IplImage*> tmp;
- IplImage mask_ipl = mask;
- tmp.push_back(&mask_ipl);
- IplImage depth_ipl = depth;
- filterPlane(&depth_ipl, tmp, chain, f);
-}
-
-std::vector<CvPoint> maskFromTemplate(const std::vector<cv::linemod::Template>& templates,
- int num_modalities, cv::Point offset, cv::Size size,
- cv::Mat& mask, cv::Mat& dst)
-{
- templateConvexHull(templates, num_modalities, offset, size, mask);
-
- const int OFFSET = 30;
- cv::dilate(mask, mask, cv::Mat(), cv::Point(-1,-1), OFFSET);
-
- CvMemStorage * lp_storage = cvCreateMemStorage(0);
- CvTreeNodeIterator l_iterator;
- CvSeqReader l_reader;
- CvSeq * lp_contour = 0;
-
- cv::Mat mask_copy = mask.clone();
- IplImage mask_copy_ipl = mask_copy;
- cvFindContours(&mask_copy_ipl, lp_storage, &lp_contour, sizeof(CvContour),
- CV_RETR_CCOMP, CV_CHAIN_APPROX_SIMPLE);
-
- std::vector<CvPoint> l_pts1; // to use as input to cv_primesensor::filter_plane
-
- cvInitTreeNodeIterator(&l_iterator, lp_contour, 1);
- while ((lp_contour = (CvSeq *)cvNextTreeNode(&l_iterator)) != 0)
- {
- CvPoint l_pt0;
- cvStartReadSeq(lp_contour, &l_reader, 0);
- CV_READ_SEQ_ELEM(l_pt0, l_reader);
- l_pts1.push_back(l_pt0);
-
- for (int i = 0; i < lp_contour->total; ++i)
- {
- CvPoint l_pt1;
- CV_READ_SEQ_ELEM(l_pt1, l_reader);
- /// @todo Really need dst at all? Can just as well do this outside
- cv::line(dst, l_pt0, l_pt1, CV_RGB(0, 255, 0), 2);
-
- l_pt0 = l_pt1;
- l_pts1.push_back(l_pt0);
- }
- }
- cvReleaseMemStorage(&lp_storage);
-
- return l_pts1;
-}
-
-// Adapted from cv_show_angles
-cv::Mat displayQuantized(const cv::Mat& quantized)
-{
- cv::Mat color(quantized.size(), CV_8UC3);
- for (int r = 0; r < quantized.rows; ++r)
- {
- const uchar* quant_r = quantized.ptr(r);
- cv::Vec3b* color_r = color.ptr<cv::Vec3b>(r);
-
- for (int c = 0; c < quantized.cols; ++c)
- {
- cv::Vec3b& bgr = color_r[c];
- switch (quant_r[c])
- {
- case 0: bgr[0]= 0; bgr[1]= 0; bgr[2]= 0; break;
- case 1: bgr[0]= 55; bgr[1]= 55; bgr[2]= 55; break;
- case 2: bgr[0]= 80; bgr[1]= 80; bgr[2]= 80; break;
- case 4: bgr[0]=105; bgr[1]=105; bgr[2]=105; break;
- case 8: bgr[0]=130; bgr[1]=130; bgr[2]=130; break;
- case 16: bgr[0]=155; bgr[1]=155; bgr[2]=155; break;
- case 32: bgr[0]=180; bgr[1]=180; bgr[2]=180; break;
- case 64: bgr[0]=205; bgr[1]=205; bgr[2]=205; break;
- case 128: bgr[0]=230; bgr[1]=230; bgr[2]=230; break;
- case 255: bgr[0]= 0; bgr[1]= 0; bgr[2]=255; break;
- default: bgr[0]= 0; bgr[1]=255; bgr[2]= 0; break;
- }
- }
- }
-
- return color;
-}
-
-// Adapted from cv_line_template::convex_hull
-void templateConvexHull(const std::vector<cv::linemod::Template>& templates,
- int num_modalities, cv::Point offset, cv::Size size,
- cv::Mat& dst)
-{
- std::vector<cv::Point> points;
- for (int m = 0; m < num_modalities; ++m)
- {
- for (int i = 0; i < (int)templates[m].features.size(); ++i)
- {
- cv::linemod::Feature f = templates[m].features[i];
- points.push_back(cv::Point(f.x, f.y) + offset);
- }
- }
-
- std::vector<cv::Point> hull;
- cv::convexHull(points, hull);
-
- dst = cv::Mat::zeros(size, CV_8U);
- const int hull_count = (int)hull.size();
- const cv::Point* hull_pts = &hull[0];
- cv::fillPoly(dst, &hull_pts, &hull_count, 1, cv::Scalar(255));
-}
-
-void drawResponse(const std::vector<cv::linemod::Template>& templates,
- int num_modalities, cv::Mat& dst, cv::Point offset, int T)
-{
- static const cv::Scalar COLORS[5] = { CV_RGB(0, 0, 255),
- CV_RGB(0, 255, 0),
- CV_RGB(255, 255, 0),
- CV_RGB(255, 140, 0),
- CV_RGB(255, 0, 0) };
-
- for (int m = 0; m < num_modalities; ++m)
- {
- // NOTE: Original demo recalculated max response for each feature in the TxT
- // box around it and chose the display color based on that response. Here
- // the display color just depends on the modality.
- cv::Scalar color = COLORS[m];
-
- for (int i = 0; i < (int)templates[m].features.size(); ++i)
- {
- cv::linemod::Feature f = templates[m].features[i];
- cv::Point pt(f.x + offset.x, f.y + offset.y);
- cv::circle(dst, pt, T / 2, color);
- }
- }
-}
+++ /dev/null
-/*
- * textdetection.cpp
- *
- * A demo program of the Extremal Region Filter algorithm described in
- * Neumann L., Matas J.: Real-Time Scene Text Localization and Recognition, CVPR 2012
- *
- * Created on: Sep 23, 2013
- * Author: Lluis Gomez i Bigorda <lgomez AT cvc.uab.es>
- */
-
-#include "opencv2/opencv.hpp"
-#include "opencv2/objdetect.hpp"
-#include "opencv2/imgcodecs.hpp"
-#include "opencv2/highgui.hpp"
-#include "opencv2/imgproc.hpp"
-
-#include <vector>
-#include <iostream>
-#include <iomanip>
-
-using namespace std;
-using namespace cv;
-
-void show_help_and_exit(const char *cmd);
-void groups_draw(Mat &src, vector<Rect> &groups);
-void er_show(vector<Mat> &channels, vector<vector<ERStat> > ®ions);
-
-int main(int argc, const char * argv[])
-{
- cout << endl << argv[0] << endl << endl;
- cout << "Demo program of the Extremal Region Filter algorithm described in " << endl;
- cout << "Neumann L., Matas J.: Real-Time Scene Text Localization and Recognition, CVPR 2012" << endl << endl;
-
- if (argc < 2) show_help_and_exit(argv[0]);
-
- Mat src = imread(argv[1]);
-
- // Extract channels to be processed individually
- vector<Mat> channels;
- computeNMChannels(src, channels);
-
- int cn = (int)channels.size();
- // Append negative channels to detect ER- (bright regions over dark background)
- for (int c = 0; c < cn-1; c++)
- channels.push_back(255-channels[c]);
-
- // Create ERFilter objects with the 1st and 2nd stage default classifiers
- Ptr<ERFilter> er_filter1 = createERFilterNM1(loadClassifierNM1("trained_classifierNM1.xml"),16,0.00015f,0.13f,0.2f,true,0.1f);
- Ptr<ERFilter> er_filter2 = createERFilterNM2(loadClassifierNM2("trained_classifierNM2.xml"),0.5);
-
- vector<vector<ERStat> > regions(channels.size());
- // Apply the default cascade classifier to each independent channel (could be done in parallel)
- cout << "Extracting Class Specific Extremal Regions from " << (int)channels.size() << " channels ..." << endl;
- cout << " (...) this may take a while (...)" << endl << endl;
- for (int c=0; c<(int)channels.size(); c++)
- {
- er_filter1->run(channels[c], regions[c]);
- er_filter2->run(channels[c], regions[c]);
- }
-
- // Detect character groups
- cout << "Grouping extracted ERs ... ";
- vector<Rect> groups;
- erGrouping(channels, regions, "trained_classifier_erGrouping.xml", 0.5, groups);
-
- // draw groups
- groups_draw(src, groups);
- imshow("grouping",src);
-
- cout << "Done!" << endl << endl;
- cout << "Press 'e' to show the extracted Extremal Regions, any other key to exit." << endl << endl;
- if( waitKey (-1) == 101)
- er_show(channels,regions);
-
- // memory clean-up
- er_filter1.release();
- er_filter2.release();
- regions.clear();
- if (!groups.empty())
- {
- groups.clear();
- }
-}
-
-
-
-// helper functions
-
-void show_help_and_exit(const char *cmd)
-{
- cout << " Usage: " << cmd << " <input_image> " << endl;
- cout << " Default classifier files (trained_classifierNM*.xml) must be in current directory" << endl << endl;
- exit(-1);
-}
-
-void groups_draw(Mat &src, vector<Rect> &groups)
-{
- for (int i=(int)groups.size()-1; i>=0; i--)
- {
- if (src.type() == CV_8UC3)
- rectangle(src,groups.at(i).tl(),groups.at(i).br(),Scalar( 0, 255, 255 ), 3, 8 );
- else
- rectangle(src,groups.at(i).tl(),groups.at(i).br(),Scalar( 255 ), 3, 8 );
- }
-}
-
-void er_show(vector<Mat> &channels, vector<vector<ERStat> > ®ions)
-{
- for (int c=0; c<(int)channels.size(); c++)
- {
- Mat dst = Mat::zeros(channels[0].rows+2,channels[0].cols+2,CV_8UC1);
- for (int r=0; r<(int)regions[c].size(); r++)
- {
- ERStat er = regions[c][r];
- if (er.parent != NULL) // deprecate the root region
- {
- int newMaskVal = 255;
- int flags = 4 + (newMaskVal << 8) + FLOODFILL_FIXED_RANGE + FLOODFILL_MASK_ONLY;
- floodFill(channels[c],dst,Point(er.pixel%channels[c].cols,er.pixel/channels[c].cols),
- Scalar(255),0,Scalar(er.level),Scalar(0),flags);
- }
- }
- char buff[10]; char *buff_ptr = buff;
- sprintf(buff, "channel %d", c);
- imshow(buff_ptr, dst);
- }
- waitKey(-1);
-}
projects / profile / ivi / opencv.git / commitdiff
