OpenCV人脸识别+模板匹配OpenCV+CMake+codebook+windows

OpenCV人脸识别+模板匹配OpenCV+CMake+codebook+windows

ABSTRACT

The basic aim of the experiment was achieving the purpose that PC could recognize the face of human by using OpenCV. In introduction this article introduced how to set up the environment of OpenCV on code::blocks as well as the basic functions of OpenCV. In second section method (template matching and face recognizer) and debug would be discussed specifically. The experiment results and performance discussion were demonstrated at the section of results, discussion and conclusion.

INTRODUCTION

ENVIRONMENT SET UP

OpenCV is a free open source library, which packaged up lot of codes to about computer graph. With OpenCV, face recognizing could be easily done by few codes.

First of all, user should download OpenCV from official website. The latest version had been updated as 4.3.0. However, latest version was not the best choice in some cases. There was possible to have bugs in new version that nobody knew. Once users had met these bugs, it was quite difficult to debug. Researcher chose 3.4.1 and downloaded it on Baidu Netdisk. Then added the path of bin into system variable (or user variable for this user only).

Why CMake?

After downloading, users had to compile OpenCV and add the header files into code::blocks. However, for a large-scale project, the files consist was too complicated. These files were written in different time and by different people, even the developers could not know all of these files. Fortunately, “makefile” was provided for users. “makefile” could clearly describe the relationships between these files so that users could only compile and generate few files instead of compiling all. In this case development efficiency was observably promoted.

CMake is a tool for makefile. It read all source files and automatically generated files.
Users could operate it in cmd, but it also had GUI version for users who were not familiar with cmd. After installing CMake-GUI, it showed as figure 1:

There were two options. The first one was “where is the source code” and it should be the path of “sources” in OpenCV file. The other one was where to deposit the files that CMake generated. Users could simply build a new file. Then click “configure” and it showed as Figure (2):

Selected “CodeBlocks – MinGW Makefiles” and “Specify native compliers”. In this case users could select compiler as below:

Users could choose either MinGW32 or TDM-GCC. It was very important to choose right version that matched computer system. Although 32-bit OpenCV (with x86 file) could also run on 64-bit system, but it was likely to run out of memory. Therefore, chose 64-bit complier if the system was 64-bit.In addition, users could also select “Use default native compiler”, which meant configure files with the compiler that code::blocks was using (set compiler in code::blocks first).

Clicked “finish” and CMake would start configuring. In this process computer would download some files. If the network speed was not quick enough, it was likely to failed. The dealing method would be discussed in next section. After first configure done, there would be some red selections on scream, ticked “build opencv world” in order to build all libs in one compressed file. So that it would be convenient to add these files
into code::blocks.

The compiling would last for some time, once it had done, there would be a file named “install” in generating file. Opened it and found three files: include, bin and lib (path: install\x86 or x64\mingw). Opened code::blocks and selected setting-compiler-compiler setting-compiler flags, ticked below option to support C++11 standard (only gcc compiler over version 4.8 supported C++11!!):

Then clicked “Search directories”. Added “include” and also two files in it (opencv and opencv2) into Compiler. Added lib into Linker and added opencv_world.dll.a into Link libraries.

In this case, environment for OpenCV had set up. Researcher would introduce some basic functions of OpenCV, and these functions would help to completed face recognizing.

BASIC FUNCTION OF OPENCV

1. Imread (“name + format”); //this function meat read a picture and give it to a Mat object.

2. Imwrite (“name+format”,Mat object); //saved a picture as “name+format”.

3. resize
   (intput_image,output_image,Size(x,y));
   //resized an image to width = x and height = y.

4. rectangle
   (image, point, Scalar(color), 1, 8, 0); //draw a rectangle at point.

5. putText
   (image, text, point, FONT_HERSHEY_COMPLEX, 1, Scalar(color)); //wrote text at point.

6. matchTemplate(image, template_image, output_image, method);
   //compared image and template image.

7. minMaxLoc (output_image, &minVal, &maxVal, &minPoint, &maxPoint); //minVal and maxVal were double format data. minPoint and maxPoint were point. With different method in match function, the double format data showed similarity of two images.

8. model.detectMultiScale(image,faces,1.1,2,0 CV_HAAR_SCALE_IMAGE,Size()); //model was the face data model. Image was the image to recognize and recommended to be transferred to gray first. Faces was the vector data that stored face data. Size() was an area to detect when face was found and unnecessary to be large.

9. Rect rect(point,x, y); new_image = image (rect) //take the part (from point, area = x * y) of image and stored it as new_image.

METHOD

Although the method of environment set up was clear, researcher had met many errors. The errors and dealing methods were recorded in this section. In the second part the face recognize method and codes were introduced.

ERROR AND DEBUG

It needed to download some files when configured with CMake. However, it was quite difficult without VPN. Researcher did not notice at the beginning and IDE report an error “could not found xxx.lib”. researcher found the lib files and they were all zero kb. Then researcher configured again and noticed that CMake reported that downloaded these files failed. For more information, researcher found “CMakeDownloadLog.txt” and all the download information was in it.

Found the lib or dll files that failed to download in txt, and it showed as below format:

do_coppy “name.dll” “ba9v90asdgb09870a” //first one was the file and second one was MD5.

“path” “website” //path was where to put the files and website was where to download.

Downloaded the files and renamed it as “MD5 + name”. If users had error in compiling or error like “could not find xxx.dll”, please checked these files and made sure they had been downloaded correctly.

In addition, code::blocks reported “thread and mutex is not defined”. There were two reasons that were possible to cause this problem. The first one was the compiler did not support C++11 standard, please upgraded it. The second reason was the code had defined thread and mutex after including them, just deleted that line.

After setting up environment, there were some error with researcher’s code. The first one was that “could not find face.hpp”. It was because face recognizing module was added into contrib file, which was not in OpenCV3 and needed to be downloaded by hand. Therefore, users downloaded it on official website and used CMake to configure again. This time users need to add the path of contrib file into CMake as below:

Then repeated the process and added the lib file of extra modules into code::blocks.

The second error was about importing the face data of OpenCV. There were many detectors in OpenCV and users only need to use an object of CascadeClassifier to load it as:

CascadeClassifier g_face_cascades;

g_face_cascades.load("haarcascade_frontalface_alt.xml")

however, the path of detectors must be relative path instead of absolute path. Researcher copy the xml file to the program file so that no path was needed.

The last error was about building face database and would be cussed in next section.

CODE

The purpose of researcher was to recognize human face and matched with template to recognize identity. In this case, researcher used the face detector in OpenCV to detect face and matched faces with the template image.

To decrease the calculate quantity, researcher used resize() to reduce pixel. Additionally, used react() to cut out faces and matched it with template image. In this case, computer only needed to match the faces in video with template image instead of matching the entire image.

As mentioned in introduction section, the double format data in minMaxLoc() showed the similarity of two images. Therefore, researcher set a threshold value of similarities. It meant it was not template person if the similarities were smaller than threshold value.

Code were shown in the end of report.

DISCUSSION

In this experiment, researcher used template matching method to identify. However, it was not accurate enough. If the threshold value was too small, computer would not be able to differentiate two people with similar appearance. In additional, computer could not recognize profile when the template image was directly opposite.

The accuracy problem was because the defect of template matching method and was difficult to solve. The second problem could be solved by using multi-images as templates. the identity was confirmed as long as threshold value was satisfied with even one image.

Another method to increase accuracy was to build face database. Using many pictures of a person to train a detector so that computer could recognize different people with different expression. To train a face model, users should prepare many pictures first (same size and gray). Put all pictures in one file and enter “dir /b/s *.jpg >at.txt” into cmd in this path. Change .jpg to the format of pictures. Then a txt named “at” was generate.

“at.txt” had recorded the path of each picture, users needed to add labels in each line (for different people: 0, 1, 2…). Researcher wrote a simple python program to finish this:

num = 1

count = 1

ff = open('D:\\facedata\\MIT-CBCL-facerec-database\\training-synthetic\\a.txt','w')

with open('D:\\facedata\\MIT-CBCL-facerec-database\\training-synthetic\\at.txt','r') as f:

    line = f.readlines()

    for line_list in line:

        line_new =line_list.replace('\n','')     //delete “/n”

        line_new=line_new+r';'+str(num)+'\n'   //add labels and “/n”

        print(line_new)

        ff.write(line_new)

        count=count + 1

        if count == 325:               //count: the number of people

            num=num+1             //num: the number of pictures (for each person)

            count = 1             

After this program done, a “a.txt” was generated. Then train these pictures by using “a.txt”:

int main()

{

    string fn_csv = "D:\\facedata\\MIT-CBCL-facerec-database\\training-synthetic\\a.txt";

    vector<Mat> images;   \\The path should be written with “\\” instead of “\”!

    vector<int> labels;

    read_csv(fn_csv, images, labels); //read data from a.txt

    Ptr<FaceRecognizer> model = createEigenFaceRecognizer();   

    model->train(images, labels);
  
    model->save("face_detect.xml");     
    }

The last error that researcher met was at this step. IDE reported “there was no createEigenFaceRecognizer in Ptr”. Researcher checked the Ptr codes and found the function should be EigenFaceRecognizer:: create(). The name was changing with different version. Therefore, codes could not run on different OpenCV version.

After training, there would be xml file generated. Used a model to read the file and use predict() to show the label. Specific code would not be discussed in this report.

CONCLUSION

In this experiment, researcher had reached the main purpose. Researcher learned how to install OpenCV(or other libraries) on code::blocks and also the basic function of OpenCV and successfully finished project of face recognizing.

Actually, coding was not as difficult as environment set up. However, this process had improved the abilities of consulting log and debugging.

APPENDICE

#include<iostream>

#include
<opencv2\opencv.hpp>

#include
<fstream>

#include
<sstream>

#include
<opencv2\face.hpp>

 

using
namespace cv;

using
namespace std;

using
namespace face;

 

 

 

int main()

 

{

    int facenum = 0;

 

    VideoCapture cap;

 

    Mat frame;

 

    Mat edges;

 

    Mat gray;

 

    Mat temframe;

 

    Mat dstImg;

 

    temframe = imread("template1.jpg");  //template image

 

    dstImg.create(temframe.dims,temframe.size,temframe.type());

 

    CascadeClassifier modelPCA;

 

    cout<<"read model"<<endl;

 

    modelPCA.load("haarcascade_frontalface_alt.xml");       //get model loaded

    

    cout<<"read done"<<endl;

 

    cap.open("me.mp4");

 

    while (1){

 

        cap>>frame;

              

        if (frame.empty())

       

              {break;}

 

        resize(frame,frame,Size(384,218));      //shrink the image by 50% to increase speed.

 

              vector<Rect> faces(0);      // face vector

 

              cvtColor(frame, gray, CV_BGR2GRAY);   //make the image gray

 

              equalizeHist(gray, gray);

 

              modelPCA.detectMultiScale(gray,faces,1.1,2,0 CV_HAAR_SCALE_IMAGE,Size(30,30));

 

              Mat face;

 

              Mat myface;

 

              Point text_lb;

 

 

              for (size_t i = 0; i <faces.size(); i++)

 

              {

 

                     if (faces[i].height > 0 && faces[i].width > 0)

 

                     {


                          face = gray(faces[i]);

                          text_lb = Point(faces[i].x, faces[i].y);

                          rectangle(frame, faces[i], Scalar(255, 0, 0), 1, 8, 0);

                          ect rect(faces[i].x,faces[i].y,faces[i].height,faces[i].width);       // take the area of face and match it with template image, in order to decrease calculated quantity

                        

       	myface = frame(rect);

                        

	resize(myface,myface,Size(57,57));

                

	cv::matchTemplate(myface, temframe, dstImg, 1);

                

	cv::Point minPoint;

                

	cv::Point maxPoint;

                

	double minVal = 0;

                

	double maxVal = 0;

                

	cv::minMaxLoc(dstImg, &minVal, &maxVal, &minPoint,&maxPoint);

                

	cout<<minVal<<endl;

                

	if(minVal<0.1)     // if matched, wrote "pepe". otherwise wrote "somebody"

                    {

                    putText(frame, "pepe", Point(faces[i].x, faces[i].y), FONT_HERSHEY_COMPLEX, 1, Scalar(0, 0, 255));

                    }

                else

                    {

                        putText(frame, "somebody", Point(faces[i].x, faces[i].y), FONT_HERSHEY_COMPLEX, 1, Scalar(0, 0, 255));

                    }

 

                     }

          }

              imshow("face.detector", frame);

              waitKey(10);

    }

   return 0;
   }