OpenCV Series — 11 — DoppelGanger App — Using Dlib’s Face Recognition resnet Model

DoppelGanger face recognizer in OpenCV

In this post , we are going to discuss about a project that I did as part of my OpenCV course. This project is used for identifying the celebrity lookalikes using Dlib’s pre-trained face-recognizer resnet model. There is no training involved , we will be just enrolling new celebrity faces to the model and use the Euclidean distance between the DoppelGanger image and the Celebrity images to identify the relevance between them for the look-alike. The model reshapes & converts the images to 128 X 1 vector , a point in 128 Dimensional space. We then use these array of vectors to calculate the Euclidean distance for the given image.

The sample dataset is used as given as below, 5 images per celebrity in each folder,

The celebrity names and the folders are mapped using a dictionary in a python numpy array file called celeb_mapping.npy

{'n00000001': 'A.J. Buckley',
'n00000002': 'A.R. Rahman',
'n00000003': 'Aamir Khan',
'n00000004': 'Aaron Staton',
'n00000005': 'Aaron Tveit',
'n00000006': 'Aaron Yoo',
'n00000007': 'Abbie Cornish',
.
.
.
}

So now let’s look at the code, Load the necessary packages.

Load the Dlib’s frontal face detector, shape predictor model and the face recognizer resnet model,

Load the celeb_mapping.npy to labelMap dictionary

Now we need to map the image files to the celebrity names

# Initialize variables.
subfolders = []
imagePaths = []
labels = []
nameLabelMap = {}
# For each subfolder in labelMap
for key in labelMap:
# Append the root folder path to the subfolder
subfolderpath = os.path.join(faceDatasetFolder, key)
# if it is a folder
if os.path.isdir(subfolderpath):
# iterate through the files in the subfolder
for x in os.listdir(subfolderpath):
# if it is a JPEG file
if x.endswith('JPEG'):
# Append the subfolder path to the image
xpath = os.path.join(subfolderpath,x)
# Append the values to the variables
imagePaths.append(xpath)
labels.append(labelMap[key])
nameLabelMap[xpath] = labelMap[key]

Now enroll the images to the model.

  • Read images
  • Detect faces
  • Identify the facial landmarks for each face
  • Convert the images to 128D vector using faceRecognizer resnet model, then convert it from Dlib’s format to List and finally to a numpy array, Then convert the 1D numpy array to a row vector using np.newaxis.
  • Concatenate the faceDescriptors row vectors as rows
  • Append the index array with image urls.

Save the faceDescriptors to numpy array file descriptors.npy and dump the. index content to index.pkl

# Write descriptors and index to disk
np.save('descriptors.npy', faceDescriptors)
# index has image paths in same order as descriptors in faceDescriptors
with open('index.pkl', 'wb') as f:
cPickle.dump(index, f)

Now it is time for testing, We can reload the content from the index.pkl file to index variable and the faceDescriptors to faceDescriptorsEnrolled variable. We will also define the threshold of the Euclidean distance as 0.55

THRESHOLD = 0.55# load descriptors and index file generated during enrollmentindex = np.load('index.pkl', allow_pickle=True)faceDescriptorsEnrolled = np.load('descriptors.npy')

Now let’s do some testing with doppelGanger images.

  • Load the DoppelGanger test images using glob into variable testImages
  • Iterate through the test Images
  • Record the time taken for each test image
  • Read the image and convert it from BGR to RGB . ( Dlib work with RGB images )
  • Detect faces & facial landmarks
  • Convert the image to 128D vector using faceRecognizer resnet model, then convert it from Dlib’s format to List and finally to a numpy array, Then convert the 1D numpy array to a row vector using np.newaxis.
  • Normalize the vector forms using np.linalg.norm on axis=1
  • Now identify the index of the minimum value along the axis using np.argmin
  • Identify the minimum distance value
  • If the minimum distance value is less than the threshold, get the label name
  • Use the label name to retrieve the image url

For more details on the project visit

Thanks for reading this post, i hope it helps you , see you in the next post

Previous-Post………………………………………………………Next-Post

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Rajathithan Rajasekar

Rajathithan Rajasekar

107 Followers

I like to write code in Python . Interested in cloud , dataAnalysis, computerVision, ML and deepLearning. https://www.linkedin.com/in/rajathithan-rajasekar/