Face Detection Video Analysis
In this tutorial, we will use fastdup with a pretrained face detection model to detect and crop from videos. Following that we analyze the cropped objects for issues such as duplicates, near-duplicates, outliers, bright/dark/blurry objects.
Follow along this tutorial by running this Colab Notebook.
Setting Up
!pip install pip -U
!pip install fastdup
Verify the installation
import fastdup
fastdup.__version__
'0.910'
Download & Extract Dataset
We downloaded a few random TikTok videos for the purpose of this demonstration. Feel free to use your own video dataset.
Let's download the TikTok videos into our local folder.
!gdown --fuzzy https://drive.google.com/file/d/1fzmOgmRu557aU4lEbzL7XCf78KntFCeQ/view
!unzip data.zip
From Videos to Images
fastdup works on images. We must first extract frames from the videos using a one-liner fastdup utility function.
fastdup.extract_video_frames(input_dir="data", work_dir="frames")
This should create aframes/folder which stores all the frames extracted from the videos.
Run fastdup
Now that we have the frames of images, let's run fastdup and analyze the frames.
fd = fastdup.create(input_dir='frames', work_dir='face_detection_work_dir')
fd.run(bounding_box='face', overwrite=True)
FastDup Software, (C) copyright 2022 Dr. Amir Alush and Dr. Danny Bickson.
2023-03-29 17:11:55 [INFO] Going to loop over dir frames
2023-03-29 17:11:55 [INFO] Found total 99 images to run on
FastDup Software, (C) copyright 2022 Dr. Amir Alush and Dr. Danny Bickson.utes 0 Features
2023-03-29 17:11:58 [INFO] Going to loop over dir /tmp/crops_input.csv
2023-03-29 17:11:58 [INFO] Found total 66 images to run on
2023-03-29 17:11:59 [INFO] Found total 66 images to run onEstimated: 0 Minutes 0 Features
Finished histogram 0.091
Finished bucket sort 0.106
2023-03-29 17:11:59 [INFO] 10) Finished write_index() NN model
2023-03-29 17:11:59 [INFO] Stored nn model index file face_detection_work_dir/nnf.index
2023-03-29 17:11:59 [INFO] Total time took 1017 ms
2023-03-29 17:11:59 [INFO] Found a total of 0 fully identical images (d>0.990), which are 0.00 %
2023-03-29 17:11:59 [INFO] Found a total of 2 nearly identical images(d>0.980), which are 1.01 %
2023-03-29 17:11:59 [INFO] Found a total of 28 above threshold images (d>0.900), which are 14.14 %
2023-03-29 17:11:59 [INFO] Found a total of 6 outlier images (d<0.050), which are 3.03 %
2023-03-29 17:11:59 [INFO] Min distance found 0.528 max distance 0.982
2023-03-29 17:11:59 [INFO] Running connected components for ccthreshold 0.960000
.0
########################################################################################
Dataset Analysis Summary:
Dataset contains 66 images
Valid images are 100.00% (66) of the data, invalid are 0.00% (0) of the data
Similarity: 3.03% (2) belong to 2 similarity clusters (components).
96.97% (64) images do not belong to any similarity cluster.
Largest cluster has 6 (9.09%) images.
For a detailed analysis, use `.connected_components()`
(similarity threshold used is 0.9, connected component threshold used is 0.96).
Outliers: 6.06% (4) of images are possible outliers, and fall in the bottom 5.00% of similarity values.
For a detailed list of outliers, use `.outliers()`.
Components Gallery
We can visualize the cluster of similar detections using the components gallery view. Specify draw_bbox=True to see the detection bounding box on the original image.
fd.vis.component_gallery(draw_bbox=True)
If you'd like to view just the cropped bounding box images, specify draw_bbox=False
fd.vis.component_gallery(draw_bbox=False)
Similarity Gallery
Using the similarity_gallery view, we can find similar looking faces (bounding boxes) across all the extracted frames.
fd.vis.similarity_gallery(draw_bbox=False)
Duplicates Gallery
With the duplicates_gallery view, visualize duplicate image pairs across videos.
fd.vis.duplicates_gallery()
Outliers Gallery
Using the outliers_gallery we can also visualize faces (detections) that looks visually different from others.
fd.vis.outliers_gallery()
Stats Gallery
You can visualize the faces using various metrics using the stats_gallery.
Dark Faces
Specifying metric='dark' sorts the detections in an ascending order of image mean value.
fd.vis.stats_gallery(metric='dark')
Bright Faces
Conversely, specifying metric='bright'sorts the detections in descending order of image mean value.
fd.vis.stats_gallery(metric='bright')
Blurry Faces
Finally, specifying metric='blur'ranks the images in ascending 'blurriness'.
fd.vis.stats_gallery(metric='blur')
Updated about 1 year ago