From febdfbbd3256ec2740e2162ed8f84bc25b3f001a Mon Sep 17 00:00:00 2001 From: Damien De Paoli Date: Sat, 4 Jan 2025 21:59:56 +1100 Subject: [PATCH] added deepface notes --- TODO | 189 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 185 insertions(+), 4 deletions(-) diff --git a/TODO b/TODO index 6477095..3a5617e 100644 --- a/TODO +++ b/TODO @@ -1,4 +1,10 @@ ### GENERAL + * optimisation: + keep track of just new files since scan (even if we did this from the DB), + then we could just feed those eid's explicitly into a 'get_file_details_on_new_files'. + ALSO use new_eids list IF no new refimgs to do a 'run_ai_on_new_files' :) + + * allow changing dates in move dbox and then re-scan for existing folders OR just have a browse for existing... - for use on scanned photos, they register as 2010, but are datestamped in visuals for 95 (is there a library for this???) @@ -13,10 +19,6 @@ * read this: https://flask.palletsprojects.com/en/2.2.x/testing/#faking-resources-and-context - * could get better AI optim, by keeping track of just new files since scan (even if we did this from the DB), - then we could just feed those eid's explicitly into a 'run_ai_on_new_files' :) -- maybe particularly - if count('new files') < say 1000 do eids, otherwise do path AND no new refimgs - * ignore face should ignore ALL matching faces (re: Declan) * should allow context menu from View thumbs (particularly useful on search) to show other files around this one by date (maybe that folder or something?) @@ -207,3 +209,182 @@ def AddRefimgToPerson(person_id: int, ref_images: List[str]) -> bool: >>> AddRefimgToPerson(123, ['http://example.com/img1.jpg', '/path/to/img2.png']) True """ + + + + +checkout deepface: + + +User: I have this code I wrote using pythons face_recognition library, but I would like to alter it to work with deepface. My code is: im = face_recognition.load_image_file(e.FullPathOnFS()) + face_locations = face_recognition.face_locations(im, model=model.name ) + unknown_encodings = face_recognition.face_encodings(im, known_face_locations=face_locations) + +Assistant: To alter your existing code to work with **DeepFace**, the changes involve replacing the methods of the `face_recognition` library with similar functionalities provided by DeepFace. + +In **DeepFace**, the equivalent functionalities include: + +1. **Detecting faces**: This is automatically handled by the `analyze` method in DeepFace. You don't need to manually specify the face locations. + +2. **Encoding faces**: DeepFace internally computes embeddings for faces through its supported models (like VGG-Face, Facenet, OpenFace, DeepID, etc.). + +Here's how you can modify your code to use DeepFace instead of `face_recognition`: + +### Original Code (using `face_recognition`): +```python +im = face_recognition.load_image_file(e.FullPathOnFS()) +face_locations = face_recognition.face_locations(im, model=model.name) +unknown_encodings = face_recognition.face_encodings(im, known_face_locations=face_locations) +``` + +### Modified Code (using `DeepFace`): +```python +from deepface import DeepFace +import cv2 + +# Load the image +im = cv2.imread(e.FullPathOnFS()) + +# Analyze the image to get face embeddings (encodings) and other details +results = DeepFace.represent(img_path=e.FullPathOnFS(), model_name=model.name) + +# `results` will contain a list of dictionaries where each dictionary contains the embedding (encoding) of a detected face. +# Example of accessing the embeddings (face encodings) +unknown_encodings = [res['embedding'] for res in results] +``` + +### Explanation: + +- `DeepFace.represent()` takes the path to the image (`e.FullPathOnFS()`) and the name of the model (`model.name`). It processes the image and extracts the embeddings (encodings) for each face detected. + +- **Note**: In DeepFace, the model name can be one of several options: `VGG-Face`, `Facenet`, `OpenFace`, `DeepFace`, `DeepID`, etc. Make sure that `model.name` corresponds to one of these supported models. + +- The result is a list of dictionaries. Each dictionary contains a detected face's encoding (embedding), which you can access using the `'embedding'` key. + +DeepFace abstracts away a lot of the manual face detection work that you were handling with `face_recognition`. Let me know if you'd like to handle anything else, like visualizing the detected faces or working with other DeepFace models. + + +User: I want to use Deepface to take a set of reference images and match against all the images I have to find the best matches, how would you go about that? + +Assistant: To use **DeepFace** to match a set of reference images against other images to find the best matches, you can follow these general steps: + +1. **Compute embeddings (encodings) for the reference images**. +2. **Compute embeddings for the target images** (the ones you're searching through). +3. **Compare the embeddings** of the reference images with those of the target images to find the best matches based on a distance metric. +4. **Rank the matches** by similarity and return the best ones. + +Here's a step-by-step guide on how you can achieve this with **DeepFace**: + +### Step 1: Compute embeddings for reference images +You will load the reference images, compute their embeddings using DeepFace, and store them. + +### Step 2: Compute embeddings for all target images +Do the same for the target images you want to match against. + +### Step 3: Compare embeddings +You will compare the embeddings from the reference images to the embeddings of each target image. The smaller the distance between the embeddings, the more similar the faces are. + +### Step 4: Return the best matches + +### Example Code + +```python +from deepface import DeepFace +import os +import cv2 +import numpy as np + +# Function to compute embeddings for a given image path +def compute_embeddings(image_path, model_name='VGG-Face'): + try: + # This returns a list of embeddings for each face found in the image + embeddings = DeepFace.represent(img_path=image_path, model_name=model_name) + return [res['embedding'] for res in embeddings] + except Exception as e: + print(f"Error processing image {image_path}: {e}") + return [] + +# Function to find the best matches between reference and target images +def find_best_matches(reference_images, target_images, model_name='VGG-Face', metric='cosine'): + # Store embeddings for reference images + reference_embeddings = {} + + for ref_img in reference_images: + ref_embeddings = compute_embeddings(ref_img, model_name=model_name) + if ref_embeddings: + reference_embeddings[ref_img] = ref_embeddings + + # Store best matches + best_matches = [] + + # Loop through each target image + for target_img in target_images: + target_embeddings = compute_embeddings(target_img, model_name=model_name) + + for target_emb in target_embeddings: + for ref_img, ref_emb_list in reference_embeddings.items(): + # Compare each reference embedding with the target embedding + for ref_emb in ref_emb_list: + # Compute the distance between embeddings using DeepFace's distance functions + distance = DeepFace.find_distance(ref_emb, target_emb, distance_metric=metric) + + # Store match details (target image, reference image, and distance) + best_matches.append({ + 'target_image': target_img, + 'reference_image': ref_img, + 'distance': distance + }) + + # Sort matches by the distance (smallest distances indicate best matches) + best_matches = sorted(best_matches, key=lambda x: x['distance']) + + return best_matches + +# Example usage +reference_images = [ + "path_to_reference_image1.jpg", + "path_to_reference_image2.jpg" +] + +target_images = [ + "path_to_target_image1.jpg", + "path_to_target_image2.jpg", + # Add more target images here... +] + +# Find the best matches +matches = find_best_matches(reference_images, target_images, model_name='VGG-Face', metric='cosine') + +# Display the best match +for match in matches[:5]: # Display top 5 matches + print(f"Reference Image: {match['reference_image']}, Target Image: {match['target_image']}, Distance: {match['distance']}") +``` + +### Explanation: + +1. **`compute_embeddings(image_path, model_name)`**: This function uses DeepFace's `represent` function to compute face embeddings for each face found in the image. + +2. **`find_best_matches()`**: + - It first computes and stores embeddings for all reference images in the `reference_embeddings` dictionary. + - For each target image, it computes the embeddings, then compares those embeddings with all the reference embeddings using the chosen distance metric (default is `'cosine'`). + - The function then stores the matches and sorts them by distance (the smaller the distance, the closer the match). + +3. **Distance metrics**: You can choose between several distance metrics like `'cosine'`, `'euclidean'`, or `'euclidean_l2'`, depending on the model and your preference. + +4. **Matching logic**: It finds the best matches by comparing each face embedding in the target images with each face embedding in the reference images and storing the distances. + +### DeepFace Distance Metrics: + +- **Cosine**: Measures the cosine of the angle between two vectors (common for face embeddings). +- **Euclidean**: Measures the straight-line distance between two vectors. +- **Euclidean L2**: A variant of Euclidean where embeddings are normalized before distance calculation. + +### Example of top match output: +```bash +Reference Image: path_to_reference_image1.jpg, Target Image: path_to_target_image1.jpg, Distance: 0.256 +Reference Image: path_to_reference_image2.jpg, Target Image: path_to_target_image2.jpg, Distance: 0.310 +... +``` + +This approach will give you the best matching images based on facial similarity. You can adjust the model used (`VGG-Face`, `Facenet`, etc.) and the distance metric to fine-tune results. Let me know if you'd like to visualize the results or handle larger datasets more efficiently! +