ddp/photoassistant
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

added deepface notes

Browse Source
This commit is contained in:
Damien De Paoli
2025-01-04 21:59:56 +11:00
parent 37c0125fbd
commit febdfbbd32
1 changed files with 185 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files

189
TODO
View File

@@ -1,4 +1,10 @@
### GENERAL
* optimisation:
keep track of just new files since scan (even if we did this from the DB),
then we could just feed those eid's explicitly into a 'get_file_details_on_new_files'.
ALSO use new_eids list IF no new refimgs to do a 'run_ai_on_new_files' :)
* allow changing dates in move dbox and then re-scan for existing folders OR just have a browse for existing...
- for use on scanned photos, they register as 2010, but are datestamped in visuals for 95
(is there a library for this???)
@@ -13,10 +19,6 @@
* read this: https://flask.palletsprojects.com/en/2.2.x/testing/#faking-resources-and-context
* could get better AI optim, by keeping track of just new files since scan (even if we did this from the DB),
then we could just feed those eid's explicitly into a 'run_ai_on_new_files' :) -- maybe particularly
if count('new files') < say 1000 do eids, otherwise do path AND no new refimgs
* ignore face should ignore ALL matching faces (re: Declan)
* should allow context menu from View thumbs (particularly useful on search) to show other files around this one by date (maybe that folder or something?)
@@ -207,3 +209,182 @@ def AddRefimgToPerson(person_id: int, ref_images: List[str]) -> bool:
>>> AddRefimgToPerson(123, ['http://example.com/img1.jpg', '/path/to/img2.png'])
True
"""
checkout deepface:
User: I have this code I wrote using pythons face_recognition library, but I would like to alter it to work with deepface. My code is: im = face_recognition.load_image_file(e.FullPathOnFS())
face_locations = face_recognition.face_locations(im, model=model.name )
unknown_encodings = face_recognition.face_encodings(im, known_face_locations=face_locations)
Assistant: To alter your existing code to work with **DeepFace**, the changes involve replacing the methods of the `face_recognition` library with similar functionalities provided by DeepFace.
In **DeepFace**, the equivalent functionalities include:
1. **Detecting faces**: This is automatically handled by the `analyze` method in DeepFace. You don't need to manually specify the face locations.
2. **Encoding faces**: DeepFace internally computes embeddings for faces through its supported models (like VGG-Face, Facenet, OpenFace, DeepID, etc.).
Here's how you can modify your code to use DeepFace instead of `face_recognition`:
### Original Code (using `face_recognition`):
```python
im = face_recognition.load_image_file(e.FullPathOnFS())
face_locations = face_recognition.face_locations(im, model=model.name)
unknown_encodings = face_recognition.face_encodings(im, known_face_locations=face_locations)
```
### Modified Code (using `DeepFace`):
```python
from deepface import DeepFace
import cv2
# Load the image
im = cv2.imread(e.FullPathOnFS())
# Analyze the image to get face embeddings (encodings) and other details
results = DeepFace.represent(img_path=e.FullPathOnFS(), model_name=model.name)
# `results` will contain a list of dictionaries where each dictionary contains the embedding (encoding) of a detected face.
# Example of accessing the embeddings (face encodings)
unknown_encodings = [res['embedding'] for res in results]
```
### Explanation:
- `DeepFace.represent()` takes the path to the image (`e.FullPathOnFS()`) and the name of the model (`model.name`). It processes the image and extracts the embeddings (encodings) for each face detected.
- **Note**: In DeepFace, the model name can be one of several options: `VGG-Face`, `Facenet`, `OpenFace`, `DeepFace`, `DeepID`, etc. Make sure that `model.name` corresponds to one of these supported models.
- The result is a list of dictionaries. Each dictionary contains a detected face's encoding (embedding), which you can access using the `'embedding'` key.
DeepFace abstracts away a lot of the manual face detection work that you were handling with `face_recognition`. Let me know if you'd like to handle anything else, like visualizing the detected faces or working with other DeepFace models.
User: I want to use Deepface to take a set of reference images and match against all the images I have to find the best matches, how would you go about that?
Assistant: To use **DeepFace** to match a set of reference images against other images to find the best matches, you can follow these general steps:
1. **Compute embeddings (encodings) for the reference images**.
2. **Compute embeddings for the target images** (the ones you're searching through).
3. **Compare the embeddings** of the reference images with those of the target images to find the best matches based on a distance metric.
4. **Rank the matches** by similarity and return the best ones.
Here's a step-by-step guide on how you can achieve this with **DeepFace**:
### Step 1: Compute embeddings for reference images
You will load the reference images, compute their embeddings using DeepFace, and store them.
### Step 2: Compute embeddings for all target images
Do the same for the target images you want to match against.
### Step 3: Compare embeddings
You will compare the embeddings from the reference images to the embeddings of each target image. The smaller the distance between the embeddings, the more similar the faces are.
### Step 4: Return the best matches
### Example Code
```python
from deepface import DeepFace
import os
import cv2
import numpy as np
# Function to compute embeddings for a given image path
def compute_embeddings(image_path, model_name='VGG-Face'):
try:
# This returns a list of embeddings for each face found in the image
embeddings = DeepFace.represent(img_path=image_path, model_name=model_name)
return [res['embedding'] for res in embeddings]
except Exception as e:
print(f"Error processing image {image_path}: {e}")
return []
# Function to find the best matches between reference and target images
def find_best_matches(reference_images, target_images, model_name='VGG-Face', metric='cosine'):
# Store embeddings for reference images
reference_embeddings = {}
for ref_img in reference_images:
ref_embeddings = compute_embeddings(ref_img, model_name=model_name)
if ref_embeddings:
reference_embeddings[ref_img] = ref_embeddings
# Store best matches
best_matches = []
# Loop through each target image
for target_img in target_images:
target_embeddings = compute_embeddings(target_img, model_name=model_name)
for target_emb in target_embeddings:
for ref_img, ref_emb_list in reference_embeddings.items():
# Compare each reference embedding with the target embedding
for ref_emb in ref_emb_list:
# Compute the distance between embeddings using DeepFace's distance functions
distance = DeepFace.find_distance(ref_emb, target_emb, distance_metric=metric)
# Store match details (target image, reference image, and distance)
best_matches.append({
'target_image': target_img,
'reference_image': ref_img,
'distance': distance
})
# Sort matches by the distance (smallest distances indicate best matches)
best_matches = sorted(best_matches, key=lambda x: x['distance'])
return best_matches
# Example usage
reference_images = [
"path_to_reference_image1.jpg",
"path_to_reference_image2.jpg"
]
target_images = [
"path_to_target_image1.jpg",
"path_to_target_image2.jpg",
# Add more target images here...
]
# Find the best matches
matches = find_best_matches(reference_images, target_images, model_name='VGG-Face', metric='cosine')
# Display the best match
for match in matches[:5]: # Display top 5 matches
print(f"Reference Image: {match['reference_image']}, Target Image: {match['target_image']}, Distance: {match['distance']}")
```
### Explanation:
1. **`compute_embeddings(image_path, model_name)`**: This function uses DeepFace's `represent` function to compute face embeddings for each face found in the image.
2. **`find_best_matches()`**:
- It first computes and stores embeddings for all reference images in the `reference_embeddings` dictionary.
- For each target image, it computes the embeddings, then compares those embeddings with all the reference embeddings using the chosen distance metric (default is `'cosine'`).
- The function then stores the matches and sorts them by distance (the smaller the distance, the closer the match).
3. **Distance metrics**: You can choose between several distance metrics like `'cosine'`, `'euclidean'`, or `'euclidean_l2'`, depending on the model and your preference.
4. **Matching logic**: It finds the best matches by comparing each face embedding in the target images with each face embedding in the reference images and storing the distances.
### DeepFace Distance Metrics:
- **Cosine**: Measures the cosine of the angle between two vectors (common for face embeddings).
- **Euclidean**: Measures the straight-line distance between two vectors.
- **Euclidean L2**: A variant of Euclidean where embeddings are normalized before distance calculation.
### Example of top match output:
```bash
Reference Image: path_to_reference_image1.jpg, Target Image: path_to_target_image1.jpg, Distance: 0.256
Reference Image: path_to_reference_image2.jpg, Target Image: path_to_target_image2.jpg, Distance: 0.310
...
```
This approach will give you the best matching images based on facial similarity. You can adjust the model used (`VGG-Face`, `Facenet`, etc.) and the distance metric to fine-tune results. Let me know if you'd like to visualize the results or handle larger datasets more efficiently!