ddp/photoassistant
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

unmatched faces now sorts size of face desc, and is slightly prettier -- still slow and only takes you to the file anyway, might optimise that later. still no code to auto deal with unmatched face, but will add some capabilities next. Also, remembered last dir when file_ip/sp/bin. Also throw error if try to find unknown person - happened since I allowed the back/forward.

Browse Source
This commit is contained in:
Damien De Paoli
2022-01-18 20:59:39 +11:00
parent 4c3aae770d
commit de81db9412
11 changed files with 80 additions and 73 deletions
Download Patch File Download Diff File Expand all files Collapse all files

29
TODO
View File

@@ -1,19 +1,17 @@
## GENERAL
*** Need to double-check scheduled jobs running in PROD (can use new pa_job_manager.log)
* remember last import dir, so you can just go straight back to it
* when hitting back button to a search, it doesnt handle the post, etc.
$(document).ready(function() {
window.onpopstate = function() {
# this seems to work, but feels like no protection at all???
# (what about back when it goes onto a POST of deleting a file!)
window.history.back()
};
});
-- maybe window.history.replace() is needed on unsafe URLs?
* optimise run_ai_on (and for that matter getfiledetails, etc.)...
- e.g. with last scan*, STORE: new files?
- if last scan no new files, dont getfiledetails, don't re-run ai job
* per file you could select an unknown face and add it as a ref img to an existing person, or make a new person and attach?
* [DONE] order/ find face with largest size and at least show that as unmatched
- could also try to check it vs. other faces, if it matches more than say 10? we offer it up as a required ref img, then cut that face (with margin) out and use it is a new ref image / person
* on viewer: allow face to be used to create person, add to existing person, and allow 'ignore', mark as 'not a face', etc. -> all into DB
- so need face 'treatment' -> could be matched via face_refimg_link, but also could be 'ignore' or 'not a face', in each case we could exclude those faces from
matching for the future, and reporting on matches, etc.
- context-menu with rects on a canvas
https://stackoverflow.com/questions/31601393/create-context-menu-using-jquery-with-html-5-canvas
- also allow joblog search from the viewer for that file...
* delete folder
@@ -46,14 +44,15 @@
* comment your code -> only html files remaining
* from menu, we could try to get smart/fancy... say find face with largest size, check it vs. other faces, if it matches more than say 10? we offer it up as a required ref img, then cut that face (with margin) out and use it is a new ref image / person
- read that guys face matching / clustering / nearest neighbour examples, for a whole new AI capability
* read that guys face matching / clustering / nearest neighbour examples, for a whole new AI capability
https://www.pyimagesearch.com/2018/07/09/face-clustering-with-python/
* fix up logging in general
* support animated gifs in html5 canvas
* think about security - in job_mgr anywhere I can os.replace/remove NEED to protect, etc
## DB
* Dir can have date in the DB, so we can do Oldest/Newest dirs in Folder view

3
ai.py
View File

@@ -91,7 +91,7 @@ def run_ai_on_storage():
@app.route("/unmatched_faces")
@login_required
def unmatched_faces():
faces=Face.query.join(FaceFileLink).join(FaceRefimgLink, isouter=True).filter(FaceRefimgLink.refimg_id==None).limit(10).all()
faces=Face.query.join(FaceFileLink).join(FaceRefimgLink, isouter=True).filter(FaceRefimgLink.refimg_id==None).order_by(Face.h.desc()).limit(10).all()
imgs={}
for face in faces:
face.locn=json.loads("["+face.locn+"]")
@@ -102,6 +102,7 @@ def unmatched_faces():
y=face.locn[0][0]*0.95
x2=face.locn[0][1]*1.05
y2=face.locn[0][2]*1.05
im = Image.open(f.FullPathOnFS())
region = im.crop((x, y, x2, y2))
img_bytearray = io.BytesIO()

9
face.py
View File

@@ -2,6 +2,12 @@ from main import db, app, ma
from sqlalchemy import Sequence
from sqlalchemy.exc import SQLAlchemyError
# DEL ME SOON
from flask_login import login_required
from flask import render_template
import json
# pylint: disable=no-member
################################################################################
@@ -17,6 +23,8 @@ class Face(db.Model):
id = db.Column(db.Integer, db.Sequence('face_id_seq'), primary_key=True )
face = db.Column( db.LargeBinary )
locn = db.Column( db.String )
w = db.Column( db.Integer )
h = db.Column( db.Integer )
refimg_lnk = db.relationship("FaceRefimgLink", uselist=False, viewonly=True)
facefile_lnk = db.relationship("FaceFileLink", uselist=False, viewonly=True)
refimg =db.relationship("Refimg", secondary="face_refimg_link", uselist=False)
@@ -53,4 +61,3 @@ class FaceRefimgLink(db.Model):
def __repr__(self):
return f"<face_id: {self.face_id}, refimg_id={self.refimg_id}, face_distance: {self.face_distance}"

16
internal/js/face.js
View File

@@ -1,29 +1,15 @@
// Define this once and before it will be called, hence at the top of this file
function DrawRefimg(fig, img, canvas, orig_face )
{
// FIXME: should get this from shared.py, not sure why this doesnt work at present
thumbsize=256
context=canvas.getContext('2d')
// another call to this func will occur on load, so skip this one
if( img.width == 0 )
return
// only set canvas.width once we have valid img dimensions
canvas.width=img.width/2
canvas.width=img.width/(img.height/canvas.height)
// actually draw the pixel images to the canvas at the right size
context.drawImage(img, 0, 0, img.width/(img.height/canvas.height), canvas.height);
fig.width(canvas.width)
// draw rectangle on face
context.beginPath();
new_x=(orig_face.x/orig_face.orig_w)*img.width/(img.height/canvas.height)
new_y=(orig_face.y/orig_face.orig_h)*thumbsize/(img.height/canvas.height)
new_w=(orig_face.w/orig_face.orig_w)*img.width/(img.height/canvas.height)
new_h=(orig_face.h/orig_face.orig_h)*thumbsize/(img.height/canvas.height)
context.rect(new_x, new_y, new_w, new_h)
context.lineWidth = 2;
context.strokeStyle = 'green';
context.stroke();
}

16
options.py
View File

@@ -21,9 +21,11 @@ class PA_PREF(db.Model):
size = db.Column(db.Integer, unique=False, nullable=False )
folders = db.Column(db.Boolean, unique=False, nullable=False )
fullscreen = db.Column(db.Boolean, unique=False, nullable=False )
root = db.Column(db.String, unique=False, nullable=False )
cwd = db.Column(db.String, unique=False, nullable=False )
def __repr__(self):
return f"<pa_user_dn: {self.pa_user_dn}, path_type: {self.path_type}, noo: {self.noo}, grouping: {self.grouping}, how_many: {self.how_many}, st_offset: {self.st_offset}, size: {self.size}, folders: {self.folders}>"
return f"<pa_user_dn: {self.pa_user_dn}, path_type: {self.path_type}, noo: {self.noo}, grouping: {self.grouping}, how_many: {self.how_many}, st_offset: {self.st_offset}, size: {self.size}, folders: {self.folders}, root: {self.root}, cwd: {self.cwd}>"
################################################################################
@@ -85,14 +87,16 @@ class Options(PA):
self.how_many=pref.how_many
self.offset=pref.st_offset
self.size=pref.size
self.root=pref.root
self.cwd=pref.cwd
else:
self.grouping="None"
self.how_many="50"
self.offset="0"
self.size="128"
self.root='static/' + self.path_type
self.cwd=self.root
self.cwd='static/' + self.path_type
self.root=self.cwd
# the above are defaults, if we are here, then we have current values, use them instead if they are set -- AI: searches dont set them so then we use those in the DB first
if request.method=="POST":
@@ -134,7 +138,7 @@ class Options(PA):
pref=PA_PREF.query.filter(PA_PREF.pa_user_dn==current_user.dn,PA_PREF.path_type==self.path_type).first()
if not pref:
pref=PA_PREF( pa_user_dn=current_user.dn, path_type=self.path_type, noo=self.noo, grouping=self.grouping, how_many=self.how_many,
st_offset=self.offset, size=self.size, folders=self.folders)
st_offset=self.offset, size=self.size, folders=self.folders, root=self.root, cwd=self.cwd)
else:
pref.noo=self.noo
pref.grouping=self.grouping
@@ -142,10 +146,14 @@ class Options(PA):
pref.st_offset=self.offset
pref.size=self.size
pref.folders=self.folders
pref.root = self.root
pref.cwd = self.cwd
db.session.add(pref)
db.session.commit()
return
################################################################################
# /prefs -> GET only -> prints out list of all prefs (simple for now)
################################################################################

6
pa_job_manager.py
View File

@@ -320,6 +320,8 @@ class Face(Base):
id = Column(Integer, Sequence('face_id_seq'), primary_key=True )
face = Column( LargeBinary )
locn = Column(String)
w = Column(Integer)
h = Column(Integer)
def __repr__(self):
return f"<id: {self.id}, face={self.face}"
@@ -1848,7 +1850,9 @@ def InitialValidationChecks():
# AddFaceToFile(): adds the specified face, location & model_used to the specified file
####################################################################################################################################
def AddFaceToFile( locn_data, face_data, file_eid, model_id ):
face = Face( face=face_data.tobytes(), locn=json.dumps(locn_data) )
w = locn_data[1] - locn_data[3]
h = locn_data[2] - locn_data[0]
face = Face( face=face_data.tobytes(), locn=json.dumps(locn_data), w=w, h=h )
session.add(face)
session.commit()
ffl = FaceFileLink( face_id=face.id, file_eid=file_eid, model_used=model_id )

4
person.py
View File

@@ -178,6 +178,10 @@ def person(id):
return render_template("person.html", form=form, page_title=page_title)
else:
person = Person.query.get(id)
if not person:
st.SetMessage( f"No such person with id: {id}", "danger" )
return render_template("base.html" )
for r in person.refimg:
r.face_locn=json.loads(r.face_locn)
form = PersonForm(request.values, obj=person)

4
tables.sql
View File

@@ -15,7 +15,7 @@ create table SETTINGS(
constraint FK_DEFAULT_REFIMG_MODEL foreign key (DEFAULT_REFIMG_MODEL) references AI_MODEL(ID),
constraint FK_DEFAULT_SCAN_MODEL foreign key (DEFAULT_SCAN_MODEL) references AI_MODEL(ID) );
create table PA_PREF ( PA_USER_DN varchar(128), PATH_TYPE varchar(16), NOO varchar(16), GROUPING varchar(16), HOW_MANY integer, ST_OFFSET integer, SIZE integer, FOLDERS Boolean, FULLSCREEN Boolean,
create table PA_PREF ( PA_USER_DN varchar(128), PATH_TYPE varchar(16), NOO varchar(16), GROUPING varchar(16), HOW_MANY integer, ST_OFFSET integer, SIZE integer, FOLDERS Boolean, FULLSCREEN Boolean, ROOT varchar, CWD varchar,
constraint PK_PA_USER_DN_PATH_TYPE primary key(PA_USER_DN, PATH_TYPE ) );
create table PA_USER( ID integer, dn varchar,
@@ -61,7 +61,7 @@ create table REFIMG ( ID integer, FNAME varchar(128), FACE bytea, ORIG_W integer
constraint PK_REFIMG_ID primary key(ID),
constraint FK_REFIMG_MODEL_USED foreign key (MODEL_USED) references AI_MODEL(ID) );
create table FACE( ID integer, FACE bytea, LOCN varchar(32), constraint PK_FACE_ID primary key(ID) );
create table FACE( ID integer, FACE bytea, LOCN varchar(32), W integer, H integer, constraint PK_FACE_ID primary key(ID) );
create table FACE_FILE_LINK( FACE_ID integer, FILE_EID integer, MODEL_USED integer,
constraint PK_FFL_FACE_ID_FILE_ID primary key(FACE_ID, FILE_EID),

4
templates/base.html
View File

@@ -163,6 +163,10 @@
{% if not InDBox %}
<script>
// do our own back button handling, TODO: 'dangerous' URLs, will be replaced with GETs first via history.replaceState()
window.onpopstate = function(e) {
window.history.back()
}
function SetViewingOptionsForSearchForm()
{
if( $('#noo').length )

58
templates/faces.html
View File

@@ -6,42 +6,34 @@
<h3>Unmatched Faces</h3>
<div class="row mt-3">
{% for f in faces %}
<div id="F{{f.id}}" class="col-2 px-0">
<div id="F{{f.id}}" class="col col-auto mt-3 pr-1">
<form id="_fm" method="POST" action="/view/{{f.file_eid}}">
<input type="hidden" name="eids" value="{{f.file_eid}},">
<input type="hidden" name="noo" value="newest">
<input type="hidden" name="cwd" value="/">
<input type="hidden" name="root" value="/">
<input type="hidden" name="size" value="128">
<input type="hidden" name="grouping" value="1">
<input type="hidden" name="offset" value="0">
<input type="hidden" name="folders" value="false">
<input type="hidden" name="how_many" value="1">
<input type="hidden" name="orig_url" value="{{request.path}}">'
<input type="hidden" name="eids" value="{{f.file_eid}},">
<figure id="fig_{{f.id}}" class="mb-0">
<div style="position:relative">
<canvas id="c_{{f.id}}" height="128"></canvas>
<script>
var im_{{f.id}}=new Image();
im_{{f.id}}.src="data:image/jpeg;base64,{{f.img}}";
fig_{{f.id}}=$('#fig_{{f.id}}')
// store this stuff in an javascript Object to use when document is ready event is triggered
var orig_face_{{f.id}}=new Object;
orig_face_{{f.id}}.x = (({{f.locn[0][1]}}*1.05 - {{f.locn[0][3]}}*.95) - {{f.w}}) / 2
orig_face_{{f.id}}.y = (({{f.locn[0][2]}}*1.05 - {{f.locn[0][0]}}*.95) - {{f.h}}) / 2
orig_face_{{f.id}}.w = {{f.w}}
orig_face_{{f.id}}.h = {{f.h}}
orig_face_{{f.id}}.orig_w = {{f.locn[0][1]}}*1.05 - {{f.locn[0][3]}}*.95
orig_face_{{f.id}}.orig_h = {{f.locn[0][2]}}*1.05 - {{f.locn[0][0]}}*.95
<figure id="fig_{{f.id}}">
<div style="position:relative">
<canvas id="c_{{f.id}}" height="128"></canvas>
<script>
var im_{{f.id}}=new Image();
im_{{f.id}}.src="data:image/jpeg;base64,{{f.img}}";
fig_{{f.id}}=$('#fig_{{f.id}}')
// store this stuff in an javascript Object to use when document is ready event is triggered
var orig_face_{{f.id}}=new Object;
orig_face_{{f.id}}.x = {{f.locn[0][3]}}
orig_face_{{f.id}}.y = {{f.locn[0][0]}}
orig_face_{{f.id}}.w = {{f.locn[0][1]}}-{{f.locn[0][3]}}
orig_face_{{f.id}}.h = {{f.locn[0][2]}}-{{f.locn[0][0]}}
orig_face_{{f.id}}.orig_w = orig_face_{{f.id}}.w
orig_face_{{f.id}}.orig_h = orig_face_{{f.id}}.h
console.log( orig_face_{{f.id}} )
// when the document is ready, then DrawRefimg
$(function() { DrawRefimg( fig_{{f.id}}, im_{{f.id}}, c_{{f.id}}, orig_face_{{f.id}} ) });
</script>
<figcaption>{{f.id}}</figcation>
</div>
</figure>
<button>Go</button>
// when the document is ready, then DrawRefimg
$(function() { DrawRefimg( fig_{{f.id}}, im_{{f.id}}, c_{{f.id}}, orig_face_{{f.id}} ) });
</script>
<figcaption>Face #{{f.id}}</figcation>
</div>
</figure>
<button class="btn btn-outline-info">Go</button>
</form>
</div id="/F*">
{% endfor %}

4
templates/prefs.html
View File

@@ -5,7 +5,7 @@
<table id="prefs_table" class="table table-striped table-sm" data-toolbar="#toolbar" data-search="true">
<thead>
<tr class="table-primary"><th>Path</th><th>New or Oldest</th><th>How Many</th><th>Folders?</th><th>Group by</th><th>Thumb size</th><th>Fullscreen</th><th>DB retrieve offset</th></tr>
<tr class="table-primary"><th>Path</th><th>New or Oldest</th><th>How Many</th><th>Folders?</th><th>Group by</th><th>Thumb size</th><th>Fullscreen</th><th>DB retrieve offset</th><th>Root</th><th>cwd</th></tr>
</thead>
<tbody>
{% for pref in prefs %}
@@ -18,6 +18,8 @@
<td>{{pref.size}}</td>
<td>{{pref.fullscreen}}</td>
<td>{{pref.st_offset}}</td>
<td>{{pref.root}}</td>
<td>{{pref.cwd}}</td>
</tr>
{% endfor %}
</tbody>