remove duplicate lines of code

update amendments in tables.sql to include job_id in entry_ammendment
added amend.py to move amendment-related code into its own file when we create a job (NewJob)
  and that job matches an amendmentType (via job_name or job_name:amt <- where amt relates to how we do a transform_image), then
  we enter a new EntryAmendment pa_job_mgr knows when a Transform job ends, and removes relevant EntryAmendment
files*.js use EntryAmendment data to render thumbnails with relevant AmendmentType
if a normal page load (like /files_ip), and there is an EntryAmendment, mark up the thumb, run the check jobs to look for completion of the job,
  removeal of the EntryAmendment and update the entry based on 'transformed' image

OVERALL: this is a functioning version that uses EntryAmendments and can handle loading a new page with outstanding amendments
  and 'deals' with it.  This is a good base, but does not cater for remove_files or move_files

BUGs

@@ -1,41 +1,9 @@
-### Next: 141
+### Next: 146
 BUG-140: When db is restarted underneath PA, it crashes job mgr... It should just accept timeouts, and keep trying to reconnect every 2? mins
-BUG-139: using any large entry list and going next a few times, ends say 4 pages of 50 into 4000 matches (entries from DB < 50)... 
-    - confirmed this is when person has 2 or more refimgs:
-      - on page "2", we get 49 pulled back in the ORM instead of the 50 expected -- b/c I use that to indicate we must be at the end of the list if not 50 found
-    -- really, need to fix once and for all the eids / re-running query.
-        do GetEntries as we do now, once done however, get all entry ids.  Stick those into the DB with a unique query-id and datestamp
-        new func to get all details needed for entries in an eid list (of 1-page) - show this page of entries
-        use current, full eidlist and to work our start/end of list (next/prev), disabling.
-        then client can keep current page of data, if you hit next/prev, use DB unique query id / full list and page of eids, and give full data for new page of entries
-      Implications though, are if a search is invalidated (maybe delete / move a photo), need to remove them from the list on the DB too OR let user know/decide to fix/wait.
-
-
-BUG-100: I managed to get 2 photos matching mich in the NOT_WORKING photo (probably dif refimgs but same p.tag?)
-    = /photos/2012/20120414-damien/IMG_8467.JPG
+BUG-118: can move files from Bin path, but it leaves the del_file entry for it - need to remove it
+BUG-117: when search returns files that can be deleted and/or restored, the icon stays as delete and tries to delete!
 BUG-106: cant add trudy /pat? as refimgs via FaceDBox
     - seems the cropped trudy face is not sufficient to find a face, how odd...
       (it came from a face bbox, BUT, I have grown the face seln by 10%?)
-BUG-117: when search returns files that can be deleted and/or restored, the icon stays as delete and tries to delete!
-BUG-118: can move files from Bin path, but it leaves the del_file entry for it - need to remove it
-BUG-119: "Uncaught (in promise) Error: A listener indicated an asynchronous
-            response by returning true, but the message channel closed before a response
-            was received"
-            investigate this (possible I'm calling check_for_jobs and maybe not doing the async right?)
-
-BUG-123: pa_job_manager crashed with timeout on connection (probably when I turned off traefik for a bit?).  Regardless, should be more fault tolerant --> maybe offer to restart pa_job_manager IF its crashed?
-        this definitely happened also, when I shutdown the DB back-end mid job, and it was able to be restarted, so could get f/e to at least suggest a restart of the contianer, or auto-restart job_mgr?
-
-BUG-125: when an image is highlighted, then post the contextmenu on a different image - the highlight does not move to the new image
-         and the selected menu function processes the original or the new depending on the way the code works.
-         There is a chance we need to change the document on click to a mouse down (or whatever the context menu 
-         uses for default), rather than just fix the highlight
-
-BUG-130: moving files and then trying to go next page and it got confused...
-BUG-132: right arrow to go to next photo in viewer ALSO scrolls to the right, needs a return somewhere in the jscript
-BUG-133: when rebuilding pa[dev], the first run fails to have symlinks to the right paths for Import/Storage, etc.  a simple restart fixes - so potentially the intial copy or some other race condition?
-BUG-134: when moving set of photos on page, then move another set of photos on page, the first set reappears. Could really delete them from the dom?
-BUG-135: failed to rotate: 2006/20061215-ITS-xmas-KP/DSC00582.JPG - not sure why && not repeatable, so its not the image, timing/race condition maybe?
-BUG-137: after moving/refiling photos, the next shift-click is out of order (reload fixes it)
-BUG-138: Placeholder for all the ways we can get the front-end confused:
-        ---> JUST fix all these BUGs (relating to confused/lost state) by revisiting the overally complex way I remember state and my position in a list (probably FAR easier, to make an initial sql just save all eids, and then not try to recreate that list ever again and not care how I got into the list).  Can attach a "running server-side sequence number", and if old sequence, and the original eid list results in a failure, then just pop up that the saved list is no longer valid, and ask user to re-do their search/list..."
+BUG-100: I managed to get 2 photos matching mich in the NOT_WORKING photo (probably dif refimgs but same p.tag?)
+    = /photos/2012/20120414-damien/IMG_8467.JPG

TODO

@@ -1,29 +1,13 @@
-###
-# get override data into view
-# think view_transform might need to be included?
-# should start with an empty DB and test - definitely no dirs in storage_sp gives:
-#        dir_id=dir_arr[0]
-#        IndexError: list index out of range
-# empty directories are sometimes showing "No matches for: 'undefined'" <- should only comes up for search in URL???
-# transforms in files_* also fails (old js in *transform uses current)
-# also all the add ref img/add override, etc are non-functional - FIX the override* stuff first to get table/naming consistency as that is half the problem
-# delete button also uses current? (eid is empty anyway)
-###
-
-### major fix - go to everywhere I call GetEntries(), and redo the logic totally...
-    * client side: 
-        * for real chance to stop confusion, instead of removing deleted images from DOM, we should gray them out and put a big Del (red circle with line?) though it as overlay.
-        * Create another table is entry_ammendments - note the deletions, rotations, flips of specific eids - then reproduce that on the client side visually as needed 
-          - at least grayed-out, to indicate a pending action is not complete.
-          - When job that flips, rotates, deletes completes then lets update the query details (e.g. remove eids, or remove the ammendments)
-          - this actually is quite an improvement, if someone is deleting 2 as per above, I will see that as a pending change in my unrelated query, ditto flips, etc.
-
 ### GENERAL
-    * jobs for AI should show path name
-    * rm dups job should show progress bar
+* jobs for AI should show path name
+* rm dups job should show progress bar
     * in viewer, there is no move button (maybe add one?)
+    * think I killed pa_job_manager without passing an eid to a transform job, shouldn't crash
+        - SHOULD JUST get AI to help clean-up and write defensive code here...
     * consider doing duplicates before AI, and if there are say 100s+, then maybe pause the AI work
         - had 5000+ new photos, took 8 hours to finish, for me to just delete them anyway
+    * consider how to better version jscript - across all html files, consistently
+        - mtime, didnt work anyway, my phone still wont pick up the change, it was adding any ?v= changed this (once)
     * optimisation:
         - keep track of just new files since scan (even if we did this from the DB), 
         then we could just feed those eid's explicitly into a 'get_file_details_on_new_files'.
@@ -34,9 +18,7 @@
         (is there a library for this???)
 
     * sqlalchemy 2 migration:
-        * fix unmapped (in fact make all the code properly sqlachemy 2.0 compliant)
-          -- path.py has the __allow_unmapped__ = True
-        * remove all '.execute' from *.py
+        - get AI to help
 
     * allow actions for wrong person:
         -> someone else? OR override no match for this person ever for this image?
@@ -65,10 +47,6 @@
         - rename (does this work already somehow? see issue below)
         - dont allow me to stupidly move a folder to itself
 
-    * browser back/forward buttons dont work -- use POST -> redirect to GET
-      - need some sort of clean-up of pa_user_state -- I spose its triggered by browser session, so maybe just after a week is lazy/good enough 
-        - pa_user_state has last_used as a timestamp so can be used to delete old entries
-
     * back button will fail if we do these POSTs:
         job.py:@app.route("/jobs", methods=["GET", "POST"])
         job.py:@app.route("/job/<id>", methods=["GET","POST"])
@@ -76,8 +54,8 @@
         * if on jobs page and jobs increase, then 'rebuild' the content of the page to show new jobs, and potentially do that every 5 seconds...
             - THINK: could also 'refresh' /job/id via Ajax not a reload, to avoid the POST issue above needing to remember job prefs somewhere?
 
-        files.py:@app.route("/fix_dups", methods=["POST"])
-            ???
+    * files.py:@app.route("/fix_dups", methods=["POST"])
+        - ???
 
     * GUI overhaul?
         * on a phone, the files.html page header is a mess "Oldest.." line is too  large to fit on 1 line (make it a hamburger?)
@@ -112,7 +90,6 @@
     * revisit SymlinkName() and make it simpler (see comment in shared.py)
 
     *** Need to use thread-safe sessions per Thread, half-assed version did not work
-
         Admin
             -> do I want to have admin roles/users?
             -> purge deleted files (and associated DB data) needs a dbox or privs

amend.py

@@ -0,0 +1,65 @@
+from sqlalchemy import select
+from flask import request, jsonify
+from flask_login import login_required
+
+from shared import PA
+from main import db, app
+
+################################################################################
+# Amendments are used to define types of changes being made to an entry (e.g.
+# rotate, flip) should contain relatively transient content (e.g. we might be
+# processing a long-running job now, and then add a rotate, the rotate wont
+# finish for minutes, so these classes allow the UI to handle that gracefully
+################################################################################
+
+################################################################################
+# Class describing AmendmentType in the DB (via sqlalchemy)
+################################################################################
+class AmendmentType(PA,db.Model):
+    __tablename__ = "amendment_type"
+    id = db.Column(db.Integer, db.Sequence('file_type_id_seq'), primary_key=True )
+    job_name = db.Column(db.String, nullable=False )
+    which = db.Column(db.String, nullable=False )
+    what = db.Column(db.String, nullable=False )
+    colour = db.Column(db.String, nullable=False )
+
+################################################################################
+# Class describing which Entry has a pending Amendment in the DB (via sqlalchemy)
+################################################################################
+class EntryAmendment(PA,db.Model):
+    __tablename__ = "entry_amendment"
+    eid = db.Column(db.Integer, db.ForeignKey("entry.id"), primary_key=True )
+    job_id = db.Column(db.Integer, db.ForeignKey("job.id"), primary_key=True )
+    amend_type = db.Column(db.Integer, db.ForeignKey("amendment_type.id"))
+    type = db.relationship("AmendmentType", backref="entry_amendment")
+    job = db.relationship("Job", back_populates="amendments")
+
+
+################################################################################
+# check if this job is something we need to log an EntryAmendment for, based on
+# job name and potentially amt in extras, to find the type of amendment
+################################################################################
+def inAmendmentTypes(job):
+    if not hasattr(job, 'extra' ) or not job.extra:
+        return None
+    amt=None
+    for jex in job.extra:
+        if jex.name == "amt":
+            amt=jex.value
+
+    # FIXME: should just cache this once per build, only would change with code updates
+    for at in getAmendments():
+        # for transform_image, amt=flip*, 90/180/270 - so amt will be set, use it, otherwise just use job.name
+        if (amt and f"{job.name}:{amt}" == at.job_name) or (at.job_name == job.name):
+            return at.id
+    return None
+
+
+################################################################################
+# Class describing which Entry has a pending Amendment in the DB (via sqlalchemy)
+################################################################################
+def getAmendments():
+    # get Amend types (get EAT data once - used in inAmendmentTypes()
+    stmt=select(AmendmentType)
+    eat=db.session.execute(stmt).scalars().all()
+    return eat

face.py

@@ -31,6 +31,8 @@ class Face(PA,db.Model):
     refimg_lnk = db.relationship("FaceRefimgLink", uselist=False, viewonly=True )
     facefile_lnk = db.relationship("FaceFileLink", uselist=False, viewonly=True )
     refimg =db.relationship("Refimg", secondary="face_refimg_link", uselist=False)
+    fnmo = db.relationship("FaceNoMatchOverride",  back_populates="face")
+    ffmo = db.relationship("FaceForceMatchOverride",  back_populates="face")
 
 
 ################################################################################
@@ -104,6 +106,7 @@ class FaceNoMatchOverride(PA, db.Model):
     face_id = db.Column(db.Integer, db.ForeignKey("face.id"), primary_key=True )
     type_id = db.Column(db.Integer, db.ForeignKey("face_override_type.id"))
     type = db.relationship("FaceOverrideType")
+    face = db.relationship("Face", back_populates="fnmo")
 
 
 ################################################################################
@@ -123,3 +126,4 @@ class FaceForceMatchOverride(PA, db.Model):
     face_id = db.Column(db.Integer, db.ForeignKey("face.id"), primary_key=True )
     person_id = db.Column(db.Integer, db.ForeignKey("person.id"), primary_key=True )
     person = db.relationship("Person")
+    face = db.relationship("Face", back_populates="ffmo")

files.py

@@ -1,10 +1,11 @@
 from flask_wtf import FlaskForm
-from flask import request, render_template, redirect, send_from_directory, url_for, jsonify, make_response
+from flask import request, render_template, redirect, send_from_directory, url_for, jsonify
 from marshmallow import Schema, fields
 from main import db, app, ma 
-from sqlalchemy import Sequence, text, select, union
+from sqlalchemy import Sequence, text, select, union, or_
 from sqlalchemy.exc import SQLAlchemyError
 from sqlalchemy.orm import joinedload
+import numbers
 import os
 import glob
 import json
@@ -22,17 +23,19 @@ import pytz
 import html
 from flask_login import login_required, current_user
 from types import SimpleNamespace
+from amend import EntryAmendment, AmendmentType
 
 # Local Class imports
 ################################################################################
 from states import States, PA_UserState
 from query import Query
 from job import Job, JobExtra, Joblog, NewJob, SetFELog
-from path import PathType, Path, MovePathDetails
+from path import PathType, Path
 from person import Refimg, Person, PersonRefimgLink
 from settings import Settings, SettingsIPath, SettingsSPath, SettingsRBPath
-from shared import SymlinkName
+from shared import SymlinkName, ICON, PA
 from dups import Duplicates
+from amend import getAmendments
 from face import Face, FaceFileLink, FaceRefimgLink, FaceOverrideType, FaceNoMatchOverride, FaceForceMatchOverride
 
 # pylint: disable=no-member
@@ -41,41 +44,32 @@ from face import Face, FaceFileLink, FaceRefimgLink, FaceOverrideType, FaceNoMat
 # Class describing PathDirLink  and in the DB (via sqlalchemy)
 # connects the entry (dir) with a path
 ################################################################################
-class PathDirLink(db.Model):
+class PathDirLink(PA,db.Model):
     __tablename__ = "path_dir_link"
     path_id = db.Column(db.Integer, db.ForeignKey("path.id"), primary_key=True )
     dir_eid = db.Column(db.Integer, db.ForeignKey("dir.eid"), primary_key=True )
 
-    def __repr__(self):
-        return f"<path_id: {self.path_id}, dir_eid: {self.dir_eid}>"
-
 ################################################################################
 # Class describing EntryDirLInk  and in the DB (via sqlalchemy)
 # connects (many) entry contained in a directory (which is also an entry)
 ################################################################################
-class EntryDirLink(db.Model):
+class EntryDirLink(PA,db.Model):
     __tablename__ = "entry_dir_link"
     entry_id = db.Column(db.Integer, db.ForeignKey("entry.id"), primary_key=True )
     dir_eid = db.Column(db.Integer, db.ForeignKey("dir.eid"), primary_key=True )
 
-    def __repr__(self):
-        return f"<entry_id: {self.entry_id}, dir_eid: {self.dir_eid}>"
-
 ################################################################################
 # Class describing Dir and in the DB (via sqlalchemy)
 #   rel_path: rest of dir after path, e.g. if path = /..../storage, then
 #             rel_path could be 2021/20210101-new-years-day-pics
 #   in_path: only in this structure, not DB, quick ref to the path this dir is in
 ################################################################################
-class Dir(db.Model):
+class Dir(PA,db.Model):
     __tablename__ = "dir"
     eid = db.Column(db.Integer, db.ForeignKey("entry.id"), primary_key=True )
     rel_path = db.Column(db.String, unique=True )
     in_path = db.relationship("Path", secondary="path_dir_link", uselist=False)
 
-    def __repr__(self):
-        return f"<eid: {self.eid}, rel_path: {self.rel_path}, in_path: {self.in_path}>"
-
 ################################################################################
 # Class describing Entry and in the DB (via sqlalchemy)
 # an entry is the common bits between files and dirs
@@ -85,7 +79,7 @@ class Dir(db.Model):
 # in_dir - is the Dir that this entry is located in (convenience for class only)
 # FullPathOnFS(): method to get path on the FS for this Entry
 ################################################################################
-class Entry(db.Model):
+class Entry(PA,db.Model):
     __tablename__ = "entry"
     id = db.Column(db.Integer, db.Sequence('file_id_seq'), primary_key=True )
     name = db.Column(db.String, unique=False, nullable=False )
@@ -106,9 +100,6 @@ class Entry(db.Model):
             s=self.dir_details.in_path.path_prefix
         return s
 
-    def __repr__(self):
-        return f"<id: {self.id}, name: {self.name}, type={self.type}, dir_details={self.dir_details}, file_details={self.file_details}, in_dir={self.in_dir}"
-
 ################################################################################
 # Class describing File and in the DB (via sqlalchemy)
 # all files are entries, this is the extra bits only for a file, of note:
@@ -117,7 +108,7 @@ class Entry(db.Model):
 #          info can be from exif, or file system, or file name (rarely)
 #   faces: convenience field to show connected face(s) for this file
 ################################################################################
-class File(db.Model):
+class File(PA,db.Model):
     __tablename__ = "file"
     eid = db.Column(db.Integer, db.ForeignKey("entry.id"), primary_key=True )
     size_mb = db.Column(db.Integer, unique=False, nullable=False)
@@ -129,22 +120,15 @@ class File(db.Model):
     woy = db.Column(db.Integer)
     faces = db.relationship ("Face", secondary="face_file_link" )
 
-    def __repr__(self):
-        return f"<eid: {self.eid}, size_mb={self.size_mb}, hash={self.hash}, year={self.year}, month={self.month}, day={self.day}, woy={self.woy}, faces={self.faces}>"
-
 ################################################################################
 # Class describing FileType and in the DB (via sqlalchemy)
 # pre-defined list of file types (image, dir, etc.)
 ################################################################################
-class FileType(db.Model):
+class FileType(PA,db.Model):
     __tablename__ = "file_type"
     id = db.Column(db.Integer, db.Sequence('file_type_id_seq'), primary_key=True )
     name = db.Column(db.String, unique=True, nullable=False )
 
-    def __repr__(self):
-        return f"<id: {self.id}, name={self.name}>"
-
-
 ################################################################################
 # this is how we order all queries based on value of 'noo' - used with
 # access *order_map.get(OPT.noo)
@@ -170,59 +154,106 @@ class PathSchema(ma.SQLAlchemyAutoSchema):
     class Meta: model = Path
     load_instance = True
     type = ma.Nested(PathType) 
+    root_dir = fields.Method("get_root_dir")
+    icon_url = fields.Method("get_icon_url")
+    def get_icon_url(self, obj):
+        return url_for("internal", filename="icons.svg") + "#" + ICON[obj.type.name]
+    def get_root_dir(self, obj):
+        parts = obj.path_prefix.split('/')
+        return ''.join(parts[2:])
+
 
 class FileTypeSchema(ma.SQLAlchemyAutoSchema):
-    class Meta: model = FileType
-    load_instance = True
+    class Meta: 
+        model = FileType
+        load_instance = True
 
 class DirSchema(ma.SQLAlchemyAutoSchema):
-    class Meta: model = Dir
-    load_instance = True
+    class Meta: 
+        model = Dir
+        load_instance = True
     eid = ma.auto_field()  # Explicitly include eid
     in_path = ma.Nested(PathSchema)
 
 class FaceFileLinkSchema(ma.SQLAlchemyAutoSchema):
-    class Meta: model = FaceFileLink
+    class Meta: 
+        model = FaceFileLink
+        load_instance = True
     model_used = ma.auto_field()
-    load_instance = True
 
 class PersonSchema(ma.SQLAlchemyAutoSchema):
-    class Meta: model=Person
-    load_instance = True
+    class Meta:
+        model=Person
+        load_instance = True
 
 class RefimgSchema(ma.SQLAlchemyAutoSchema):
     class Meta:
         model = Refimg
         exclude = ('face',)
-    load_instance = True
+        load_instance = True
     person = ma.Nested(PersonSchema)
 
 class FaceRefimgLinkSchema(ma.SQLAlchemyAutoSchema):
-    class Meta: model = FaceRefimgLink
-    load_instance = True
+    class Meta: 
+        model = FaceRefimgLink
+        load_instance = True
+
+class FaceOverrideTypeSchema(ma.SQLAlchemyAutoSchema):
+    class Meta: 
+        model = FaceOverrideType
+        load_instance = True
+
+class FaceNoMatchOverrideSchema(ma.SQLAlchemyAutoSchema):
+    class Meta: 
+        model = FaceNoMatchOverride
+        load_instance = True
+    type = ma.Nested(FaceOverrideTypeSchema)
+
+class FaceForceMatchOverrideSchema(ma.SQLAlchemyAutoSchema):
+    class Meta: 
+        model = FaceForceMatchOverride
+        load_instance = True
+    person = ma.Nested(PersonSchema) 
 
 class FaceSchema(ma.SQLAlchemyAutoSchema):
     class Meta:
         model=Face
         exclude = ('face',)
-    load_instance = True
+        load_instance = True
     refimg = ma.Nested(RefimgSchema,allow_none=True)
     # faces have to come with a file connection
     facefile_lnk = ma.Nested(FaceFileLinkSchema)
     refimg_lnk = ma.Nested(FaceRefimgLinkSchema,allow_none=True)
+    fnmo = ma.Nested( FaceNoMatchOverrideSchema, allow_none=True, many=True )
+    ffmo = ma.Nested( FaceForceMatchOverrideSchema, allow_none=True, many=True )
 
 class FileSchema(ma.SQLAlchemyAutoSchema):
-    class Meta: model = File
-    load_instance = True
+    class Meta: 
+        model = File
+        load_instance = True
     faces = ma.Nested(FaceSchema,many=True,allow_none=True)
 
+class AmendmentTypeSchema(ma.SQLAlchemyAutoSchema):
+    class Meta: 
+        model = AmendmentType
+        load_instance = True
+
+class EntryAmendmentSchema(ma.SQLAlchemyAutoSchema):
+    class Meta: 
+        model = EntryAmendment
+        load_instance = True
+    eid = ma.auto_field()
+    job_id = ma.auto_field()
+    type = ma.Nested(AmendmentTypeSchema)
+
 ################################################################################
 # Schema for Entry so we can json for data to the client
 ################################################################################
 class EntrySchema(ma.SQLAlchemyAutoSchema):
     # gives id, name, type_id
-    class Meta: model = Entry
-    load_instance = True
+    class Meta:
+        model = Entry
+        load_instance = True
 
     type = ma.Nested(FileTypeSchema)
     file_details = ma.Nested(FileSchema,allow_none=True)
@@ -236,27 +267,31 @@ class EntrySchema(ma.SQLAlchemyAutoSchema):
     def get_full_path(self, obj):
         return obj.FullPathOnFS()
 
+class JobExtraSchema(ma.SQLAlchemyAutoSchema):
+    class Meta:
+        model = JobExtra
+        load_instance = True
+    name = ma.auto_field()
+    value = ma.auto_field()
+
+class JobSchema(ma.SQLAlchemyAutoSchema):
+    class Meta:
+        model = Job
+        load_instance = True
+    id = ma.auto_field()
+    name = ma.auto_field()
+    extra = ma.Nested(JobExtraSchema, many=True)
+    amendments = ma.Nested(EntryAmendmentSchema, many=True)
+
 # global - this will be use more than once below, so do it once for efficiency
 entries_schema = EntrySchema(many=True)
-
-################################################################################
-# util function to just update the current/first/last positions needed for
-# viewing / using pa_user_state DB table
-################################################################################
-def UpdatePref( pref, OPT ):
-    last_used=datetime.now(pytz.utc)
-    if OPT.current>0:
-        pref.current=OPT.current
-    if OPT.first_eid>0:
-        pref.first_eid=OPT.first_eid
-    if OPT.last_eid>0:
-        pref.last_eid=OPT.last_eid
-    if OPT.num_entries>0:
-        pref.num_entries=OPT.num_entries
-    pref.last_used=last_used
-    db.session.add(pref)
-    db.session.commit()
-    return
+FOT_Schema = FaceOverrideTypeSchema(many=True)
+path_Schema = PathSchema(many=True)
+person_Schema = PersonSchema(many=True)
+et_schema = AmendmentTypeSchema(many=True)
+ea_schema = EntryAmendmentSchema(many=True)
+job_schema = JobSchema(many=False)
+job_schemas = JobSchema(many=True)
 
 ################################################################################
 # /get_entries_by_ids -> route where we supply list of entry ids (for next/prev
@@ -275,6 +310,8 @@ def process_ids():
             joinedload(Entry.file_details).joinedload(File.faces).joinedload(Face.refimg).joinedload(Refimg.person),
             joinedload(Entry.file_details).joinedload(File.faces).joinedload(Face.refimg_lnk),
             joinedload(Entry.file_details).joinedload(File.faces).joinedload(Face.facefile_lnk),
+            joinedload(Entry.file_details).joinedload(File.faces).joinedload(Face.fnmo).joinedload(FaceNoMatchOverride.type),
+            joinedload(Entry.file_details).joinedload(File.faces).joinedload(Face.ffmo).joinedload(FaceForceMatchOverride.person),
         )
         .where(Entry.id.in_(ids))
     )
@@ -289,44 +326,87 @@ def process_ids():
     # Sort the entries according to the order of ids
     sorted_data = [entry_map[id_] for id_ in ids if id_ in entry_map]
 
-    return jsonify(entries_schema.dump(sorted_data))
+    # get any pending entry amendments
+    stmt = select(EntryAmendment).join(AmendmentType)
+    ea = db.session.execute(stmt).unique().scalars().all()
+    ea_data=ea_schema.dump(ea)
+
+    return jsonify(entries=entries_schema.dump(sorted_data), amendments=ea_data)
 
 
 ################################################################################
-# /get_dir_entries -> show thumbnail view of files from import_path(s)
+# /get_dir_entries:
+# -> if back is false - returns list of eids inside this dir
+# -> if back is true - returns list of eids inside the parent of this dir
 ################################################################################
-@app.route("/get_dir_entries", methods=["POST"])
+@app.route("/get_dir_eids", methods=["POST"])
 @login_required
 def get_dir_entries():
     data = request.get_json()  # Parse JSON body
     dir_id = data.get('dir_id', [])  # Extract list of ids
     back = data.get('back', False)  # Extract back boolean
+    noo = data.get('noo', "A to Z")  # Extract noo ordering
 
     # if we are going back, find the parent id and use that instead
     if back:
-        stmt=( select(EntryDirLink.dir_eid).filter(EntryDirLink.entry_id==dir_id) )
-        dir_id = db.session.execute(stmt).scalars().all() [0]
+        # get parent of this dir, to go back
+        stmt=select(EntryDirLink.dir_eid).filter(EntryDirLink.entry_id==dir_id)
+        dir_id = db.session.execute(stmt).scalars().one_or_none()
+        if not dir_id:
+            # return valid as false, we need to let user know this is not an empty dir, it does not exist
+            return jsonify( valid=False, entry_list=[] )
+
+    # Just double-check this is still in the DB, in case it got deleted since client made view
+    stmt=select(Entry.id).where(Entry.id==dir_id)
+    ent_id = db.session.execute(stmt).scalars().one_or_none()
+    if not ent_id:
+        # return valid as false, we need to let user know this is not an empty dir, it does not exist
+        return jsonify( valid=False, entry_list=[] )
 
     # get content of dir_id
-    stmt=( select(Entry.id).join(EntryDirLink).filter(EntryDirLink.dir_eid==dir_id) )
-    ids=db.session.execute(stmt).scalars().all()
-    entries_schema = EntrySchema(many=True)
-    entries = Entry.query.filter(Entry.id.in_(ids)).all()
-    return jsonify(entries_schema.dump(entries))
+    stmt=select(Entry.id).join(EntryDirLink).filter(EntryDirLink.dir_eid==dir_id)
+    stmt=stmt.order_by(*order_map.get(noo) )
+    return jsonify( valid=True, entry_list=db.session.execute(stmt).scalars().all() )
 
+# get Face override details
+def getFOT():
+    stmt = select(FaceOverrideType)
+    fot=db.session.execute(stmt).scalars().all()
+    return FOT_Schema.dump(fot)
+
+
+# get import/storage path details for move dbox
+def getMoveDetails():
+    stmt = select(Path).where( or_( Path.type.has(name="Import"), Path.type.has(name="Storage")))
+    mp=db.session.execute(stmt).scalars().all()
+    return path_Schema.dump(mp)
+
+# get people data for the menu for AI matching (of person.tag)
+def getPeople():
+    stmt = select(Person)
+    people=db.session.execute(stmt).scalars().all()
+    return person_Schema.dump(people)
+
+def initQueryData():
+    query_data={}
+    query_data['entry_list']=None 
+    query_data['root_eid']=0
+    query_data['NMO'] = getFOT()
+    query_data['move_paths'] = getMoveDetails()
+    query_data['people'] = getPeople()
+    query_data['amendTypes'] = et_schema.dump( getAmendments() )
+    return query_data
 
 ################################################################################
 # Get all relevant Entry.ids based on search_term passed in and OPT visuals
 ################################################################################
 def GetSearchQueryData(OPT):
-    query_data={}
-    query_data['entry_list']=None 
-    query_data['root_eid']=0
+    query_data=initQueryData()
 
     search_term = OPT.search_term
     # turn * wildcard into sql wildcard of %
     search_term = search_term.replace('*', '%')
-    if 'AI:' in search_term:
+    if 'AI:' in OPT.search_term:
         search_term = search_term.replace('AI:', '')
 
     # AI searches are for specific ppl/joins in the DB AND we do them for ALL types of searches, define this once
@@ -337,7 +417,7 @@ def GetSearchQueryData(OPT):
         .order_by(*order_map.get(OPT.noo) )
     )
 
-    if 'AI:' in search_term:
+    if 'AI:' in OPT.search_term:
         all_entries = db.session.execute(ai_query).scalars().all()
     else:
         # match name of File
@@ -345,20 +425,20 @@ def GetSearchQueryData(OPT):
         # match name of Dir
         dir_query = select(Entry.id).join(File).join(EntryDirLink).join(Dir).where(Dir.rel_path.ilike(f'%{search_term}%')).order_by(*order_map.get(OPT.noo))
 
+        ai_entries = db.session.execute(ai_query).scalars().all()
+        file_entries = db.session.execute(file_query).scalars().all()
+        dir_entries = db.session.execute(dir_query).scalars().all()
         # Combine ai, file & dir matches with union() to dedup and then order them 
-        combined_query = union( file_query, dir_query, ai_query )
-        all_entries = db.session.execute(combined_query).scalars().all()
+        all_entries =  list(dict.fromkeys(ai_entries + dir_entries + file_entries))
 
     query_data['entry_list']=all_entries
     return query_data
 
-
 #################################################################################
 # Get all relevant Entry.ids based on files_ip/files_sp/files_rbp and OPT visuals
 #################################################################################
 def GetQueryData( OPT ):
-    query_data={}
-    query_data['entry_list']=None 
+    query_data=initQueryData()
 
     # always get the top of the (OPT.prefix) Path's eid and keep it for OPT.folders toggling/use
     dir_stmt=( 
@@ -368,13 +448,16 @@ def GetQueryData( OPT ):
     )
     # this should return the 1 Dir (that we want to see the content of) - and with only 1, no need to worry about order
     dir_arr=db.session.execute(dir_stmt).scalars().all()
-    dir_id=dir_arr[0]
+    if dir_arr:
+        dir_id=dir_arr[0]
+    else:
+        dir_id=0
     # used to know the parent/root (in folder view), in flat view - just ignore/safe though
     query_data['root_eid']=dir_id
 
     if OPT.folders:
         # start folder view with only the root folder
-        stmt=( select(Entry.id).join(EntryDirLink).filter(EntryDirLink.dir_eid==dir_id) )
+        stmt=select(Entry.id).join(EntryDirLink).filter(EntryDirLink.dir_eid==dir_id)
     else:
         # get every File that is in the OPT.prefix Path
         stmt=( 
@@ -385,7 +468,6 @@ def GetQueryData( OPT ):
 
     stmt=stmt.order_by(*order_map.get(OPT.noo) )
     query_data['entry_list']=db.session.execute(stmt).scalars().all()
-
     return query_data
 
 ################################################################################
@@ -403,8 +485,11 @@ def change_file_opts():
     else:
         OPT.folders=False
     # so create a new entryList, and handle that on the client
-    query_data = GetQueryData( OPT )
-    return make_response( jsonify( query_data=query_data ) )
+    if 'search' in request.referrer:
+        query_data = GetSearchQueryData( OPT )
+    else:
+        query_data = GetQueryData( OPT )
+    return jsonify( query_data=query_data )
     
 
 ################################################################################
@@ -415,7 +500,8 @@ def change_file_opts():
 def file_list_ip():
     OPT=States( request )
     query_data = GetQueryData( OPT )
-    return render_template("file_list.html", page_title='View File Details (Import Path)', query_data=query_data, OPT=OPT )
+    js_vers = getVersions()
+    return render_template("file_list.html", page_title='View File Details (Import Path)', query_data=query_data, OPT=OPT, js_vers=js_vers )
 
 ################################################################################
 # /files -> show thumbnail view of files from import_path(s)
@@ -424,10 +510,9 @@ def file_list_ip():
 @login_required
 def files_ip():
     OPT=States( request )
-    people = Person.query.all()
-    move_paths = MovePathDetails()
     query_data = GetQueryData( OPT )
-    return render_template("files.html", page_title=f"View Files ({OPT.path_type} Path)", OPT=OPT, people=people, move_paths=move_paths, query_data=query_data )
+    js_vers = getVersions()
+    return render_template("files.html", page_title=f"View Files ({OPT.path_type} Path)", OPT=OPT, query_data=query_data, js_vers=js_vers )
 
 ################################################################################
 # /files -> show thumbnail view of files from storage_path
@@ -436,10 +521,9 @@ def files_ip():
 @login_required
 def files_sp():
     OPT=States( request )
-    people = Person.query.all()
-    move_paths = MovePathDetails()
     query_data = GetQueryData( OPT )
-    return render_template("files.html", page_title=f"View Files ({OPT.path_type} Path)", OPT=OPT, people=people, move_paths=move_paths, query_data=query_data )
+    js_vers = getVersions()
+    return render_template("files.html", page_title=f"View Files ({OPT.path_type} Path)", OPT=OPT, query_data=query_data, js_vers=js_vers )
 
 
 ################################################################################
@@ -449,10 +533,9 @@ def files_sp():
 @login_required
 def files_rbp():
     OPT=States( request )
-    people = Person.query.all()
-    move_paths = MovePathDetails()
     query_data = GetQueryData( OPT )
-    return render_template("files.html", page_title=f"View Files ({OPT.path_type} Path)", OPT=OPT, people=people, move_paths=move_paths, query_data=query_data )
+    js_vers = getVersions()
+    return render_template("files.html", page_title=f"View Files ({OPT.path_type} Path)", OPT=OPT, query_data=query_data, js_vers=js_vers )
 
 ################################################################################
 # search -> GET version -> has search_term in the URL and is therefore able to
@@ -464,11 +547,9 @@ def files_rbp():
 def search(search_term):
     OPT=States( request )
     OPT.search_term = search_term
-    OPT.folders = False
-
     query_data=GetSearchQueryData( OPT )
-    move_paths = MovePathDetails()
-    return render_template("files.html", page_title='View Files', search_term=search_term, query_data=query_data, OPT=OPT, move_paths=move_paths )
+    js_vers = getVersions()
+    return render_template("files.html", page_title='View Files', search_term=search_term, query_data=query_data, OPT=OPT, js_vers=js_vers )
 
 ################################################################################
 # /files/scan_ip -> allows us to force a check for new files
@@ -567,7 +648,7 @@ def restore_files():
         jex.append( JobExtra( name=f"{el}", value=str(request.form[el]) ) )
 
     job=NewJob( name="restore_files", num_files=0, wait_for=None, jex=jex, desc="to restore selected file(s)" )
-    return redirect("/jobs")
+    return jsonify( job=job_schema.dump(job) )
 
 ################################################################################
 # /delete_files -> create a job to delete files for the b/e to process
@@ -580,7 +661,7 @@ def delete_files():
         jex.append( JobExtra( name=f"{el}", value=str(request.form[el]) ) )
 
     job=NewJob( name="delete_files", num_files=0, wait_for=None, jex=jex, desc="to delete selected file(s)" )
-    return redirect("/jobs")
+    return jsonify( job=job_schema.dump(job) )
 
 ################################################################################
 # /move_files -> create a job to move files for the b/e to process
@@ -593,8 +674,7 @@ def move_files():
     for el in request.form:
         jex.append( JobExtra( name=f"{el}", value=str(request.form[el]) ) )
     job=NewJob( name="move_files", num_files=0, wait_for=None, jex=jex, desc="to move selected file(s)" )
-    # data is not used, but send response to trigger CheckForJobs()
-    return make_response( jsonify( job_id=job.id ) )
+    return jsonify( job=job_schema.dump(job) )
 
 @login_required
 @app.route("/view/", methods=["POST"])
@@ -621,32 +701,9 @@ def view():
     data=db.session.execute(stmt).unique().scalars().all()
     return jsonify(entries_schema.dump(data))
 
-
-####
-"""
-        # process any overrides
-        for face in e.file_details.faces:
-            # now get any relevant override and store it in objs...
-            fnmo = FaceNoMatchOverride.query.filter(FaceNoMatchOverride.face_id==face.id).first()
-            if fnmo:
-                face.no_match_override=fnmo
-            mo = FaceForceMatchOverride.query.filter(FaceForceMatchOverride.face_id==face.id).first()
-            if mo:
-                mo.type = FaceOverrideType.query.filter( FaceOverrideType.name== 'Manual match to existing person' ).first()
-                face.manual_override=mo
-
-        NMO_data = FaceOverrideType.query.all()
-        setting = Settings.query.first()
-        imp_path = setting.import_path
-        st_path  = setting.storage_path
-        bin_path = setting.recycle_bin_path
-#        print( f"BUG-DEBUG: /view/id GET route - OPT={OPT}, eids={eids}, current={int(id)} ")
-        return render_template("viewer.html", current=int(id), eids=eids, objs=objs, OPT=OPT, NMO_data=NMO_data, imp_path=imp_path, st_path=st_path, bin_path=bin_path )
-"""
-
 # route called from front/end - if multiple images are being transformed, each transorm == a separate call
 # to this route (and therefore a separate transorm job.  Each reponse allows the f/e to check the
-# specific transorm job is finished (/check_transform_job)  which will be called (say) every 1 sec. from f/e
+# specific transorm job is finished (/check_amend_job_status)  which will be called (say) every 1 sec. from f/e
 # with a spinning wheel, then when pa_job_mgr has finished it will return the transformed thumb
 @app.route("/transform", methods=["POST"])
 @login_required
@@ -659,25 +716,29 @@ def transform():
         jex.append( JobExtra( name=f"{el}", value=str(request.form[el]) ) )
 
     job=NewJob( name="transform_image", num_files=0, wait_for=None, jex=jex, desc="to transform selected file(s)" )
-    return make_response( jsonify( job_id=job.id ) )
+    return jsonify( job=job_schema.dump(job) )
 
 ################################################################################
-# /check_transform_job -> URL that is called repeatedly by front-end waiting for the
-#                    b/e to finish the transform job.  Once done, the new / now
-#                    transformed image's thumbnail is returned so the f/e can
-#                    update with it
+# /check_amend_job_status -> URL that is called repeatedly by front-end waiting
+#   for the b/e to finish the amendment job (delete/restore/move file).
+#   Once done, return "ok"
 ################################################################################
-@app.route("/check_transform_job", methods=["POST"])
+@app.route("/check_amend_job_status", methods=["POST"])
 @login_required
-def check_transform_job():
+def check_amend_job_status():
     job_id = request.form['job_id']
-    job = Job.query.get(job_id)
-    j=jsonify( finished=False )
-    if job.pa_job_state == 'Completed':
-        id=[jex.value for jex in job.extra if jex.name == "id"][0]
-        e=Entry.query.join(File).filter(Entry.id==id).first()
-        j=jsonify( finished=True, thumbnail=e.file_details.thumbnail )
-    return make_response( j )
+    stmt = select(Job).options(joinedload(Job.amendments)).where(Job.id == job_id)
+    job=db.session.execute(stmt).unique().scalars().first()
+    # FIXME: should validate job_id is real from UI
+    if job.name == 'transform_image':
+        eid=[jex.value for jex in job.extra if jex.name == "id"][0]
+        stmt=select(Entry).where(Entry.id==eid)
+        ent=db.session.execute(stmt).scalars().all()
+        ent_data=entries_schema.dump(ent)
+        j=jsonify(finished=(job.pa_job_state == 'Completed'), job=job_schema.dump(job), entry=ent_data[0] )
+    else:
+        j=jsonify(finished=(job.pa_job_state == 'Completed'), job=job_schema.dump(job))
+    return j
 
 ################################################################################
 # /include -> return contents on /include and does not need a login, so we
@@ -723,7 +784,7 @@ def get_existing_paths(dt):
         except:
             # this is not a date, so we cant work out possible dirs, just
             # return an empty set
-            return make_response( '[]' )
+            return jsonify( '[]' )
         new_dt=new_dtime.strftime('%Y%m%d')
         # find dirs named with this date
         dirs_arr+=Dir.query.filter(Dir.rel_path.ilike('%'+new_dt+'%')).all();
@@ -737,8 +798,8 @@ def get_existing_paths(dt):
     ret='[ '
     first_dir=1
     for dir in dirs:
-        # this can occur if there is a file with this date name in the top-levle of the path, its legit, but only really happens in DEV
-        # regardless, it cant be used for a existpath button in the F/E, ignore it
+        # this can occur if there is a file with this date name in the top-level of the path, its legit, but only really happens in DEV
+        # regardless, it cant be used for a existing path button in the F/E, ignore it
         if dir.rel_path == '':
             continue
         if not first_dir:
@@ -756,4 +817,18 @@ def get_existing_paths(dt):
         ret+= ' } '
         first_dir=0
     ret+= ' ]'
-    return make_response( ret )
+    return jsonify ( ret )
+
+# quick helper func to return timestamps of jscript files
+# we use this as a quick/hacky way of versioning them
+def getVersions():
+    js_vers={}
+    js_vers['fs'] = int(os.path.getmtime( "."+url_for( 'internal', filename='js/files_support.js') ))
+    js_vers['vs'] = int(os.path.getmtime( "."+url_for( 'internal', filename='js/view_support.js') ))
+    js_vers['ft'] = int(os.path.getmtime( "."+url_for( 'internal', filename='js/files_transform.js') ))
+    js_vers['ic'] = int(os.path.getmtime( "."+url_for( 'internal', filename='icons.svg') ))
+    js_vers['r180'] = int(os.path.getmtime( "."+url_for( 'internal', filename='rot180.png') ))
+    js_vers['r270'] = int(os.path.getmtime( "."+url_for( 'internal', filename='rot270.png') ))
+    js_vers['r90'] = int(os.path.getmtime( "."+url_for( 'internal', filename='rot90.png') ))
+    js_vers['th'] = int(os.path.getmtime( "."+url_for( 'internal', filename='throbber.gif') ))
+    return js_vers

internal/icons.svg

@@ -161,13 +161,13 @@
             c4.142,0,7.5-3.357,7.5-7.5S339.642,328,335.5,328z"/>
 <g style="fill:#00000025;" transform="matrix(16, 0, 0, 16, 120, 115)"><path d="M4.502 9a1.5 1.5 0 1 0 0-3 1.5 1.5 0 0 0 0 3z"/><path d="M14.002 13a2 2 0 0 1-2 2h-10a2 2 0 0 1-2-2V5A2 2 0 0 1 2 3a2 2 0 0 1 2-2h10a2 2 0 0 1 2 2v8a2 2 0 0 1-1.998 2zM14 2H4a1 1 0 0 0-1 1h9.002a2 2 0 0 1 2 2v7A1 1 0 0 0 15 11V3a1 1 0 0 0-1-1zM2.002 4a1 1 0 0 0-1 1v8l2.646-2.354a.5.5 0 0 1 .63-.062l2.66 1.773 3.71-3.71a.5.5 0 0 1 .577-.094l1.777 1.947V5a1 1 0 0 0-1-1h-10z"/></g>
     </svg>
-    <svg id="flip_h" fill="currentColor" viewBox='0 0 512 512'>
+    <svg id="flip_h" viewBox='0 0 512 512'>
         <path fill='none' stroke='currentColor' stroke-linecap='round' stroke-linejoin='round' stroke-width='32' d='M304 48l112 112-112 112M398.87 160H96M208 464L96 352l112-112M114 352h302'/>
     </svg>
-    <svg id="flip_v" fill="currentColor" viewBox='0 0 512 512'>
+    <svg id="flip_v" viewBox='0 0 512 512'>
         <path fill='none' stroke='currentColor' stroke-linecap='round' stroke-linejoin='round' stroke-width='32' d='M464 208L352 96 240 208M352 113.13V416M48 304l112 112 112-112M160 398V96'/>
     </svg>
-    <svg id="fullscreen" fill="currentColor" viewBox="0 0 16 16">
+    <svg id="fullscreen" viewBox="0 0 16 16">
       <path fill-rule="evenodd" d="M5.828 10.172a.5.5 0 0 0-.707 0l-4.096 4.096V11.5a.5.5 0 0 0-1 0v3.975a.5.5 0 0 0 .5.5H4.5a.5.5 0 0 0 0-1H1.732l4.096-4.096a.5.5 0 0 0 0-.707zm4.344 0a.5.5 0 0 1 .707 0l4.096 4.096V11.5a.5.5 0 1 1 1 0v3.975a.5.5 0 0 1-.5.5H11.5a.5.5 0 0 1 0-1h2.768l-4.096-4.096a.5.5 0 0 1 0-.707zm0-4.344a.5.5 0 0 0 .707 0l4.096-4.096V4.5a.5.5 0 1 0 1 0V.525a.5.5 0 0 0-.5-.5H11.5a.5.5 0 0 0 0 1h2.768l-4.096 4.096a.5.5 0 0 0 0 .707zm-4.344 0a.5.5 0 0 1-.707 0L1.025 1.732V4.5a.5.5 0 0 1-1 0V.525a.5.5 0 0 1 .5-.5H4.5a.5.5 0 0 1 0 1H1.732l4.096 4.096a.5.5 0 0 1 0 .707z"/>
     </svg>
     <svg id="unknown_ftype" fill="grey" viewBox="0 0 16 16">
@@ -208,4 +208,7 @@
       <path d="M4.406 1.342A5.53 5.53 0 0 1 8 0c2.69 0 4.923 2 5.166 4.579C14.758 4.804 16 6.137 16 7.773 16 9.569 14.502 11 12.687 11H10a.5.5 0 0 1 0-1h2.688C13.979 10 15 8.988 15 7.773c0-1.216-1.02-2.228-2.313-2.228h-.5v-.5C12.188 2.825 10.328 1 8 1a4.53 4.53 0 0 0-2.941 1.1c-.757.652-1.153 1.438-1.153 2.055v.448l-.445.049C2.064 4.805 1 5.952 1 7.318 1 8.785 2.23 10 3.781 10H6a.5.5 0 0 1 0 1H3.781C1.708 11 0 9.366 0 7.318c0-1.763 1.266-3.223 2.942-3.593.143-.863.698-1.723 1.464-2.383z"/>
       <path d="M7.646 15.854a.5.5 0 0 0 .708 0l3-3a.5.5 0 0 0-.708-.708L8.5 14.293V5.5a.5.5 0 0 0-1 0v8.793l-2.146-2.147a.5.5 0 0 0-.708.708l3 3z"/>
     </svg>
+    <svg id="back" viewBox="0 0 16 16">
+      <path d="m7.247 4.86-4.796 5.481c-.566.647-.106 1.659.753 1.659h9.592a1 1 0 0 0 .753-1.659l-4.796-5.48a1 1 0 0 0-1.506 0z"/>
+    </svg>
 </svg>

internal/js/files_support.js

@@ -4,6 +4,13 @@ ICON["Import"]="import"
 ICON["Storage"]="db"
 ICON["Bin"]="trash"
 
+// function called when we get another page from inside the files view
+function getPageFigures(res, viewingIdx)
+{
+    // add all the figures to files_div
+    drawPageOfFigures()
+}
+
 // grab all selected thumbnails and return a <div> containing the thumbnails
 // with extra yr and date attached as attributes so we can set the default
 // dir name for a move directory - not used in del, but no harm to include them
@@ -80,21 +87,9 @@ function GetExistingDirsAsDiv( dt, divname, ptype )
     } )
 }
 
-// wrapper to do some clean up before POST to /move_files or /delete_files
-// used to remove the highlighted item(s) && reset the numbering so highlighting continues to work
-function MoveOrDelCleanUpUI()
-{
-    // remove the images being moved (so UI immediately 'sees' the move)
-    $("[name^=eid-]").each( function() { $('#'+$(this).attr('value')).remove() } )
-    // reorder the images via ecnt again, so future highlighting can work
-    document.mf_id=0; $('.figure').each( function() { $(this).attr('ecnt', document.mf_id ); document.mf_id++ } )
-    $('#dbox').modal('hide')
-}
-
-
 // show the DBox for a move file, includes all thumbnails of selected files to move
 // and a pre-populated folder to move them into, with text field to add a suffix
-function MoveDBox(path_details, db_url)
+function MoveDBox()
 {
     $('#dbox-title').html('Move Selected File(s) to new directory in Storage Path')  
     div =`
@@ -104,21 +99,21 @@ function MoveDBox(path_details, db_url)
         <form id="mv_fm" class="form form-control-inline col-12">
         <input id="move_path_type" name="move_path_type" type="hidden"
         `
-    div += ' value="' + path_details[0].type + '"></input>'
+    div += ' value="' + move_paths[0].type.name + '"></input>'
     div+=GetSelnAsDiv()
     yr=$('.highlight').first().attr('yr')
     dt=$('.highlight').first().attr('date')
     div+='<div class="row">Use Existing Directory (in the chosen path):</div><div id="existing"></div>'
-    GetExistingDirsAsDiv( dt, "existing", path_details[0].type )
+    GetExistingDirsAsDiv( dt, "existing", 'Storage' )
     div+=`
         <div class="input-group my-3">
         <alert class="alert alert-primary my-auto py-1">
         `
     // NB: alert-primary here is a hack to get the bg the same color as the alert primary by
-    div+= '<svg id="move_path_icon" width="20" height="20" fill="currentColor"><use xlink:href="' + path_details[0].icon_url + '"></svg>'
+    div+= '<svg id="move_path_icon" width="20" height="20" fill="currentColor"><use xlink:href="' + move_paths[0].icon_url + '"></svg>'
     div+= '<select id="rp_sel" name="rel_path" class="text-primary alert-primary py-1 border border-primary rounded" onChange="change_rp_sel()">'
-    for(p of path_details) {
-        div+= '<option path_type="'+p.type+'" icon_url="'+p.icon_url+'">'+p.path+'</option>'
+    for(p of move_paths) {
+        div+= `<option path_type="${p.type.name}" icon_url="${p.icon_url}">${p.root_dir}</option>`
     }
     div+= '</select>'
     div+=`
@@ -132,11 +127,26 @@ function MoveDBox(path_details, db_url)
         </div>
         <div class="form-row col-12 mt-2">
             <button onClick="$('#dbox').modal('hide'); return false;" class="btn btn-outline-secondary offset-1 col-2">Cancel</button>
-            <button id="move_submit" onClick="MoveOrDelCleanUpUI(); $.ajax({ type: 'POST', data: $('#mv_fm').serialize(), url: '/move_files', success: function(data) { 
-                if( $(location).attr('pathname').match('search') !== null ) { window.location='/' }; CheckForJobs() } }); return false" class="btn btn-outline-primary col-2">Ok</button>
+            <button onClick="
+                $.ajax({ type: 'POST', data: $('#mv_fm').serialize(), url: '/move_files',
+                        success: function(data) {
+                            processAmendments( data.job.amendments )
+                            checkForAmendmentJobToComplete(data.job.id)
+                        }
+                });
+                $('#dbox').modal('hide')
+                return false"
+                class="btn btn-outline-secondary col-2">Ok</button>
         </div>
         </form>
     `
+    // force to Storage always - if in Import, liekly storing, if in Storage, likely moving, user can always override
+    div+=`
+        <script>
+            storage_rp = move_paths.find(item => item.type.name === "Storage")?.root_dir;
+            $('#rp_sel').val(storage_rp);change_rp_sel()
+        </script>
+        `
 
     $('#dbox-content').html(div)
     $('#dbox').modal('show')
@@ -144,36 +154,71 @@ function MoveDBox(path_details, db_url)
     $("#suffix").keypress(function (e) { if (e.which == 13) { $("#move_submit").click(); return false; } } )
 }
 
+// This function is called anytime we have a job that returns amendments
+// (visually we want to show this entry is being amended by a job)
+// as we check for a job to end every second, we can call this multiple times
+// during the runtime of a job, so only redraw/react to a new amendment
+// NOTE: we update all views, as we might go into one via jscript before the job ends
+function processAmendments( ams )
+{
+    for (const am of ams)
+    {
+        // if we return anything here, we already have this amendment, so continue to next
+        if( document.amendments.filter(obj => obj.eid === am.eid).length > 0 )
+            continue
+
+        document.amendments.push(am)
+
+        if( document.viewing && document.viewing.id == am.eid )
+        {
+            im.src=im.src + '?t=' + new Date().getTime();
+            DrawImg()
+        }
+
+        // find where in the page this image is being viewed
+        idx = pageList.indexOf(am.eid)
+        // createFigureHtml uses matching document.amendments to show thobber, etc
+        html = createFigureHtml( document.entries[idx] )
+        $('#'+am.eid).replaceWith( html )
+    }
+}
+
+// function to add data for document.amendment based on id and amt
+// used when we transform several images in files_*, or single image in viewer
 // show the DBox for a delete/restore file, includes all thumbnails of selected files 
-// with appropriate coloured button to Delete or Restore files`
+// with appropriate coloured button to Delete or Restore files
 function DelDBox(del_or_undel)
 {
     to_del = GetSelnAsData()
     $('#dbox-title').html(del_or_undel+' Selected File(s)')
     div ='<div class="row col-12"><p class="col">' + del_or_undel + ' the following files?</p></div>'
     div+=GetSelnAsDiv()
+    if( del_or_undel == "Delete" )
+    {
+        which="delete"
+        col="danger"
+    }
+    else
+    {
+        which="restore"
+        col="success"
+    }
+
+    document.ents_to_del=[]
+    $('.highlight').each(function( cnt ) { document.ents_to_del[cnt]=parseInt($(this).attr('id')) } )
     div+=`<div class="row col-12 mt-3">
             <button onClick="$('#dbox').modal('hide')" class="btn btn-outline-secondary col-2">Cancel</button>
-    `
-    div+=`
-    <button onClick="MoveOrDelCleanUpUI(); $.ajax({ type: 'POST', data: to_del, url:
-    `
-    if( del_or_undel == "Delete" )
-        div+=`
-        '/delete_files',
-        success: function(data){ 
-            if( $(location).attr('pathname').match('search') !== null ) { window.location='/' }; CheckForJobs() } }); return false" class="btn btn-outline-danger col-2">Ok</button>
-        </div>
-        `
-    else
-        // just force page reload to / for now if restoring files from a search path -- a search (by name) 
-        // would match the deleted/restored file, so it would be complex to clean up the UI (and can't reload, as DB won't be changed yet)
-        div+=`
-        '/restore_files',
-        success: function(data){ 
-            if( $(location).attr('pathname').match('search') !== null ) { window.location='/' }; CheckForJobs() } }); return false" class="btn btn-outline-success col-2">Ok</button>
-        </div>
-        `
+            <button onClick="
+                $.ajax({ type: 'POST', data: to_del, url: '/${which}_files',
+                        success: function(data) {
+                            processAmendments( data.job.amendments )
+                            checkForAmendmentJobToComplete(data.job.id)
+                        }
+                });
+                $('#dbox').modal('hide')
+                return false"
+                class="btn btn-outline-${col} col-2">Ok</button>
+        </div>`
     $('#dbox-content').html(div)
     $('#dbox').modal('show')
 }
@@ -206,58 +251,82 @@ function DetailsDBox()
 }
 
 
-// function to change the size of thumbnails (and resets button bar to newly
-// selected size)
-function ChangeSize(clicked_button,sz)
-{
-	$('.sz-but.btn-info').removeClass('btn-info text-white').addClass('btn-outline-info')
-	$(clicked_button).addClass('btn-info text-white').removeClass('btn-outline-info')
-	$('.thumb').attr( {height: sz, style: 'font-size:'+sz+'px' } )
-	$('#size').val(sz)
-    sz=sz-22
-    $('.svg').height(sz);
-    $('.svg').width(sz);
-    $('.svg_cap').width(sz);
-}
-
 // DoSel is called when a click event occurs, and sets the selection via adding 
 // 'highlight' to the class of the appropriate thumbnails
 // e == event (can see if shift/ctrl held down while left-clicking
 // el == element the click is on
 // this allows single-click to select, ctrl-click to (de)select 1 item, and
-// shift-click to add all elements between highlighted area and clicked area,
-// whether you click after highlight or before
-function DoSel(e, el)
-{
-    if( e.ctrlKey || document.fake_ctrl === 1 )
-    {
-        $(el).toggleClass('highlight')
-        if( document.fake_ctrl === 1 )
-            document.fake_ctrl=0
-        return
-    }
-    if( e.shiftKey || document.fake_shift === 1 )
-    {
-        st=Number($('.highlight').first().attr('ecnt'))
-        end=Number($('.highlight').last().attr('ecnt'))
-        clicked=Number($(el).attr('ecnt'))
-        // if we shift-click first element, then st/end are NaN, so just highlightthe one clicked
-        if( isNaN(st) )
-        {
-            $('.entry').slice( clicked, clicked+1 ).addClass('highlight')
-            return
-        }
-        if( clicked > end )
-            $('.entry').slice( end, clicked+1 ).addClass('highlight')
-        else
-            $('.entry').slice( clicked, st ).addClass('highlight')
+// shift-click to add all elements between highlighted area and clicked el,
+// whether you click before highlight or after, or inside a gap and then back
+// or forward to the closest higlighted entry - also, only works on entry class,
+// so it ignores figures that we take entry off while we transform, etc it
+function DoSel(e, el) {
+    const id = $(el).attr('id');
+    const entries = $('.entry');
 
-        if( document.fake_shift === 1 )
-            document.fake_shift=0
-        return
+    // Collect currently highlighted entries
+    const currentHighlights = $('.highlight');
+    const highlighted = new Set();
+    currentHighlights.each(function() {
+        highlighted.add($(this).attr('id'));
+    });
+
+    // Ctrl+click: toggle highlight for the clicked entry
+    if (e.ctrlKey || document.fake_ctrl === 1) {
+        $(el).toggleClass('highlight');
+        if (highlighted.has(id)) {
+            highlighted.delete(id);
+        } else {
+            highlighted.add(id);
+        }
+        if (document.fake_ctrl === 1) {
+            document.fake_ctrl = 0;
+        }
+        return;
+    }
+    // Shift+click: select a range
+    else if (e.shiftKey || document.fake_shift === 1) {
+        if (currentHighlights.length === 0) {
+            // If no highlights, just highlight the clicked entry
+            $(el).addClass('highlight');
+            highlighted.add(id);
+        } else {
+            // Find the nearest highlighted entry
+            const clickedIndex = entries.index($(el));
+            let nearestHighlightIndex = -1;
+            let minDistance = Infinity;
+
+            currentHighlights.each(function() {
+                const highlightIndex = entries.index($(this));
+                const distance = Math.abs(highlightIndex - clickedIndex);
+                if (distance < minDistance) {
+                    minDistance = distance;
+                    nearestHighlightIndex = highlightIndex;
+                }
+            });
+
+            // Highlight the range between the nearest highlighted entry and the clicked entry
+            const from = Math.min(clickedIndex, nearestHighlightIndex);
+            const to = Math.max(clickedIndex, nearestHighlightIndex);
+
+            for (let i = from; i <= to; i++) {
+                const entryId = entries.eq(i).attr('id');
+                highlighted.add(entryId);
+                entries.eq(i).addClass('highlight');
+            }
+        }
+        if (document.fake_shift === 1) {
+            document.fake_shift = 0;
+        }
+        return;
+    }
+    // Single click: clear all highlights and highlight the clicked entry
+    else {
+        $('.highlight').removeClass('highlight');
+        highlighted.clear();
+        $(el).addClass('highlight');
+        highlighted.add(id);
     }
-    $('.highlight').removeClass('highlight')
-    $(el).addClass('highlight')
 }
 
 // if a selection exists, enable move & del/restore buttons otherwise disable them
@@ -323,129 +392,120 @@ function NoSel() {
         return true
 }
 
+// quick wrapper to add a single <figure> to the #figures div
+function addFigure( obj )
+{
+    html=createFigureHtml( obj )
+    $('#figures').append( html )
+}
+
 /**
  * Renders a group header or entry based on the object and options.
- * @param {Object} obj - The object containing file/directory details.
- * @param {Object} last - Tracks the last printed group (e.g., { printed: null }).
- * @param {Object} ecnt - Entry counter (e.g., { val: 0 }).
- * @returns {string} - Generated HTML string.
+ * obj - The object containing file/directory details.
+ * returns {string} - Generated HTML string.
  */
-function addFigure( obj, last, ecnt) 
+function createFigureHtml( obj )
 {
-  let html = "";
+    // if am is null, no amendment for this obj, otherwise we have one
+    var am=null
+    for (const tmp of document.amendments)
+        if( tmp.eid == obj.id )
+            am=tmp
 
-  // Grouping logic
-  if (OPT.grouping === "Day") {
-    if (last.printed !== obj.file_details.day) {
-      html += `<div class="row ps-3"><h6>Day: ${obj.file_details.day} of ${obj.file_details.month}/${obj.file_details.year}</h6></div>`;
-      last.printed = obj.file_details.day;
-    }
-  } else if (OPT.grouping === "Week") {
-    if (last.printed !== obj.file_details.woy) {
-      html += `<div class="row ps-3"><h6>Week #: ${obj.file_details.woy} of ${obj.file_details.year}</h6></div>`;
-      last.printed = obj.file_details.woy;
-    }
-  } else if (OPT.grouping === "Month") {
-    if (last.printed !== obj.file_details.month) {
-      html += `<div class="row ps-3"><h6>Month: ${obj.file_details.month} of ${obj.file_details.year}</h6></div>`;
-      last.printed = obj.file_details.month;
-    }
-  }
+    let html = "";
 
-  // Image/Video/Unknown entry
-  if (obj.type.name === "Image" || obj.type.name === "Video" || obj.type.name === "Unknown") {
-    if (!OPT.folders || isTopLevelFolder(obj.in_dir.in_path.path_prefix + '/' + obj.in_dir.rel_path + '/' + obj.name, OPT.cwd)) {
-      const pathType = obj.in_dir.in_path.type.name;
-      const size = obj.file_details.size_mb;
-      const hash = obj.file_details.hash;
-      const inDir = `${obj.in_dir.in_path.path_prefix}/${obj.in_dir.rel_path}`;
-      const fname = obj.name;
-      const yr = obj.file_details.year;
-      const date = `${yr}${String(obj.file_details.month).padStart(2, '0')}${String(obj.file_details.day).padStart(2, '0')}`;
-      const prettyDate = `${obj.file_details.day}/${obj.file_details.month}/${obj.file_details.year}`;
-      const type = obj.type.name;
+    // Image/Video/Unknown entry
+    if (obj.type.name === "Image" || obj.type.name === "Video" || obj.type.name === "Unknown") {
+        const pathType = obj.in_dir.in_path.type.name;
+        const size = obj.file_details.size_mb;
+        const hash = obj.file_details.hash;
+        const inDir = `${obj.in_dir.in_path.path_prefix}/${obj.in_dir.rel_path}`;
+        const fname = obj.name;
+        const yr = obj.file_details.year;
+        const date = `${yr}${String(obj.file_details.month).padStart(2, '0')}${String(obj.file_details.day).padStart(2, '0')}`;
+        const prettyDate = `${obj.file_details.day}/${obj.file_details.month}/${obj.file_details.year}`;
+        const type = obj.type.name;
 
-      html += `
-        <figure id="${obj.id}" ecnt="${ecnt}" class="col col-auto g-0 figure entry m-1"
+        // if amendment for this obj, do not add entry class - prevents highlighting
+        if( am ) {
+            ent=""
+            gs="style='filter: grayscale(100%);'"
+            am_html ='<img class="position-absolute top-50 start-50 translate-middle" height="60" src="/internal/white-circle.png">'
+            am_html+='<img class="position-absolute top-50 start-50 translate-middle" height="64" src="/internal/throbber.gif">'
+            if( am.type.which == 'icon' )
+                am_html+=`<svg class="position-absolute top-50 start-50 translate-middle" height="32" style="color:${am.type.colour}" fill="${am.type.colour}"><use xlink:href="/internal/icons.svg#${am.type.what}"></use></svg>`
+            else
+                am_html+=`<img class="position-absolute top-50 start-50 translate-middle" src="/internal/${am.type.what}?v={{js_vers['r270']}}" height="32">`
+        } else {
+            ent="entry"
+            gs=""
+            am_html=""
+        }
+        html += `
+            <figure id="${obj.id}" class="col col-auto g-0 figure ${ent} m-1"
                 path_type="${pathType}" size="${size}" hash="${hash}" in_dir="${inDir}"
                 fname="${fname}" yr="${yr}" date="${date}" pretty_date="${prettyDate}" type="${type}">
-          ${renderMedia(obj)}
-        </figure>
-      `;
+                ${renderMedia(obj,gs,am_html)}
+            `
     }
-  }
-  // Directory entry
-  else if (obj.type.name === "Directory" && OPT.folders) {
-    const dirname = obj.dir_details.rel_path.length
-      ? `${obj.dir_details.in_path.path_prefix}/${obj.dir_details.rel_path}`
-      : obj.dir_details.in_path.path_prefix;
+    // Directory entry
+    else if (obj.type.name === "Directory" && OPT.folders) {
+        const dirname = obj.dir_details.rel_path.length
+          ? `${obj.dir_details.in_path.path_prefix}/${obj.dir_details.rel_path}`
+          : obj.dir_details.in_path.path_prefix;
 
-    if (isTopLevelFolder(dirname, OPT.cwd)) {
-      html += `
-        <figure class="col col-auto g-0 dir entry m-1" id="${obj.id}" ecnt="${ecnt}" dir="${dirname}" type="Directory">
-          <svg class="svg" width="${OPT.size - 22}" height="${OPT.size - 22}" fill="currentColor">
+        html += `
+            <figure class="col col-auto g-0 dir entry m-1" id="${obj.id}" dir="${dirname}" type="Directory">
+            <svg class="svg" width="${OPT.size - 22}" height="${OPT.size - 22}" fill="currentColor">
             <use xlink:href="/internal/icons.svg#Directory"></use>
-          </svg>
-          <figcaption class="svg_cap figure-caption text-center text-wrap text-break">${obj.name}</figcaption>
-        </figure>
-      `;
-      html += `<script>f=$('#${obj.id}'); w=f.find('svg').width(); f.find('figcaption').width(w);</script>`;
+            </svg>
+            <figcaption class="svg_cap figure-caption text-center text-wrap text-break">${obj.name}</figcaption>
+        `;
+        html += `<script>f=$('#${obj.id}'); w=f.find('svg').width(); f.find('figcaption').width(w);</script>`;
     }
-  }
-
-  $('#figures').append( html )
-  return
+    // moved the bindings to here as we need to reset them if we recreate this Figure (after a transform job)
+    html += `<script>
+        if( "${obj.type.name}" === "Directory" ) {
+            $("#${obj.id}").click( function(e) { document.back_id=this.id; getDirEntries(this.id,false) } )
+        } else {
+            $('#${obj.id}').click( function(e) { DoSel(e, this ); SetButtonState(); return false; });
+            $('#${obj.id}').dblclick( function(e) { startViewing( $(this).attr('id') ) } )
+        }
+    </script>
+    </figure>`
+    return html
 }
 
 // Helper function to render media (image/video/unknown)
-function renderMedia(obj) {
+function renderMedia(obj,gs,am_html) {
   const isImageOrUnknown = obj.type.name === "Image" || obj.type.name === "Unknown";
   const isVideo = obj.type.name === "Video";
   const path = `${obj.in_dir.in_path.path_prefix}/${obj.in_dir.rel_path}/${obj.name}`;
   const thumb = obj.file_details.thumbnail
-    ? `<a href="${path}"><img alt="${obj.name}" class="thumb" height="${OPT.size}" src="data:image/jpeg;base64,${obj.file_details.thumbnail}"></a>`
+    ? `<a href="${path}"><img alt="${obj.name}" ${gs} class="thumb" height="${OPT.size}" src="data:image/jpeg;base64,${obj.file_details.thumbnail}"></a>`
     : `<a href="${path}"><svg width="${OPT.size}" height="${OPT.size}" fill="white"><use xlink:href="/internal/icons.svg#unknown_ftype"/></svg></a>`;
 
-  let mediaHtml = `<div style="position:relative; width:100%">${thumb}`;
+  let mediaHtml = `<div style="position:relative; width:100%">${thumb}${am_html}`;
 
-  if (isImageOrUnknown) {
-    if (OPT.search_term) {
-      mediaHtml += `
-        <div style="position:absolute; bottom: 0px; left: 2px;">
-          <svg width="16" height="16" fill="white"><use xlink:href="/internal/icons.svg#${getLocationIcon(obj)}"/></svg>
-        </div>
-      `;
-    }
-    mediaHtml += `
-      <div id="s${obj.id}" style="display:none; position:absolute; top: 50%; left:50%; transform:translate(-50%, -50%);">
-        <img height="64px" src="/internal/throbber.gif">
-      </div>
-    `;
-  } else if (isVideo) {
+  if (isVideo) {
     mediaHtml += `
       <div style="position:absolute; top: 0px; left: 2px;">
         <svg width="16" height="16" fill="white"><use xlink:href="/internal/icons.svg#film"/></svg>
       </div>
     `;
-    if (OPT.search_term) {
+  }
+  if (OPT.search_term) {
       mediaHtml += `
         <div style="position:absolute; bottom: 0px; left: 2px;">
           <svg width="16" height="16" fill="white"><use xlink:href="/internal/icons.svg#${getLocationIcon(obj)}"/></svg>
         </div>
       `;
-    }
   }
 
   mediaHtml += `</div>`;
   return mediaHtml;
 }
 
-// Helper: Check if path is a top-level folder of cwd
-function isTopLevelFolder(path, cwd) {
-  // Implement your logic here
-  return true; // Placeholder
-}
-
 // Helper: Get location icon (placeholder)
 function getLocationIcon(obj) {
     return ICON[obj.in_dir.in_path.type.name]
@@ -457,21 +517,24 @@ function getDirEntries(dir_id, back)
     data={}
     data.dir_id=dir_id
     data.back=back
+    data.noo=OPT.noo
 
     $.ajax({
         type: 'POST',
-        url: '/get_dir_entries',
+        url: '/get_dir_eids',
         data: JSON.stringify(data),
         contentType: 'application/json',
         dataType: 'json',
         success: function(res) {
-            document.entries=res
-            // rebuild entryList/pageList as each dir comes with new entries
-            entryList=res.map(obj => obj.id);
+            if( res.valid === false )
+            {
+                $('#figures').html( "<alert class='alert alert-danger'>ERROR! directory has changed since you loaded this view.  You have to reload and reset your view (probably someone deleted the directory or its parent since you loaded this page)" )
+                return
+            }
+            entryList=res.entry_list
             pageList=entryList.slice(0, OPT.how_many)
-            if( back )
-                document.back_id = res[0].in_dir.eid
-            drawPageOfFigures()
+            // now go get actual data/entries
+            getPage(1,getPageFigures)
         },
         error: function(xhr, status, error) {
             console.error("Error:", error);
@@ -485,53 +548,86 @@ function drawPageOfFigures()
 {
     $('#figures').empty()
     var last = { printed: null }
-    var ecnt=0
+
+    // something is up, let the user know
+    if( document.alert )
+        $('#figures').append( document.alert )
 
     if( OPT.folders )
     {
-        if( document.entries.length && document.entries[0].in_dir.rel_path == '' )
+        // it root_eid is 0, then no entries in this path - cant go up
+        if( OPT.root_eid == 0  || (document.entries.length && document.entries[0].in_dir.eid == OPT.root_eid ) ) 
         {
             gray="_gray"
             back=""
             cl=""
+            back_id=0
         }
         else
         {
             gray=""
             back="Back"
             cl="back"
+            if( document.entries.length > 0 )
+                back_id = document.entries[0].in_dir.eid
+            else
+                back_id = document.back_id
         }
         // back button, if gray/back decide if we see grayed out folder and/or the name of the folder we go back to
+        // with clas "back" this gets a different click handler which flags server to return data by 'going back/up' in dir tree
+        // we give the server the id of the first item on the page so it can work out how to go back
         html=`<div class="col col-auto g-0 m-1">
-                <figure id="${document.back_id}" ecnt="0" class="${cl} entry m-1" type="Directory">
+                <figure id="${back_id}" class="${cl} entry m-1" type="Directory">
                     <svg class="svg" width="${OPT.size-22}" height="${OPT.size-22}">
                         <use xlink:href="internal/icons.svg#folder_back${gray}"/>
                     </svg>
                     <figcaption class="figure-caption text-center">${back}</figcaption>
                 </figure>
              </div>`
-        ecnt++
-        /*
-            <script>f=$('#_back'); w=f.find('svg').width(); f.find('figcaption').width(w);</script>
-        */
         $('#figures').append(html)
     }
     for (const obj of document.entries) {
-        addFigure( obj, last, ecnt )
-        ecnt++
+        // Grouping logic
+        if (OPT.grouping === "Day") {
+            if (last.printed !== obj.file_details.day) {
+                $('#figures').append(`<div class="row ps-3"><h6>Day: ${obj.file_details.day} of ${obj.file_details.month}/${obj.file_details.year}</h6></div>` );
+                last.printed = obj.file_details.day;
+            }
+        } else if (OPT.grouping === "Week") {
+            if (last.printed !== obj.file_details.woy) {
+                $('#figures').append(`<div class="row ps-3"><h6>Week #: ${obj.file_details.woy} of ${obj.file_details.year}</h6></div>` );
+                last.printed = obj.file_details.woy;
+            }
+        } else if (OPT.grouping === "Month") {
+            if (last.printed !== obj.file_details.month) {
+                $('#figures').append(`<div class="row ps-3"><h6>Month: ${obj.file_details.month} of ${obj.file_details.year}</h6></div>` );
+                last.printed = obj.file_details.month;
+            }
+        }
+        addFigure( obj )
     }
-    if( document.entries.length == 0 && OPT.search_term != '' )
-        $('#figures').append( `<span class="alert alert-danger p-2 col-auto"> No matches for: '${OPT.search_term}'</span>` )
-    $('.figure').click( function(e) { DoSel(e, this ); SetButtonState(); return false; });
-    $('.figure').dblclick( function(e) { dblClickToViewEntry( $(this).attr('id') ) } )
-    // for dir, getDirEntries 2nd param is back (or "up" a dir)
-    $(".dir").click( function(e) { document.back_id=this.id; getDirEntries(this.id,false) } )
     $(".back").click( function(e) { getDirEntries(this.id,true) } )
+    if( document.entries.length == 0 )
+        if( OPT.search_term )
+            $('#figures').append( `<span class="alert alert-danger p-2 col-auto"> No matches for: '${OPT.search_term}'</span>` )
+        else if( OPT.root_eid == 0 )
+            $('#figures').append( `<span class="alert alert-danger p-2 col-auto d-flex align-items-center">No files in Path!</span>` )
 }
 
+// emtpy out file_list_div, and repopulate it with new page of content
 function getPageFileList(res, viewingIdx)
 {
     $('#file_list_div').empty()
+
+    // something is up, let the user know
+    if( document.alert )
+        $('#file_list_div').append( '<div class="row">' + document.alert + '</div>' )
+
+    if( OPT.root_eid == 0 )
+    {
+        $('#file_list_div').append( `<span class="alert alert-danger p-2">No files in Path!</span>` )
+        return
+    }
     html='<table class="table table-striped table-sm col-12">'
     html+='<thead><tr class="table-primary"><th>Name</th><th>Size (MB)</th><th>Path Prefix</th><th>Hash</th></tr></thead><tbody>'
     for (const obj of res) {
@@ -553,19 +649,28 @@ function getPageFileList(res, viewingIdx)
     $('#file_list_div').append(html)
 }
 
-// function called when we get another page from inside the files view
-function getPageFigures(res, viewingIdx)
+// wrapper function as we want to handle real DB query success, but also do the 
+// same when we just use cache
+function getEntriesByIdSuccessHandler(res,pageNumber,successCallback,viewingIdx)
 {
-    // add all the figures to files_div
-    drawPageOfFigures()
-}
+    if( res.length != pageList.length )
+        document.alert="<alert class='alert alert-warning'>WARNING: something has changed since viewing this page (likely someone deleted content in another view), strongly suggest a page reload to get the latest data</alert>"
     
-// function called when we get another page from inside the viewer
-function getPageViewer(res, viewingIdx)
-{
-    document.viewing=document.entries[viewingIdx]
-    // update viewing, arrows and image/video too
-    ViewImageOrVideo()
+    document.entries=res;
+    // cache this
+    document.page[pageNumber]=res
+    // FIXME: I want to remove successCallback, instead: if viewing, or files_*, or file_list, then call relevant draw routine
+    successCallback(res,viewingIdx)
+    resetNextPrevButtons()
+    // if search, disable folders
+    if( OPT.search_term )
+         $('#folders').prop('disabled', 'disabled').removeClass('border-info').addClass('border-secondary').removeClass('text-info').addClass('text-secondary');
+    else if( document.entries.length == 0 )
+    {
+        html=`<span class="alert alert-danger p-2 col-auto">No files in Path</span>`
+        $('#file_list_div').append(html)
+        $('#figures').append(html)
+    }
 }
 
 // Function to get the 'page' of entry ids out of entryList
@@ -583,13 +688,28 @@ function getPage(pageNumber, successCallback, viewingIdx=0)
     data={}
     data.ids = pageList
 
+    // assume nothing wrong, but if the data goes odd, then this will be non-null and displayed later (cant add here, as later code does .empty() of file divs)
+    document.alert=null
+    // see if we can use cache, and dont reload from DB
+    if( !OPT.folders && document.page.length && document.page[pageNumber] )
+    {
+        getEntriesByIdSuccessHandler( document.page[pageNumber], pageNumber, successCallback, viewingIdx )
+        return
+    }
+
     $.ajax({
         type: 'POST', url: '/get_entries_by_ids',
         data: JSON.stringify(data), contentType: 'application/json',
         dataType: 'json',
-        success: function(res) { document.entries=res; successCallback(res,viewingIdx); },
+        success: function(res) {
+            document.amendments=res.amendments;
+            // only called when an amendment is pending & we are viewing a page in files/list view
+            // so check for amendment job(s) ending...
+            for (const tmp of document.amendments)
+                checkForAmendmentJobToComplete(tmp.job_id)
+            getEntriesByIdSuccessHandler( res.entries, pageNumber, successCallback, viewingIdx ) 
+        },
         error: function(xhr, status, error) { console.error("Error:", error); } });
-    resetNextPrevButtons()
     return
 }
 
@@ -610,9 +730,7 @@ function isLastPage(pageNumber)
 function getPageNumberForId(id) {
     const idx = entryList.indexOf(id);
     // should be impossible but jic
-    if (idx === -1) {
-        return -1; // or null, if you prefer
-    }
+    if (idx === -1) { return -1 }
     return Math.floor(idx / OPT.how_many) + 1;
 }
 
@@ -620,6 +738,13 @@ function getPageNumberForId(id) {
 // if we are on last page, disable next, it not ensure prev is enabled
 function resetNextPrevButtons()
 {
+    // no data, so disabled both
+    if( getPageNumberForId(pageList[0]) == -1 )
+    {
+        $('.prev').prop('disabled', true).addClass('disabled');
+        $('.next').prop('disabled', true).addClass('disabled');
+        return
+    }
     if ( isFirstPage( getPageNumberForId(pageList[0]) ) )
         $('.prev').prop('disabled', true).addClass('disabled');
     else
@@ -634,12 +759,16 @@ function resetNextPrevButtons()
 // get list of eids for the next page, also make sure next/prev buttons make sense for page we are on
 function nextPage(successCallback)
 {
+     // start with disabling more next presses until we are ready to process them
+     $('.prev').prop('disabled', true).addClass('disabled');
+     $('.next').prop('disabled', true).addClass('disabled');
+
     // pageList[0] is the first entry on this page
     const currentPage=getPageNumberForId( pageList[0] )
     // should never happen / just return pageList unchanged
     if ( currentPage === -1 || isLastPage( currentPage ) )
     {
-        console.error( "WARNING: seems first on pg=" + firstEntryOnPage + " of how many=" + OPT.how_many + " gives currentPage=" + currentPage + " and we cant go next page?" )
+        console.error( "WARNING: seems first on pg=" + pageList[0] + " of how many=" + OPT.how_many + " gives currentPage=" + currentPage + " and we cant go next page?" )
         return
     }
     getPage( currentPage+1, successCallback )
@@ -649,26 +778,34 @@ function nextPage(successCallback)
 // get list of eids for the prev page, also make sure next/prev buttons make sense for page we are on
 function prevPage(successCallback)
 {
+    // start with disabling more prev presses until we are ready to process them
+    $('.prev').prop('disabled', true).addClass('disabled');
+    $('.next').prop('disabled', true).addClass('disabled');
+
     // pageList[0] is the first entry on this page
     const currentPage=getPageNumberForId( pageList[0] )
     // should never happen / just return pageList unchanged
     if (currentPage === 1 || currentPage === -1 )
     {
-        console.error( "WARNING: seems first on pg=" + firstEntryOnPage + " of how many=" + OPT.how_many + " gives currentPage=" + currentPage + " and we cant go prev page?" )
+        console.error( "WARNING: seems first on pg=" + pageList[0] + " of how many=" + OPT.how_many + " gives currentPage=" + currentPage + " and we cant go prev page?" )
         return
     }
     getPage( currentPage-1, successCallback )
     return
 }
 
+// function to see if we are on a phone or tablet (where we dont have ctrl or shift keys - helps to display fake buttons to allow multiselect on mobiles)
 function isMobile() {
     try{ document.createEvent("TouchEvent"); return true; }
     catch(e){ return false; }
 }
 
-
+// when we change one of the options (noo, how_many, folders) - then update '{how_many} files' str,
+// tweak noo menu for folders/flat view then reset the page contents based on current OPT values
 function changeOPT(successCallback) {
     OPT.how_many=$('#how_many').val()
+    // changes invalidate page cache so clear it out
+    document.page.length=0
     new_f=$('#folders').val()
     new_f=( new_f == 'True' )
     // if change to/from folders, also fix the noo menu
@@ -689,7 +826,7 @@ function changeOPT(successCallback) {
     OPT.folders=new_f
     OPT.folders=$('#folders').val()
     OPT.grouping=$('#grouping').val()
-    OPT.size=$('#size').val()
+    OPT.size=$('input[name="size"]:checked').val();
     $.ajax({
         type: 'POST',
         url: '/change_file_opts',
@@ -697,13 +834,180 @@ function changeOPT(successCallback) {
         contentType: 'application/json',
         success: function(resp) {
             entryList=resp.query_data.entry_list
+            OPT.how_many=parseInt(OPT.how_many)
+            pageList=entryList.slice(0, OPT.how_many)
             // put data back into booleans, ints, etc
             OPT.folders=( OPT.folders == 'True' )
-            OPT.how_many=parseInt(OPT.how_many)
             $('.how_many_text').html( `&nbsp;${OPT.how_many} files&nbsp;` )
-            OPT.root_eid=parseInt(OPT.root_eid)
             OPT.size=parseInt(OPT.size)
             getPage(1,successCallback)
         }
     })
 }
+
+// function to change the size of thumbnails when user clicks xs/s/m/l/xl buttons
+function changeSize()
+{
+    sz=$('input[name="size"]:checked').val();
+    OPT.size=sz
+	$('.thumb').attr( {height: sz, style: 'font-size:'+sz+'px' } )
+	$('#size').val(sz)
+    sz=sz-22
+    $('.svg').height(sz);
+    $('.svg').width(sz);
+    $('.svg_cap').width(sz);
+}
+
+// when a delete or restore files job has completed successfullly, then get ids
+// find the page we are on, remove amendments & ids from entryList and re-get page
+// which will reset pageList and the UI of images for that page
+function handleMoveOrDeleteOrRestoreFileJobCompleted(job)
+{
+    // this grabs the values from the object attributes of eid-0, eid-1, etc.
+    const ids = job.extra.filter(item => item.name.startsWith("eid-")).map(item => item.value);
+    
+    // find page number of first element to delete (this is the page we will return too)
+    pnum=getPageNumberForId( parseInt(ids[0]) )
+
+    // remove amendment data
+    for (const ent of ids)
+    {
+        id=parseInt(ent)
+        removeAmendment( id )
+        // remove the item in the entryList
+        index=entryList.indexOf(id);
+        if( index != -1 )
+            entryList.splice(index, 1); // Remove the element
+        else
+        {
+            return; // have to get out of here, or calling getPage() below will loop forever
+        }
+    }
+
+    // re-create pageList by reloading the page
+    getPage(pnum,getPageFigures)
+}
+
+// POST to a check URL, that will tell us if the amendment job has completed,
+// it also calls CheckForJobs() which will fix up the Active Jobs badge,
+function checkForAmendmentJobToComplete(job_id)
+{
+    CheckForJobs()
+    $.ajax( { type: 'POST', data: '&job_id='+job_id, url: '/check_amend_job_status',
+        success: function(res) { handleCheckAmendmentJobStatus(res); } } )
+}
+
+// the status of a Amendment Job has been returned, finished is True/False
+// if not finished try again in 1 second... If finished then invalidate page
+// cache and based on job type call code correct func to update the UI appropriately
+function handleCheckAmendmentJobStatus(data)
+{
+    if( data.finished )
+    {
+        // invalidate the cache
+        document.page.length=0
+
+        // transforms contain the single transformed entry data for convenience
+        if( data.job.name == 'transform_image' )
+            handleTransformImageJobCompleted(data.job, data.entry)
+        else if ( data.job.name == 'delete_files' || data.job.name == 'restore_files' || data.job.name == 'move_files' )
+            handleMoveOrDeleteOrRestoreFileJobCompleted(data.job)
+            // if we are viewing this file, then just go up / back,b/c this file is "gone" from this view
+            if( document.viewing )
+                goOutOfViewer()
+    }
+    else { setTimeout( function() { checkForAmendmentJobToComplete(data.job.id) }, 1000 ); }
+}
+
+// different context menu on files
+$.contextMenu({
+    selector: '.entry',
+    itemClickEvent: "click",
+    build: function($triggerElement, e) {
+        // if we are not in the highlight set, then move the highlight to this element
+        if( ! $(e.currentTarget).is('.highlight') )
+        {
+            $('.highlight').removeClass('highlight');
+            $(e.currentTarget).addClass('highlight')
+        }
+        // when right-clicking & no selection add one OR deal with ctrl/shift right-lick as it always changes seln
+        if( NoSel() || e.ctrlKey || e.shiftKey )
+        {
+            DoSel(e, e.currentTarget )
+            SetButtonState();
+        }
+
+        if( FiguresOrDirsOrBoth() == "figure" )
+        {
+            item_list = {
+                details: { name: "Details..." },
+                view: { name: "View File" },
+                sep: "---",
+            }
+            if( e.currentTarget.getAttribute('type') == 'Image' )
+            {
+                item_list['transform'] = {
+                    name: "Transform",
+                    items: {
+                        "r90": { "name" : "Rotate 90 degrees" },
+                        "r180": { "name" : "Rotate 180 degrees" },
+                        "r270": { "name" : "Rotate 270 degrees" },
+                        "fliph": { "name" : "Flip horizontally" },
+                        "flipv": { "name" : "Flip vertically" }
+                    }
+                }
+
+            }
+            item_list['move'] = { name: "Move selected file(s) to new folder" }
+            item_list['sep2'] = { sep: "---" }
+        }
+        else
+            item_list = {
+                move: { name: "Move selection(s) to new folder" }
+            }
+
+        item_list['ai'] = {
+            name: "Scan file for faces",
+            items: {
+                "ai-all": { name: "all" }
+            }
+        };
+
+        // Dynamically add entries for each person in the `people` array
+        people.forEach(person => {
+            item_list['ai'].items[`ai-${person.tag}`] = { name: person.tag };
+        });
+
+        if( SelContainsBinAndNotBin() ) {
+            item_list['both']= { name: 'Cannot delete and restore at same time', disabled: true }
+        } else {
+            if (e.currentTarget.getAttribute('path_type') == 'Bin' )
+                item_list['undel']= { name: "Restore selected file(s)" }
+            else if( e.currentTarget.getAttribute('type') != 'Directory' )
+                item_list['del']= { name: "Delete Selected file(s)" }
+        }
+
+        return {
+            callback: function( key, options) {
+                if( key == "details" ) { DetailsDBox() }
+                if( key == "view" ) { startViewing( $(this).attr('id') ) }
+                if( key == "move" ) { MoveDBox() }
+                if( key == "del"  ) { DelDBox('Delete') }
+                if( key == "undel") { DelDBox('Restore') }
+                if( key == "r90"  ) { Transform(90) }
+                if( key == "r180" ) { Transform(180) }
+                if( key == "r270" ) { Transform(270) }
+                if( key == "fliph" ) { Transform("fliph") }
+                if( key == "flipv" ) { Transform("flipv") }
+                if( key.startsWith("ai")) { RunAIOnSeln(key) }
+                // dont flow this event through the dom
+                e.stopPropagation()
+            },
+            items: item_list
+        };
+    }
+});
+
+// finally, for files_ip/files_sp/files_rbp - set click inside document (NOT an entry) to remove seln
+$(document).on('click', function(e) { $('.highlight').removeClass('highlight') ; SetButtonState() });
+document.page=[]

internal/js/files_transform.js

@@ -1,27 +1,31 @@
-// POST to a check URL, that will tell us if the transformation has completed,
-// if not, try again in 1 second... If it has finished then reset the thumbnail
-// to full colour, put it back to being an entry and reset the thumbnail to the
-// newly created one that was sent back in the response to the POST
-function CheckTransformJob(id,job_id)
+// This function will remove the matching amendment for this entry (id)
+// can only have 1 ammendment per image, its grayed out for other changes
+function removeAmendment( id )
 {
-    CheckForJobs()
-    $.ajax(
-        {
-            type: 'POST', data: '&job_id='+job_id, url: '/check_transform_job', success: function(data) { 
-                if( data.finished )
-                {
-                    $('#s'+id).hide()
-                    $('#'+id).find('img.thumb').attr('style', 'filter: color(100%);' );
-                    $('#'+id).addClass('entry')
-                    $('#'+id).find('.thumb').attr('src', 'data:image/jpeg;base64,'+data.thumbnail)
-                    return false;
-                }
-                else
-                {
-                    setTimeout( function() { CheckTransformJob(id,job_id) }, 1000,id, job_id ); 
-                }
-            }, 
-        } )
+    document.amendments=document.amendments.filter(obj => obj.eid !== id)
+}
+
+// If Transform job has finished then reset relevant document.entries
+// with updated from DB, remove the amendment and redraw image
+function handleTransformImageJobCompleted(job, entry)
+{
+    removeAmendment( entry.id )
+    // update viewer  if we are viewing an image
+    if( document.viewing )
+    {
+        // force reload with timestamped version of im.src
+        im.src=im.src + '?t=' + new Date().getTime();
+        DrawImg()
+    }
+
+    // ALWAYS update files* div as we could go back to this from a viewer, and
+    // the thumbnail needs the updated data
+    idx = entryList.indexOf(entry.id)
+    // replace data for this entry now its been transformed
+    document.entries[idx]=entry
+    // redraw into figure html in dom
+    html = createFigureHtml( entry )
+    $('#'+entry.id).replaceWith( html )
 }
 
 // for each highlighted image, POST the transform with amt (90, 180, 270,
@@ -31,9 +35,25 @@ function CheckTransformJob(id,job_id)
 // to finish
 function Transform(amt)
 {
-    $('.highlight').each(function( id, e ) { 
-        post_data = '&amt='+amt+'&id='+e.id
-        // send /transform for this image, grayscale the thumbmail, add color spinning wheel overlay, and start checking for job end
-        $.ajax({ type: 'POST', data: post_data, url: '/transform', success: function(data){ $('#'+e.id).find('img.thumb').attr('style', 'filter: grayscale(100%);' ); $('#'+e.id).removeClass('entry'); $('#s'+e.id).show(); CheckTransformJob(e.id,data.job_id); return false; } })
-    } )
+    // we are in the viewer with 1 image only...
+    if( $('#viewer_div').length && ! $('#viewer_div').hasClass('d-none') )
+    {
+        post_data = '&amt='+amt+'&id='+document.viewing.id
+        // POST /transform for image, grayscale the image, add throbber, & start checking for end of job
+        $.ajax({ type: 'POST', data: post_data, url: '/transform', success: function(data) { 
+            processAmendments(data.job.amendments)
+            checkForAmendmentJobToComplete(data.job.id)
+        } })
+    }
+    else
+    {
+        $('.highlight').each(function( cnt, e ) {
+            post_data = '&amt='+amt+'&id='+e.id
+            // POST /transform for image, grayscale the thumbnail, add throbber, & start checking for end of job
+            $.ajax({ type: 'POST', data: post_data, url: '/transform', success: function(data){ 
+                processAmendments(data.job.amendments)
+                checkForAmendmentJobToComplete(data.job.id)
+            } })
+        } )
+    }
 }

internal/js/jobs.js

@@ -71,22 +71,15 @@ function SetActiveJobsBadge(num_jobs)
 // after a 1 second timeout
 function CheckForJobs()
 {
-    $.ajax(
-        {
-            type: 'POST', url: '/check_for_jobs', 
-            success: function(data) { 
-                data.sts.forEach( 
-                    function(el)
-                    {
-                        StatusMsg(el)
-                    }
-                )
-                SetActiveJobsBadge(data.num_active_jobs)
-                if( data.num_active_jobs > 0 )
-                {
-                    setTimeout( function() { CheckForJobs() }, 1000 );
-                }
-            }, 
-        } )
+    $.ajax( {
+        type: 'POST', url: '/check_for_jobs',
+        success: function(data) {
+            // for each status, handle it/make toast in UI
+            data.sts.forEach( function(el) { StatusMsg(el) } )
+            SetActiveJobsBadge(data.num_active_jobs)
+            // still active job(s), keep checking for them to end
+            if( data.num_active_jobs > 0 ) { setTimeout( function() { CheckForJobs() }, 1000 ); }
+        },
+    } )
     return false;
 }

internal/js/view_support.js

@@ -25,6 +25,8 @@ function NewHeight()
         return im.height*gap / (im.width/window.innerWidth)
 }
 
+// draw 'str' as a label above the bounding box of the face (with a white
+// transparent background to enhance readability of str)
 function DrawLabelOnFace(str)
 {
 	// finish face box, need to clear out new settings for // transparent backed-name tag
@@ -60,6 +62,11 @@ function DrawImg()
     if( im.width == 0 )
         return
 
+    // find any matching ammendment
+    am=document.amendments.filter(obj => obj.eid === document.viewing.id)
+    if( am.length )
+        am=am[0]
+
     canvas.width=NewWidth(im)
     canvas.height=NewHeight(im)
 
@@ -67,14 +74,32 @@ function DrawImg()
 	$('#img-cap').width(canvas.width)
 
     // actually draw the pixel images to the canvas at the right size
-    if( grayscale )
+    if (!Array.isArray(am))
         context.filter='grayscale(1)'
     context.drawImage(im, 0, 0, canvas.width, canvas.height )
     // -50 is a straight up hack, no idea why this works, but its good enough for me
-    if( throbber )
-        $('#throbber').attr('style', 'display:show; position:absolute; left:'+canvas.width/2+'px; top:'+(canvas.height/2-50)+'px' )
-    else
-        $('#throbber').hide();
+    if (!Array.isArray(am))
+    {
+        $('#throbber').show()
+        $('#white-circle').show()
+        if(am.type.which == 'img' )
+        {
+            $('#inside-img').attr('src', '/internal/'+am.type.what );
+            $('#inside-img').show()
+        }
+        else
+        {
+            $('#inside-icon').attr('style', `color:${am.type.colour};height:64px` )
+            $('#inside-icon').attr('fill', am.type.colour )
+            $('#inside-icon use').attr('xlink:href', `/internal/icons.svg#${am.type.what}`);
+            $('#inside-icon').show()
+        }
+    } else {
+        $('#throbber').hide()
+        $('#white-circle').hide()
+        $('#inside-img').hide()
+        $('#inside-icon').hide()
+    }
 
     // show (or not) the whole figcaption with fname in it - based on state of fname_toggle
     if( $('#fname_toggle').prop('checked' ) )
@@ -85,9 +110,8 @@ function DrawImg()
     else
         $('.figcaption').hide()
 
-    // if we have faces, the enable the toggles, otherwise disable them
-    // and reset model select too
-    if( document.viewing.file_details.faces )
+    // if we have faces, the enable the toggles, otherwise disable them and reset model select too
+    if( document.viewing.file_details.faces.length )
     {
         $('#faces').attr('disabled', false)
         $('#distance').attr('disabled', false)
@@ -118,10 +142,13 @@ function DrawImg()
             context.lineWidth = 2
 
 			// this face has an override so diff colour
-			if( faces[i].override )
+			if( faces[i].fnmo.length || faces[i].ffmo.length )
 			{
 				context.strokeStyle = 'blue'
-				DrawLabelOnFace( faces[i].override.who )
+                if( faces[i].ffmo.length )
+                    DrawLabelOnFace( faces[i].ffmo[0].person.tag )
+                else
+                    DrawLabelOnFace( faces[i].fnmo[0].type.name )
 			}
 			else
 			{
@@ -162,7 +189,7 @@ function ViewImageOrVideo()
     if( ! document.viewing ) return
     if( document.viewing.type.name == 'Image' )
     {
-        im.src='../' + document.viewing.FullPathOnFS
+        im.src='../' + document.viewing.FullPathOnFS + '?t=' + new Date().getTime();
         $('#video_div').hide()
         if( $('#fname_toggle').prop('checked' ) )
             $('#img-cap').show()
@@ -193,6 +220,8 @@ function ViewImageOrVideo()
 
 var offsetX,offsetY;
 
+// find the edge of the canvas, so when we have a PAGE event with x,y we can see
+// where we clicked in it (PAGE.x - canvas.x to see where in canvas, etc)
 function reOffset()
 {
   var BB=$('#canvas').get(0).getBoundingClientRect();
@@ -200,23 +229,27 @@ function reOffset()
   offsetY=BB.top;
 }
 
-window.onscroll=function(e){ reOffset(); }
-window.onresize=function(e){ reOffset(); }
-
+// when we are ready, 
 $(document).ready( function()
 {
     var cw=$('#canvas').width;
     var ch=$('#canvas').height;
     reOffset();
+    // if we scroll or resize the window, the canvas moves on the page, reset the offsets
     window.onscroll=function(e){ reOffset(); }
     window.onresize=function(e){ reOffset(); }
 
+    // clicking in the viewer canvas gets its own handlers to handle faces (or not)
     $.contextMenu({
         selector: '#canvas',
         trigger: 'left',
 //        trigger: 'none',
         hideOnSecondTrigger: true,
 
+        // go through each face, and add appropriate 'left-click' menu.
+        // e.g if known face, say name, offer add refimg to person, etc.
+        // this is a bit complex, the item_list var has a key (which is what we
+        // will do if we are chosen from the menu), and data to process the action
         build: function($triggerElement, e) {
             reOffset();
             // get mouse position relative to the canvas (left-click uses page*)
@@ -235,10 +268,10 @@ $(document).ready( function()
 
                 if( x >= fx && x <= fx+fw && y >= fy && y <= fy+fh )
                 {
-                    if( faces[i].override )
+                    if( faces[i].ffmo.length || faces[i].fnmo.length )
 					{
                         item_list['remove_force_match_override']={ 'name': 'Remove override for this face', 'which_face': i, 'id': faces[i].id }
-					}
+                    }
 					else if( faces[i].refimg )
 					{
                         item_list['match']={ 'name': faces[i].refimg.person.tag, 'which_face': i, 'id': faces[i].id }
@@ -251,7 +284,7 @@ $(document).ready( function()
                         item_list['no_match_new_person']={ 'name': 'Add as reference image to NEW person', 'which_face': i, 'id': faces[i].id }
                         item_list['no_match_new_refimg']={ 'name': 'Add as reference image to EXISTING person', 'which_face': i, 'id': faces[i].id }
 						for( var el in NMO ) {
-							item_list['NMO_'+el]={'type_id': NMO[el].type_id, 'name': 'Override: ' + NMO[el].name, 'which_face': i, 'id': faces[i].id }
+							item_list['NMO_'+el]={'type_id': NMO[el].id, 'name': 'Override: ' + NMO[el].name, 'which_face': i, 'id': faces[i].id }
 						}
                     }
                     delete item_list['not_a_face']
@@ -271,26 +304,15 @@ $(document).ready( function()
     } )
 } );
 
-// quick wrapper function to make calling this ajax code simpler in SearchForPerson
+// POST to the server to force a match for this face to person_id
+// FIXME: could I not pass person_id, and use // ...[item[key].which_face].refimg.person.id
 function OverrideForceMatch( person_id, key )
 {
-	// we have type_id passed in, so dig the NMO out, and use that below (its really just for name, but in case we change that in the DB)
-	for( el in NMO )
-	{
-		if( NMO[el].type_id == item[key].type_id )
-		{
-			fm_idx=el
-			break
-		}
-	}
 	ofm='&person_id='+person_id+'&face_id='+item[key].id
 	$.ajax({ type: 'POST', data: ofm, url: '/add_force_match_override', success: function(data) { 
-		document.viewing.file_details.faces[item[key].which_face].override={}
-		document.viewing.file_details.faces[item[key].which_face].override.who=data.person_tag
-		document.viewing.file_details.faces[item[key].which_face].override.distance='N/A'
-		document.viewing.file_details.faces[item[key].which_face].override.type_id=NMO[fm_idx].id
-		document.viewing.file_details.faces[item[key].which_face].override.type_name=NMO[fm_idx].name
-
+		document.viewing.file_details.faces[item[key].which_face].ffmo=[]
+		document.viewing.file_details.faces[item[key].which_face].ffmo[0]={}
+        document.viewing.file_details.faces[item[key].which_face].ffmo[0].person=data.person
 		$('#dbox').modal('hide')
 		$('#faces').prop('checked',true)
 		DrawImg()
@@ -299,6 +321,23 @@ function OverrideForceMatch( person_id, key )
 	} )
 }
 
+// function that handles the POSTed data that comes back when we add a
+// reference image to a new or existing person (right-click on a face)
+// used in success callbacks from CreatePersonAndRefimg() and AddRefimgTo()
+function handleAddRefimgData(key, data)
+{
+    document.viewing.file_details.faces[item[key].which_face].refimg=data.refimg
+    document.viewing.file_details.faces[item[key].which_face].refimg_lnk={}
+    // if we used this img, for now set distance to 0 - it is an exact match!
+    document.viewing.file_details.faces[item[key].which_face].refimg_lnk.face_distance=0.0
+    $('#dbox').modal('hide')
+    $('#faces').prop('checked',true)
+    DrawImg()
+    CheckForJobs()
+}
+
+// when we right-click a face and make a new person, this code creates and
+// associates the face 
 function CreatePersonAndRefimg( key )
 {
 	d='&face_id='+item[key].id
@@ -307,29 +346,17 @@ function CreatePersonAndRefimg( key )
 		+'&surname='+$('#surname').val()
 		+'&refimg_data='+item[key].refimg_data
 	$.ajax({ type: 'POST', data: d, url: '/match_with_create_person',
-        success: function(data) {
-			document.viewing.file_details.faces[item[key].which_face].refimg.person.tag=data.who
-			document.viewing.file_details.faces[item[key].which_face].facefile_lnk.face_distance=data.distance
-			$('#dbox').modal('hide')
-			$('#faces').prop('checked',true)
-			DrawImg()
-            CheckForJobs()
-		}
+        success: function(data) { handleAddRefimgData(key, data ) },
 	})
 }
 
+// when we right-click a face and connect to an existing person, this connects
+// the refimg and associates the face
 function AddRefimgTo( person_id, key, search )
 {
 	d='&face_id='+item[key].id+'&person_id='+person_id+'&refimg_data='+item[key].refimg_data+'&search='+search
     $.ajax({ type: 'POST', data: d, url: '/add_refimg_to_person',
-        success: function(data) {
-			document.viewing.file_details.faces[item[key].which_face].refimg.person.tag=data.who
-			document.viewing.file_details.faces[item[key].which_face].facefile_lnk.face_distance=data.distance
-			$('#dbox').modal('hide')
-			$('#faces').prop('checked',true)
-			DrawImg()
-            CheckForJobs()
-		}
+        success: function(data) { handleAddRefimgData(key, data ) },
 	})
 }
 
@@ -346,8 +373,7 @@ function SearchForPerson(content, key, face_id, face_pos, type_id)
 			for( var el in data ) {
 				content+='<div class="row">'
 				var person = data[el];
-				// NMO_1 is  a non-match-override type_id==1 (or force match to existing person)
-				if( key == "NMO_1" )
+				if( item[key].name == "Override: Manual match to existing person" )
 				{
 					func='OverrideForceMatch('+person.id+',\''+key+'\' )'
                     content+= '<div class="col">' + person.tag + ' (' + person.firstname+' '+person.surname+ ') </div>'
@@ -370,17 +396,19 @@ function SearchForPerson(content, key, face_id, face_pos, type_id)
 	return false
 }
 
+// if we force a match, this func allows us to POST to the server to remove the override
 function RemoveOverrideForceMatch(face_pos)
 {
-    if( document.viewing.file_details.faces[face_pos].override )
-        who=document.viewing.file_details.faces[face_pos].override.who
+    if( document.viewing.file_details.faces[face_pos].ffmo.length )
+        who=document.viewing.file_details.faces[face_pos].ffmo[0].person.tag
     else
 	    who=document.viewing.file_details.faces[face_pos].refimg.person.tag
 
-	d='&face_id='+document.viewing.file_details.faces[face_pos].id+'&person_tag='+document.viewing.file_details.faces[face_pos].refimg.person.tag+'&file_eid='+document.viewing.id
+	d='&face_id='+document.viewing.file_details.faces[face_pos].id+'&person_tag='+who+'&file_eid='+document.viewing.id
     $.ajax({ type: 'POST', data: d, url: '/remove_force_match_override',
         success: function(data) {
-			delete document.viewing.file_details.faces[face_pos].override
+            // force/delete the ffmo cleanly
+			document.viewing.file_details.faces[face_pos].ffmo.length=0
 			$('#dbox').modal('hide')
 			DrawImg()
             CheckForJobs()
@@ -390,12 +418,13 @@ function RemoveOverrideForceMatch(face_pos)
 	return false
 }
 
+// if we force NO match, this func allows us to POST to the server to remove the override
 function RemoveOverrideNoMatch(face_pos, type_id)
 {
 	d='&face_id='+document.viewing.file_details.faces[face_pos].id+'&type_id='+type_id
     $.ajax({ type: 'POST', data: d, url: '/remove_no_match_override',
         success: function(data) {
-			delete document.viewing.file_details.faces[face_pos].override
+			document.viewing.file_details.faces[face_pos].fnmo.length=0
 			$('#dbox').modal('hide')
 			DrawImg()
             CheckForJobs()
@@ -405,16 +434,13 @@ function RemoveOverrideNoMatch(face_pos, type_id)
 	return false
 }
 
+// POST to the server to force NO match for this face
 function AddNoMatchOverride(type_id, face_id, face_pos, type_id)
 {
 	d='&type_id='+type_id+'&face_id='+face_id
     $.ajax({ type: 'POST', data: d, url: '/add_no_match_override',
         success: function(data) {
-			document.viewing.file_details.faces[face_pos].override={}
-			document.viewing.file_details.faces[face_pos].override.who=NMO[type_id].name
-			document.viewing.file_details.faces[face_pos].override.distance='N/A'
-			document.viewing.file_details.faces[face_pos].override.type_id=type_id
-			document.viewing.file_details.faces[face_pos].override.type_name=NMO[type_id].name
+			document.viewing.file_details.faces[face_pos].fnmo[0]=data
 			$('#dbox').modal('hide')
 			$('#faces').prop('checked',true)
 			DrawImg()
@@ -423,6 +449,9 @@ function AddNoMatchOverride(type_id, face_id, face_pos, type_id)
     } )
 }
 
+// generate html for the appropriate content to search for a person when adding
+// override DBox. has a button that when clicked calls SeachForPerson() which
+// POSTs to the server, and fills in the 'search_person_results' div with content
 function AddSearch( content, key, face_pos )
 {
 	html='<h5>search for existing person:</h5>'
@@ -462,17 +491,17 @@ function FaceDBox(key, item)
     div+='</div><div class="col-6">'
     if ( key == 'remove_force_match_override' )
 	{
-		if( document.viewing.file_details.faces[face_pos].override.type_name == 'Manual match to existing person' )
-			div+='<div class="row col-12">remove this override (force match to: ' + document.viewing.file_details.faces[face_pos].override.who + ')</div>'
+		if( document.viewing.file_details.faces[face_pos].ffmo.length )
+			div+='<div class="row col-12">remove this override (force match to: ' + document.viewing.file_details.faces[face_pos].ffmo[0].person.tag + ')</div>'
 		else
-			div+='<div class="row col-12">remove this override (no match)</div>'
+			div+='<div class="row col-12">remove this override (' + document.viewing.file_details.faces[face_pos].fnmo[0].type.name + ')</div>'
 		div+='<div class="row">'
 		div+='<button class="btn btn-outline-info col-6" type="button" onClick="$(\'#dbox\').modal(\'hide\'); return false">Cancel</button>'
 		div+='<button class="btn btn-outline-danger col-6" type="button" '
-		if( document.viewing.file_details.faces[face_pos].override.type_name == 'Manual match to existing person' )
+		if( document.viewing.file_details.faces[face_pos].ffmo.length )
 			div+='onClick="RemoveOverrideForceMatch(' +face_pos+ ')">Remove</button>'
 		else
-			div+='onClick="RemoveOverrideNoMatch(' +face_pos+','+document.viewing.file_details.faces[face_pos].override.type_id+ ')">Remove</button>'
+			div+='onClick="RemoveOverrideNoMatch(' +face_pos+','+document.viewing.file_details.faces[face_pos].fnmo[0].type.id+ ')">Remove</button>'
 		div+='</div>'
 	}
     if ( key == 'no_match_new_person' )
@@ -506,7 +535,6 @@ function FaceDBox(key, item)
         func='AddRefimgTo('+item[key]['person_id']+',\''+key+'\''
         func_sn=func+ ', true )'
         func_ao=func+ ', false )'
-        div+=`<script>console.log( "AddExistingFaceAsRefimgToMatchedPerson()" )</script>`
         div+="Confirm you wish to add this face as a reference image for " + item[key]['who']
         div+= '<div class="col">' + item[key]['who'] + '</div><div class="col input-group">'
         div+= '<button onClick="'+func_sn+'" class="btn btn-success py-1 input-group-prepend">Add & search now</button>&nbsp;'
@@ -565,7 +593,167 @@ function JoblogSearch()
     })
 }
 
+// helper func to resert the src on the video div
 function setVideoSource(newSrc) {
     $('#videoSource').attr('src', newSrc);
     $('#video')[0].load();
 }
+
+// function called when we get another page from inside the viewer
+function getPageViewer(res, viewingIdx)
+{
+    document.viewing=document.entries[viewingIdx]
+    // update viewing, arrows and image/video too
+    ViewImageOrVideo()
+}
+
+// handler used when we double click an entry to show it in the viewer
+function dblClickToViewEntry(id) {
+    $('#files_div').addClass('d-none')
+    $('#viewer_div').removeClass('d-none')
+    setEntryById( id )
+    ViewImageOrVideo()
+}
+
+// quick function that allows us to go out of the viewer and back, the viewercomes from files_ip/sp
+// so just redraw the page with drawPageOfFigures() as we have all the data 
+function goOutOfViewer()
+{
+    // if this returns -1, we have used arrows to go onto a new page(s)
+    if( getPageNumberForId( $('#figures').find('.figure').first().prop('id') ) == -1 )
+        drawPageOfFigures()
+
+    // hide viewer div, then show files_div
+    $('#viewer_div').addClass('d-none')
+    $('#files_div').removeClass('d-none')
+    // no longer viewing an image too
+    document.viewing=null
+}
+
+// change the viewer to the previous entry (handle page change too)
+function getPreviousEntry() {
+    var currentIndex = entryList.indexOf(document.viewing.id);
+
+    oldPageOffset=Math.floor(currentIndex / OPT.how_many) 
+    if (currentIndex > 0) {
+        currentIndex--;
+        pageOffset=Math.floor(currentIndex / OPT.how_many)
+        currentIndex=currentIndex-(pageOffset*OPT.how_many)
+        // pref page, load it
+        if( oldPageOffset != pageOffset )
+            // pref page is pageOffset+1 now
+            getPage(pageOffset+1,getPageViewer,currentIndex)
+        else
+            document.viewing=document.entries[currentIndex]
+    }
+}
+
+// change the viewer to the next entry (handle page change too)
+function getNextEntry() {
+    var currentIndex = entryList.indexOf(document.viewing.id);
+
+    oldPageOffset=Math.floor(currentIndex / OPT.how_many) 
+    if (currentIndex < entryList.length - 1) {
+        currentIndex++
+        pageOffset=Math.floor(currentIndex / OPT.how_many) 
+        currentIndex=currentIndex-(pageOffset*OPT.how_many)
+        // next page, load it
+        if( oldPageOffset != pageOffset )
+            // next page is pageOffset+1 now
+            getPage(pageOffset+1,getPageViewer,currentIndex)
+        else
+            document.viewing=document.entries[currentIndex]
+    }
+}
+
+// check if we are viewing the very first entry (helps to disable la)
+function entryIsAtStart() {
+    return document.viewing.id === entryList[0];
+}
+
+// check if we are viewing the very last entry (helps to disable ra)
+function entryIsAtEnd() {
+    return document.viewing.id === entryList[entryList.length - 1];
+}
+
+// helper func to ensure document.viewing is the right entry from document.entries array
+function setEntryById(id) {
+    var currentIndex = entryList.indexOf(parseInt(id));
+    // if we are on a different page, adjust as document.entries only has <= how_many
+    pageOffset=Math.floor(currentIndex / OPT.how_many) 
+    currentIndex = currentIndex-(pageOffset*OPT.how_many)
+    document.viewing=document.entries[currentIndex]
+}
+
+// disable la button if we are viewing first entry and/or ra button if we are viewing last entry
+function setDisabledForViewingNextPrevBttons()
+{
+    $('#la').attr('disabled', entryIsAtStart());
+    $('#ra').attr('disabled', entryIsAtEnd());
+}
+
+// when we go into the view, the keybindings are set here for items like 'f' for face box/name
+function addViewerKeyHandler() {
+    // allow a keypress on the viewer_div
+    $(document).keydown(function(event) {
+        // if dbox is visible, dont process this hot-key, we are inputting text into inputs instead
+        if( $("#dbox").is(':visible') )
+            return
+        switch (event.key) 
+        {
+            case "Left": // IE/Edge specific value
+            case "ArrowLeft":
+                $('#la').click()
+                break;
+            case "Right": // IE/Edge specific value
+            case "ArrowRight":
+                $('#ra').click()
+                break;
+            case "d":
+                $('#distance').click()
+                break;
+            case "f":
+                $('#faces').click()
+                break;
+            case "n":
+                $('#fname_toggle').click()
+                break;
+            case "F":
+                fullscreen=!document.fullscreen
+                ViewImageOrVideo()
+                break;
+            case "l":
+                JoblogSearch()
+                break;
+            case "Delete":
+                $('#del').click()
+            default:
+                return; // Quit when this doesn't handle the key event.
+        }
+    });
+}
+
+// left arrow onclick handler to go to prev image from inside the viewer
+function prevImageInViewer()
+{
+    getPreviousEntry()
+    setDisabledForViewingNextPrevBttons()
+    ViewImageOrVideo()
+}
+
+// right arrow onclick handler to go to next image from inside the viewer
+function nextImageInViewer()
+{
+    getNextEntry()
+    setDisabledForViewingNextPrevBttons()
+    ViewImageOrVideo()
+}
+
+// wrapper func to start the viewer - needed as we have a dbl-click & View file
+// to start the viewer
+function startViewing(eid)
+{
+    dblClickToViewEntry( eid );
+    setDisabledForViewingNextPrevBttons();
+    addViewerKeyHandler()
+}

internal/js/view_transform.js

@@ -1,37 +0,0 @@
-// POST to a check URL, that will tell us if the transformation has completed,
-// if not, try again in 1 second... If it has finished then reset the thumbnail
-// to full colour, put it back to being an entry and reset the thumbnail to the
-// newly created one that was sent back in the response to the POST
-function CheckTransformJob(id,job_id)
-{
-    CheckForJobs()
-    $.ajax(
-        {
-            type: 'POST', data: '&job_id='+job_id, url: '/check_transform_job', success: function(data) { 
-                if( data.finished )
-                {
-                    // stop throbber, remove grayscale & then force reload with timestamped version of im.src
-                    grayscale=0
-                    throbber=0
-                    im.src=im.src + '?t=' + new Date().getTime();
-                    return false;
-                }
-                else
-                {
-                    setTimeout( function() { CheckTransformJob(id,job_id) }, 1000,id, job_id ); 
-                }
-            }, 
-        } )
-}
-
-// for each highlighted image, POST the transform with amt (90, 180, 270,
-// fliph, flipv) which will let the job manager know what to do to this file.
-// we also grayscale the thumbnail out, remove the entry class for now, show
-// the spinning wheel, and finally kick of the checking for the transform job
-// to finish
-function Transform(amt)
-{
-    post_data = '&amt='+amt+'&id='+current
-    // send /transform for this image, grayscale the thumbmail, add color spinning wheel overlay, and start checking for job end
-    $.ajax({ type: 'POST', data: post_data, url: '/transform', success: function(data) { grayscale=1; throbber=1; DrawImg(); CheckTransformJob(current,data.job_id); return false; } })
-}

job.py

@@ -3,12 +3,14 @@ from flask_wtf import FlaskForm
 from flask import request, render_template, redirect, make_response, jsonify, url_for
 from settings import Settings
 from main import db, app, ma
-from sqlalchemy import Sequence, func
+from sqlalchemy import Sequence, func, select
+from sqlalchemy.orm import joinedload
 from sqlalchemy.exc import SQLAlchemyError
 from datetime import datetime, timedelta
 import pytz
 import socket
 from shared import PA, PA_JOB_MANAGER_HOST, PA_JOB_MANAGER_PORT, NEWEST_LOG_LIMIT, OLDEST_LOG_LIMIT
+from amend import EntryAmendment, inAmendmentTypes
 from flask_login import login_required, current_user
 from sqlalchemy.dialects.postgresql import INTERVAL
 from sqlalchemy.sql.functions import concat
@@ -57,10 +59,12 @@ class Job(db.Model):
 
     extra = db.relationship( "JobExtra")
     logs = db.relationship( "Joblog")
+    amendments = db.relationship("EntryAmendment", back_populates="job")
 
     def __repr__(self):
         return "<id: {}, start_time: {}, last_update: {}, name: {}, state: {}, num_files: {}, current_file_num: {}, current_file: {}, pa_job_state: {}, wait_for: {}, extra: {}, logs: {}>".format(self.id, self.start_time, self.last_update, self.name, self.state, self.num_files, self.current_file_num, self.current_file, self.pa_job_state, self.wait_for, self.extra, self.logs)
 
+
 ################################################################################
 # Class describing PA_JobManager_Message and in the DB (via sqlalchemy)
 # the job manager can send a message back to the front end (this code) via the 
@@ -82,7 +86,7 @@ class PA_JobManager_Message(PA,db.Model):
 # Used in main html to show a red badge of # jobs to draw attention there are
 # active jobs being processed in the background
 ################################################################################
-def GetNumActiveJobs():
+def getNumActiveJobs():
     ret=Job.query.filter(Job.pa_job_state != 'Completed').with_entities(func.count(Job.id).label('count') ).first()
     return ret[0]
 
@@ -114,8 +118,23 @@ def NewJob(name, num_files="0", wait_for=None, jex=None, desc="No description pr
 
     db.session.add(job)
     db.session.commit()
-    SetFELog( message=f'Created <a class="link-light" href="/job/{job.id}">Job #{job.id}</a> to {desc}', level="success" )
 
+    # if this job changes an eid we store that in DB and client shows until it finishes the job
+    at_id = inAmendmentTypes(job)
+    if at_id:
+        if job.name == 'transform_image':
+            id=[jex.value for jex in job.extra if jex.name == "id"][0]
+            ea=EntryAmendment( eid=id, job_id=job.id, amend_type=at_id )
+            db.session.add(ea)
+            job.amendments.append(ea)
+        elif job.name == 'delete_files' or job.name == 'restore_files' or job.name == 'move_files':
+            for j in jex:
+                if 'eid-' in j.name:
+                    ea=EntryAmendment( eid=j.value, job_id=job.id, amend_type=at_id )
+                    db.session.add(ea)
+                    job.amendments.append(ea)
+
+    SetFELog( message=f'Created <a class="link-light" href="/job/{job.id}">Job #{job.id}</a> to {desc}', level="success" )
     WakePAJobManager(job.id)
     return job
 
@@ -280,7 +299,8 @@ def joblog_search():
     from sqlalchemy import text
 
     eid=request.form['eid']
-    ent=Entry.query.get(eid)
+    stmt = select(Entry).where(Entry.id == eid)
+    ent = db.session.scalars(stmt).one_or_none()
     logs=Joblog.query.join(Job).filter(Joblog.log.ilike(text(f"'%%{ent.name}%%'"))).with_entities(Joblog.log, Job.id, Job.name, Job.state, Joblog.log_date).all()
 
     # turn DB output into json and return it to the f/e
@@ -308,14 +328,22 @@ def joblog_search():
 @app.route("/check_for_jobs", methods=["POST"])
 @login_required
 def check_for_jobs():
-    num=GetNumActiveJobs()
+    from files import job_schemas
+
+    num=getNumActiveJobs()
+    messages = PA_JobManager_Message.query.all()
     sts=[]
-    for msg in PA_JobManager_Message.query.all():
+    for msg in messages:
         u=''
         if 'Job #' not in msg.message and msg.job_id:
             u='<a class="link-light" href="' + url_for('joblog', id=msg.job_id) + '">Job #' + str(msg.job_id) + '</a>: '
         sts.append( { 'id': msg.id, 'message': u+msg.message, 'level': msg.level, 'job_id': msg.job_id, 'persistent': msg.persistent, 'cant_close': msg.cant_close } )
-    return make_response( jsonify( num_active_jobs=num, sts=sts ) )
+
+    # get jobs mentioned in messages as we may need to process the by client for UI
+    job_list=[obj.job_id for obj in messages]
+    stmt = select(Job).options(joinedload(Job.amendments)).where(Job.id.in_(job_list))
+    jobs=db.session.execute(stmt).unique().scalars().all()
+    return make_response( jsonify( num_active_jobs=num, sts=sts, jobs=job_schemas.dump(jobs) ) )
 
 ###############################################################################
 # /clear_msg -> POST -> clears out a F/E message based on passed in <id>

main.py

@@ -66,8 +66,6 @@ app.config['LDAP_USER_DN'] = 'ou=users'
 app.config['LDAP_GROUP_DN'] = 'ou=groups'
 app.config['LDAP_USER_RDN_ATTR'] = 'uid'
 app.config['LDAP_USER_LOGIN_ATTR'] = 'uid'
-app.config['LDAP_BIND_USER_DN'] = None
-app.config['LDAP_BIND_USER_PASSWORD'] = None
 app.config['LDAP_GROUP_OBJECT_FILTER'] = '(objectclass=posixGroup)'
 app.config['LDAP_BIND_USER_DN'] = None
 app.config['LDAP_BIND_USER_PASSWORD'] = None
@@ -246,6 +244,10 @@ def logout():
     logout_user()
     return redirect('/login')
 
+# quick health route so traefik knows we are up
+@app.route('/health')
+def health():
+    return {"status": "ok"}, 200
 
 ###############################################################################
 # main to be called via Flask/Gunicorn

pa_job_manager.py

@@ -1,4 +1,3 @@
-
 #
 # This file controls the 'external' job control manager, that (periodically #
 # looks / somehow is pushed an event?) picks up new jobs, and processes them.
@@ -15,7 +14,7 @@
 
 ### SQLALCHEMY IMPORTS ###
 from sqlalchemy.ext.declarative import declarative_base
-from sqlalchemy import Column, Integer, String, Sequence, Float, ForeignKey, DateTime, LargeBinary, Boolean, func, text
+from sqlalchemy import Column, Integer, String, Sequence, Float, ForeignKey, DateTime, LargeBinary, Boolean, func, text, select
 from sqlalchemy.exc import SQLAlchemyError
 from sqlalchemy.orm import relationship
 from sqlalchemy import create_engine
@@ -23,7 +22,7 @@ from sqlalchemy.orm import sessionmaker
 from sqlalchemy.orm import scoped_session
 
 ### LOCAL FILE IMPORTS ###
-from shared import DB_URL, PA_JOB_MANAGER_HOST, PA_JOB_MANAGER_PORT, THUMBSIZE, SymlinkName, GenThumb, SECS_IN_A_DAY, PA_EXIF_ROTATER
+from shared import DB_URL, PA_JOB_MANAGER_HOST, PA_JOB_MANAGER_PORT, THUMBSIZE, SymlinkName, GenThumb, SECS_IN_A_DAY, PA_EXIF_ROTATER, PA
 from datetime import datetime, timedelta, date
 
 ### PYTHON LIB IMPORTS ###
@@ -46,6 +45,8 @@ import re
 import sys
 import ffmpeg
 import subprocess
+# FIXME: remove this
+import time
 
 
 # global debug setting
@@ -512,6 +513,15 @@ class PA_JobManager_FE_Message(Base):
     def __repr__(self):
         return "<id: {}, job_id: {}, level: {}, message: {}".format(self.id, self.job_id, self.level, self.message)
 
+################################################################################
+# Class describing which Entry has a pending Amendment in the DB (via sqlalchemy)
+################################################################################
+class EntryAmendment(PA,Base):
+    __tablename__ = "entry_amendment"
+    eid = Column(Integer, ForeignKey("entry.id"), primary_key=True )
+    job_id = Column(Integer, ForeignKey("job.id"), primary_key=True )
+    # don't over think this, we just use eid to delete this entry anyway
+    amend_type = Column(Integer)
 
 ##############################################################################
 # PAprint(): convenience function to prepend a timestamp to a printed string
@@ -1863,9 +1873,22 @@ def JobRunAIOn(job):
     FinishJob(job, "Finished Processesing AI")
     return
 
+################################################################################
+# removeEntryAmendment(): helper routine to remove an Etnry Amendment for a
+# given job and eid (called after Transform or Delete/Restore/Move files
+################################################################################
+def removeEntryAmendment( job, eid ):
+    # now remove the matching amendment for the transform job
+    stmt=select(EntryAmendment).where(EntryAmendment.eid==eid)
+    ea=session.execute(stmt).scalars().one_or_none()
+    if ea:
+        session.delete(ea)
+    else:
+        AddLogForJob( job, f"ERROR: failed to remove entry amendment in DB for this transformation? (eid={id})" )
+        PAprint( f"ERROR: failed to remove entry amendment in DB for this transformation? (eid={id}, job={job} )" )
+
 ####################################################################################################################################
 # JobTransformImage(): transform an image by the amount requested (can also flip horizontal or vertical)
-# TODO: should be JobTransformImage() ;)
 ####################################################################################################################################
 def JobTransformImage(job):
     JobProgressState( job, "In Progress" )
@@ -1874,6 +1897,11 @@ def JobTransformImage(job):
     amt=[jex.value for jex in job.extra if jex.name == "amt"][0]
     e=session.query(Entry).join(File).filter(Entry.id==id).first()
     PAprint( f"JobTransformImage: job={job.id}, id={id}, amt={amt}" )
+    # cant transfer non-image, but may get here if multi-select includes non-Image
+    if e.type.name != 'Image':
+        removeEntryAmendment( job, id )
+        FinishJob(job, "Cannot rotate file as it is not an Image","Failed")
+        return
 
     if amt == "fliph":
         AddLogForJob(job, f"INFO: Flipping {e.FullPathOnFS()} horizontally" )
@@ -1897,6 +1925,10 @@ def JobTransformImage(job):
     e.file_details.hash = md5( job, e )
     PAprint( f"JobTransformImage DONE thumb: job={job.id}, id={id}, amt={amt}" )
     session.add(e)
+    # any faces in this file are no longer valid, remove them
+    session.query(FaceFileLink).filter(FaceFileLink.file_eid==e.id).delete()
+    removeEntryAmendment( job, id )
+
     FinishJob(job, "Finished Processesing image rotation/flip")
     return
     
@@ -2174,6 +2206,7 @@ def JobMoveFiles(job):
         if 'eid-' in jex.name:
             move_me=session.query(Entry).get(jex.value)
             MoveEntriesToOtherFolder( job, move_me, dst_storage_path, f"{prefix}{suffix}" )
+            removeEntryAmendment( job, move_me.id )
     NewJob( name="check_dups", num_files=0, wait_for=None, jex=None, parent_job=None, desc="check for duplicate files" )
     FinishJob(job, f"Finished move selected file(s)")
     return
@@ -2188,6 +2221,7 @@ def JobDeleteFiles(job):
         if 'eid-' in jex.name:
             del_me=session.query(Entry).join(File).filter(Entry.id==jex.value).first()
             MoveFileToRecycleBin(job,del_me)
+            removeEntryAmendment(job,del_me.id)
     NewJob( name="check_dups", num_files=0, wait_for=None, jex=None, parent_job=None, desc="check for duplicate files" )
     FinishJob(job, f"Finished deleting selected file(s)")
     return
@@ -2202,6 +2236,7 @@ def JobRestoreFiles(job):
         if 'eid-' in jex.name:
             restore_me=session.query(Entry).join(File).filter(Entry.id==jex.value).first()
             RestoreFile(job,restore_me)
+            removeEntryAmendment(job,restore_me.id)
     NewJob( name="check_dups", num_files=0, wait_for=None, jex=None, parent_job=None, desc="check for duplicate files" )
     FinishJob(job, f"Finished restoring selected file(s)")
     return
@@ -2707,7 +2742,6 @@ def ScheduledJobs():
         created_jobs=True
     return created_jobs
 
-
 ####################################################################################################################################
 # MAIN - start with validation, then grab any jobs in the DB to process, then
 # go into waiting on a socket to be woken up (and then if woken, back into HandleJobs()

path.py

@@ -42,50 +42,3 @@ class Path(db.Model):
 
     def __repr__(self):
         return f"<id: {self.id}, path_prefix: {self.path_prefix}, num_files={self.num_files}, type={self.type}>"
-
-
-################################################################################
-# Class describing PathDetail (quick connvenence class for MovePathDetails())
-################################################################################
-class PathDetail(PA):
-    """Class describing details of a Path [internal class used in MovePathDetais()]"""
-
-    def __init__(self,ptype,path):
-        """Initialisation function for PathDetail class
-
-        Args:
-            id (int): database id of row in PathDetail table / primary key
-            ptype (int): database id of row in PathType table / foreign key
-        """
-
-        self.type:int=ptype
-        self.path:str=path
-        # construct icon_url based on type of storage path (icons.svg contains icons for each)
-        self.icon_url:str=url_for("internal", filename="icons.svg") + "#" + ICON[self.type]
-
-################################################################################
-# helper function to find path details for move destinations - used in html
-# for move DBox to show potential storage paths to move files into
-################################################################################
-def MovePathDetails():
-    """helper function to find path details for move destinations
-
-       used in html/javascript for move Dialog Box to show potential storage paths to move files into
-
-       Args:
-           None
-
-       Returns:
-           ret (List[PathDetail]): a list of Path Details for where files can be moved
-
-    """
-    ret=[]
-    sps=Path.query.join(PathType).filter(PathType.name=="Storage").all()
-    for p in sps:
-        obj = PathDetail( ptype="Storage", path=p.path_prefix.replace("static/Storage/","") )
-        ret.append( obj )
-    ips=Path.query.join(PathType).filter(PathType.name=="Import").all()
-    for p in ips:
-        obj = PathDetail( ptype="Import", path=p.path_prefix.replace("static/Import/","") )
-        ret.append( obj )
-    return ret

person.py

@@ -3,8 +3,9 @@ from flask_wtf import FlaskForm
 from flask import request, render_template, redirect, url_for, make_response, jsonify
 from main import db, app, ma
 from settings import Settings, AIModel
-from sqlalchemy import Sequence, func
+from sqlalchemy import Sequence, func, select
 from sqlalchemy.exc import SQLAlchemyError
+from sqlalchemy.orm import joinedload
 from flask_login import login_required, current_user
 from werkzeug.utils import secure_filename
 from shared import GenFace, GenThumb, PA
@@ -114,7 +115,7 @@ def AddRefimgToPerson( filename, person ):
         SetFELog( f"<b>Failed to add Refimg:</b>&nbsp;{e.orig}", "danger" )
     except Exception as e:
         SetFELog( f"<b>Failed to modify Refimg:</b>&nbsp;{e}", "danger" )
-    return
+    return refimg
 
 ################################################################################
 # TempRefimgFile: helper function that takes data POST'd (from dialog box to
@@ -182,9 +183,12 @@ def match_with_create_person():
     p = Person( tag=request.form["tag"], surname=request.form["surname"], firstname=request.form["firstname"] )
     # add this fname (of temp refimg) to person
     fname=TempRefimgFile( request.form['refimg_data'], p.tag )
-    AddRefimgToPerson( fname, p )
+    r=AddRefimgToPerson( fname, p )
     SetFELog( f"Created person: {p.tag}" )
-    return make_response( jsonify( who=p.tag, distance='0.0' ) )
+    refimg_schema=RefimgSchema(many=False)
+    r_data=refimg_schema.dump(r)
+
+    return make_response( jsonify( refimg=r_data, who=p.tag, distance='0.0' ) )
 
 ################################################################################
 # /person/<id> -> GET/POST(save or delete) -> shows/edits/delets a single person
@@ -267,7 +271,7 @@ def add_refimg():
     except Exception as e:
         SetFELog( f"<b>Failed to load reference image:</b>&nbsp;{e}", "danger" )
 
-    AddRefimgToPerson( fname, person )
+    r=AddRefimgToPerson( fname, person )
     return redirect( url_for( 'person', id=person.id) )
 
 ################################################################################
@@ -289,6 +293,29 @@ def find_persons(who):
 
     return make_response( resp )
 
+class FaceRefimgLinkSchema(ma.SQLAlchemyAutoSchema):
+    class Meta: model = FaceRefimgLink
+    face_distance = ma.auto_field()  # Explicitly include face_distance
+    load_instance = True
+
+class PersonSchema(ma.SQLAlchemyAutoSchema):
+    class Meta: model=Person
+    load_instance = True
+
+class RefimgSchema(ma.SQLAlchemyAutoSchema):
+    class Meta:
+        model = Refimg
+        exclude = ('face',)
+    load_instance = True
+    person = ma.Nested(PersonSchema)
+
+class FaceSchema(ma.SQLAlchemyAutoSchema):
+    class Meta:
+        model=Face
+        exclude = ('face',)
+    load_instance = True
+    refimg = ma.Nested(RefimgSchema,allow_none=True)
+    refimg_lnk = ma.Nested(FaceRefimgLinkSchema,allow_none=True)
 
 ################################################################################
 # /add_refimg_to_person/ -> POST 
@@ -296,12 +323,19 @@ def find_persons(who):
 @app.route("/add_refimg_to_person", methods=["POST"])
 @login_required
 def add_refimg_to_person():
-    f = Face.query.get( request.form['face_id'] )
-    p = Person.query.get( request.form['person_id'] )
+    stmt = select(Face).options( joinedload(Face.refimg_lnk) ).where(Face.id == request.form['face_id'])
+    f=db.session.execute(stmt).scalars().first()
+    stmt = select(Person).options( joinedload(Person.refimg) ).where(Person.id == request.form['person_id'])
+    p=db.session.execute(stmt).scalars().first()
 
     # add this fname (of temp refimg) to person
     fname=TempRefimgFile( request.form['refimg_data'], p.tag )
-    AddRefimgToPerson( fname, p )
+    r=AddRefimgToPerson( fname, p )
+
+    # connect the refimg to the face in the db, now we have added this refimg to the person
+    frl=FaceRefimgLink( face_id=f.id, refimg_id=r.id, face_distance=0 )
+    db.session.add(frl)
+    db.session.commit()
 
     if request.form['search'] == "true":
         jex=[]
@@ -316,7 +350,12 @@ def add_refimg_to_person():
         jex.append( JobExtra( name=f"path_type", value=str(ptype.id) ) )
         job=NewJob( name="run_ai_on_path", num_files=0, wait_for=None, jex=jex, desc="Look for face(s) in storage path(s)" )
 
-    return make_response( jsonify( who=p.tag, distance='0.0' ) )
+    refimg_schema=RefimgSchema(many=False)
+    r_data=refimg_schema.dump(r)
+    frl_schema=FaceRefimgLinkSchema(many=False)
+    frl_data=refimg_schema.dump(r)
+
+    return make_response( jsonify( refimg=r_data, frl=frl_data ) )
 
 ################################################################################
 # /add_force_match_override -> POST
@@ -346,7 +385,9 @@ def add_force_match_override():
     NewJob( "metadata", num_files=0, wait_for=None, jex=jex, desc="create metadata for adding forced match" )
 
     # this will reply to the Ajax / POST, and cause the page to re-draw with new face override to person_tag
-    return make_response( jsonify( person_tag=p.tag ) )
+    person_schema = PersonSchema(many=False)
+    p_data = person_schema.dump(p)
+    return make_response( jsonify( person=p_data ) )
  
 ################################################################################
 # /remove_force_match_override -> POST
@@ -397,6 +438,11 @@ def remove_no_match_override():
     return make_response( jsonify( face_id=face_id  ) )
 
 
+class FaceOverrideTypeSchema(ma.SQLAlchemyAutoSchema):
+    class Meta:
+        model = FaceOverrideType
+        load_instance = True
+
 ################################################################################
 # /add_no_match_override -> POST
 ################################################################################
@@ -424,5 +470,6 @@ def add_no_match_override():
     # dont do status update here, the F/E is in the middle of a dbox, just send metadata through to the B/E
     NewJob( "metadata", num_files=0, wait_for=None, jex=jex, desc="create metadata for adding forced non-match" )
 
-    # this will reply to the Ajax / POST, and cause the page to re-draw with new face override to person_tag
-    return make_response( jsonify( type=t.name  ) )
+    fot_schema = FaceOverrideTypeSchema(many=False)
+    t_data=fot_schema.dump(t)
+    return make_response( jsonify( type_id=t.id, type=t_data ) )

states.py

@@ -27,10 +27,9 @@ class PA_UserState(db.Model):
     folders = db.Column(db.Boolean, unique=False, nullable=False )
     root = db.Column(db.String, unique=False, nullable=False )
     cwd = db.Column(db.String, unique=False, nullable=False )
-    search_term = db.Column(db.String, unique=False, nullable=False )
 
     def __repr__(self):
-        return f"<pa_user_dn: {self.pa_user_dn}, path_type: {self.path_type}, noo: {self.noo}, grouping: {self.grouping}, how_many: {self.how_many}, size: {self.size}, folders: {self.folders}, root: {self.root}, cwd: {self.cwd}, search_term: {self.orig_search_term}>"
+        return f"<pa_user_dn: {self.pa_user_dn}, path_type: {self.path_type}, noo: {self.noo}, grouping: {self.grouping}, how_many: {self.how_many}, size: {self.size}, folders: {self.folders}, root: {self.root}, cwd: {self.cwd}>"
 
 
 ################################################################################

tables.sql

@@ -1,4 +1,4 @@
-ALTER DATABASE pa SET TIMEZONE TO 'aUSTRALIA/vICTORIA'; 
+ALTER DATABASE pa SET TIMEZONE TO 'Australia/Victoria'; 
 
 CREATE SEQUENCE pa_user_id_seq;
 CREATE SEQUENCE pa_user_state_id_seq;
@@ -21,8 +21,8 @@ CREATE SEQUENCE query_id_seq;
 
 -- these are hard-coded at present, not sure I can reflexively find models from API?
 CREATE TABLE ai_model ( id INTEGER, name VARCHAR(24), description VARCHAR(80), CONSTRAINT pk_ai_model PRIMARY KEY(id) );
-INSERT INTO ai_model VALUES ( 1, 'HOG', 'NORMAL' );
-INSERT INTO ai_model VALUES ( 2, 'CNN', 'MORE ACCURATE / MUCH SLOWER' );
+INSERT INTO ai_model VALUES ( 1, 'hog', 'normal' );
+INSERT INTO ai_model VALUES ( 2, 'cnn', 'more accurate / much slower' );
 
 CREATE TABLE settings( 
     id INTEGER, 
@@ -50,11 +50,10 @@ CREATE TABLE pa_user(
     default_storage_folders BOOLEAN,
     CONSTRAINT pk_pa_user_id PRIMARY KEY(id) );
 
--- this is totally not 3rd normal form, but when I made it that, it was so complex, it was stupid
--- so for the little data here, I'm deliberately doing a redundant data structure
+-- FIXME: NEED TO RETHINK THIS, not sure this even needs to be in the DB
 CREATE TABLE pa_user_state ( id INTEGER, pa_user_dn VARCHAR(128), path_type VARCHAR(16), 
     noo VARCHAR(16), grouping VARCHAR(16), how_many INTEGER, size INTEGER, folders BOOLEAN,
-    root VARCHAR, cwd VARCHAR, search_term VARCHAR,
+    root VARCHAR, cwd VARCHAR,
     CONSTRAINT fk_pa_user_dn FOREIGN KEY (pa_user_dn) REFERENCES pa_user(dn),
     CONSTRAINT pk_pa_user_states_id PRIMARY KEY(id ) );
 
@@ -123,10 +122,10 @@ CREATE TABLE face_refimg_link( face_id INTEGER, refimg_id INTEGER, face_distance
     CONSTRAINT fk_frl_refimg_id FOREIGN KEY (refimg_id) REFERENCES refimg(id) );
 
 CREATE TABLE face_override_type ( id INTEGER, name VARCHAR UNIQUE, CONSTRAINT pk_face_override_type_id PRIMARY KEY(id) );
-INSERT INTO face_override_type VALUES ( (SELECT NEXTVAL('face_override_type_id_seq')), 'mANUAL MATCH TO EXISTING PERSON' );
-INSERT INTO face_override_type VALUES ( (SELECT NEXTVAL('face_override_type_id_seq')), 'nOT A FACE' );
-INSERT INTO face_override_type VALUES ( (SELECT NEXTVAL('face_override_type_id_seq')), 'tOO YOUNG' );
-INSERT INTO face_override_type VALUES ( (SELECT NEXTVAL('face_override_type_id_seq')), 'iGNORE FACE' );
+INSERT INTO face_override_type VALUES ( (SELECT NEXTVAL('face_override_type_id_seq')), 'Manual match to existing person' );
+INSERT INTO face_override_type VALUES ( (SELECT NEXTVAL('face_override_type_id_seq')), 'Not a face' );
+INSERT INTO face_override_type VALUES ( (SELECT NEXTVAL('face_override_type_id_seq')), 'Too young' );
+INSERT INTO face_override_type VALUES ( (SELECT NEXTVAL('face_override_type_id_seq')), 'Ignore face' );
 
 -- keep non-redundant FACE because, when we rebuild data we may have a null FACE_ID, but still want to connect to this override
 -- from a previous AI pass... (would happen if we delete a file and then reimport/scan it), OR, more likely we change (say) a threshold, etc.
@@ -166,20 +165,36 @@ CREATE TABLE joblog ( id INTEGER, job_id INTEGER, log_date TIMESTAMPTZ, log VARC
     CONSTRAINT pk_jl_id PRIMARY KEY(id), CONSTRAINT fk_jl_job_id FOREIGN KEY(job_id) REFERENCES job(id) );
 
 CREATE TABLE pa_job_manager_fe_message ( id INTEGER, job_id INTEGER, level VARCHAR(16), message VARCHAR(8192), persistent BOOLEAN, cant_close BOOLEAN,
-    CONSTRAINT pa_job_manager_fe_acks_id PRIMARY KEY(id),
+    CONSTRAINT pk_pa_job_manager_fe_acks_id PRIMARY KEY(id),
     CONSTRAINT fk_pa_job_manager_fe_message_job_id FOREIGN KEY(job_id) REFERENCES job(id) );
 
+CREATE TABLE amendment_type ( id INTEGER, job_name VARCHAR(64), which VARCHAR(8), what VARCHAR(32), colour VARCHAR(32),
+    CONSTRAINT pk_amendment_type_id PRIMARY KEY(id) );
+INSERT INTO amendment_type ( id, job_name, which, what, colour ) VALUES ( 1, 'delete_files', 'icon', 'trash', 'var(--bs-danger)' );
+INSERT INTO amendment_type ( id, job_name, which, what, colour ) VALUES ( 2, 'restore_files', 'icon', 'trash', 'var(--bs-success)' );
+INSERT INTO amendment_type ( id, job_name, which, what, colour ) VALUES ( 3, 'transform_image:90', 'img', 'rot90.png', '#009EFF' );
+INSERT INTO amendment_type ( id, job_name, which, what, colour ) VALUES ( 4, 'transform_image:180', 'img', 'rot180.png', '#009EFF' );
+INSERT INTO amendment_type ( id, job_name, which, what, colour ) VALUES ( 5, 'transform_image:270', 'img', 'rot270.png', '#009EFF' );
+INSERT INTO amendment_type ( id, job_name, which, what, colour ) VALUES ( 6, 'transform_image:fliph', 'icon', 'flip_h', '#009EFF' );
+INSERT INTO amendment_type ( id, job_name, which, what, colour ) VALUES ( 7, 'transform_image:flipv', 'icon', 'flip_v', '#009EFF' );
+INSERT INTO amendment_type ( id, job_name, which, what, colour ) VALUES ( 8, 'move_files', 'icon', 'folder_plus', 'var(--bs-primary)' );
+
+CREATE TABLE entry_amendment ( amend_type INTEGER, eid INTEGER, job_id INTEGER,
+    CONSTRAINT pk_entry_amendment_eid_job_id PRIMARY KEY(eid,job_id),
+    CONSTRAINT fk_entry_amendment_amendment_type FOREIGN KEY(amend_type) REFERENCES amendment_type(id),
+    CONSTRAINT fk_entry_amendment_job_id FOREIGN KEY(job_id) REFERENCES job(id) );
+
 -- default data for types of paths
-INSERT INTO path_type VALUES ( (SELECT NEXTVAL('path_type_id_seq')), 'iMPORT' );
-INSERT INTO path_type VALUES ( (SELECT NEXTVAL('path_type_id_seq')), 'sTORAGE' );
-INSERT INTO path_type VALUES ( (SELECT NEXTVAL('path_type_id_seq')), 'bIN' );
-INSERT INTO path_type VALUES ( (SELECT NEXTVAL('path_type_id_seq')), 'mETADATA' );
+INSERT INTO path_type VALUES ( (SELECT NEXTVAL('path_type_id_seq')), 'Import' );
+INSERT INTO path_type VALUES ( (SELECT NEXTVAL('path_type_id_seq')), 'Storage' );
+INSERT INTO path_type VALUES ( (SELECT NEXTVAL('path_type_id_seq')), 'Bin' );
+INSERT INTO path_type VALUES ( (SELECT NEXTVAL('path_type_id_seq')), 'Metadata' );
 
 -- default data for types of files
-INSERT INTO file_type VALUES ( (SELECT NEXTVAL('file_type_id_seq')), 'iMAGE' );
-INSERT INTO file_type VALUES ( (SELECT NEXTVAL('file_type_id_seq')), 'vIDEO' );
-INSERT INTO file_type VALUES ( (SELECT NEXTVAL('file_type_id_seq')), 'dIRECTORY' );
-INSERT INTO file_type VALUES ( (SELECT NEXTVAL('file_type_id_seq')), 'uNKNOWN' );
+INSERT INTO file_type VALUES ( (SELECT NEXTVAL('file_type_id_seq')), 'Image' );
+INSERT INTO file_type VALUES ( (SELECT NEXTVAL('file_type_id_seq')), 'Video' );
+INSERT INTO file_type VALUES ( (SELECT NEXTVAL('file_type_id_seq')), 'Directory' );
+INSERT INTO file_type VALUES ( (SELECT NEXTVAL('file_type_id_seq')), 'Unknown' );
 
 -- fake data only for making testing easier
 --INSERT INTO person VALUES ( (SELECT NEXTVAL('person_id_seq')), 'dad', 'Damien', 'De Paoli' );
@@ -187,7 +202,7 @@ INSERT INTO file_type VALUES ( (SELECT NEXTVAL('file_type_id_seq')), 'uNKNOWN' )
 --INSERT INTO person VALUES ( (SELECT NEXTVAL('person_id_seq')), 'cam', 'Cameron', 'De Paoli' );
 --INSERT INTO person VALUES ( (SELECT NEXTVAL('person_id_seq')), 'mich', 'Michelle', 'De Paoli' );
 -- DEV(ddp):
-INSERT INTO settings ( ID, BASE_PATH, IMPORT_PATH, STORAGE_PATH, RECYCLE_BIN_PATH, METADATA_PATH, AUTO_ROTATE, DEFAULT_REFIMG_MODEL, DEFAULT_SCAN_MODEL, DEFAULT_THRESHOLD, FACE_SIZE_LIMIT, SCHEDULED_IMPORT_SCAN, SCHEDULED_STORAGE_SCAN, SCHEDULED_BIN_CLEANUP, BIN_CLEANUP_FILE_AGE, JOB_ARCHIVE_AGE ) VALUES ( (SELECT NEXTVAL('settings_id_seq')), '/HOME/DDP/SRC/PHOTOASSISTANT/', 'IMAGES_TO_PROCESS/', 'PHOTOS/', '.PA_BIN/', '.PA_METADATA/', TRUE, 1, 1, '0.55', 43, 1, 1, 7, 30, 3 );
+INSERT INTO settings ( id, base_path, import_path, storage_path, recycle_bin_path, metadata_path, auto_rotate, default_refimg_model, default_scan_model, default_threshold, face_size_limit, scheduled_import_scan, scheduled_storage_scan, scheduled_bin_cleanup, bin_cleanup_file_age, job_archive_age ) VALUES ( (SELECT NEXTVAL('settings_id_seq')), '/home/ddp/src/photoassistant/', 'images_to_process/', 'photos/', '.pa_bin/', '.pa_metadata/', true, 1, 1, '0.55', 43, 1, 1, 7, 30, 3 );
 -- DEV(cam):
 --INSERT INTO settings ( id, base_path, import_path, storage_path, recycle_bin_path, metadata_path, auto_rotate, default_refimg_model, default_scan_model, default_threshold, face_size_limit, scheduled_import_scan, scheduled_storage_scan, scheduled_bin_cleanup, bin_cleanup_file_age, job_archive_age ) VALUES ( (select nextval('SETTINGS_ID_SEQ')), 'c:/Users/cam/Desktop/code/python/photoassistant/', 'c:\images_to_process', 'photos/', '.pa_bin/', '.pa_metadata/', TRUE, 1, 1, '0.55', 43, 1, 1, 7, 30, 3 );
 -- PROD:

templates/base.html

@@ -136,7 +136,7 @@
 
 {% if not InDBox %}
 	{%block script_content %}{% endblock script_content %}
-        <div id="status_container" class="position-fixed top-0 end-0 p-0 my-5" "z-index: 11"> </div>
+        <div id="status_container" class="position-fixed top-0 end-0 p-0 my-5" style="z-index: 9999"> </div>
         <!-- CheckForJobs(), will see if there are any messages/jobs and keep doing this until there are 0 more and then stop -->
         <script>
             $(document).ready(function() { CheckForJobs() } )

templates/file_list.html

@@ -1,5 +1,5 @@
 {% extends "base.html" %} {% block main_content %}
-    <script src="{{ url_for( 'internal', filename='js/files_support.js')}}"></script>
+    <script src="{{ url_for( 'internal', filename='js/files_support.js')}}?v={{js_vers['fs']}}"></script>
 
     <div class="container-fluid">
         <h3 class="offset-2">{{page_title}}</h3>
@@ -9,28 +9,28 @@
                     {{CreateSelect( "how_many", OPT.how_many|string, ["10", "25", "50", "75", "100", "150", "200", "500"], "changeOPT(getPageFileList); return false", "rounded-end py-1 my-1" )|safe }}
                     <div class="mb-1 col my-auto d-flex justify-content-center">
                         <button id="prev" name="prev" class="prev sm-txt btn btn-outline-secondary" onClick="prevPage(getPageFileList)">
-                            <svg width="16" height="16" fill="currentColor"><use xlink:href="{{url_for('internal', filename='icons.svg')}}#prev"/></svg>
+                            <svg width="16" height="16" fill="currentColor"><use xlink:href="{{url_for('internal', filename='icons.svg')}}?v={{js_vers['ic']}}#prev"/></svg>
                         </button>
                         <span class="how_many_text sm-txt my-auto">&nbsp;{{OPT.how_many}} files&nbsp;</span>
                         <button id="next" name="next" class="next sm-txt btn btn-outline-secondary" onClick="nextPage(getPageFileList)">
-                            <svg width="16" height="16" fill="currentColor"><use xlink:href="{{url_for('internal', filename='icons.svg')}}#next"/></svg>
+                            <svg width="16" height="16" fill="currentColor"><use xlink:href="{{url_for('internal', filename='icons.svg')}}?v={{js_vers['ic']}}#next"/></svg>
                         </button>
                     </div class="col...">
                 </div class="input-group...">
             </div class="col col-auto">
     </div class="container">
-    <div id="file_list_div" class="container-fluid">
+    <div id="file_list_div" class="container-fluid pt-2">
     </div class="container">
     <div class="container-fluid">
         <input type="hidden" name="cwd" id="cwd" value="{{OPT.cwd}}">
         <div class="row">
             <div class="col my-auto d-flex justify-content-center">
                 <button aria-label="prev" id="prev" name="prev" class="prev sm-txt btn btn-outline-secondary disabled" onClick="prevPage(getPageFileList)" disabled>
-                    <svg width="16" height="16" fill="currentColor"><use xlink:href="{{url_for('internal', filename='icons.svg')}}#prev"/></svg>
+                    <svg width="16" height="16" fill="currentColor"><use xlink:href="{{url_for('internal', filename='icons.svg')}}?v={{js_vers['ic']}}#prev"/></svg>
                 </button>
                 <span class="how_many_text sm-txt my-auto">&nbsp;{{OPT.how_many}} files&nbsp;</span>
                 <button aria-label="next" id="next" name="next" class="next sm-txt btn btn-outline-secondary" onClick="nextPage(getPageFileList)">
-                    <svg width="16" height="16" fill="currentColor"><use xlink:href="{{url_for('internal', filename='icons.svg')}}#next"/></svg>
+                    <svg width="16" height="16" fill="currentColor"><use xlink:href="{{url_for('internal', filename='icons.svg')}}?v={{js_vers['ic']}}#next"/></svg>
                 </button>
             </div class="col my-auto">                                                                                                                                                                                   </div class="row">
     </div class="container-fluid">
@@ -40,6 +40,9 @@
     // this is the list of entry ids for the images for ALL matches for this query
     var entryList={{query_data.entry_list}}
     var OPT = {{ OPT.to_dict()|tojson }};
+    // set from query data and stored in OPT for convenience.  It can be 0 -
+    // this implies no content in the Path at all
+    OPT.root_eid = {{ query_data.root_eid }};
 
     // pageList is just those entries shown on this page from the full entryList
     var pageList=[]

templates/files.html

@@ -1,45 +1,28 @@
 {% extends "base.html" %}
 {% block main_content %}
 
-    <script src="{{ url_for( 'internal', filename='js/files_support.js')}}"></script>
-    <script src="{{ url_for( 'internal', filename='js/files_transform.js')}}"></script>
-
-    <script>
-        // FIXME: used by viewer code - should probably get rid of this?
-        var fullscreen=false;
-        document.fake_shift=0
-        document.fake_ctrl=0
-        var move_paths=[]
-        {% for p in move_paths %}
-            p = new Object()
-            p.type = '{{p.type}}'
-            p.path = '{{p.path}}'
-            p.icon_url = '{{p.icon_url}}'
-            move_paths.push(p)
-        {% endfor %}
-
-        // GLOBALS
-        // this is which eid we are viewing an image/video (when we dbl-click & then next/prev)
-        document.viewing_eid=null;
-        document.viewing=null;
-
-        var OPT = {{ OPT.to_dict()|tojson }};
-        OPT.root_eid = {{ query_data.root_eid }};
-
-        // this is the list of entry ids for the images for ALL matches for this query 
-        var entryList={{query_data.entry_list}}
-
-        // pageList is just those entries shown on this page from the full entryList
-        var pageList=[]
-        // force pageList to set pageList for & render the first page
-        getPage(1,getPageFigures)
-
-        function changeSize() 
-        {
-            sz=$('input[name="size"]:checked').val();
-            $('.thumb').prop('height',sz);
+    <style>
+        @media (max-width: 576px) {
+            #la, #ra {
+                padding: 5% !important;
+            }
         }
-    </script>
+        .norm-txt { font-size: 1.0rem }
+        .form-check-input:checked {
+            background-color: #39C0ED;
+            border-color: #CFF4FC;
+        }
+        .form-switch .form-check-input {
+          background-image: url("data:image/svg+xml,%3csvg xmlns='http://www.w3.org/2000/svg' viewBox='-4 -4 8 8'%3e%3ccircle r='3' fill='%2339C0ED'/%3e%3c/svg%3e");
+        }
+        .form-switch .form-check-input:focus {
+          background-image: url("data:image/svg+xml,%3csvg xmlns='http://www.w3.org/2000/svg' viewBox='-4 -4 8 8'%3e%3ccircle r='3' fill='%23CFF4FC'/%3e%3c/svg%3e");
+        }
+        #tst90:hover,#tst90:focus { filter: invert(73%) sepia(27%) saturate(3970%) hue-rotate(146deg) brightness(94%) contrast(100%); }
+    </style>
+    <script src="{{ url_for( 'internal', filename='js/files_transform.js')}}?v={{ js_vers['ft'] }}"></script>
+    <script src="{{ url_for( 'internal', filename='js/files_support.js')}}?v={{ js_vers['fs'] }}"></script>
+    <script src="{{ url_for( 'internal', filename='js/view_support.js')}}?v={{ js_vers['vs'] }}"></script>
 
     <div id="files_div">
         <div class="container-fluid">
@@ -48,13 +31,13 @@
                     <div class="my-auto col col-auto">
                         <span class="alert alert-primary py-2">
                         {% if "files_ip" in request.url %}
-                            <svg width="20" height="20" fill="currentColor"><use xlink:href="{{url_for('internal', filename='icons.svg')}}#import"/></svg>
+                            <svg width="20" height="20" fill="currentColor"><use xlink:href="{{url_for('internal', filename='icons.svg')}}?v={{js_vers['ic']}}#import"/></svg>
                           {% set tmp_path=OPT.cwd | replace( "static/Import", "" ) + "/" %}
                         {% elif "files_sp" in request.url %}
-                            <svg width="20" height="20" fill="currentColor"><use xlink:href="{{url_for('internal', filename='icons.svg')}}#db"/></svg>
+                            <svg width="20" height="20" fill="currentColor"><use xlink:href="{{url_for('internal', filename='icons.svg')}}?v={{js_vers['ic']}}#db"/></svg>
                           {% set tmp_path=OPT.cwd | replace( "static/Storage", "" ) + "/" %}
                         {% elif "files_rbp" in request.url %}
-                            <svg width="20" height="20" fill="currentColor"><use xlink:href="{{url_for('internal', filename='icons.svg')}}#trash"/></svg>
+                            <svg width="20" height="20" fill="currentColor"><use xlink:href="{{url_for('internal', filename='icons.svg')}}?v={{js_vers['ic']}}#trash"/></svg>
                           {% set tmp_path=OPT.cwd | replace( "static/Bin", "" ) + "/" %}
                         {% endif %}
                         {{tmp_path}}</span>
@@ -77,27 +60,24 @@
                     <div class="col col-auto my-auto">
                         <span class="alert alert-primary p-2">Searched for: '{{search_term}}'</span>
                     </div class="col my-auto">
-                    <script>
-                        $('#folders').prop('disabled', 'disabled').removeClass('border-info').addClass('border-secondary').removeClass('text-info').addClass('text-secondary');
-                    </script>
                 {% endif %}
                 <div class="col flex-grow-1 my-auto d-flex justify-content-center w-100">
                     <button aria-label="prev" id="prev" name="prev" class="prev sm-txt btn btn-outline-secondary disabled" onClick="prevPage(getPageFigures)" disabled>
-                        <svg width="16" height="16" fill="currentColor"><use xlink:href="{{url_for('internal', filename='icons.svg')}}#prev"/></svg>
+                        <svg width="16" height="16" fill="currentColor"><use xlink:href="{{url_for('internal', filename='icons.svg')}}?v={{js_vers['ic']}}#prev"/></svg>
                     </button>
                     <span class="how_many_text sm-txt my-auto">&nbsp;{{OPT.how_many}} files&nbsp;</span>
                     <button aria-label="next" id="next" name="next" class="next sm-txt btn btn-outline-secondary" onClick="nextPage(getPageFigures)">
-                        <svg width="16" height="16" fill="currentColor"><use xlink:href="{{url_for('internal', filename='icons.svg')}}#next"/></svg>
+                        <svg width="16" height="16" fill="currentColor"><use xlink:href="{{url_for('internal', filename='icons.svg')}}?v={{js_vers['ic']}}#next"/></svg>
                     </button>
-                    <button aria-label="move" id="move" disabled name="move" class="sm-txt btn btn-outline-primary ms-4" onClick="MoveDBox(move_paths,'{{url_for('internal', filename='icons.svg')}}'); return false;">
-                        <svg width="16" height="16" fill="currentColor"><use xlink:href="{{url_for('internal', filename='icons.svg')}}#folder_plus"/></svg>
+                    <button aria-label="move" id="move" disabled name="move" class="sm-txt btn btn-outline-primary ms-4" onClick="MoveDBox(move_paths); return false;">
+                        <svg width="16" height="16" fill="currentColor"><use xlink:href="{{url_for('internal', filename='icons.svg')}}?v={{js_vers['ic']}}#folder_plus"/></svg>
                     </button>
                     {% if "files_rbp" in request.url %}
                         <button aria-label="delete" id="del" disabled name="del" class="sm-txt btn btn-outline-success mx-1" onClick="DelDBox('Restore'); return false;">
-                            <svg width="16" height="16" fill="currentColor"><use xlink:href="{{url_for('internal', filename='icons.svg')}}#trash-fill"/></svg>
+                            <svg width="16" height="16" fill="currentColor"><use xlink:href="{{url_for('internal', filename='icons.svg')}}?v={{js_vers['ic']}}#trash-fill"/></svg>
                     {% else %}
                         <button aria-label="delete" id="del" disabled name="del" class="sm-txt btn btn-outline-danger mx-1" onClick="DelDBox('Delete'); return false;">
-                            <svg width="16" height="16" fill="currentColor"><use xlink:href="{{url_for('internal', filename='icons.svg')}}#trash-fill"/></svg>
+                            <svg width="16" height="16" fill="currentColor"><use xlink:href="{{url_for('internal', filename='icons.svg')}}?v={{js_vers['ic']}}#trash-fill"/></svg>
                     {% endif %}
                     </button>
                     <button style="visibility:hidden" class="btn btn-outline-secondary" aria-label="shift-key" id="shift-key" onclick="document.fake_shift=1-document.fake_shift; event.stopPropagation(); return false">shift</button>
@@ -129,392 +109,229 @@
             <div class="row">
                 <div class="col my-auto d-flex justify-content-center">
                     <button aria-label="prev" id="prev" name="prev" class="prev sm-txt btn btn-outline-secondary disabled" onClick="prevPage(getPageFigures)" disabled>
-                        <svg width="16" height="16" fill="currentColor"><use xlink:href="{{url_for('internal', filename='icons.svg')}}#prev"/></svg>
+                        <svg width="16" height="16" fill="currentColor"><use xlink:href="{{url_for('internal', filename='icons.svg')}}?v={{js_vers['ic']}}#prev"/></svg>
                     </button>
                     <span class="how_many_text sm-txt my-auto">&nbsp;{{OPT.how_many}} files&nbsp;</span>
                     <button aria-label="next" id="next" name="next" class="next sm-txt btn btn-outline-secondary" onClick="nextPage(getPageFigures)">
-                        <svg width="16" height="16" fill="currentColor"><use xlink:href="{{url_for('internal', filename='icons.svg')}}#next"/></svg>
+                        <svg width="16" height="16" fill="currentColor"><use xlink:href="{{url_for('internal', filename='icons.svg')}}?v={{js_vers['ic']}}#next"/></svg>
                     </button>
                 </div class="col my-auto">
             </div class="row">
         </div class="container-fluid">
     </div id="files_div">
     <div id="viewer_div" class="d-none">
+        <div id="viewer" class="container-fluid">
+            <div class="row flex-nowrap">
+                <!-- Left Buttons Column -->
+                <div class="col-auto d-flex flex-column min-width-0">
+                    <!-- Up Button (Small) -->
+                    <button title="Back to list" class="btn btn-outline-info btn-sm p-1 mb-1" onclick="goOutOfViewer()">
+                        <svg width="16" height="16" fill="currentColor">
+                            <use xlink:href="{{url_for('internal', filename='icons.svg')}}?v={{js_vers['ic']}}#back"></use>
+                        </svg>
+                    </button>
+                    <!-- Left Button (large/flex-grow-1) -->
+                    <button title="Show previous image" class="btn btn-outline-info px-2 flex-grow-1 overflow-hidden"
+                        style="padding: 10%" id="la" onClick="prevImageInViewer()">
+                        <svg width="16" height="16" fill="currentColor">
+                            <use xlink:href="{{url_for('internal', filename='icons.svg')}}?v={{js_vers['ic']}}#prev"/></svg>
+                    </button>
+                </div>
+                <figure style="position: relative;" class="col col-auto border border-info rounded m-0 p-1" id="figure">
+                    <canvas id="canvas"></canvas>
+                    <!-- next 4 are placeholders and called on during amendments only in viewer code -->
+                    <img id="throbber" src="{{url_for('internal', filename='throbber.gif')}}?v={{js_vers[th]}}" style="display:none;height:96px"
+                        class="position-absolute top-50 start-50 translate-middle">
+                    <img id="white-circle" src="{{url_for('internal', filename='white-circle.png')}}?v={{js_vers[th]}}" style="display:none;height:72px"
+                        class="position-absolute top-50 start-50 translate-middle">
+                    <img id="inside-img" style="display:none;height:64px" class="position-absolute top-50 start-50 translate-middle">
+                    <svg id="inside-icon" style="display:none;height:64px" class="position-absolute top-50 start-50 translate-middle">
+                        <use xlink:href="{{url_for('internal', filename='icons.svg')}}?v={{js_vers['ic']}}#flip_v">
+                        </use></svg>
+                    <script>
+                        var im=new Image();
+                        im.onload=DrawImg
+                        var context = canvas.getContext('2d')
+                    </script>
+                    <figcaption id="img-cap" class="figure-caption text-center text-wrap text-break">
+                        <span id="fname_i"></span></figcaption>
+                </figure>
+                <div id="video_div" class="col col-auto">
+                    <video id="video" class="col col-auto" controls>
+                        <source id="videoSource" src="" type="video/mp4">
+                        Your browser does not support the video tag.
+                    </video>
+                    <figcaption id="vid-cap" class="figure-caption text-center text-wrap text-break">
+                        <span id="fname_v"></span></figcaption>
+                </div>
+                <!-- Right-hand Buttons Column -->
+                <div class="col-auto d-flex flex-column min-width-0">
+                    <!-- Up Button (Small) -->
+                    <button title="Back to list" class="btn btn-outline-info btn-sm p-1 mb-1" onclick="goOutOfViewer()">
+                        <svg width="16" height="16" fill="currentColor">
+                            <use xlink:href="{{url_for('internal', filename='icons.svg')}}?v={{js_vers['ic']}}#back"></use>
+                        </svg>
+                    </button>
+                    <!-- Right Button (large/flex-grow-1) -->
+                    <button title="Show next image" class="btn btn-outline-info px-2 flex-grow-1 overflow-hidden" 
+                        style="padding: 10%" id="ra" onClick="nextImageInViewer()">
+                        <svg width="16" height="16" fill="currentColor">
+                            <use xlink:href="{{url_for('internal', filename='icons.svg')}}?v={{js_vers['ic']}}#next"/></svg>
+                    </button>
+                </div>
+            </div class="row">
+            <div class="row">
+                {# this whole div, just takes up the same space as the left button and is hidden for alignment only #}
+                <div class="col-auto px-0">
+                    <button class="btn btn-outline-info px-2 invisible" disabled>
+                        <svg width="16" height="16" fill="currentColor"><use xlink:href="{{url_for('internal', filename='icons.svg')}}?v={{js_vers['ic']}}#next"/></svg>
+                    </button>
+                </div>
+                <span class="col-auto my-auto">Show:</span>
+                <div title="Toggle showing filename (hotkey: n)" class="d-flex form-check form-switch border border-info rounded col col-auto my-auto py-1 justify-content-center ps-5">
+                  <input class="form-check-input" type="checkbox" id="fname_toggle" onChange="$('.figure-caption').toggle()" checked>
+                  <label class="form-check-label ps-1" for="fname_toggle">Filename</label>
+                </div>
+                <div title="Toggle showing matched faces (hotkey: f)" class="d-flex form-check form-switch border border-info rounded col col-auto my-auto py-1 justify-content-center ps-5">
+                  <input class="form-check-input" type="checkbox" onChange="FaceToggle()" id="faces">
+                  <label class="form-check-label ps-1" for="faces">Faces</label>
+                </div>
+                <div title="Toggle showing 'distance' on matched faces (hotkey: d)" class="d-flex form-check form-switch border border-info rounded col col-auto my-auto py-1 justify-content-center ps-5">
+                  <input class="form-check-input" type="checkbox" onChange="DrawImg()" id="distance">
+                  <label class="form-check-label ps-1" for="distance">Distance</label>
+                </div>
+                <div title="Change the model used to detect faces" class="col col-auto my-auto">
+                    AI Model:
+                    {# can use 0 as default, it will be (re)set correctly in DrawImg() anyway #}
+                    {{CreateSelect( "model", 0, ["N/A", "normal", "slow/accurate"], "", "rounded norm-txt", [0,1,2])|safe }}
+                </div>
+                <div class="col col-auto pt-1">
+                    <button class="btn btn-outline-info p-1" title="Rotate by 90 degrees" onClick="Transform(90)">
+                        <img src="{{url_for('internal', filename='rot90.png')}}?v={{js_vers['r90']}}" width="32" height="32" onMouseOver="this.src='{{url_for('internal', filename='rot90-invert.png')}}'" 
+                    onMouseOut="this.src='{{url_for('internal', filename='rot90.png')}}?v={{js_vers['r90']}}'" />
+                    </button>
+                    <button class="btn btn-outline-info p-1" title="Rotate by 180 degrees" onClick="Transform(180)">
+                        <img src="{{url_for('internal', filename='rot180.png')}}?v={{js_vers['r180']}}" width="32" height="32" onMouseOver="this.src='{{url_for('internal', filename='rot180-invert.png')}}'"
+                    onMouseOut="this.src='{{url_for('internal', filename='rot180.png')}}?v={{js_vers['r180']}}'" />
+                    </button>
+                    <button class="btn btn-outline-info p-1" title="Rotate by 270 degrees" onClick="Transform(270)">
+                        <img src="{{url_for('internal', filename='rot270.png')}}?v={{js_vers['r270']}}" width="32" height="32" onMouseOver="this.src='{{url_for('internal', filename='rot270-invert.png')}}'"
+                    onMouseOut="this.src='{{url_for('internal', filename='rot270.png')}}?v={{js_vers['r270']}}'" />
+                    </button>
+                    <button class="btn btn-outline-info p-1" title="Flip horizontally" onClick="Transform('fliph')">
+                        <svg width="32" height="32" fill="currentColor"><use xlink:href="{{url_for('internal', filename='icons.svg')}}?v={{js_vers['ic']}}#flip_h"/></svg>
+                    </button>
+                    <button class="btn btn-outline-info p-1" title="Flip vertically" onClick="Transform('flipv')">
+                        <svg width="32" height="32" fill="currentColor"><use xlink:href="{{url_for('internal', filename='icons.svg')}}?v={{js_vers['ic']}}#flip_v"/></svg>
+                    </button>
+                    <button class="btn btn-outline-info p-1" title="View in Fullscreen mode (hotkey: F)" onClick="fullscreen=true; ViewImageOrVideo()">
+                        <svg width="32" height="32" fill="currentColor"><use xlink:href="{{url_for('internal', filename='icons.svg')}}?v={{js_vers['ic']}}#fullscreen"/></svg>
+                    </button>
+                    <button class="btn btn-outline-info p-1" title="Show logs relating to this filename (hotkey: l)" onClick="JoblogSearch()">
+                        <svg width="32" height="32" fill="currentColor"><use xlink:href="{{url_for('internal', filename='icons.svg')}}?v={{js_vers['ic']}}#log"/></svg>
+                    </button>
+                    <button class="btn btn-outline-info p-1" title="View Original" onClick="window.location='/'+document.viewing.FullPathOnFS">
+                        <svg width="32" height="32" fill="currentColor"><use xlink:href="{{url_for('internal', filename='icons.svg')}}?v={{js_vers['ic']}}#download"/></svg>
+                    </button>
+                    <button id="viewer_del" class="btn btn-outline-danger p-1" title="Delete (hotkey: Del)" onClick="DelDBox('Delete')">
+                        <svg id="viewer_bin" width="32" height="32" fill="currentColor"><use xlink:href="{{url_for('internal', filename='icons.svg')}}?v={{js_vers['ic']}}#trash"/></svg>
+                    </button>
+                </div>
+            </div class="row">
+        </div id="viewer">
+    </div id="viewer_div">
+{% endblock main_content %}
 
-<style>
-
-
-.norm-txt { font-size: 1.0rem }
-.form-check-input:checked {
-    background-color: #39C0ED;
-    border-color: #CFF4FC;
-}
-.form-switch .form-check-input {
-  background-image: url("data:image/svg+xml,%3csvg xmlns='http://www.w3.org/2000/svg' viewBox='-4 -4 8 8'%3e%3ccircle r='3' fill='%2339C0ED'/%3e%3c/svg%3e");
-}
-.form-switch .form-check-input:focus {
-  background-image: url("data:image/svg+xml,%3csvg xmlns='http://www.w3.org/2000/svg' viewBox='-4 -4 8 8'%3e%3ccircle r='3' fill='%23CFF4FC'/%3e%3c/svg%3e");
-}
-#tst90:hover,#tst90:focus { filter: invert(73%) sepia(27%) saturate(3970%) hue-rotate(146deg) brightness(94%) contrast(100%); }
-</style>
-
-<script src="{{ url_for( 'internal', filename='js/view_transform.js')}}"></script>
-<script src="{{ url_for( 'internal', filename='js/view_support.js')}}"></script>
-
+{% block script_content %}
 <script>
-    var gap=0.8
-    var grayscale=0
-    var throbber=0
+    // GLOBALS
+    document.fake_shift=0
+    document.fake_ctrl=0
 
-    var objs=[]
-    var NMO=[]
-	var imp_path="static/Import/{{imp_path}}"
-	var st_path="static/Storage/{{st_path}}"
-	var bin_path="static/Bin/{{bin_path}}"
+    // FIXME: used by viewer code - should probably get rid of this?
+    var fullscreen=false;
+
+    // this is the current entry (object) we are viewing - an image/video (used when we dbl-click to view & then in next/prev in view)
+    document.viewing=null;
+
+    var OPT = {{ OPT.to_dict()|tojson }};
+    // set from query data and stored in OPT for convenience.  It can be 0 -
+    // this implies no content in the Path at all
+    OPT.root_eid = {{ query_data.root_eid }};
+
+    // amendment types are stable per code release, store them once and use as
+    // needed when we amend entrys in Transforms, removes, etc.
+    document.amendTypes = {{ query_data.amendTypes|tojson }};
+
+    // get items out of query_data into convenience javascript vars...
+    var move_paths = {{ query_data.move_paths|tojson }};
+    var NMO={{query_data.NMO|tojson}}
+    var people={{query_data.people|tojson}}
+
+    // this is the list of entry ids for the images for ALL matches for this query 
+    var entryList={{query_data.entry_list}}
+
+    // pageList is just those entries shown on this page from the full entryList
+    var pageList=[]
+    // force pageList to set pageList for & render the first page
+    getPage(1,getPageFigures)
+
+    // gap is used to keep some space around video in viewer - tbh, not sure why anymore
+    var gap=0.8
 
     function PrettyFname(fname)
     {
         s='<span class="alert alert-secondary py-2">'
         if( fname.indexOf( "static/Import" ) == 0 ) 
         {
-            s+='<svg width="20" height="20" fill="currentColor"><use xlink:href="{{url_for('internal', filename='icons.svg')}}#import"/></svg>'
-            tmp_path=fname.replace(imp_path,"" )
+            s+='<svg width="20" height="20" fill="currentColor"><use xlink:href="{{url_for('internal', filename='icons.svg')}}?v={{js_vers['ic']}}#import"/></svg>'
+            tmp_path=fname.replace("statuc/Import","" )
         }
         if( fname.indexOf( "static/Storage" ) == 0 ) 
         {
-            s+='<svg width="20" height="20" fill="currentColor"><use xlink:href="{{url_for('internal', filename='icons.svg')}}#db"/></svg>'
+            s+='<svg width="20" height="20" fill="currentColor"><use xlink:href="{{url_for('internal', filename='icons.svg')}}?v={{js_vers['ic']}}#db"/></svg>'
             tmp_path=fname.replace("static/Storage","" )
         }
         if( fname.indexOf( "static/Bin" ) == 0 ) 
         {
-            s+='<svg width="20" height="20" fill="currentColor"><use xlink:href="{{url_for('internal', filename='icons.svg')}}#trash-fill"/></svg>'
+            s+='<svg width="20" height="20" fill="currentColor"><use xlink:href="{{url_for('internal', filename='icons.svg')}}?v={{js_vers['ic']}}#trash-fill"/></svg>'
             tmp_path=fname.replace("static/Bin","" )
         }
         s+=tmp_path+'</span>'
         return s
     }
 
-    function getPreviousEntry() {
-        var currentIndex = entryList.indexOf(document.viewing.id);
+    // check the size radiobutton
+    $(`input[name="size"][value="${OPT.size}"]`).prop('checked', true)
 
-        oldPageOffset=Math.floor(currentIndex / OPT.how_many) 
-        if (currentIndex > 0) {
-            currentIndex--;
-            pageOffset=Math.floor(currentIndex / OPT.how_many)
-            currentIndex=currentIndex-(pageOffset*OPT.how_many)
-            // pref page, load it
-            if( oldPageOffset != pageOffset )
-                // pref page is pageOffset+1 now
-                getPage(pageOffset+1,getPageViewer,currentIndex)
-            else
-                document.viewing=document.entries[currentIndex]
-        }
-    }
+    window.addEventListener('resize', DrawImg, false);
+    window.addEventListener('resize', ResizeVideo, false);
 
-    function getNextEntry() {
-        var currentIndex = entryList.indexOf(document.viewing.id);
-
-        oldPageOffset=Math.floor(currentIndex / OPT.how_many) 
-        if (currentIndex < entryList.length - 1) {
-            currentIndex++
-            pageOffset=Math.floor(currentIndex / OPT.how_many) 
-            currentIndex=currentIndex-(pageOffset*OPT.how_many)
-            // next page, load it
-            if( oldPageOffset != pageOffset )
-                // next page is pageOffset+1 now
-                getPage(pageOffset+1,getPageViewer,currentIndex)
-            else
-                document.viewing=document.entries[currentIndex]
-        }
-    }
-
-    function entryIsAtStart() {
-        return document.viewing.id === entryList[0];
-    }
-
-    function entryIsAtEnd() {
-        return document.viewing.id === entryList[entryList.length - 1];
-    }
-
-    function setEntryById(id) {
-        var currentIndex = entryList.indexOf(parseInt(id));
-        // if we are on a different page, adjust as document.entries only has <= how_many
-        pageOffset=Math.floor(currentIndex / OPT.how_many) 
-        currentIndex = currentIndex-(pageOffset*OPT.how_many)
-        document.viewing=document.entries[currentIndex]
-    }
-
-    function setDisabledForViewingNextPrevBttons()
+    // when we are in recycle bin, change colours to green & func to restore
+    if( window.location.href.includes('files_rbp') )
     {
-        $('#la').attr('disabled', entryIsAtStart());
-        $('#ra').attr('disabled', entryIsAtEnd());
+        $('#viewer_bin').attr('fill', 'var(--bs-success)')
+        // fill with bg-success colour
+        $('#viewer_bin use').attr('fill', 'var(--bs-success)')
+        $('#viewer_del').removeClass('btn-outline-danger').addClass('btn-outline-success')
+        $('#viewer_del').on('mouseenter', function() {
+          // Set the SVG fill to white
+          $('#viewer_bin use').attr('fill', 'white');
+        });
+
+        // When mouse leaves the button
+        $('#viewer_del').on('mouseleave', function() {
+          // Revert the SVG fill to the bg-success colour
+          $('#viewer_bin use').attr('fill', 'var(--bs-success)');
+        });
+        $('#viewer_del').on('click', function() { DelDBox('Restore') } )
     }
-</script>
 
-    <div id="viewer" class="container-fluid">
-        <div class="row">
-            <button title="Show previous image" class="col-auto btn btn-outline-info px-2" 
-                style="padding: 10%" id="la" 
-                onClick="
-                    getPreviousEntry()
-                    setDisabledForViewingNextPrevBttons()
-                    ViewImageOrVideo()
-                    ">
-                <svg width="16" height="16" fill="currentColor">
-                    <use xlink:href="{{url_for('internal', filename='icons.svg')}}#prev"/></svg>
-            </button>
-            <figure class="col col-auto border border-info rounded m-0 p-1" id="figure">
-                <canvas id="canvas"></canvas>
-                <img id="throbber" src="{{url_for('internal', filename='throbber.gif')}}" style="display:none;">
-                <script>
-                    var im=new Image();
-                    im.onload=DrawImg
-                    var context = canvas.getContext('2d')
-                </script>
-                <figcaption id="img-cap" class="figure-caption text-center text-wrap text-break">
-                    <span id="fname_i"></span></figcaption>
-            </figure>
-            <div id="video_div" class="col col-auto">
-                <video id="video" class="col col-auto" controls>
-                    <source id="videoSource" src="" type="video/mp4">
-                    Your browser does not support the video tag.
-                </video>
-                <figcaption id="vid-cap" class="figure-caption text-center text-wrap text-break">
-                    <span id="fname_v"></span></figcaption>
-            </div>
-            <script>
-                window.addEventListener('resize', DrawImg, false);
-                window.addEventListener('resize', ResizeVideo, false);
-            </script>
-
-            <button title="Show next image" class="col-auto btn btn-outline-info px-2" style="padding: 10%" id="ra" 
-                onClick="
-                    getNextEntry()
-                    setDisabledForViewingNextPrevBttons()
-                    ViewImageOrVideo()
-                ">
-                <svg width="16" height="16" fill="currentColor">
-                    <use xlink:href="{{url_for('internal', filename='icons.svg')}}#next"/></svg>
-            </button>
-        </div class="row">
-        {# use this for color of toggles: https://www.codeply.com/p/4sL9uhevwJ #}
-        <div class="row">
-            {# this whole div, just takes up the same space as the left button and is hidden for alignment only #}
-            <div class="col-auto px-0">
-                <button class="btn btn-outline-info px-2 invisible" disabled>
-                    <svg width="16" height="16" fill="currentColor"><use xlink:href="{{url_for('internal', filename='icons.svg')}}#next"/></svg>
-                </button>
-            </div>
-            <span class="col-auto my-auto">Show:</span>
-            <div title="Toggle showing filename (hotkey: n)" class="d-flex form-check form-switch border border-info rounded col col-auto my-auto py-1 justify-content-center ps-5">
-              <input class="form-check-input" type="checkbox" id="fname_toggle" onChange="$('.figure-caption').toggle()" checked>
-              <label class="form-check-label ps-1" for="fname_toggle">Filename</label>
-            </div>
-            <div title="Toggle showing matched faces (hotkey: f)" class="d-flex form-check form-switch border border-info rounded col col-auto my-auto py-1 justify-content-center ps-5">
-              <input class="form-check-input" type="checkbox" onChange="FaceToggle()" id="faces">
-              <label class="form-check-label ps-1" for="faces">Faces</label>
-            </div>
-            <div title="Toggle showing 'distance' on matched faces (hotkey: d)" class="d-flex form-check form-switch border border-info rounded col col-auto my-auto py-1 justify-content-center ps-5">
-              <input class="form-check-input" type="checkbox" onChange="DrawImg()" id="distance">
-              <label class="form-check-label ps-1" for="distance">Distance</label>
-            </div>
-            <div title="Change the model used to detect faces" class="col col-auto my-auto">
-                AI Model:
-                {# can use 0 as default, it will be (re)set correctly in DrawImg() anyway #}
-                {{CreateSelect( "model", 0, ["N/A", "normal", "slow/accurate"], "", "rounded norm-txt", [0,1,2])|safe }}
-            </div>
-            <div class="col col-auto pt-1">
-                <button class="btn btn-outline-info p-1" title="Rotate by 90 degrees" onClick="Transform(90)">
-                    <img src="{{url_for('internal', filename='rot90.png')}}" width="32" height="32" onMouseOver="this.src='{{url_for('internal', filename='rot90-invert.png')}}'" 
-                onMouseOut="this.src='{{url_for('internal', filename='rot90.png')}}'" />
-                </button>
-                <button class="btn btn-outline-info p-1" title="Rotate by 180 degrees" onClick="Transform(180)">
-                    <img src="{{url_for('internal', filename='rot180.png')}}" width="32" height="32" onMouseOver="this.src='{{url_for('internal', filename='rot180-invert.png')}}'"
-                onMouseOut="this.src='{{url_for('internal', filename='rot180.png')}}'" />
-                </button>
-                <button class="btn btn-outline-info p-1" title="Rotate by 270 degrees" onClick="Transform(270)">
-                    <img src="{{url_for('internal', filename='rot270.png')}}" width="32" height="32" onMouseOver="this.src='{{url_for('internal', filename='rot270-invert.png')}}'"
-                onMouseOut="this.src='{{url_for('internal', filename='rot270.png')}}'" />
-                </button>
-                <button class="btn btn-outline-info p-1" title="Flip horizontally" onClick="Transform('fliph')">
-                    <svg width="32" height="32" fill="currentColor"><use xlink:href="{{url_for('internal', filename='icons.svg')}}#flip_h"/></svg>
-                </button>
-                <button class="btn btn-outline-info p-1" title="Flip vertically" onClick="Transform('flipv')">
-                    <svg width="32" height="32" fill="currentColor"><use xlink:href="{{url_for('internal', filename='icons.svg')}}#flip_v"/></svg>
-                </button>
-                <button class="btn btn-outline-info p-1" title="View in Fullscreen mode (hotkey: F)" onClick="fullscreen=true; ViewImageOrVideo()">
-                    <svg width="32" height="32" fill="currentColor"><use xlink:href="{{url_for('internal', filename='icons.svg')}}#fullscreen"/></svg>
-                </button>
-                <button class="btn btn-outline-info p-1" title="Show logs relating to this filename (hotkey: l)" onClick="JoblogSearch()">
-                    <svg width="32" height="32" fill="currentColor"><use xlink:href="{{url_for('internal', filename='icons.svg')}}#log"/></svg>
-                </button>
-                <button class="btn btn-outline-info p-1" title="View Original" onClick="window.location='/'+objs[current].url">
-                    <svg width="32" height="32" fill="currentColor"><use xlink:href="{{url_for('internal', filename='icons.svg')}}#download"/></svg>
-                </button>
-                <button id="del" class="btn btn-outline-danger p-1" title="Delete (hotkey: Del)" 
-                    onClick="$.ajax({ type: 'POST', data: '&eid-0={{current}}', url: '/delete_files', success: function(data){ window.location='/'; return false; } })">
-                    <svg width="32" height="32" fill="currentColor"><use xlink:href="{{url_for('internal', filename='icons.svg')}}#trash"/></svg>
-                </button>
-            </div>
-        </div class="row">
-    </div id="viewer">
-</div id="viewer_div">
-{% endblock main_content %}
-
-{% block script_content %}
-<script>
-$( document ).keydown(function(event) {
-    // if dbox is visible, dont process this hot-key, we are inputting text
-    // into inputs instead
-    if( $("#dbox").is(':visible') )
-        return
-    switch (event.key) 
+    if( isMobile() )
     {
-        case "Left": // IE/Edge specific value
-        case "ArrowLeft":
-            if( $('#la').prop('disabled') == false )
-                $('#la').click()
-            break;
-        case "Right": // IE/Edge specific value
-        case "ArrowRight":
-            if( $('#ra').prop('disabled') == false )
-                $('#ra').click()
-            break;
-        case "d":
-            $('#distance').click()
-            break;
-        case "f":
-            $('#faces').click()
-            break;
-        case "n":
-            $('#fname_toggle').click()
-            break;
-        case "F":
-            fullscreen=!document.fullscreen
-            ViewImageOrVideo()
-            break;
-        case "l":
-            JoblogSearch()
-            break;
-        case "Delete":
-            $('#del').click()
-        default:
-            return; // Quit when this doesn't handle the key event.
+        $('#shift-key').css('visibility', 'visible');
+        $('#ctrl-key').css('visibility', 'visible');
     }
-});
 
-var fullscreen=false;
-
-$(document).on('click', function(e) { $('.highlight').removeClass('highlight') ; SetButtonState() });
-
-function dblClickToViewEntry(id) {
-    $('#files_div').addClass('d-none')
-    $('#viewer_div').removeClass('d-none')
-    setEntryById( id )
-    ViewImageOrVideo()
-}
-
-// different context menu on files 
-$.contextMenu({
-    selector: '.entry',
-    itemClickEvent: "click",
-    build: function($triggerElement, e) {
-        // when right-clicking & no selection add one OR deal with ctrl/shift right-lick as it always changes seln
-        if( NoSel() || e.ctrlKey || e.shiftKey )
-        {
-            DoSel(e, e.currentTarget )
-            SetButtonState();
-        }
-
-        if( FiguresOrDirsOrBoth() == "figure" )
-        {
-            item_list = {
-                details: { name: "Details..." },
-                view: { name: "View File" },
-                sep: "---",
-            }
-            if( e.currentTarget.getAttribute('type') == 'Image' )
-            {
-                item_list['transform'] = { 
-                    name: "Transform",
-                    items: {
-                        "r90": { "name" : "Rotate 90 degrees" },
-                        "r180": { "name" : "Rotate 180 degrees" },
-                        "r270": { "name" : "Rotate 270 degrees" },
-                        "fliph": { "name" : "Flip horizontally" },
-                        "flipv": { "name" : "Flip vertically" }
-                    }
-                }
-
-            }
-            item_list['move'] = { name: "Move selected file(s) to new folder" }
-            item_list['sep2'] = { sep: "---" }
-        }
-        else
-            item_list = {
-                move: { name: "Move selection(s) to new folder" }
-            }
-        
-        item_list['ai'] = {
-                name: "Scan file for faces",
-                items: {
-                    "ai-all": {"name": "all"},
-                    {% for p in people %}
-                        "ai-{{p.tag}}": {"name": "{{p.tag}}"},
-                    {% endfor %}
-                }
-            }
-        
-        if( SelContainsBinAndNotBin() ) {
-            item_list['both']= { name: 'Cannot delete and restore at same time', disabled: true }
-        } else {
-            if (e.currentTarget.getAttribute('path_type') == 'Bin' )
-                item_list['undel']= { name: "Restore selected file(s)" }
-            else if( e.currentTarget.getAttribute('type') != 'Directory' )
-                item_list['del']= { name: "Delete Selected file(s)" }
-        }
-
-        return {
-            callback: function( key, options) {
-                if( key == "details" ) { DetailsDBox() }
-                if( key == "view" ) { dblClickToViewEntry( $(this).attr('id') ) }
-                if( key == "move" ) { MoveDBox(move_paths, "{{url_for('internal', filename='icons.svg')}}") }
-                if( key == "del"  ) { DelDBox('Delete') }
-                if( key == "undel") { DelDBox('Restore') }
-                if( key == "r90"  ) { Transform(90) }
-                if( key == "r180" ) { Transform(180) }
-                if( key == "r270" ) { Transform(270) }
-                if( key == "fliph" ) { Transform("fliph") }
-                if( key == "flipv" ) { Transform("flipv") }
-                if( key.startsWith("ai")) { RunAIOnSeln(key) }
-                // dont flow this event through the dom
-                e.stopPropagation()
-            },
-            items: item_list
-        };
-    }
-});
-
-
-$( document ).keydown(function(event) {
-    switch (event.key)
-    {
-        case "Delete":
-            {% if "files_rbp" in request.url %}
-                if( ! NoSel() ) DelDBox('Restore');
-            {% else %}
-                if( ! NoSel() ) DelDBox('Delete');
-            {% endif %}
-            break;
-    } })
-
-if( isMobile() )
-{
-    $('#shift-key').css('visibility', 'visible');
-    $('#ctrl-key').css('visibility', 'visible');
-}
-
-  // check the size radiobutton
-  $(`input[name="size"][value="${OPT.size}"]`).prop('checked', true)
 </script>
 {% endblock script_content %}

templates/states.html

@@ -56,32 +56,6 @@
             </div>
         </div>
     </div class="col-7">
-
-    <div class="row pt-5">
-        <alert class="alert alert-warning">The following values are based on the defaults above and subsequent changes as you navigate the application and are not set by hand.  The following content is for checking/debugging only.</alert>
-    </div class="row">
-
-    <div class="row">
-        <table id="pa_user_state_tbl" class="table table-striped table-sm" data-toolbar="#toolbar" data-search="true">
-            <thead>
-                <tr class="table-primary"><th>Path</th><th>New or Oldest</th><th>How Many</th><th>Folders?</th><th>Group by</th><th>Thumb size</th><th>DB retrieve offset</th><th>Root</th><th>cwd</th></tr>
-            </thead>
-            <tbody>
-            {% for st in states %}
-                <tr>
-                    <td>{{st.path_type}}</td>
-                    <td>{{st.noo}}</td>
-                    <td>{{st.how_many}}</td>
-                    <td>{{st.folders}}</td>
-                    <td>{{st.grouping}}</td>
-                    <td>{{st.size}}</td>
-                    <td>{{st.root}}</td>
-                    <td>{{st.cwd}}</td>
-                </tr>
-            {% endfor %}
-            </tbody>
-        </table>
-    </div class="row">
 </div class="container-fluid">
 {% endblock main_content %}
 {% block script_content %}

wrapper.sh

@@ -13,6 +13,9 @@ else
     sudo -u pauser gunicorn --bind=0.0.0.0:80 --workers=1 --threads=1 main:app --env ENV="development" --error-logfile gunicorn.error.log --access-logfile gunicorn.log --capture-output --enable-stdio-inheritance --reload
 fi
 
+# warm the cache to see if this helps with odd restart 404s
+curl -sf http://localhost/health
+
 # this should never be invoked unless gunicorn fails -- in that case, at least
 # we will keep the container can login by hand and check the issue/error
 sleep 99999