works with new ldap now, and put forward details of next big change into TODO

BUGs

@@ -1,4 +1,15 @@
-### Next: 139
+### Next: 140
+BUG-139: since AI:mich not %AI:mich% ? going next a few times, ends say 4 pages of 50 into 4000 matches (entries from DB < 50)... Either search is no longer returning all those matches, or the stats page is wildly wrong?
+    - confirmed this is when person has 2 or more refimgs:
+      - on page "2", we get 49 pulled back in the ORM instead of the 50 expected -- b/c I use that to indicate we must be at the end of the list if not 50 found
+    -- really, need to fix once and for all the eids / re-running query.
+        do GetEntries as we do now, once done however, get all entry ids.  Stick those into the DB with a unique query-id and datestamp
+        new func to get all details needed for entries in an eid list (of 1-page) - show this page of entries
+        use current, full eidlist and to work our start/end of list (next/prev), disabling.
+        then client can keep current page of data, if you hit next/prev, use DB unique query id / full list and page of eids, and give full data for new page of entries
+      Implications though, are if a search is invalidated (maybe delete / move a photo), need to remove them from the list on the DB too OR let user know/decide to fix/wait.
+
+
 BUG-100: I managed to get 2 photos matching mich in the NOT_WORKING photo (probably dif refimgs but same p.tag?)
     = /photos/2012/20120414-damien/IMG_8467.JPG
 BUG-106: cant add trudy /pat? as refimgs via FaceDBox

TODO

@@ -1,4 +1,28 @@
+### major fix - go to everywhere I call GetEntries(), and redo the logic totally...
+    * firstly, run the query as per normal, but get just the matched eids into an entry_lst
+    * make a unique query_id for this entry_lst, and store entry_ids into "query" table, with a unique query_id
+    * take most of pa_user_state that relates to query state and move it to the "query" table per query_id
+    * pa_user_state then becomes defaults for next query (so how_many, noo, etc)
+
+    * we can age out queries form the query_table after a few months?
+        * client side always has query_id.  IF DB does not have query_id, then its really old? - just say so...
+
+    * client side takes query_id, entry_lst, current_eid, offset, first/last_eid, etc. as part of its first route / html creation.
+        * it then decides based on all this to GetEntryDetails( subset of entry_lst ) <- needs new route
+        * IN THEORY some of the subset of entry_lst don't exist -- BUT, we can handle that on response, e.g. say my query used to have 1,2,3, and since then another user/action deleted 2:
+            - I ask for details on 1,2,3 and get back details on 1,3 only.
+            - On client-side, I can say, since you ran this query, data in PA has changed - image#2 is no longer in PA.   
+              Please run a new query (or bonus points, maybe have enough of the original query to note this and ask, do you want to ignore changes, or re-run query and get latest data?)
+    * client can go fwd or back in the entry_lst same as now (disabling buttons as needed), BUT as entry_lst is NOT recreated per page move, then no chance to get confused about first/last
+    * client side: 
+        * for real chance to stop confusion, instead of removing deleted images from DOM, we should gray them out and put a big Del (red circle with line?) though it as overlay.
+        * Create another table is entry_ammendments - note the deletions, rotations, flips of specific eids - then reproduce that on the client side visually as needed 
+          - at least grayed-out, to indicate a pending action is not complete.
+          - When job that flips, rotates, deletes completes then lets update the query details (e.g. remove eids, or remove the ammendments)
+          - this actually is quite an improvement, if someone is deleting 2 as per above, I will see that as a pending change in my unrelated query, ditto flips, etc.
+
 ### GENERAL
+    * jobs for AI should show path name
     * rm dups job should show progress bar
     * in viewer, there is no move button (maybe add one?)
     * consider doing duplicates before AI, and if there are say 100s+, then maybe pause the AI work

files.py

@@ -280,7 +280,7 @@ def GetEntriesInSearchView( OPT ):
     if OPT.last_eid == 0:
         by_fname=  f"select e.id from entry e where e.name ilike '%%{search_term}%%'"
         by_dirname=f"select e.id from entry e, entry_dir_link edl where edl.entry_id = e.id and edl.dir_eid in ( select d.eid from dir d where d.rel_path ilike '%%{search_term}%%' )"
-        by_ai     =f"select e.id from entry e, face_file_link ffl, face_refimg_link frl, person_refimg_link prl, person p where e.id =  ffl.file_eid and frl.face_id = ffl.face_id and frl.refimg_id = prl.refimg_id and prl.person_id = p.id and  p.tag ilike '%%{search_term}%%'"
+        by_ai     =f"select e.id from entry e, face_file_link ffl, face_refimg_link frl, person_refimg_link prl, person p where e.id =  ffl.file_eid and frl.face_id = ffl.face_id and frl.refimg_id = prl.refimg_id and prl.person_id = p.id and p.tag = '{search_term}'"
 
         if 'AI:' in OPT.orig_search_term:
             sel_no_order=f"select e.*, f.* from entry e, file f where e.id=f.eid and e.id in ( {by_ai} ) "

main.py

@@ -60,15 +60,18 @@ except Exception:
     app.config['SECRET_KEY'] = b'my_insecure_PA_token_with_random_2134876adsfjhlkasdf87'
 
 # ldap config vars: (the last one is required, or python ldap freaks out)
-app.config['LDAP_HOST'] = 'mara.ddp.net'
+app.config['LDAP_HOST'] = 'ldap.ddp.net'
 app.config['LDAP_BASE_DN'] = 'dc=depaoli,dc=id,dc=au'
 app.config['LDAP_USER_DN'] = 'ou=users'
 app.config['LDAP_GROUP_DN'] = 'ou=groups'
-app.config['LDAP_USER_RDN_ATTR'] = 'cn'
+app.config['LDAP_USER_RDN_ATTR'] = 'uid'
 app.config['LDAP_USER_LOGIN_ATTR'] = 'uid'
 app.config['LDAP_BIND_USER_DN'] = None
 app.config['LDAP_BIND_USER_PASSWORD'] = None
 app.config['LDAP_GROUP_OBJECT_FILTER'] = '(objectclass=posixGroup)'
+app.config['LDAP_BIND_USER_DN'] = None
+app.config['LDAP_BIND_USER_PASSWORD'] = None
+
 
 db = SQLAlchemy(app)                           # create the (flask) sqlalchemy connection
 ma = Marshmallow(app)                          # set up Marshmallow - data marshalling / serialising

templates/files.html

@@ -15,6 +15,11 @@
             p.icon_url = '{{p.icon_url}}'
             move_paths.push(p)
         {% endfor %}
+
+        document.OPT = '{{OPT}}'
+        document.entries = '{{entry_data}}'
+        document.how_many = '{{OPT.how_many}}'
+        document.entries_len = '{{entry_data|length}}'
     </script>
 
     <div class="container-fluid">
@@ -68,7 +73,7 @@
                     </button>
                     <span class="sm-txt my-auto">&nbsp;{{OPT.how_many}} files&nbsp;</span>
                     {% set nxt_disabled="" %}
-                    {% if not entry_data or entry_data and entry_data|length < OPT.how_many|int %}
+                    {% if not entry_data or entry_data|length < OPT.how_many|int %}
                         {% set nxt_disabled="disabled" %}
                     {% endif %}
                     <button aria-label="next" id="next" {{nxt_disabled}} name="next" class="next sm-txt btn btn-outline-secondary">