Commit Graph

192 Commits

Author SHA1 Message Date
d39f1734a3 fix BUG-124 - pa job mgr does not have timestamps, vi creating/use of PAprint() 2024-03-11 00:40:25 +11:00
2665ab086e properly define regex strings 2024-03-10 23:52:24 +11:00
0ed56d6a1a added a debug for rotate code, and put quotes in the right spots so paths with spaces are covered - D'oh! 2023-11-11 22:25:26 +11:00
1005acb339 use jpegtran to do lossless 90 rotations 2023-11-11 21:28:04 +11:00
62cfe6305e use text() for newer sqlalchemy 2023-09-27 15:02:57 +10:00
51492efe04 replaced a couple of raw sqls with ORM, making sure JobExtra value is a str(), for newer pg/sqlalchmemy 2023-09-27 14:31:22 +10:00
2273066f04 remove duplicate import of re 2023-05-06 11:55:52 +10:00
2b478ed505 made SafePath and ensured initital paths and move_paths have valid paths, and all filenames have to be found via os.walk so should be impossible to write to parts of the FS that are unsafe 2023-02-03 16:57:39 +11:00
4b1bbcb2bf broad (small) changes to make style of all routes and jobs to be consistent, e.g. use underscords between words 2023-01-15 23:17:59 +11:00
13bfedb1a8 when init jobs fail and HandleJobs() has a catchall fail, then turn them into persistent danger popups 2023-01-15 13:29:05 +11:00
2dac6125c4 change from UUID to md5sum of face_data to allow disconnected metadata to be rematched in the future. This also removes BUG-109 where a file in DEV was deleted them copied back and deleted again, and it caused duplicate metadata reading on a subsequent pa_job_manager restart 2023-01-13 17:35:30 +11:00
a29cbb143c Huge change, removed Status class and all "alert" messages are now shown as BS toast() and are via the DB and handled async in the F/E in jscript via Ajax. Fixed BUG-113 where toasts() were repeating. Removed many of the explicit alert messages (other than errors) and hooked {New|Finish}Job to consistently send messages to the F/E. Other messages (F/E without a job, like save settings) now use this model as well. Finally converted most of the older POST responses to formal json 2023-01-11 13:50:05 +11:00
0784861331 added persistent and cant_close to PA_JobManager_FE_Message, used them from pa_job_manager to set status messages with persistence/close buttons appropriately for items like fix_dups/stale_jobs. When "fixing" now, the persistent Status message stays, but its now positioned approx. below the navbar on the right and is ok. Started on changing status to a more sensible naming conventions (away from alert to level) - more work to complete this 2023-01-10 17:45:02 +11:00
b952fe82f6 large refactor of code to use venv (in .python) for DEV, to use consistent python versions with PROD, sepcifically flask 2.2.2 to address the deprecation of FLASK_ENV -> replaced with ENV. Fixed up git and dockerignores to be more stringent 2022-12-31 15:42:03 +11:00
2a1e2d7d5d cleaned up code around json, *locn -> transitioned to top, right, bottom, left to finally fix ORM replacing [ with { <- FIXES BUG-108 2022-12-23 16:18:27 +11:00
81fea4f6f3 remove temporary code to fix faces (move coords from locn to top, right, etc.) 2022-12-23 16:04:51 +11:00
ac24c98bed store individual coords for face data - partial fix for BUG-108 2022-12-23 14:51:28 +11:00
d673c138e7 add explicit coords, sizes - partial BUG-108 fix 2022-12-23 13:42:08 +11:00
2bf8f08228 fix BUG-107: view / pagination gets out of whack 2022-09-22 14:25:07 +10:00
71ec54807a remove last remnants of Paths, now only 1 directory per path AND created a new db-container subdir that takes the latest postgres image and adds a crontab / uses cron to run a bkup_users.sh script that dumps the person, refimg and linkage table so that if we ever blow the DB away, we dont lose the users 2022-09-17 22:40:50 +10:00
11925cd08f remove path arrays, back to simple string for a path 2022-08-08 21:18:28 +10:00
20c76609cd fixed up some missing try/excepts on os. calls 2022-08-06 12:02:10 +10:00
a620024de6 remove debug, fix BUG-105 by setting last_update on job always 2022-08-03 17:59:41 +10:00
a8af00fe66 big change to get metadata working fully in DB and on Filesystem, and recover from most common scenarios, improved GUI as well for allowing an immediate search after adding refimg as well 2022-08-01 23:44:38 +10:00
391b61f3c4 first pass at keeping overrides on face delete - should at least stop code crashing if you try to delete faces from DB and they still had a matching override 2022-07-28 21:23:34 +10:00
24bcfa0aab now have a NewJob used in pa_job_mgr and it does not set start_time until it starts, caters for BUG-97 (restarting jobs resetting start times) 2022-07-23 00:33:04 +10:00
9164b2c9f6 use python-ffmpeg (run external ffmpeg) to generate video thumb as it also auto-rotates 2022-07-21 20:59:05 +10:00
9f0dd2ba94 fix up extra debug / file_cnts on RunAIOn job -- when we fix logging, fix this properly 2022-07-17 16:45:56 +10:00
a1245151d3 when restart JobRunAIOnPath, dont add the path again, move a couple of debug lines to an actual debug 2022-07-17 15:32:59 +10:00
34c759b7a0 fix up video thumbnailing issue, seems I repalce cv2 with cv2.cv2 in a couple of spots 2022-07-15 22:50:11 +10:00
84a4cf7cf8 added ability to auto-rotate jpegs as we import them. The auto-rotation uses /usr/bin/exifautotran which rotates losslessly, and we optimised to then not also re-rotate the thumbmail. This address a few bugs in the 90s, including the one where Mandys photos were not getting faces (they were rotated), and without really doing anything the odd one where we sometimes lost tmp_locn on first load after db recreation - I cant reproduce so ignoring it 2022-07-15 17:18:31 +10:00
37bea367f5 fixed BUG-90 (not finding best match, but last match) 2022-07-10 20:09:22 +10:00
226f6fdb3e remove old/wrong faces binary from FILE in DB 2022-02-27 00:40:06 +11:00
694aef038b if we use run_ai_on_path, then we remove all existing matches on the path, then go again -- allows us to change threshold and reset 2022-02-04 22:03:04 +11:00
e1a6348967 face_size_limit added so we ignore faces that are too small 2022-02-02 21:12:14 +11:00
0751cc6010 clarified bug-82, fixed issue with AI scan optimising still scanning when it should not, finalised back button work for view/viewlist 2022-01-27 21:44:29 +11:00
08ca9b4e74 partial implementation of first_eid, last_eid -- I think the vals work -- they do for searches anyway, but not stored in pa_user_state yet 2022-01-25 00:48:14 +11:00
65ebfe2d31 DONE - todo to have force rescan delete face from DB 2022-01-23 14:54:49 +11:00
f8cff5f73e make optimisation from scan do run_ai_on_path IF there are newer reference images than the last scan even if we dont find new files in the scan itself 2022-01-19 15:37:27 +11:00
2b923f0b60 fix BUG that made job mgr break in prod (with DEBUG not even defined, its now False in Prod) 2022-01-19 01:20:40 +11:00
2af93312e1 fix minor syntax, and make sure checkdups waits for getfiledetails not run_ai_on, to avoid the ai job failing due to no ref images, and then withdrawing the checkdups and the ai job cannot create a duplicate 2022-01-18 22:11:39 +11:00
b327ddff7b optimise to not go past scan job if it does not find any new files, also now print out if we find new files into scan log, fix up first attempt at debug being off/on with "production" or not 2022-01-18 22:01:39 +11:00
4a1fbf3f44 debug off for prod, on for rest by default 2022-01-18 21:01:47 +11:00
de81db9412 unmatched faces now sorts size of face desc, and is slightly prettier -- still slow and only takes you to the file anyway, might optimise that later. still no code to auto deal with unmatched face, but will add some capabilities next. Also, remembered last dir when file_ip/sp/bin. Also throw error if try to find unknown person - happened since I allowed the back/forward. 2022-01-18 20:59:39 +11:00
096d4f79cf scans now put AI back on by default, and optimise to only call path-level scans once per path, rather than once per dir in path 2022-01-16 12:51:25 +11:00
a3c75c699d fixed BUGs with deleting old files out of the recyle bin, when they were already deleted from the FS and added timeout debugs, still not 100% the scheduled jobs are working 2022-01-15 16:37:40 +11:00
bd7cae6037 fixed BUG-78 (deleted recursively too far - took out parent_dir), also deleting from the viewer page works with Delete key too. Removed debugs in viewer for fullscreen 2022-01-14 13:49:39 +11:00
10866c3147 scheduled jobs implemented, after X days in settings we scan import, storage paths, and check the Bin for old files and actually delete them from the file system 2022-01-13 13:27:55 +11:00
a67c20d72b new BUG (very minor), reordered TODOs and now have basic stale job handling - they are detected, and can be cancelled or restarted from GUI 2022-01-11 13:18:21 +11:00
33ce78cb17 fixed Bug-75, moving dir between paths now works 2022-01-09 18:35:37 +11:00