09b6863f0c
moved generating refimgs to once per import dir, rather than once per file (duh), and fixed BUG-18 (file counting wrong in ProcessAI)
Damien De Paoli2021-01-25 10:42:25 +11:00
18b8a30140
Fixed BUG-11: we have changed to file_refimg_link table (from file_person_link), this means we now have some optimisations, and can definitely re-run AI jobs without crashing. Several optims could still be done - see TODO
Damien De Paoli2021-01-25 01:05:30 +11:00
0829a98376
updated install instructions to get newer PIL and fix exif issues
Damien De Paoli2021-01-25 01:01:51 +11:00
56f19ff3b7
first pass/thoughts on faces per file in DB, mostyl commented out. Also put a poor hack in to allow AI jobs to re-run without error, its limited, need to really fix this properly
Damien De Paoli2021-01-24 00:16:58 +11:00
469745ce3b
partial fix/optimisation for refimgs, only recalc if file is newer than when we build the refimg encoding
Damien De Paoli2021-01-23 21:21:29 +11:00
f253760d26
update start time of jobs to when they actually are starting to run, should also help with durations being more accurate. Also added a catch-all force job finish if one is not done already. Will stop it enetering an endless loop
Damien De Paoli2021-01-23 20:19:44 +11:00
e17d6c41fe
just use python time calls for jobs, now db and python calls, as on Cams box, this can results in the last update (Cams box) occurring before the job is created in the db (maras time) - and that shows up as negative durations
Damien De Paoli2021-01-23 20:05:15 +11:00
afc4eda233
last commit or the night, a feel-good one, now have search by file name and refimg tag, can find ppl :)
Damien De Paoli2021-01-23 00:48:00 +11:00
221ab03dbb
fixed BUG-8 job failures should cascade cancels. Also put back dependency for AI job to wait for GetFileDetails (might not really be needed as I think Cams code just calls exif rotate itself anyway, also took out my temporary block of running the slow AI code. Need to discuss how to optimise it so it does not re-run parts it should know, e.g. refimg stuff once / not again unless ref imgs changed, ditto file/ai scan , etc.
Damien De Paoli2021-01-23 00:32:17 +11:00
ecb276b4c7
new thumbnail code Cam wrote fixes BUG-7 too - image orientation
Damien De Paoli2021-01-23 00:08:57 +11:00
5c34aa9cc7
BUG-10 also squashed by removing dir-level optimisation code
Damien De Paoli2021-01-23 00:05:48 +11:00
bfe0613167
fixed BUGS 12/13/14, fixed new thumbnail code, fixed duplicate db entries for directories, fixed optimising code not working, by just ditching the directory optimising anyway
Damien De Paoli2021-01-23 00:00:47 +11:00
8e9c7408a8
BUG-14 added - optimisation at dir level doesnt work at all, only dir modifications occur with NEW files, this probably would work for importing, but its not that bad to stat files, so Im just ditching the added complexity
Damien De Paoli2021-01-22 23:58:54 +11:00
17447f9314
fixed thumbnail issues on linux, and JobImportDir bug with adding empty and then duplicate dir entries
Damien De Paoli2021-01-22 23:43:50 +11:00
954132cf1a
BUG-2 (FixPath) solved, because we no longer fix paths :)
Damien De Paoli2021-01-22 22:33:44 +11:00
20a302e3d6
BUG-3 is wrong, removing it... we can never know numfiles before we walk the fs, so just live with it, the code is quick anyway, it will be 0 then 100% before you notice anyway
Damien De Paoli2021-01-22 22:32:21 +11:00
4cf1613949
new fix for path shenanigans... python deals with it anyway, so just leave it "linux" pathed for simplicity
Damien De Paoli2021-01-21 15:18:12 +11:00
bdef281d3b
remove FixPaths until we are sure we know the bug / fix
Damien De Paoli2021-01-21 14:28:48 +11:00
a716ed6007
use FixPath on keep_dirs key - a path, as the backslashes in windows cause python errors
Damien De Paoli2021-01-21 11:04:57 +11:00
72a892fd17
added BUG-3 (import dir job on 1st run does not track num_files - even at end), and now have wakeup route expose in menu
Damien De Paoli2021-01-21 01:08:52 +11:00
498e3d383f
put menu item in for route to wakeup of job manager
Damien De Paoli2021-01-21 01:07:21 +11:00
db056e08aa
remove pa_job_manager table, its not needed, and fixed bug where Gen hash optimisation did not finish job when it shoud have
Damien De Paoli2021-01-21 00:58:02 +11:00
2cc023bd10
cleaned up debugs, used if debug before print debug, also ran into threading issues, so for now RunJob is non-threaded
Damien De Paoli2021-01-20 23:22:44 +11:00
153be75302
cleaned up TODOs, still need to deal with jobs_active, jobs_completed...
Damien De Paoli2021-01-20 23:21:59 +11:00
5b99855cb5
call HandleJobs() when RunJob() finished to run any dependant jobs
Damien De Paoli2021-01-20 22:01:14 +11:00
f7a94f0b7d
allow job log to have links embedded, and label num files when N/A better
Damien De Paoli2021-01-20 21:49:04 +11:00
30195e9597
removed filedata*, moved ProcessImportDir out into its own function, used it, also in scannow job, ensure job states are set appropriately to enforce all of the jobs are processed right
Damien De Paoli2021-01-20 16:37:00 +11:00
6523500d54
use file progress as overall process in job list if applicable
Damien De Paoli2021-01-20 16:35:58 +11:00
41c2af2cd2
removed fake job data, and pass* from jobs, moved last_import_date into Dir from settings
Damien De Paoli2021-01-20 16:35:35 +11:00
0de95d56d5
fixed up JobExtra printout in class, also put it in job.py, and passed extras to templates/jobs.html to show the extra details on the jobs list page so we can see what path is imported for example
Damien De Paoli2021-01-20 00:05:56 +11:00
9474c12a0d
JobGetFileDetails now works - first pass, need to improve handling of files in dirs - I optimised/assumed last dir is what you are in, but that is not always true, so need to set actual dir each time
Damien De Paoli2021-01-19 23:34:12 +11:00
31d45e80b6
moved over to new ENTRY based data structures for loading from DB and viewing content
Damien De Paoli2021-01-19 17:33:10 +11:00
2bd25c3e18
set up FileType properly, select on in DB, rather than add new ones each time, ouch! Also renamed type to type_id (better as it is a new foreign key) and use type as the relationship to hold the object anyway
Damien De Paoli2021-01-19 17:32:33 +11:00
5eb967c77f
added foreign key constraint for file type in ENTRY
Damien De Paoli2021-01-19 17:30:51 +11:00
5285bf66ab
alternate database structure with entry, (new_)file, dir works in terms of DB and SQL alchemy, tomorrow we rewrite the jobs to really use the new structure
Damien De Paoli2021-01-18 21:40:46 +11:00
4053919def
comment out sleep again, also removed some debugs, and converted some to AddLogForJob
Damien De Paoli2021-01-18 13:20:15 +11:00
3951b3e0a1
do not need to send any data down the client conn to wake up the pa_job_manager
Damien De Paoli2021-01-18 13:19:50 +11:00
ab3eff4e62
starting on new DB structure, with a new branch, with thoughts in TODO, file_types in tables.sql
Damien De Paoli2021-01-18 13:19:25 +11:00
507336c2b2
commented out sleep which I use for testing of active jobs/job refresh page, and re-title the job list html
Damien De Paoli2021-01-17 12:38:28 +11:00
Damien De Paoli
Cam