|
|
3fa6a83a1a
|
had to set recursion limit higher to get code to work, also debugs
|
2021-02-28 19:16:59 +11:00 |
|
|
|
477bd6f099
|
added code to split out storage_path and import_path, still a bit clunky, but functional. SymlinkName also now in shared
|
2021-02-27 18:30:54 +11:00 |
|
|
|
fe1c01e2f4
|
fixed issue where we cant do for loop over os.walk more than once, so now convert it to a list, and then go through that once to get overal_file_cnt, and then commit this to the job and Dir structure so that it is known on the joblog page, e.g. we get the progress bar on the first import once the os.walk is finished. Also cleaned the code to remove unnecessary string manipulation around symlink name in loop, it cant happen there. Finally updated TODO to remove this todo item (progress bar on first import) and a few other old TODOs, added new section on scanning storage_dir, the next big thing...
|
2021-02-22 18:24:34 +11:00 |
|
|
|
26c2ef717c
|
cleaned up some debugs, added logs and finally committed code that does its best to make sure the os.remove(file) matches the duplicate detected, and it now actually does the file remove
|
2021-02-17 21:46:29 +11:00 |
|
|
|
7fa0d3e02b
|
fixed primary key bug with PA_JobManager_FE_Message, made Message return id, so it can be used in link to rm_dups, added in code to have rm_dups as a job, it does everthing but the actual deletes now, but the data is ready to be used
|
2021-02-14 16:20:17 +11:00 |
|
|
|
236beb70d8
|
minor formatting fix
|
2021-02-13 20:18:55 +11:00 |
|
|
|
1499f2ca61
|
fixed BUG-21: file stucture broken after rescan, and beginnings of new job for checking for duplicate files, and having the back-end job indicate to the front-end that there are duplicates, and the basic route is in the f/e, but not built yet
|
2021-02-11 20:09:24 +11:00 |
|
|
|
da93c80d4a
|
exif for dates added
|
2021-01-31 14:48:53 +11:00 |
|
|
|
2db7f59a40
|
made basic newest/oldest week/month/year part of gui actually retreive what we want, and have a title per grouping, but the grouping title is hard-coded for now, will need more data to do a better test, also hacky submit button for now until I decide how to enact the gui change requested
|
2021-01-30 23:26:52 +11:00 |
|
|
|
98d4c7d43e
|
Fixed BUG-19 - video thumbs being out of ratio, using THUMBSIZE in shared.py to also make sure images and videos have at least one of their width / height be 256 in their thumbnail
|
2021-01-30 20:16:36 +11:00 |
|
|
|
c329d51450
|
first pass of new viewing files "button bar"
|
2021-01-30 15:40:17 +11:00 |
|
|
|
38c05099bc
|
fixed Bug-19 same fname, diff dir not going into DB - fixed due to use of Unique on name field in Entry class, optimised to only commit logs to DB 1 per 100. Also have importdir job on second and subsequent runs show progress as num_files has been stored into the Dir object for the import path
|
2021-01-29 00:14:52 +11:00 |
|
|
|
9f2943a4d6
|
try/except for video thumb, it fails on 2 videos so far, also fixed same name diff file in AddFile()
|
2021-01-28 13:11:02 +11:00 |
|
|
|
60c39bf553
|
wrapped ProcessFilesInDir to just take a job, path, file_func to run, updated TODOs appropriately - added sql for duplicates check to TODO
|
2021-01-25 13:57:22 +11:00 |
|
|
|
ac9bb7ee2e
|
fixed BUG-17 (deleting files/dirs) - currently 0 BUGS :)
|
2021-01-25 13:40:53 +11:00 |
|
|
|
96b9a6b5ca
|
fixed BUG-16 - Hash optimisation (last_hast_date is now in File not Dir), also converted a few .format() to f"
|
2021-01-25 12:08:08 +11:00 |
|
|
|
09b6863f0c
|
moved generating refimgs to once per import dir, rather than once per file (duh), and fixed BUG-18 (file counting wrong in ProcessAI)
|
2021-01-25 10:42:25 +11:00 |
|
|
|
18b8a30140
|
Fixed BUG-11: we have changed to file_refimg_link table (from file_person_link), this means we now have some optimisations, and can definitely re-run AI jobs without crashing. Several optims could still be done - see TODO
|
2021-01-25 01:05:30 +11:00 |
|
|
|
56f19ff3b7
|
first pass/thoughts on faces per file in DB, mostyl commented out. Also put a poor hack in to allow AI jobs to re-run without error, its limited, need to really fix this properly
|
2021-01-24 00:16:58 +11:00 |
|
|
|
469745ce3b
|
partial fix/optimisation for refimgs, only recalc if file is newer than when we build the refimg encoding
|
2021-01-23 21:21:29 +11:00 |
|
|
|
f253760d26
|
update start time of jobs to when they actually are starting to run, should also help with durations being more accurate. Also added a catch-all force job finish if one is not done already. Will stop it enetering an endless loop
|
2021-01-23 20:19:44 +11:00 |
|
|
|
e17d6c41fe
|
just use python time calls for jobs, now db and python calls, as on Cams box, this can results in the last update (Cams box) occurring before the job is created in the db (maras time) - and that shows up as negative durations
|
2021-01-23 20:05:15 +11:00 |
|
|
|
221ab03dbb
|
fixed BUG-8 job failures should cascade cancels. Also put back dependency for AI job to wait for GetFileDetails (might not really be needed as I think Cams code just calls exif rotate itself anyway, also took out my temporary block of running the slow AI code. Need to discuss how to optimise it so it does not re-run parts it should know, e.g. refimg stuff once / not again unless ref imgs changed, ditto file/ai scan , etc.
|
2021-01-23 00:32:17 +11:00 |
|
|
|
bfe0613167
|
fixed BUGS 12/13/14, fixed new thumbnail code, fixed duplicate db entries for directories, fixed optimising code not working, by just ditching the directory optimising anyway
|
2021-01-23 00:00:47 +11:00 |
|
|
|
954132cf1a
|
BUG-2 (FixPath) solved, because we no longer fix paths :)
|
2021-01-22 22:33:44 +11:00 |
|
|
|
685c626fd7
|
updated AI job, to show progress in joblog, and fixed BUG-12 (can now forcescan again)
|
2021-01-22 22:27:35 +11:00 |
|
|
|
aa81d46a09
|
fixed BUG-9, current_file_num uninitd
|
2021-01-22 21:15:47 +11:00 |
|
|
|
1cf65f4c31
|
Merge branch 'master' of mara.ddp.net:photoassistant
|
2021-01-22 18:48:34 +11:00 |
|
|
|
605b26af89
|
implemented ai in our pa_job_manager, with refimgs, a job, and the whole deal
|
2021-01-22 18:46:18 +11:00 |
|
|
|
cfd4a3071f
|
fix current_file_num and num_files being wrong in import and gen*details jobs
|
2021-01-22 16:32:30 +11:00 |
|
|
|
f73d488ee7
|
mid-way through new Import function, has buggy counts, updated bugs and todos
|
2021-01-22 15:33:35 +11:00 |
|
|
|
2f3d7c1ae2
|
ProcessFilesInDir - now is generic and takes funcs to call on files
|
2021-01-21 22:22:42 +11:00 |
|
|
|
22cab9b090
|
fix BUG-5 (job file num cnt wrong when processing unknown file type)
|
2021-01-21 16:09:37 +11:00 |
|
|
|
a9819c613b
|
more path fixes
|
2021-01-21 15:25:19 +11:00 |
|
|
|
586c8e8edf
|
debug back on by default
|
2021-01-21 15:18:27 +11:00 |
|
|
|
4cf1613949
|
new fix for path shenanigans... python deals with it anyway, so just leave it "linux" pathed for simplicity
|
2021-01-21 15:18:12 +11:00 |
|
|
|
e56b7f51cc
|
quick fix
|
2021-01-21 15:04:09 +11:00 |
|
|
|
197c864624
|
fix bug with unknown files
|
2021-01-21 14:59:33 +11:00 |
|
|
|
bdef281d3b
|
remove FixPaths until we are sure we know the bug / fix
|
2021-01-21 14:28:48 +11:00 |
|
|
|
a716ed6007
|
use FixPath on keep_dirs key - a path, as the backslashes in windows cause python errors
|
2021-01-21 11:04:57 +11:00 |
|
|
|
db056e08aa
|
remove pa_job_manager table, its not needed, and fixed bug where Gen hash optimisation did not finish job when it shoud have
|
2021-01-21 00:58:02 +11:00 |
|
|
|
2cc023bd10
|
cleaned up debugs, used if debug before print debug, also ran into threading issues, so for now RunJob is non-threaded
|
2021-01-20 23:22:44 +11:00 |
|
|
|
5b99855cb5
|
call HandleJobs() when RunJob() finished to run any dependant jobs
|
2021-01-20 22:01:14 +11:00 |
|
|
|
574c12f7f5
|
cleaned up debugs, and optimised gen hash / thumbs to only process dirs with new content
|
2021-01-20 21:46:41 +11:00 |
|
|
|
76e37a2dc0
|
changed all new_file to file, also make JobForceScan work
|
2021-01-20 20:02:31 +11:00 |
|
|
|
30195e9597
|
removed filedata*, moved ProcessImportDir out into its own function, used it, also in scannow job, ensure job states are set appropriately to enforce all of the jobs are processed right
|
2021-01-20 16:37:00 +11:00 |
|
|
|
0de95d56d5
|
fixed up JobExtra printout in class, also put it in job.py, and passed extras to templates/jobs.html to show the extra details on the jobs list page so we can see what path is imported for example
|
2021-01-20 00:05:56 +11:00 |
|
|
|
9474c12a0d
|
JobGetFileDetails now works - first pass, need to improve handling of files in dirs - I optimised/assumed last dir is what you are in, but that is not always true, so need to set actual dir each time
|
2021-01-19 23:34:12 +11:00 |
|
|
|
2bd25c3e18
|
set up FileType properly, select on in DB, rather than add new ones each time, ouch! Also renamed type to type_id (better as it is a new foreign key) and use type as the relationship to hold the object anyway
|
2021-01-19 17:32:33 +11:00 |
|
|
|
47085b7a43
|
cleaned up code, before rewrite for viewing
|
2021-01-19 11:10:25 +11:00 |
|
