|
|
a9819c613b
|
more path fixes
|
2021-01-21 15:25:19 +11:00 |
|
|
|
586c8e8edf
|
debug back on by default
|
2021-01-21 15:18:27 +11:00 |
|
|
|
4cf1613949
|
new fix for path shenanigans... python deals with it anyway, so just leave it "linux" pathed for simplicity
|
2021-01-21 15:18:12 +11:00 |
|
|
|
e56b7f51cc
|
quick fix
|
2021-01-21 15:04:09 +11:00 |
|
|
|
197c864624
|
fix bug with unknown files
|
2021-01-21 14:59:33 +11:00 |
|
|
|
bdef281d3b
|
remove FixPaths until we are sure we know the bug / fix
|
2021-01-21 14:28:48 +11:00 |
|
|
|
a716ed6007
|
use FixPath on keep_dirs key - a path, as the backslashes in windows cause python errors
|
2021-01-21 11:04:57 +11:00 |
|
|
|
72a892fd17
|
added BUG-3 (import dir job on 1st run does not track num_files - even at end), and now have wakeup route expose in menu
|
2021-01-21 01:08:52 +11:00 |
|
|
|
498e3d383f
|
put menu item in for route to wakeup of job manager
|
2021-01-21 01:07:21 +11:00 |
|
|
|
db056e08aa
|
remove pa_job_manager table, its not needed, and fixed bug where Gen hash optimisation did not finish job when it shoud have
|
2021-01-21 00:58:02 +11:00 |
|
|
|
dace66473e
|
remove pa_job_manager table, its not needed
|
2021-01-21 00:57:32 +11:00 |
|
|
|
2cc023bd10
|
cleaned up debugs, used if debug before print debug, also ran into threading issues, so for now RunJob is non-threaded
|
2021-01-20 23:22:44 +11:00 |
|
|
|
153be75302
|
cleaned up TODOs, still need to deal with jobs_active, jobs_completed...
|
2021-01-20 23:21:59 +11:00 |
|
|
|
5b99855cb5
|
call HandleJobs() when RunJob() finished to run any dependant jobs
|
2021-01-20 22:01:14 +11:00 |
|
|
|
f7a94f0b7d
|
allow job log to have links embedded, and label num files when N/A better
|
2021-01-20 21:49:04 +11:00 |
|
|
|
aaeb4a8ffe
|
now have a last_hash_date
|
2021-01-20 21:48:13 +11:00 |
|
|
|
021aeb69c7
|
use correct seqs and order jobs by id
|
2021-01-20 21:47:53 +11:00 |
|
|
|
574c12f7f5
|
cleaned up debugs, and optimised gen hash / thumbs to only process dirs with new content
|
2021-01-20 21:46:41 +11:00 |
|
|
|
8c8eb83893
|
removed last_import_date from settings, its now per Dir
|
2021-01-20 20:04:05 +11:00 |
|
|
|
5644cd4952
|
changed all new_file to file and updated sql to be correct case
|
2021-01-20 20:03:08 +11:00 |
|
|
|
cc3a6cebd7
|
changed all new_file to file, also make JobForceScan work
|
2021-01-20 20:02:40 +11:00 |
|
|
|
76e37a2dc0
|
changed all new_file to file, also make JobForceScan work
|
2021-01-20 20:02:31 +11:00 |
|
|
|
0bc87e346e
|
update of where Im at with TODO
|
2021-01-20 16:37:56 +11:00 |
|
|
|
30195e9597
|
removed filedata*, moved ProcessImportDir out into its own function, used it, also in scannow job, ensure job states are set appropriately to enforce all of the jobs are processed right
|
2021-01-20 16:37:00 +11:00 |
|
|
|
6523500d54
|
use file progress as overall process in job list if applicable
|
2021-01-20 16:35:58 +11:00 |
|
|
|
41c2af2cd2
|
removed fake job data, and pass* from jobs, moved last_import_date into Dir from settings
|
2021-01-20 16:35:35 +11:00 |
|
|
|
4be0ef2a73
|
removed pass* from jobs
|
2021-01-20 16:34:56 +11:00 |
|
|
|
0de95d56d5
|
fixed up JobExtra printout in class, also put it in job.py, and passed extras to templates/jobs.html to show the extra details on the jobs list page so we can see what path is imported for example
|
2021-01-20 00:05:56 +11:00 |
|
|
|
0c70a0ca89
|
more TODOs
|
2021-01-20 00:05:02 +11:00 |
|
|
|
9474c12a0d
|
JobGetFileDetails now works - first pass, need to improve handling of files in dirs - I optimised/assumed last dir is what you are in, but that is not always true, so need to set actual dir each time
|
2021-01-19 23:34:12 +11:00 |
|
|
|
5c840273c2
|
added hidden route to wake up the job manager
|
2021-01-19 23:33:08 +11:00 |
|
|
|
1c181668e8
|
small renames of id to eid, added num_files to DIR and removed comments
|
2021-01-19 23:32:47 +11:00 |
|
|
|
3a73b9546d
|
KLUDGE around num_passes for now, jobs need a clean up before I finish the new wait_for jobs
|
2021-01-19 23:31:37 +11:00 |
|
|
|
325bd95f25
|
stopping for the night, so keeping track of where I am at
|
2021-01-19 23:30:53 +11:00 |
|
|
|
21678fdaad
|
fixed BUG-1 ages ago (timezones)
|
2021-01-19 23:29:51 +11:00 |
|
|
|
31d45e80b6
|
moved over to new ENTRY based data structures for loading from DB and viewing content
|
2021-01-19 17:33:10 +11:00 |
|
|
|
2bd25c3e18
|
set up FileType properly, select on in DB, rather than add new ones each time, ouch! Also renamed type to type_id (better as it is a new foreign key) and use type as the relationship to hold the object anyway
|
2021-01-19 17:32:33 +11:00 |
|
|
|
5eb967c77f
|
added foreign key constraint for file type in ENTRY
|
2021-01-19 17:30:51 +11:00 |
|
|
|
45c06dd218
|
remove unnecessary import
|
2021-01-19 17:30:13 +11:00 |
|
|
|
558b43870a
|
added instructions on how to run all this
|
2021-01-19 17:29:41 +11:00 |
|
|
|
47085b7a43
|
cleaned up code, before rewrite for viewing
|
2021-01-19 11:10:25 +11:00 |
|
|
|
5285bf66ab
|
alternate database structure with entry, (new_)file, dir works in terms of DB and SQL alchemy, tomorrow we rewrite the jobs to really use the new structure
|
2021-01-18 21:40:46 +11:00 |
|
|
|
4053919def
|
comment out sleep again, also removed some debugs, and converted some to AddLogForJob
|
2021-01-18 13:20:15 +11:00 |
|
|
|
3951b3e0a1
|
do not need to send any data down the client conn to wake up the pa_job_manager
|
2021-01-18 13:19:50 +11:00 |
|
|
|
ab3eff4e62
|
starting on new DB structure, with a new branch, with thoughts in TODO, file_types in tables.sql
|
2021-01-18 13:19:25 +11:00 |
|
|
|
0e01b132ab
|
removed debug
|
2021-01-18 10:53:28 +11:00 |
|
|
|
c543f21f3e
|
added TODO thoughts
|
2021-01-17 15:20:32 +11:00 |
|
|
|
507336c2b2
|
commented out sleep which I use for testing of active jobs/job refresh page, and re-title the job list html
|
2021-01-17 12:38:28 +11:00 |
|
|
|
2b9dedb9b9
|
change how we calculate active jobs, probably will remove it from pa_job_engine, safer with threads I think. But, mostyle, added in client / server socket comms between web FE and job manager, with better job creation message (including link to job detail) and when you view job detail it auto-refreshes every few seconds until job complete)
|
2021-01-17 12:35:28 +11:00 |
|
|
|
abff2d8bab
|
finished moving GenerateFileData out of files.py into pa_job_manager.py
|
2021-01-17 11:31:06 +11:00 |
|
