8649edfe7c
added page_size on reducing and hardcoded stich of prefix of dups (STILL needs fixing)
Damien De Paoli2021-03-03 22:03:55 +11:00
0c583aa868
finished containers, added new items on reducing dups per page and prefix of dups
Damien De Paoli2021-03-03 22:03:06 +11:00
df954e1e1a
created wrapper.sh to also run pa_job_manager in paweb container, exposed port 55432, have not tested that the job manager is connectable and we prob. should not expose that port in the long run, but for another night
Damien De Paoli2021-03-03 20:33:51 +11:00
8074225a60
fixed Dockerfile and requirements.txt (for pip) so that pa.depaoli.id.au is now a working docker container on mara
Damien De Paoli2021-03-03 01:04:57 +11:00
477bd6f099
added code to split out storage_path and import_path, still a bit clunky, but functional. SymlinkName also now in shared
Damien De Paoli2021-02-27 18:30:54 +11:00
98207dfd9c
added TODO for includes when not on internet
Damien De Paoli
2021-02-27 10:58:45 +11:00
4042fdef77
put explicit use of locahost:5432 for pg, and localhost:5000 for flask for lappy
Damien De Paoli
2021-02-27 10:57:57 +11:00
a82226b082
use DB_URL from shared
Damien De Paoli
2021-02-27 10:57:27 +11:00
f7d78c2257
also need flask
Damien De Paoli
2021-02-27 00:22:02 +11:00
fe1c01e2f4
fixed issue where we cant do for loop over os.walk more than once, so now convert it to a list, and then go through that once to get overal_file_cnt, and then commit this to the job and Dir structure so that it is known on the joblog page, e.g. we get the progress bar on the first import once the os.walk is finished. Also cleaned the code to remove unnecessary string manipulation around symlink name in loop, it cant happen there. Finally updated TODO to remove this todo item (progress bar on first import) and a few other old TODOs, added new section on scanning storage_dir, the next big thing...
Damien De Paoli2021-02-22 18:24:34 +11:00
1320ecba20
fixed BUG-24, only appends per_path_dups if we have one to append
Damien De Paoli2021-02-22 18:22:18 +11:00
2482a2e829
fixed BUG-24, only appends per_path_dups if we have one to append
Damien De Paoli2021-02-22 17:26:32 +11:00
7fd6a2eddb
fixed BUGs 22 and 23 - both relate to the HTML forms being different for the search and viewing data, so changing one when the other was set, did not pass the values through, e.g. search term not there when you change a viewing option, or after searching, it did not know the grouping options. Fixed from the search side by showing a label of what you are searching for and storing it statically in the viewing form. Sorted by jscript onSubmit func to take 5 viewing options and store the last value in hidden inputs in the search form
Damien De Paoli2021-02-21 15:40:47 +11:00
33ad544354
removed extra stuff from rmdups job list view, to keep the job list useful
Damien De Paoli2021-02-17 21:52:58 +11:00
26c2ef717c
cleaned up some debugs, added logs and finally committed code that does its best to make sure the os.remove(file) matches the duplicate detected, and it now actually does the file remove
Damien De Paoli2021-02-17 21:46:29 +11:00
6ef383f732
update status message to link to job created for del dup files
Damien De Paoli2021-02-17 21:42:59 +11:00
9e84236f49
moved to storing DB ids rather than names of files/dirs for deletions
Damien De Paoli2021-02-14 16:20:48 +11:00
7fa0d3e02b
fixed primary key bug with PA_JobManager_FE_Message, made Message return id, so it can be used in link to rm_dups, added in code to have rm_dups as a job, it does everthing but the actual deletes now, but the data is ready to be used
Damien De Paoli2021-02-14 16:20:17 +11:00
0bfd3b58b3
moved to storing DB ids rather than names of files/dirs for deletions
Damien De Paoli2021-02-14 16:18:22 +11:00
09d56c5cc5
added comments, also remove url_for on request.endpoint, was breaking when needing a param to be passed
Damien De Paoli2021-02-14 16:02:20 +11:00
039b6fd805
value with hashes is large for job extra, so remove 1024 limit
Damien De Paoli2021-02-14 16:01:53 +11:00
27e881ea8c
minor clean up on headings, alerts at top of /fix_dups route
Damien De Paoli2021-02-13 20:30:21 +11:00
49e0a5a797
now have a partial fix_dups path, it shows the content in a much more reasonable manner, and allows the GUI to select the files/paths to keep, HOWEVER, the form POST is not enabled, and I still need to process the form data -- right now, now sure how I know which files to delete vs keep -> will need hidden vars of options, not just the to_keep - then process them
Damien De Paoli2021-02-13 20:21:26 +11:00
7014eb0f35
now have a partial fix_dups path, it shows the content in a much more reasonable manner, and allows the GUI to select the files/paths to keep, HOWEVER, the form POST is not enabled, and I still need to process the form data -- right now, now sure how I know which files to delete vs keep -> will need hidden vars of options, not just the to_keep - then process them
Damien De Paoli2021-02-13 20:21:08 +11:00
0fe5d97317
fix missing self. in prints of Settings __repr__
Damien De Paoli2021-02-13 20:19:14 +11:00
1499f2ca61
fixed BUG-21: file stucture broken after rescan, and beginnings of new job for checking for duplicate files, and having the back-end job indicate to the front-end that there are duplicates, and the basic route is in the f/e, but not built yet
Damien De Paoli2021-02-11 20:09:24 +11:00
2d88403756
new BUG-21: second pass, the file structures in the DB are broken
Damien De Paoli2021-02-11 20:07:15 +11:00
a2775baeeb
trying diff size for sm-txt - better for larger monitor
Damien De Paoli2021-02-09 19:37:11 +11:00
b168b0c77b
add year as path preprend for storage suggestion
Damien De Paoli2021-02-09 19:36:56 +11:00
893d48c8d5
using sm-txt to shrink action buttons at top of page, created basic move/del dbox to be more or less partially functional, without doing anything real
Damien De Paoli2021-02-07 20:44:55 +11:00
ca7064dd92
txt height modifier style for shrinking title bar text
Damien De Paoli2021-02-07 20:43:34 +11:00
d4e3a1d93e
use vars, not form to set initial form values, use new CreateSelect() to neaten up form creation, redo formatting to have 3 cols, left / centre / right justified, also disable prev button when offset is 0, to stop the button just reloading the same page over again
Damien De Paoli2021-02-02 00:24:47 +11:00
96bcada5c5
CreateSelect() in py called in jinja to neaten up selects
Damien De Paoli2021-02-02 00:18:52 +11:00
5703664019
changed to fixed number of images per load, rather than complicating with week/month/year retrieval
Damien De Paoli2021-02-01 23:13:38 +11:00
2a5f607597
updated instructions to use full paths, and extra step to mv backup to a non-generic name
Damien De Paoli2021-02-01 23:10:51 +11:00
282c5ae721
fix up month, etc. view not having proper divs for heading rows
Damien De Paoli2021-01-31 18:32:30 +11:00
24437a5589
fix new bug where size button submits form and goes back to M with just return false on them
Damien De Paoli2021-01-31 18:29:42 +11:00
2db7f59a40
made basic newest/oldest week/month/year part of gui actually retreive what we want, and have a title per grouping, but the grouping title is hard-coded for now, will need more data to do a better test, also hacky submit button for now until I decide how to enact the gui change requested
Damien De Paoli2021-01-30 23:26:52 +11:00
98d4c7d43e
Fixed BUG-19 - video thumbs being out of ratio, using THUMBSIZE in shared.py to also make sure images and videos have at least one of their width / height be 256 in their thumbnail
Damien De Paoli2021-01-30 20:16:36 +11:00
89cc7aad2d
fix bug with unique on both entry name and dir path_prefix
Damien De Paoli2021-01-29 00:15:30 +11:00
38c05099bc
fixed Bug-19 same fname, diff dir not going into DB - fixed due to use of Unique on name field in Entry class, optimised to only commit logs to DB 1 per 100. Also have importdir job on second and subsequent runs show progress as num_files has been stored into the Dir object for the import path
Damien De Paoli2021-01-29 00:14:52 +11:00
9d1370cd62
allow video files to be clickable, in FF it just works, imagine my chromes are just broke as they have partial nvidia optims that are broken
Damien De Paoli2021-01-29 00:10:05 +11:00
e3dcce4a53
if we have current_file_num > 0 but dont know total in num_files, at least show count to show progress
Damien De Paoli2021-01-29 00:02:56 +11:00
9f2943a4d6
try/except for video thumb, it fails on 2 videos so far, also fixed same name diff file in AddFile()
Damien De Paoli2021-01-28 13:11:02 +11:00
60c39bf553
wrapped ProcessFilesInDir to just take a job, path, file_func to run, updated TODOs appropriately - added sql for duplicates check to TODO
Damien De Paoli2021-01-25 13:57:22 +11:00
ac9bb7ee2e
fixed BUG-17 (deleting files/dirs) - currently 0 BUGS :)
Damien De Paoli2021-01-25 13:40:53 +11:00
96b9a6b5ca
fixed BUG-16 - Hash optimisation (last_hast_date is now in File not Dir), also converted a few .format() to f"
Damien De Paoli2021-01-25 12:08:08 +11:00
ec4033e0fa
order jobs by descending order, more useful to see newest at top
Damien De Paoli2021-01-25 10:44:13 +11:00
Damien De Paoli
Damien De Paoli