Compare commits
19 Commits
b7b2eaeb1f
...
master
| Author | SHA1 | Date | |
|---|---|---|---|
| a4526e2b70 | |||
| cf9b0996be | |||
| 57e736a495 | |||
| aaf04e7f9b | |||
| eb9e7f2ce1 | |||
| df7a81df09 | |||
| 4742509d56 | |||
| 391fd52332 | |||
| 1831c49b15 | |||
| fa63e08b59 | |||
| e36fc170f1 | |||
| e3c94bf5eb | |||
| db8cf68d4b | |||
| 1a182d30b5 | |||
| c7c08c1c32 | |||
| 5fa35b90eb | |||
| ae718fe249 | |||
| 48ca6c5e64 | |||
| 6ceb4c4604 |
49
BUGs
49
BUGs
@@ -1,4 +1,15 @@
|
||||
### Next: 138
|
||||
### Next: 140
|
||||
BUG-139: using any large entry list and going next a few times, ends say 4 pages of 50 into 4000 matches (entries from DB < 50)...
|
||||
- confirmed this is when person has 2 or more refimgs:
|
||||
- on page "2", we get 49 pulled back in the ORM instead of the 50 expected -- b/c I use that to indicate we must be at the end of the list if not 50 found
|
||||
-- really, need to fix once and for all the eids / re-running query.
|
||||
do GetEntries as we do now, once done however, get all entry ids. Stick those into the DB with a unique query-id and datestamp
|
||||
new func to get all details needed for entries in an eid list (of 1-page) - show this page of entries
|
||||
use current, full eidlist and to work our start/end of list (next/prev), disabling.
|
||||
then client can keep current page of data, if you hit next/prev, use DB unique query id / full list and page of eids, and give full data for new page of entries
|
||||
Implications though, are if a search is invalidated (maybe delete / move a photo), need to remove them from the list on the DB too OR let user know/decide to fix/wait.
|
||||
|
||||
|
||||
BUG-100: I managed to get 2 photos matching mich in the NOT_WORKING photo (probably dif refimgs but same p.tag?)
|
||||
= /photos/2012/20120414-damien/IMG_8467.JPG
|
||||
BUG-106: cant add trudy /pat? as refimgs via FaceDBox
|
||||
@@ -24,38 +35,6 @@ BUG-132: right arrow to go to next photo in viewer ALSO scrolls to the right, ne
|
||||
BUG-133: when rebuilding pa[dev], the first run fails to have symlinks to the right paths for Import/Storage, etc. a simple restart fixes - so potentially the intial copy or some other race condition?
|
||||
BUG-134: when moving set of photos on page, then move another set of photos on page, the first set reappears. Could really delete them from the dom?
|
||||
BUG-135: failed to rotate: 2006/20061215-ITS-xmas-KP/DSC00582.JPG - not sure why && not repeatable, so its not the image, timing/race condition maybe?
|
||||
|
||||
|
||||
Example of issue when I restart and lose state:
|
||||
|
||||
id | job_id | level | message | cant_close | persistent
|
||||
-------+--------+--------+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+------------+------------
|
||||
16989 | | danger | ERROR: pref not found - dn=uid=mandy,ou=users,dc=depaoli,dc=id,dc=au, st=Sydney%20, s=<States(path_type='View', orig_search_term='', url='/view/16406', view_eid='16406', current=0, first_eid=0, last_eid=0, num_entries=0)>???? | t | t
|
||||
16991 | | danger | ERROR: viewing an id, but its not in eids OPT=<States(path_type='Search', orig_search_term='', url='/view/16406', view_eid='16406', current=0, first_eid=69336, last_eid=63222, num_entries=48440, orig_url='/view/16406', grouping='None', how_many=500, offset=0, size=128, root='static/Search', noo='Newest', folders=False, cwd='static/Search', orig_ptype=None, order='order_by(File.year.desc(),File.month.desc(),File.day.desc(),Entry.name.desc())', last_order='order_by(File.year,File.month,File.day,Entry.name)', last_order_raw='f.year, f.month, f.day, e.name')>, id=16406, eids=69336,69334,69328,69329,69330,69326,69327,69321,69304,69311,69320,69305,69316,69315,69314,69318,69312,69306,69310,69308,69309,69319,69317,69307,69313,69335,69337,69332,69331,69333,69299,69300,69302,69301,69303,69298,69297,69295,69290,69296,69275,69282,69285,69291,69283,69276,69279,69293,69287,69294,69281,69288,69284,69286,69292,69289,69277,69280,69274,69232,69257,69270,69230,69247,69258,69261,69256,69238,69248,69253,69236,69252,69267,69255,69235,69264,69233,69245,69229,69271,69273,69234,69278,69268,69249,69260,69244,69241,69265,69266,69272,69251,69242,69269,69237,69240,69239,69243,69250,69231,69262,69246,69254,69263,69259,69219,69227,69218,69225,69224,69222,69226,69220,69223,69216,69221,69228,69217,69212,69215,69213,69214,69211,69208,69158,69173,68992,69113,69130,69050,68990,68993,69039,69122,69192,69194,69165,69137,69105,69074,69162,69005,69029,69072,68988,69087,69037,69129,69053,69176,69181,69075,69034,69179,69120,69046,68966,69043,69161,69188,69002,69077,69020,69146,69013,69026,69182,69205,69001,69078,68961,69068,69044,68975,69090,69169,69140,69142,69110,69187,69057,69159,68971,69104,68994,69207,68953,68956,68959,68960,68954,69007,69059,69084,69106,68948,68946,68947,68945,69012,69081,69008,69028,69088,68979,69171,69112,69083,69131,69127,69040,68998,69185,68977,69069,69204,69089,69199,69153,69038,69082,69079,69186,69117,69196,69063,69064,69067,69070,69108,69086,69124,69160,69152,69115,68991,69189,69060,69111,69095,69114,69177,69109,69133,69048,69030,69172,69031,68989,69163,69103,69101,69009,69011,69151,69000,69123,68973,69051,69099,69094,69184,69144,69085,68986,69024,69125,69202,68981,69003,69098,69018,69021,68970,69004,69150,69167,69054,69128,69073,69096,68969,69056,69080,69134,69066,69132,69015,69170,69014,69154,69065,69035,69155,69033,69093,68968,69016,69006,68967,69168,69036,69097,69126,68978,69023,69136,69062,69141,69195,69191,69047,69166,69190,69049,69092,68995,69019,69156,69121,68983,69206,69148,69025,69198,69119,69032,68985,69200,69027,68996,69135,69178,69045,68987,69107,69091,69042,69197,69175,68976,68997,69183,68974,69076,69061,69201,69203,69100,69174,69193,69145,69041,69147,68982,69055,68972,68999,69071,69139,69022,69058,68980,69164,69149,69118,69052,69143,69010,69102,69180,69116,69138,69017,68984,68859,68698,68679,68738,68682,68826,68685,68684,68855,68777,68758,68839,68780,68806,68796,68814,68748,68772,68745,68829,68834,68819,68730,68848,68687,68713,68734,68737,68825,68721,68776,68769,68790,68723,68681,68704,68803,68851,68690,68795,68786,68775,68802,68840,68696,68736,68715,68788,68739,68703,68789,68699,68686,68756,68726,68809,68728,68702,68722,68697,68753,68824,68692,68707,68705,68847,68678,68755,68845,68846,68852,68799,68833,68676,68680,68880,68881,68882,68879,68816,68787,68749,68791,68773,68818,68771,68710,68843,68763,68770,68792,68751,68812,68779,68701,68783,68732,68683,68766,68742,68782,68836,68823,68815,68842,68804,68706,68693,68733,68752,68757,68844,68767,68817,68759,68828,68740,68700,68800,68794,68837,68764,68858,68822,68781,68761,68831,68808,68821,68854,68688,68709,68729,68765,68747,68716,68719,68857,68724,68750,68741,68762,68744,68689,68798,68850,68849,68793,68827,68841,68717,68718,68774,68810,68720,68691,68830,68760,68768,68735,68743,68805,68856,68853,68731,68601,68609,68562,68575,68557,68592,68572,68574,68589,68501,68516,68521,68518,68504,68519,68510,68514,68512,68524,68517,68560,68591,68582,68606,68594,68565,68586,68579,68554,68573,68600,68603,68567,68552,68578,68577,68471 | f | t
|
||||
16993 | | danger | ERROR: DDP messed up 2, failed to find orig_pref for a view pt=View for search=Sydney | t | t
|
||||
16994 | | danger | referrer=https://pa.depaoli.id.au/search/Sydney%20 | t | t
|
||||
16995 | | danger | ERROR: pref not found - dn=uid=mandy,ou=users,dc=depaoli,dc=id,dc=au, st=Sydney%20, s=<States(path_type='View', orig_search_term='', url='/view/16406', view_eid='16406', current=0, first_eid=0, last_eid=0, num_entries=0)>???? | t | t
|
||||
16997 | | danger | ERROR: viewing an id, but its not in eids OPT=<States(path_type='Search', orig_search_term='', url='/view/16406', view_eid='16406', current=0, first_eid=69336, last_eid=63222, num_entries=48440, orig_url='/view/16406', grouping='None', how_many=500, offset=0, size=128, root='static/Search', noo='Newest', folders=False, cwd='static/Search', orig_ptype=None, order='order_by(File.year.desc(),File.month.desc(),File.day.desc(),Entry.name.desc())', last_order='order_by(File.year,File.month,File.day,Entry.name)', last_order_raw='f.year, f.month, f.day, e.name')>, id=16406, eids=69336,69334,69328,69329,69330,69326,69327,69321,69304,69311,69320,69305,69316,69315,69314,69318,69312,69306,69310,69308,69309,69319,69317,69307,69313,69335,69337,69332,69331,69333,69299,69300,69302,69301,69303,69298,69297,69295,69290,69296,69275,69282,69285,69291,69283,69276,69279,69293,69287,69294,69281,69288,69284,69286,69292,69289,69277,69280,69274,69232,69257,69270,69230,69247,69258,69261,69256,69238,69248,69253,69236,69252,69267,69255,69235,69264,69233,69245,69229,69271,69273,69234,69278,69268,69249,69260,69244,69241,69265,69266,69272,69251,69242,69269,69237,69240,69239,69243,69250,69231,69262,69246,69254,69263,69259,69219,69227,69218,69225,69224,69222,69226,69220,69223,69216,69221,69228,69217,69212,69215,69213,69214,69211,69208,69158,69173,68992,69113,69130,69050,68990,68993,69039,69122,69192,69194,69165,69137,69105,69074,69162,69005,69029,69072,68988,69087,69037,69129,69053,69176,69181,69075,69034,69179,69120,69046,68966,69043,69161,69188,69002,69077,69020,69146,69013,69026,69182,69205,69001,69078,68961,69068,69044,68975,69090,69169,69140,69142,69110,69187,69057,69159,68971,69104,68994,69207,68953,68956,68959,68960,68954,69007,69059,69084,69106,68948,68946,68947,68945,69012,69081,69008,69028,69088,68979,69171,69112,69083,69131,69127,69040,68998,69185,68977,69069,69204,69089,69199,69153,69038,69082,69079,69186,69117,69196,69063,69064,69067,69070,69108,69086,69124,69160,69152,69115,68991,69189,69060,69111,69095,69114,69177,69109,69133,69048,69030,69172,69031,68989,69163,69103,69101,69009,69011,69151,69000,69123,68973,69051,69099,69094,69184,69144,69085,68986,69024,69125,69202,68981,69003,69098,69018,69021,68970,69004,69150,69167,69054,69128,69073,69096,68969,69056,69080,69134,69066,69132,69015,69170,69014,69154,69065,69035,69155,69033,69093,68968,69016,69006,68967,69168,69036,69097,69126,68978,69023,69136,69062,69141,69195,69191,69047,69166,69190,69049,69092,68995,69019,69156,69121,68983,69206,69148,69025,69198,69119,69032,68985,69200,69027,68996,69135,69178,69045,68987,69107,69091,69042,69197,69175,68976,68997,69183,68974,69076,69061,69201,69203,69100,69174,69193,69145,69041,69147,68982,69055,68972,68999,69071,69139,69022,69058,68980,69164,69149,69118,69052,69143,69010,69102,69180,69116,69138,69017,68984,68859,68698,68679,68738,68682,68826,68685,68684,68855,68777,68758,68839,68780,68806,68796,68814,68748,68772,68745,68829,68834,68819,68730,68848,68687,68713,68734,68737,68825,68721,68776,68769,68790,68723,68681,68704,68803,68851,68690,68795,68786,68775,68802,68840,68696,68736,68715,68788,68739,68703,68789,68699,68686,68756,68726,68809,68728,68702,68722,68697,68753,68824,68692,68707,68705,68847,68678,68755,68845,68846,68852,68799,68833,68676,68680,68880,68881,68882,68879,68816,68787,68749,68791,68773,68818,68771,68710,68843,68763,68770,68792,68751,68812,68779,68701,68783,68732,68683,68766,68742,68782,68836,68823,68815,68842,68804,68706,68693,68733,68752,68757,68844,68767,68817,68759,68828,68740,68700,68800,68794,68837,68764,68858,68822,68781,68761,68831,68808,68821,68854,68688,68709,68729,68765,68747,68716,68719,68857,68724,68750,68741,68762,68744,68689,68798,68850,68849,68793,68827,68841,68717,68718,68774,68810,68720,68691,68830,68760,68768,68735,68743,68805,68856,68853,68731,68601,68609,68562,68575,68557,68592,68572,68574,68589,68501,68516,68521,68518,68504,68519,68510,68514,68512,68524,68517,68560,68591,68582,68606,68594,68565,68586,68579,68554,68573,68600,68603,68567,68552,68578,68577,68471 | f | t
|
||||
16999 | | danger | ERROR: DDP messed up 2, failed to find orig_pref for a view pt=View for search=Sydney | t | t
|
||||
17000 | | danger | referrer=https://pa.depaoli.id.au/search/Sydney%20 | t | t
|
||||
17001 | | danger | ERROR: pref not found - dn=uid=mandy,ou=users,dc=depaoli,dc=id,dc=au, st=Sydney%20, s=<States(path_type='View', orig_search_term='', url='/view/16406', view_eid='16406', current=0, first_eid=0, last_eid=0, num_entries=0)>???? | t | t
|
||||
17003 | | danger | ERROR: viewing an id, but its not in eids OPT=<States(path_type='Search', orig_search_term='', url='/view/16406', view_eid='16406', current=0, first_eid=69336, last_eid=63222, num_entries=48440, orig_url='/view/16406', grouping='None', how_many=500, offset=0, size=128, root='static/Search', noo='Newest', folders=False, cwd='static/Search', orig_ptype=None, order='order_by(File.year.desc(),File.month.desc(),File.day.desc(),Entry.name.desc())', last_order='order_by(File.year,File.month,File.day,Entry.name)', last_order_raw='f.year, f.month, f.day, e.name')>, id=16406, eids=69336,69334,69328,69329,69330,69326,69327,69321,69304,69311,69320,69305,69316,69315,69314,69318,69312,69306,69310,69308,69309,69319,69317,69307,69313,69335,69337,69332,69331,69333,69299,69300,69302,69301,69303,69298,69297,69295,69290,69296,69275,69282,69285,69291,69283,69276,69279,69293,69287,69294,69281,69288,69284,69286,69292,69289,69277,69280,69274,69232,69257,69270,69230,69247,69258,69261,69256,69238,69248,69253,69236,69252,69267,69255,69235,69264,69233,69245,69229,69271,69273,69234,69278,69268,69249,69260,69244,69241,69265,69266,69272,69251,69242,69269,69237,69240,69239,69243,69250,69231,69262,69246,69254,69263,69259,69219,69227,69218,69225,69224,69222,69226,69220,69223,69216,69221,69228,69217,69212,69215,69213,69214,69211,69208,69158,69173,68992,69113,69130,69050,68990,68993,69039,69122,69192,69194,69165,69137,69105,69074,69162,69005,69029,69072,68988,69087,69037,69129,69053,69176,69181,69075,69034,69179,69120,69046,68966,69043,69161,69188,69002,69077,69020,69146,69013,69026,69182,69205,69001,69078,68961,69068,69044,68975,69090,69169,69140,69142,69110,69187,69057,69159,68971,69104,68994,69207,68953,68956,68959,68960,68954,69007,69059,69084,69106,68948,68946,68947,68945,69012,69081,69008,69028,69088,68979,69171,69112,69083,69131,69127,69040,68998,69185,68977,69069,69204,69089,69199,69153,69038,69082,69079,69186,69117,69196,69063,69064,69067,69070,69108,69086,69124,69160,69152,69115,68991,69189,69060,69111,69095,69114,69177,69109,69133,69048,69030,69172,69031,68989,69163,69103,69101,69009,69011,69151,69000,69123,68973,69051,69099,69094,69184,69144,69085,68986,69024,69125,69202,68981,69003,69098,69018,69021,68970,69004,69150,69167,69054,69128,69073,69096,68969,69056,69080,69134,69066,69132,69015,69170,69014,69154,69065,69035,69155,69033,69093,68968,69016,69006,68967,69168,69036,69097,69126,68978,69023,69136,69062,69141,69195,69191,69047,69166,69190,69049,69092,68995,69019,69156,69121,68983,69206,69148,69025,69198,69119,69032,68985,69200,69027,68996,69135,69178,69045,68987,69107,69091,69042,69197,69175,68976,68997,69183,68974,69076,69061,69201,69203,69100,69174,69193,69145,69041,69147,68982,69055,68972,68999,69071,69139,69022,69058,68980,69164,69149,69118,69052,69143,69010,69102,69180,69116,69138,69017,68984,68859,68698,68679,68738,68682,68826,68685,68684,68855,68777,68758,68839,68780,68806,68796,68814,68748,68772,68745,68829,68834,68819,68730,68848,68687,68713,68734,68737,68825,68721,68776,68769,68790,68723,68681,68704,68803,68851,68690,68795,68786,68775,68802,68840,68696,68736,68715,68788,68739,68703,68789,68699,68686,68756,68726,68809,68728,68702,68722,68697,68753,68824,68692,68707,68705,68847,68678,68755,68845,68846,68852,68799,68833,68676,68680,68880,68881,68882,68879,68816,68787,68749,68791,68773,68818,68771,68710,68843,68763,68770,68792,68751,68812,68779,68701,68783,68732,68683,68766,68742,68782,68836,68823,68815,68842,68804,68706,68693,68733,68752,68757,68844,68767,68817,68759,68828,68740,68700,68800,68794,68837,68764,68858,68822,68781,68761,68831,68808,68821,68854,68688,68709,68729,68765,68747,68716,68719,68857,68724,68750,68741,68762,68744,68689,68798,68850,68849,68793,68827,68841,68717,68718,68774,68810,68720,68691,68830,68760,68768,68735,68743,68805,68856,68853,68731,68601,68609,68562,68575,68557,68592,68572,68574,68589,68501,68516,68521,68518,68504,68519,68510,68514,68512,68524,68517,68560,68591,68582,68606,68594,68565,68586,68579,68554,68573,68600,68603,68567,68552,68578,68577,68471 | f | t
|
||||
(10 rows)
|
||||
|
||||
|
||||
|
||||
|
||||
Another bug - confirmed... searched for wandana, went into viewer, arrowed over
|
||||
to other images, rotated one (it had a URL of view/xxxx), reloaded page and got
|
||||
this:
|
||||
|
||||
ERROR: viewing an id, but its not in eids OPT=, id=16532, eids=69342,69341,69339,69340,69343,69336,69334,69328,69329,69330,69326,69327,69321,69304,69311,69320,69305,69316,69315,69314,69318,69312,69306,69310,69308,69309,69319,69317,69307,69313,69335,69337,69332,69331,69333,69299,69300,69302,69301,69303,69298,69297,69295,69290,69296,69275,69282,69285,69291,69283,69293,69287,69294,69281,69288,69284,69247,69258,69261,69256,69253,69267,69255,69235,69264
|
||||
|
||||
Sorry, viewing data is confused, cannot view this image nowClearing out all states. This means browser back buttons will not work, please start a new tab and try again
|
||||
|
||||
|
||||
|
||||
Also, rotated this image: dcp_0975.jpg -> 90deg, and it seemed to flip 180
|
||||
|
||||
|
||||
BUG-137: after moving/refiling photos, the next shift-click is out of order (reload fixes it)
|
||||
BUG-138: Placeholder for all the ways we can get the front-end confused:
|
||||
---> JUST fix all these BUGs (relating to confused/lost state) by revisiting the overally complex way I remember state and my position in a list (probably FAR easier, to make an initial sql just save all eids, and then not try to recreate that list ever again and not care how I got into the list). Can attach a "running server-side sequence number", and if old sequence, and the original eid list results in a failure, then just pop up that the saved list is no longer valid, and ask user to re-do their search/list..."
|
||||
|
||||
@@ -6,7 +6,7 @@ ARG PA_ID
|
||||
WORKDIR /code
|
||||
# do this first, allowing layer / build efficiencies if only code changes...
|
||||
# sudo used in dev container
|
||||
RUN apt-get update && apt-get -y install libpq-dev mediainfo cmake libgl1-mesa-glx libglib2.0-0 libjpeg-turbo-progs ffmpeg git sudo
|
||||
RUN apt-get update && apt-get -y install libpq-dev mediainfo cmake libglib2.0-0 libjpeg-turbo-progs ffmpeg git sudo
|
||||
COPY requirements.txt .
|
||||
RUN pip3 install --upgrade pip && pip3 install -r requirements.txt && pip3 install --upgrade pillow
|
||||
|
||||
@@ -16,11 +16,11 @@ RUN groupadd -g ${PA_ID} pauser && useradd -r -u ${PA_ID} -g ${PA_ID} pauser
|
||||
COPY . .
|
||||
|
||||
# set up static dir and subdirs
|
||||
RUN mkdir -p static/Bin && mkdir static/Import && mkdir static/Storage && mkdir static/Metadata
|
||||
RUN mkdir -p static/Bin && mkdir -p static/Import && mkdir -p static/Storage && mkdir -p static/Metadata
|
||||
|
||||
EXPOSE 80
|
||||
RUN cat /dev/urandom | head -c 50 | md5sum | head -c 32 > /code/.sk && chmod 600 .sk && chown pauser:pauser .sk
|
||||
RUN date > internal/build-date.txt && git log -n 15 > internal/git-log.txt && ln -s /code/TODO /code/internal/TODO && ln -s /code/BUGs /code/internal/BUGs
|
||||
RUN TZ="Australia/Melbourne" date > internal/build-date.txt && git log -n 15 > internal/git-log.txt && ln -s /code/TODO /code/internal/TODO && ln -s /code/BUGs /code/internal/BUGs
|
||||
RUN rm -rf .git
|
||||
RUN chown pauser:pauser /code && chown pauser:pauser ./static && chown pauser:pauser ./static/*
|
||||
|
||||
|
||||
6
README
6
README
@@ -4,7 +4,7 @@ to edit src:
|
||||
|
||||
git....
|
||||
cd ~/src ; # or wherever you want the code to live
|
||||
git clone git@192.168.0.2:photoassistant
|
||||
git clone https://git.depaoli.id.au/ddp/photoassistant.git
|
||||
|
||||
|
||||
ubuntu packages:
|
||||
@@ -49,10 +49,10 @@ pip3 list | tail -n +3 | grep -v mysqlclient | grep -v duplicity | grep -v gp
|
||||
To run debug version of web server (gunicorn deprecates FLASK_ENV, so co-exist for now):
|
||||
|
||||
# flask 2.2.2+ (in venv .python)
|
||||
FLASK_APP=main ENV=development ./.python/bin/flask --debug run --host=192.168.0.2
|
||||
FLASK_APP=main ENV=development ./.python/bin/flask --debug run --host=mara.ddp.net
|
||||
|
||||
to run prod version of web server:
|
||||
gunicorn --bind="192.168.0.2:5000" --threads=2 --workers=2 main:app
|
||||
gunicorn --bind="mara.ddp.net:5000" --threads=2 --workers=2 main:app
|
||||
|
||||
Also have to run the job manager for jobs to work:
|
||||
ENV="development" python3 pa_job_manager.py
|
||||
|
||||
24
TODO
24
TODO
@@ -1,4 +1,28 @@
|
||||
### major fix - go to everywhere I call GetEntries(), and redo the logic totally...
|
||||
* firstly, run the query as per normal, but get just the matched eids into an entry_lst
|
||||
* make a unique query_id for this entry_lst, and store entry_ids into "query" table, with a unique query_id
|
||||
* take most of pa_user_state that relates to query state and move it to the "query" table per query_id
|
||||
* pa_user_state then becomes defaults for next query (so how_many, noo, etc)
|
||||
|
||||
* we can age out queries form the query_table after a few months?
|
||||
* client side always has query_id. IF DB does not have query_id, then its really old? - just say so...
|
||||
|
||||
* client side takes query_id, entry_lst, current_eid, offset, first/last_eid, etc. as part of its first route / html creation.
|
||||
* it then decides based on all this to GetEntryDetails( subset of entry_lst ) <- needs new route
|
||||
* IN THEORY some of the subset of entry_lst don't exist -- BUT, we can handle that on response, e.g. say my query used to have 1,2,3, and since then another user/action deleted 2:
|
||||
- I ask for details on 1,2,3 and get back details on 1,3 only.
|
||||
- On client-side, I can say, since you ran this query, data in PA has changed - image#2 is no longer in PA.
|
||||
Please run a new query (or bonus points, maybe have enough of the original query to note this and ask, do you want to ignore changes, or re-run query and get latest data?)
|
||||
* client can go fwd or back in the entry_lst same as now (disabling buttons as needed), BUT as entry_lst is NOT recreated per page move, then no chance to get confused about first/last
|
||||
* client side:
|
||||
* for real chance to stop confusion, instead of removing deleted images from DOM, we should gray them out and put a big Del (red circle with line?) though it as overlay.
|
||||
* Create another table is entry_ammendments - note the deletions, rotations, flips of specific eids - then reproduce that on the client side visually as needed
|
||||
- at least grayed-out, to indicate a pending action is not complete.
|
||||
- When job that flips, rotates, deletes completes then lets update the query details (e.g. remove eids, or remove the ammendments)
|
||||
- this actually is quite an improvement, if someone is deleting 2 as per above, I will see that as a pending change in my unrelated query, ditto flips, etc.
|
||||
|
||||
### GENERAL
|
||||
* jobs for AI should show path name
|
||||
* rm dups job should show progress bar
|
||||
* in viewer, there is no move button (maybe add one?)
|
||||
* consider doing duplicates before AI, and if there are say 100s+, then maybe pause the AI work
|
||||
|
||||
22
files.py
22
files.py
@@ -238,7 +238,9 @@ def GetEntriesInSearchView( OPT ):
|
||||
search_term=search_term.replace('*', '%' )
|
||||
if 'AI:' in OPT.orig_search_term:
|
||||
search_term = search_term.replace('AI:','')
|
||||
join=f"Entry.query.join(File).join(FaceFileLink).join(Face).join(FaceRefimgLink).join(Refimg).join(PersonRefimgLink).join(Person).filter(Person.tag.ilike('%{search_term}%'))"
|
||||
join=f"Entry.query.join(File).join(FaceFileLink).join(Face).join(FaceRefimgLink).join(Refimg).join(PersonRefimgLink).join(Person).filter(Person.tag == search_term)"
|
||||
else:
|
||||
join=f"Entry.query.join(File).join(FaceFileLink).join(Face).join(FaceRefimgLink).join(Refimg).join(PersonRefimgLink).join(Person).filter(Person.tag.ilike('%{search_term}%'))"
|
||||
if 'AI:' in OPT.orig_search_term:
|
||||
all_entries = eval( f"{join}.{OPT.order}.offset(OPT.offset).limit(OPT.how_many).all()")
|
||||
else:
|
||||
@@ -278,7 +280,7 @@ def GetEntriesInSearchView( OPT ):
|
||||
if OPT.last_eid == 0:
|
||||
by_fname= f"select e.id from entry e where e.name ilike '%%{search_term}%%'"
|
||||
by_dirname=f"select e.id from entry e, entry_dir_link edl where edl.entry_id = e.id and edl.dir_eid in ( select d.eid from dir d where d.rel_path ilike '%%{search_term}%%' )"
|
||||
by_ai =f"select e.id from entry e, face_file_link ffl, face_refimg_link frl, person_refimg_link prl, person p where e.id = ffl.file_eid and frl.face_id = ffl.face_id and frl.refimg_id = prl.refimg_id and prl.person_id = p.id and p.tag ilike '%%{search_term}%%'"
|
||||
by_ai =f"select e.id from entry e, face_file_link ffl, face_refimg_link frl, person_refimg_link prl, person p where e.id = ffl.file_eid and frl.face_id = ffl.face_id and frl.refimg_id = prl.refimg_id and prl.person_id = p.id and p.tag = '{search_term}'"
|
||||
|
||||
if 'AI:' in OPT.orig_search_term:
|
||||
sel_no_order=f"select e.*, f.* from entry e, file f where e.id=f.eid and e.id in ( {by_ai} ) "
|
||||
@@ -448,7 +450,10 @@ def files_rbp():
|
||||
@app.route("/search/<search_term>", methods=["GET", "POST"])
|
||||
@login_required
|
||||
def search(search_term):
|
||||
# print( f"req={request}" )
|
||||
OPT=States( request )
|
||||
# print( f"OPT={OPT}" )
|
||||
|
||||
# if we posted to get here, its a change in State, so save it to pa_user_state, and go back to the GET version or URL
|
||||
if request.method=="POST":
|
||||
redirect("/search/"+search_term)
|
||||
@@ -620,7 +625,6 @@ def view_list():
|
||||
resp['objs'][e.id]['faces'] = []
|
||||
|
||||
# put face data back into array format (for js processing)
|
||||
fid=0
|
||||
for face in e.file_details.faces:
|
||||
fd= {}
|
||||
fd['x'] = face.face_left
|
||||
@@ -628,10 +632,10 @@ def view_list():
|
||||
fd['w'] = face.w
|
||||
fd['h'] = face.h
|
||||
if face.refimg:
|
||||
fd['pid'] = face.refimg.person.id
|
||||
fd['who'] = face.refimg.person.tag
|
||||
fd['distance'] = round(face.refimg_lnk.face_distance,2)
|
||||
resp['objs'][e.id]['faces'].append(fd)
|
||||
fid+=1
|
||||
|
||||
eids=eids.rstrip(",")
|
||||
lst = eids.split(',')
|
||||
@@ -648,12 +652,12 @@ def view_list():
|
||||
resp['last_eid']=OPT.last_eid
|
||||
resp['eids']=eids
|
||||
resp['offset']=OPT.offset
|
||||
print( f"BUG-DEBUG: /view_list route #1 - OPT={OPT}, eids={eids} ")
|
||||
# print( f"BUG-DEBUG: /view_list route #1 - OPT={OPT}, eids={eids} ")
|
||||
# save pref to keep the new current value, first/last
|
||||
pref=PA_UserState.query.filter(PA_UserState.pa_user_dn==current_user.dn,PA_UserState.orig_ptype==OPT.orig_ptype,PA_UserState.view_eid==OPT.view_eid).first()
|
||||
print( f"BUG-DEBUG: /view_list route #2 - OPT={OPT}, eids={eids} ")
|
||||
# print( f"BUG-DEBUG: /view_list route #2 - OPT={OPT}, eids={eids} ")
|
||||
UpdatePref( pref, OPT )
|
||||
print( f"BUG-DEBUG: /view_list route #3 - OPT={OPT}, eids={eids} ")
|
||||
# print( f"BUG-DEBUG: /view_list route #3 - OPT={OPT}, eids={eids} ")
|
||||
|
||||
return make_response( resp )
|
||||
|
||||
@@ -701,7 +705,7 @@ def view(id):
|
||||
imp_path = setting.import_path
|
||||
st_path = setting.storage_path
|
||||
bin_path = setting.recycle_bin_path
|
||||
print( f"BUG-DEBUG: /view/id GET route - OPT={OPT}, eids={eids}, current={int(id)} ")
|
||||
# print( f"BUG-DEBUG: /view/id GET route - OPT={OPT}, eids={eids}, current={int(id)} ")
|
||||
return render_template("viewer.html", current=int(id), eids=eids, objs=objs, OPT=OPT, NMO_data=NMO_data, imp_path=imp_path, st_path=st_path, bin_path=bin_path )
|
||||
|
||||
##################################################################################
|
||||
@@ -712,7 +716,7 @@ def view(id):
|
||||
def view_img_post(id):
|
||||
# set pa_user_states...
|
||||
OPT=States( request )
|
||||
print( f"BUG-DEBUG: /view/id POST route - OPT={OPT}, id={id} ")
|
||||
# print( f"BUG-DEBUG: /view/id POST route - OPT={OPT}, id={id} ")
|
||||
# then use back-button friendly URL (and use pa_user_states to view the right image in the right list
|
||||
return redirect( "/view/" + id );
|
||||
|
||||
|
||||
@@ -237,6 +237,8 @@ $(document).ready( function()
|
||||
else if( objs[current].faces[i].who )
|
||||
{
|
||||
item_list['match']={ 'name': objs[current].faces[i].who, 'which_face': i, 'id': objs[current].faces[i].id }
|
||||
item_list['match_add_refimg']={ 'name': 'Add this as refimg for ' + objs[current].faces[i].who,
|
||||
'person_id': objs[current].faces[i].pid, 'who': objs[current].faces[i].who, 'which_face': i, 'id': objs[current].faces[i].id, }
|
||||
item_list['wrong_person']={ 'name': 'wrong person', 'which_face': i, 'id': objs[current].faces[i].id }
|
||||
}
|
||||
else
|
||||
@@ -494,10 +496,23 @@ function FaceDBox(key, item)
|
||||
{
|
||||
div+=AddSearch( 'Click one of the link(s) below to add this face as a reference image to the person:<br><br>', key, face_pos );
|
||||
}
|
||||
if ( key == 'match_add_refimg' )
|
||||
{
|
||||
func='AddRefimgTo('+item[key]['person_id']+',\''+key+'\''
|
||||
func_sn=func+ ', true )'
|
||||
func_ao=func+ ', false )'
|
||||
div+=`<script>console.log( "AddExistingFaceAsRefimgToMatchedPerson()" )</script>`
|
||||
div+="Confirm you wish to add this face as a reference image for " + item[key]['who']
|
||||
div+= '<div class="col">' + item[key]['who'] + '</div><div class="col input-group">'
|
||||
div+= '<button onClick="'+func_sn+'" class="btn btn-success py-1 input-group-prepend">Add & search now</button> '
|
||||
div+= '<button onClick="'+func_ao+'" class="btn btn-outline-success py-1 input-group-append">Add only</button></div>'
|
||||
|
||||
}
|
||||
if ( key == 'wrong_person' )
|
||||
{
|
||||
div+='<br>wrong person, so mark this as the wrong person/refimg connection, for face#' + item[key]['which_face']
|
||||
div+='<br>face db id: ' + item[key]['id']
|
||||
div += '<br>not yet'
|
||||
}
|
||||
if( /NMO_/.test(key) )
|
||||
{
|
||||
|
||||
22
job.py
22
job.py
@@ -78,13 +78,6 @@ class PA_JobManager_Message(PA,db.Model):
|
||||
job = db.relationship ("Job" )
|
||||
|
||||
|
||||
################################################################################
|
||||
# GetJM_Message: used in html to display any message for this front-end
|
||||
################################################################################
|
||||
def GetJM_Message():
|
||||
msg=PA_JPA_JobManager_MessageobManager_Message.query.first()
|
||||
return msg
|
||||
|
||||
################################################################################
|
||||
# Used in main html to show a red badge of # jobs to draw attention there are
|
||||
# active jobs being processed in the background
|
||||
@@ -178,7 +171,9 @@ def jobs():
|
||||
jobs = Job.query.order_by(Job.id.desc()).all()
|
||||
else:
|
||||
page_title='Job list (recent)'
|
||||
jobs = Job.query.filter( Job.last_update >= (func.now() - func.cast(concat(settings.job_archive_age, 'DAYS'), INTERVAL)) ).order_by(Job.id.desc()).all()
|
||||
# work out cutoff in python (used to do this in sql and it was too slow)
|
||||
cutoff = datetime.now() - timedelta(days=settings.job_archive_age)
|
||||
jobs = Job.query.filter( Job.last_update >= cutoff ).order_by(Job.id.desc()).all()
|
||||
return render_template("jobs.html", jobs=jobs, page_title=page_title)
|
||||
|
||||
|
||||
@@ -344,3 +339,14 @@ def _jinja2_filter_datetime(date, fmt=None):
|
||||
return date.strftime("%d/%m/%Y %I:%M:%S %p")
|
||||
else:
|
||||
return "N/A"
|
||||
|
||||
################################################################################
|
||||
# allow a way to force the messages to be deleted if really needed - its a bit
|
||||
# lame, but a quick fix
|
||||
################################################################################
|
||||
@app.route('/force_clear')
|
||||
@login_required
|
||||
def force_clear():
|
||||
PA_JobManager_Message.query.delete();
|
||||
db.session.commit()
|
||||
return redirect("/")
|
||||
|
||||
7
main.py
7
main.py
@@ -60,15 +60,18 @@ except Exception:
|
||||
app.config['SECRET_KEY'] = b'my_insecure_PA_token_with_random_2134876adsfjhlkasdf87'
|
||||
|
||||
# ldap config vars: (the last one is required, or python ldap freaks out)
|
||||
app.config['LDAP_HOST'] = 'mara.ddp.net'
|
||||
app.config['LDAP_HOST'] = 'ldap.ddp.net'
|
||||
app.config['LDAP_BASE_DN'] = 'dc=depaoli,dc=id,dc=au'
|
||||
app.config['LDAP_USER_DN'] = 'ou=users'
|
||||
app.config['LDAP_GROUP_DN'] = 'ou=groups'
|
||||
app.config['LDAP_USER_RDN_ATTR'] = 'cn'
|
||||
app.config['LDAP_USER_RDN_ATTR'] = 'uid'
|
||||
app.config['LDAP_USER_LOGIN_ATTR'] = 'uid'
|
||||
app.config['LDAP_BIND_USER_DN'] = None
|
||||
app.config['LDAP_BIND_USER_PASSWORD'] = None
|
||||
app.config['LDAP_GROUP_OBJECT_FILTER'] = '(objectclass=posixGroup)'
|
||||
app.config['LDAP_BIND_USER_DN'] = None
|
||||
app.config['LDAP_BIND_USER_PASSWORD'] = None
|
||||
|
||||
|
||||
db = SQLAlchemy(app) # create the (flask) sqlalchemy connection
|
||||
ma = Marshmallow(app) # set up Marshmallow - data marshalling / serialising
|
||||
|
||||
@@ -101,6 +101,7 @@
|
||||
{% if config.ENV != "production" %}
|
||||
<a class="dropdown-item" href="{{url_for('force_scan')}}">Force Scan (delete data & rebuild)</a>
|
||||
{% endif %}
|
||||
<a class="dropdown-item" href="{{url_for('force_clear')}}">Clear Messages (Force)</a>
|
||||
<a class="dropdown-item" href="{{url_for('wake_up')}}">Force wake the job manager</a>
|
||||
</div class="dropdow-menu">
|
||||
</div class="nav-item dropdown">
|
||||
|
||||
@@ -15,6 +15,11 @@
|
||||
p.icon_url = '{{p.icon_url}}'
|
||||
move_paths.push(p)
|
||||
{% endfor %}
|
||||
|
||||
document.OPT = '{{OPT}}'
|
||||
document.entries = '{{entry_data}}'
|
||||
document.how_many = '{{OPT.how_many}}'
|
||||
document.entries_len = '{{entry_data|length}}'
|
||||
</script>
|
||||
|
||||
<div class="container-fluid">
|
||||
@@ -68,7 +73,7 @@
|
||||
</button>
|
||||
<span class="sm-txt my-auto"> {{OPT.how_many}} files </span>
|
||||
{% set nxt_disabled="" %}
|
||||
{% if not entry_data or entry_data and entry_data|length < OPT.how_many|int %}
|
||||
{% if not entry_data or entry_data|length < OPT.how_many|int %}
|
||||
{% set nxt_disabled="disabled" %}
|
||||
{% endif %}
|
||||
<button aria-label="next" id="next" {{nxt_disabled}} name="next" class="next sm-txt btn btn-outline-secondary">
|
||||
|
||||
@@ -46,6 +46,7 @@
|
||||
{% for face in objs[id].file_details.faces %}
|
||||
data = { 'id': '{{face.id}}', 'x': '{{face.face_left}}', 'y': '{{face.face_top}}', 'w': '{{face.w}}', 'h':'{{face.h}}' }
|
||||
{% if face.refimg %}
|
||||
data['pid']='{{face.refimg.person.id}}'
|
||||
data['who']='{{face.refimg.person.tag}}'
|
||||
data['distance']="{{face.refimg_lnk.face_distance|round(2)}}"
|
||||
{% endif %}
|
||||
|
||||
@@ -2,10 +2,12 @@
|
||||
|
||||
sudo -u pauser ENV=${ENV} python3 -u pa_job_manager.py &> /var/log/pa_job_manager.out &
|
||||
if [ "$ENV" == "production" ]; then
|
||||
# su pauser -g pauser -c 'ENV="production" python3 -u /code/pa_job_manager.py' &> /var/log/pa_job_manager.out &
|
||||
sudo -u pauser gunicorn --bind=0.0.0.0:80 --workers=4 --threads=16 main:app --env ENV="production" --error-logfile gunicorn.error.log --access-logfile gunicorn.log --capture-output
|
||||
elif [ "$ENV" == "container" ]; then
|
||||
sudo -u pauser gunicorn --bind=0.0.0.0:80 --workers=1 --threads=1 main:app --env ENV="container" --reload --capture-output
|
||||
# move into src tree so padev responds to code changes
|
||||
cd /home/ddp/src/photoassistant
|
||||
# push container logs/errors to stdout so docker logs works
|
||||
sudo -u pauser gunicorn --bind=0.0.0.0:80 --workers=1 --threads=1 main:app --env ENV="container" --reload --capture-output --log-file - --error-logfile -
|
||||
else
|
||||
echo "Not sure which ENV ($ENV) we are running, set up for DEV and dont run job manager (jic)"
|
||||
sudo -u pauser gunicorn --bind=0.0.0.0:80 --workers=1 --threads=1 main:app --env ENV="development" --error-logfile gunicorn.error.log --access-logfile gunicorn.log --capture-output --enable-stdio-inheritance --reload
|
||||
|
||||
Reference in New Issue
Block a user