new BUG-21: second pass, the file structures in the DB are broken

2021-02-11 20:07:15 +11:00
parent e6bc751844
commit 2d88403756
1 changed files with 60 additions and 0 deletions
--- a/60
+++ b/60
@@ -1 +1,61 @@
 ### Next: 21
+
+BUG-21: the datastructure of dir/files is actually quite broken on a real import (seems to be on second/subsequent imports -- the code to remove deleted files, is not 'seeing all files?'
+pa=# select count(edl.dir_eid), d.path_prefix from entry_dir_link edl, dir d where edl.dir_eid = d.eid and edl.dir_eid in ( select eid from dir ) group by d.path_prefix;
+ count |                    path_prefix
+-------+----------------------------------------------------
+   101 | static/CAM_UPLOADS/M's Phone/Camera Roll
+     1 | static/CAM_UPLOADS/M's Phone
+  2979 | static/CAM_UPLOADS/Mandy's Phone/Camera Roll
+     1 | static/CAM_UPLOADS
+   675 | static/CAM_UPLOADS/M's Galaxy A51/Camera Roll
+  3656 | static/CAM_UPLOADS/Damien's Phone/Camera Roll
+     1 | static/CAM_UPLOADS/Damien's Phone/Camera Roll/0000
+     1 | static/CAM_UPLOADS/M's Galaxy A51
+     1 | static/CAM_UPLOADS/Mandy's Phone
+     1 | static/CAM_UPLOADS/Damien's Phone
+(10 rows)
+
+pa=# select * from dir;
+ eid  |                    path_prefix                     | num_files |  last_import_date
+------+----------------------------------------------------+-----------+--------------------
+    2 | static/CAM_UPLOADS/Mandy's Phone                   |         1 | 1613024867.8238187
+    3 | static/CAM_UPLOADS/Mandy's Phone/Camera Roll       |      2999 |  1613024872.385247
+ 3003 | static/CAM_UPLOADS/M's Galaxy A51                  |         1 |  1613024872.387184
+ 3004 | static/CAM_UPLOADS/M's Galaxy A51/Camera Roll      |      2400 | 1613024875.9811678
+ 5405 | static/CAM_UPLOADS/Damien's Phone                  |         1 |  1613024875.983697
+ 5406 | static/CAM_UPLOADS/Damien's Phone/Camera Roll      |      3658 | 1613024883.1730359
+ 9058 | static/CAM_UPLOADS/Damien's Phone/Camera Roll/0000 |         1 | 1613024883.1779747
+ 9066 | static/CAM_UPLOADS/M's Phone                       |         1 | 1613024883.1806386
+ 9067 | static/CAM_UPLOADS/M's Phone/Camera Roll           |       101 | 1613024883.3877454
+    1 | static/CAM_UPLOADS                                 |      9167 | 1613024867.8217578
+
+
+############
+SO FILE counts are really broken, a slight bit off on a couple of dirs, but say M51, its 675 in DB, and the file tree walk saw 2400 -- and an ls -lR confirms...
+
+
+checking by hand for the first file in M51 dir:
+
+ddp@mara:~/src/photoassistant$ ls -l /export/docker/storage/photos/CAM_UPLOADS/M\'s\ Galaxy\ A51/Camera\ Roll/20190105_175219.jpg
+-rw-r--r-- 1 mythtv mythtv 2.6M Dec 28 20:37 "/export/docker/storage/photos/CAM_UPLOADS/M's Galaxy A51/Camera Roll/20190105_175219.jpg"
+
+pa=# select * from entry where name = '20190105_175219.jpg';
+ id  |        name         | type_id | exists_on_fs
+-----+---------------------+---------+--------------
+ 951 | 20190105_175219.jpg |       2 | t
+(1 row)
+
+pa=# select * from entry_dir_link where entry_id = 951;
+ entry_id | dir_eid
+----------+---------
+      951 |       3
+(1 row)
+
+pa=# select * from dir  where eid = 3;
+ eid |                 path_prefix                  | num_files | last_import_date
+-----+----------------------------------------------+-----------+-------------------
+   3 | static/CAM_UPLOADS/Mandy's Phone/Camera Roll |      2999 | 1613024872.385247
+
+
+so the file is really in the M51 dir, but is in the DB in Mandy's phone dir instead...