91b5e1b6d2b572bb0a5c31a9a2f28559d988e5eb
In here we can put instructions on how to run this / any general info
to edit src:
git....
CAM: fill this in pls
ubuntu packages:
sudo apt-get install -y mediainfo cmake python3-flask
pip packages:
* pymediainfo
* PIL (should be there by default)
- on ubuntu, need to do: pip3 install --upgrade pillow --user
* ExifRead
* opencv-python
* datetime
* pytz
* face_recognition
* flask_login
* flask-ldap3-login
#### dlib (might need to install this before face_recognitioin, but it might not be needed, cmake clearly was)
upstream packages...
mkdir static/upstream
cd static/upstream
mkdir bootstrap-4.6.0-dist
cd bootstrap-4.6.0-dist
mkdir css
# for boostrap:
wget https://cdn.jsdelivr.net/npm/bootstrap@4.6.0/dist/css/bootstrap.min.css
mkdir js
# to note we might need bootstrap.bundle.min.js if we use new features?
wget https://cdn.jsdelivr.net/npm/bootstrap@4.6.0/dist/js/bootstrap.min.js
# for jquery
https://code.jquery.com/jquery-3.6.0.min.js
# for jquery/smart context menu
wget https://cdnjs.cloudflare.com/ajax/libs/jquery-contextmenu/2.9.2/jquery.contextMenu.css
wget https://cdnjs.cloudflare.com/ajax/libs/jquery-contextmenu/2.9.2/jquery.contextMenu.min.js
wget https://cdnjs.cloudflare.com/ajax/libs/jquery-contextmenu/2.9.2/jquery.contextMenu.min.js.map
wget https://cdnjs.cloudflare.com/ajax/libs/jquery-contextmenu/2.9.2/jquery.ui.position.min.js
### I tried to update via pip3, with this:
sudo apt-get install libcairo2-dev libjpeg-dev libgif-dev libgirepository1.0-dev libcups2-dev
pip3 list | tail -n +3 | grep -v mysqlclient | grep -v duplicity | grep -v gpg | awk ' { print $1 } ' | xargs pip3 install --upgrade
To run debug version of web server:
FLASK_APP=main FLASK_ENV=development flask run --host=192.168.0.2
to run prod version of web server:
gunicorn --bind="192.168.0.2:5000" --threads=2 --workers=2 main:app
Also have to run the job manager for jobs to work:
FLASK_ENV="development" python3 pa_job_manager.py
To rebuild DB from scratch/empty data:
###BE CAREFUL this completely destroys the database contents and starts from
#scratch, when we get real data, we will instead cp a backup file not the
#tables.sql
( cd /srv/docker/config/ ; sudo docker-compose stop padb_dev ; yes | sudo docker-compose rm padb_dev ; sudo rm -rf /srv/docker/container/padb_dev/data/ ; sudo cp /home/ddp/src/photoassistant/tables.sql /srv/docker/container/padb_dev/docker-entrypoint-initdb.d/tables.sql ; sudo docker-compose up -d padb_dev ) &
To get back a 'working' but scanned set of data:
# make a backup and store it in DB_BACKUP:
sudo docker exec -it padb bash
root@2881f871e1c2:/# pg_dump --user=pa pa > /docker-entrypoint-initdb.d/tables.sql
cp /srv/docker/container/padb/docker-entrypoint-initdb.d/tables.sql /home/ddp/src/photoassistant/DB_BACKUP/
mv /home/ddp/src/photoassistant/DB_BACKUP/tables.sql /home/ddp/src/photoassistant/DB_BACKUP/`date +%Y%m%d-tables.sql`
gzip /home/ddp/src/photoassistant/DB_BACKUP/`date +%Y%m%d-tables.sql`
# mv /home/ddp/src/photoassistant/DB_BACKUP/`date +%Y%m%d-tables.sql`.gz /home/ddp/src/photoassistant/DB_BACKUP/<meaningfulname>......
# start db, using new tables.sql
DEV:
( cd /srv/docker/config/ ; sudo docker-compose stop padb_dev ; sudo docker-compose rm -f padb_dev ; sudo rm -rf /srv/docker/container/padb_dev/data/ ; sudo cp /home/ddp/src/photoassistant/tables.sql /srv/docker/container/padb_dev/docker-entrypoint-initdb.d/tables.sql ; sudo docker-compose up -d padb_dev )
### decide what tables.sql you want, e.g.
sudo bash
# gunzip -c /home/ddp/src/photoassistant/DB_BACKUP/20200126-all-imported-no-duplicates.sql.gz > /srv/docker/container/padb/docker-entrypoint-initdb.d/tables.sql
( cd /srv/docker/config/ ; sudo docker-compose stop padb ; yes | sudo docker-compose rm padb ; sudo rm -rf /srv/docker/container/padb/data/ ; sudo docker-compose up -d padb )
HANDY SQLs/commands:
# long-running AI job (in this case #46), which is not committing joblog per file, and isnt tracking counts properly (temporary bug)
sudo docker exec -it padb bash
echo 'select * from joblog where job_id = 46;' | psql --user=pa pa | grep 'ooking for' | awk '{ print $15 } ' | sort -u | wc -l
# how many entries are in a path
sudo docker exec -it padb bash
psql --user=pa pa
select count(entry_id) from entry_dir_link where dir_eid in ( select distinct dir_eid from path_dir_link where path_id = 2 );
# how many Images are in a path
sudo docker exec -it padb bash
psql --user=pa pa
select count(distinct e.id) from entry e, entry_dir_link edl where e.type_id = 1 and e.id = edl.entry_id and edl.dir_eid in ( select distinct dir_eid from path_dir_link where path_id = 2 );
# get abs filenames of matching files (for liz person.tag, but could easily add
# d.rel_path like 'liz' too :
select '"'||replace(replace(p.path_prefix,'static/Storage/',''),'static/Import/', '')||'/'||d.rel_path||'/'||e.name||'"' from entry e, entry_dir_link edl, path_dir_link pdl, path p, dir d where e.id = edl.entry_id and edl.dir_eid = pdl.dir_eid and pdl.path_id = p.id and d.eid = edl.dir_eid and e.id in ( select e.id from entry e, face_file_link ffl, face_refimg_link frl, person_refimg_link prl, person p where e.id = ffl.file_eid and ffl.face_id = frl.face_id and frl.refimg_id = prl.refimg_id and prl.person_id = p.id and p.tag = 'liz' );
Description
PhotoAssistant - allows me to remove duplicate photos, more easily create folders to store common photos and uses python AI libraries to find faces and allows searching across all
Languages
Python
67.5%
HTML
22.1%
JavaScript
9.3%
Shell
0.7%
Dockerfile
0.4%