Cataloging Script
Document Updated: 2020-02-09
Once data is downloaded and unpacked, the data can be cataloged in order to more quickly access key information such as GPS data for each image, file size, and date taken to help with statistical and spacial analysis.
Command-Line Quick Reference
Parameter | Argument(s) | Function | Default Value |
---|---|---|---|
--path , -p |
<path> | The path on your computer to start looking for files and save the catalog to | /data/archive_cache |
--extension , -e |
<extension> | Only add files with this extension to the catalog | jpg |
--fields , -e |
<Set> | Only include these fields from the .geom and system values | (See Note) |
--debug , -d |
Include parameter to output debug information to console | False | |
--verbosity , -v |
<level> | The amount of information to log to console (0 = only errors, 1 = low, 2 = medium, 3 = high) | 1 |
Modifying Catalog Contents
Any field within the `.geom` file of all images can be added to the catalog by simply changing the `--fields` flag.
The default fields are `{'file', 'storm_id', 'archive', 'image', 'date', 'size', 'geom_checksum', 'll_lat',
'll_lon', 'lr_lat', 'lr_lon', 'ul_lat', 'ul_lon', 'ur_lat', 'ur_lon'}`.
Warning
If the command doesn't understand one of your parameters, try
again, but with quotes around any argument. Command-line treats anything separated by a space as two separate
arguments, so something like -p C:\some dir\data\archive_cache\Florence
should instead be
-p "C:\some dir\data\archive_cache\Florence"
Catalog Snippet Example (data/catalogs/v2/Florence.csv)
file storm_id archive image size date ur_lat lr_lon ur_lon ll_lat ul_lon ll_lon lr_lat geom_checksum ul_lat
0 20180919b_jpgs/jpgs/P26276665.jpg florence 20180919b_jpgs P26276665.jpg 5921685 2018/09/19 34.191330528689896 -77.8172986188853 -77.8071351918187 34.2084190030626 -77.7966678384306 -77.8109531705992 34.2002698812517 22bfdc9e8c7dd77694051c42b73ef383 34.2044620318067
1 20180919b_jpgs/jpgs/P26275221.jpg florence 20180919b_jpgs P26275221.jpg 7856304 2018/09/19 34.7371165394654 -76.77522871514509 -76.7749606421931 34.7477255105265 -76.7604501831659 -76.7643344559212 34.745983715257395 2bf6434e14c78918a083b45d8e8c84e3 34.739330560690604
2 20180919b_jpgs/jpgs/C26276792.jpg florence 20180919b_jpgs C26276792.jpg 7416479 2018/09/19 34.1162796503269 -77.87568430993849 -77.8792119685572 34.1210497955716 -77.8652190981454 -77.86175166742471 34.1254405298099 4028837f1fd5c4d272b3fcbf086da858 34.112801566182604
3 20180919b_jpgs/jpgs/P26272498.jpg florence 20180919b_jpgs P26272498.jpg 7077670 2018/09/19 35.193880664552495 -77.1145840941805 -77.12685826117591 35.1924425087302 -77.1143746980758 -77.1067252477598 35.1994469823763 6936f486b508e47daa4170f59ec40493 35.1834462909063
4 20180919b_jpgs/jpgs/S26272099.jpg florence 20180919b_jpgs S26272099.jpg 7181557 2018/09/19 35.2185091526623 -77.2066561006074 -77.2176882132229 35.220158733872 -77.20315073385841 -77.1973995512931 35.2259529314223 7cf0657aa2514cb44a9b3a911dfd6327 35.2089104765192
Last update: 2020-02-09