adulau/hashlookup-server
1
0
Fork 0
mirror of https://github.com/adulau/hashlookup-server.git synced 2024-12-27 03:06:07 +00:00
Code Issues Projects Releases Packages Wiki Activity
hashlookup-server/bin/import-poc
History
Alexandre Dulaunoy 72b462b5ea
new: [statistics] add an optional statistic option in the server to have 
a sorted set of hashes matching and non-matching.
2021-08-13 22:13:25 +02:00
..
config.json new: [statistics] add an optional statistic option in the server to have 2021-08-13 22:13:25 +02:00
import-hashlookup-server.py Support for import of NSRL datasets in ISO and ZIP format 2021-07-29 10:46:59 +02:00
README.md Support for import of NSRL datasets in ISO and ZIP format 2021-07-29 10:46:59 +02:00
requirements chg: [requirements] remove standard lib python modules 2021-07-29 11:55:26 +02:00

README.md

hashlookup-server

PoC to better streamline the import of NSRL data.

Usage

$ python3 import-hashlookup-server.py -h
usage: import-hashlookup-server.py [-h] [-l | -i IMPORT_DATASET | -e INIT_DATASET] [-d] [-c]

optional arguments:
  -h, --help            show this help message and exit
  -l, --list            List datasets available for download and import.
  -i IMPORT_DATASET, --import-dataset IMPORT_DATASET
                        Import a dataset.
  -e INIT_DATASET, --init-dataset INIT_DATASET
                        Remove / initialize a dataset.
  -d, --skip-download   Skip downloading the dataset.
  -c, --skip-init       Skip initialization of the database.

$ python3 import-hashlookup-server.py -i nsrl_minimal

Todo

  • Test with the other data sets (currently only Android was tested) : Fetch from ZIP and not ISO file
  • Move older input scripts to "old" directory
  • Complete with sha256 and xcycl
  • Error handling (sufficient drive space, Redis active, check if there is already a db before init)
  • Multiple data sets at once?
  • Import from MISP (depends on filter)