adulau/napkin-text-analysis
1
0
Fork 0
mirror of https://github.com/adulau/napkin-text-analysis.git synced 2024-11-24 10:57:07 +00:00
Code Issues Projects Releases Packages Wiki Activity

new: [option] --binary to dump in binary format instead of UTF-8

Browse source
This commit is contained in:
Alexandre Dulaunoy 2020-10-08 23:30:57 +02:00
parent 26244739dd
commit 10049a69b6
Signed by: adulau
GPG key ID: 09E2CD4944E6CBCD
2 changed files with 7 additions and 2 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

3
README.md
View file

@ -27,7 +27,7 @@ Intermediate results are stored in a Redis database to allow the analysis of mul
~~~~ ~~~~
usage: napkin.py [-h] [-v V] [-f F] [-t T] [-s] [-o O] [-l L] [--verbatim] usage: napkin.py [-h] [-v V] [-f F] [-t T] [-s] [-o O] [-l L] [--verbatim]
[--no-flushdb] [--no-flushdb] [--binary]
Extract statistical analysis of text Extract statistical analysis of text
@ -44,6 +44,7 @@ optional arguments:
--no-flushdb Don't flush the redisdb, useful when you want to process --no-flushdb Don't flush the redisdb, useful when you want to process
multiple files and aggregate the results. (by default the multiple files and aggregate the results. (by default the
redis database is flushed at each run) redis database is flushed at each run)
--binary Output response in binary instead of UTF-8 (default)
~~~~ ~~~~
# example usage of napkin # example usage of napkin

4
bin/napkin.py
View file

@ -17,12 +17,16 @@ parser.add_argument('-o', help="output format (default is csv), json", default="
parser.add_argument('-l', help="language used for the analysis (default is en)", default="en") parser.add_argument('-l', help="language used for the analysis (default is en)", default="en")
parser.add_argument('--verbatim', help="Don't use the lemmatized form, use verbatim. (default is the lematized form)", default=False, action='store_true') parser.add_argument('--verbatim', help="Don't use the lemmatized form, use verbatim. (default is the lematized form)", default=False, action='store_true')
parser.add_argument('--no-flushdb', help="Don't flush the redisdb, useful when you want to process multiple files and aggregate the results. (by default the redis database is flushed at each run)", default=False, action='store_true') parser.add_argument('--no-flushdb', help="Don't flush the redisdb, useful when you want to process multiple files and aggregate the results. (by default the redis database is flushed at each run)", default=False, action='store_true')
parser.add_argument('--binary', help="Output response in binary instead of UTF-8 (default)", default=False, action='store_true')
args = parser.parse_args() args = parser.parse_args()
if args.f is None: if args.f is None:
parser.print_help() parser.print_help()
sys.exit() sys.exit()
if not args.binary:
redisdb = redis.Redis(host="localhost", port=6380, db=5, encoding='utf-8', decode_responses=True)
else:
redisdb = redis.Redis(host="localhost", port=6380, db=5) redisdb = redis.Redis(host="localhost", port=6380, db=5)
try: try: