11 commits

Author	SHA1	Message	Date
Alexandre Dulaunoy	9364c75477	chg: [score] scores are now integer	2020-10-09 07:27:08 +02:00
Alexandre Dulaunoy	193ad08144	fix: [bug] punctuation was not part of OOV and were not accounted	2020-10-09 07:25:26 +02:00
Alexandre Dulaunoy	ef5011a64f	chg: [cleanup] key names used in redis has been simplified	2020-10-09 07:18:16 +02:00
Alexandre Dulaunoy	10049a69b6	new: [option] --binary to dump in binary format instead of UTF-8	2020-10-08 23:30:57 +02:00
Alexandre Dulaunoy	26244739dd	new: [option] Don't flush the redisdb, useful when you want to process multiple files and aggregate the results.	2020-10-08 23:22:00 +02:00
Alexandre Dulaunoy	949e41d19f	new: [lemmatized/verbatim] displaying verbatim or lemmatized version is now an option	2020-10-08 23:13:51 +02:00
Alexandre Dulaunoy	3d71d9288e	chg: [args] add an option to force the language	2020-10-01 23:06:39 +02:00
Alexandre Dulaunoy	3a09abc80c	new: [output] JSON export added	2020-09-21 07:50:57 +02:00
Alexandre Dulaunoy	3c3760019e	chg: [feature] add punct statistics for the oov (but the punct in ... spacy.io seems super buggy or incorrect)	2020-08-20 14:33:15 +02:00
Alexandre Dulaunoy	526f88071c	new: [feature] -s option to display the overall statistics of different tokens seen	2020-08-20 13:28:49 +02:00
Alexandre Dulaunoy	dd7c796460	new: [napkin] first release ... Napkin is a Python tool to produce statistical analysis of a text. Analysis features are : - Verbs frequency - Nouns frequency - Digit frequency - Labels frequency such as (Person, organisation, product, location) as defined in spacy.io [named entities](https://spacy.io/api/annotation#named-entities) - URL frequency - Email frequency - Mention frequency (everything prefixed with an @ symbol) - Out-Of-Vocabulary (OOV) word frequency meaning any words outside English dictionary	2020-08-19 17:33:04 +02:00