mirror of
https://github.com/adulau/pdns-toolkit.git
synced 2024-12-23 01:06:00 +00:00
98 lines
2.7 KiB
Text
98 lines
2.7 KiB
Text
|
This is a description of the various data-structure used
|
||
|
|
in the Passive DNS toolkit.
|
||
|
|
|
||
|
|
= Redis Database number =
|
||
|
|
|
||
|
|
- (0) default passive dns
|
||
|
|
- (6) ranking of hostname
|
||
|
|
|
||
|
|
= format for redis db 0 =
|
||
|
|
|
||
|
|
KEY, VALUE
|
||
|
|
|
||
|
|
s:<tuple> <epoch timestamp>
|
||
|
|
l:<tuple> <epoch timestamp>
|
||
|
|
o:<tuple> <integer value>
|
||
|
|
|
||
|
|
KEY, {SET}
|
||
|
|
|
||
|
|
r:<record>:<rr-type-value> {<answer>}
|
||
|
|
v:<answer> {<record>}
|
||
|
|
|
||
|
|
# "SET" "l:www.l.google.com:72.14.204.103" "1298819557"
|
||
|
|
#1298823157.764353 "INCR" "o:www.l.google.com:72.14.204.103"
|
||
|
|
#1298823157.765310 "SADD" "r:www.l.google.com:1" "72.14.204.104"
|
||
|
|
#1298823157.765439 "SADD" "v:72.14.204.104" "www.l.google.com"
|
||
|
|
#1298823157.765567 "EXISTS" "s:www.l.google.com:72.14.204.104"
|
||
|
|
#1298823157.765696 "SET" "l:www.l.google.com:72.14.204.104" "1298819557"
|
||
|
|
#1298823157.765823 "INCR" "o:www.l.google.com:72.14.204.104"
|
||
|
|
#1298823157.766776 "SADD" "r:www.l.google.com:1" "72.14.204.147"
|
||
|
|
#1298823157.766901 "SADD" "v:72.14.204.147" "www.l.google.com"
|
||
|
|
#1298823157.767086 "EXISTS" "s:www.l.google.com:72.14.204.147"
|
||
|
|
#1298823157.767207 "SET" "l:www.l.google.com:72.14.204.147" "1298819557"
|
||
|
|
#1298823157.767333 "INCR" "o:www.l.google.com:72.14.204.147"
|
||
|
|
|
||
|
|
|
||
|
|
== sample db 0 ==
|
||
|
|
|
||
|
|
Here is the classical update process for a feeder:
|
||
|
|
|
||
|
|
Set or update the last-seen.
|
||
|
|
|
||
|
|
#1298823157.764221 "SET" "l:www.l.google.com:72.14.204.103" "1298819557"
|
||
|
|
|
||
|
|
Update the occurence of the answer seen.
|
||
|
|
|
||
|
|
#1298823157.764353 "INCR" "o:www.l.google.com:72.14.204.103"
|
||
|
|
|
||
|
|
Add to a set "record:rr-type" the seen answer.
|
||
|
|
|
||
|
|
#1298823157.765310 "SADD" "r:www.l.google.com:1" "72.14.204.104"
|
||
|
|
|
||
|
|
Add the reverse for fast lookup.
|
||
|
|
|
||
|
|
#1298823157.765439 "SADD" "v:72.14.204.104" "www.l.google.com"
|
||
|
|
|
||
|
|
If there is no existing first-seen for the tuple,
|
||
|
|
|
||
|
|
#1298823157.765567 "EXISTS" "s:www.l.google.com:72.14.204.104"
|
||
|
|
|
||
|
|
A first-seen is set:
|
||
|
|
|
||
|
|
#1298823157.765696 "SET" "l:www.l.google.com:72.14.204.104" "1298819557"
|
||
|
|
|
||
|
|
and so on...
|
||
|
|
|
||
|
|
#1298823157.765823 "INCR" "o:www.l.google.com:72.14.204.104"
|
||
|
|
#1298823157.766776 "SADD" "r:www.l.google.com:1" "72.14.204.147"
|
||
|
|
#1298823157.766901 "SADD" "v:72.14.204.147" "www.l.google.com"
|
||
|
|
#1298823157.767086 "EXISTS" "s:www.l.google.com:72.14.204.147"
|
||
|
|
#1298823157.767207 "SET" "l:www.l.google.com:72.14.204.147" "1298819557"
|
||
|
|
#1298823157.767333 "INCR" "o:www.l.google.com:72.14.204.147"
|
||
|
|
|
||
|
|
= format for redis db 6 =
|
||
|
|
|
||
|
|
This db index hold a ranking value per hostname.
|
||
|
|
|
||
|
|
KEY, VALUE
|
||
|
|
|
||
|
|
hostname, float double precision
|
||
|
|
|
||
|
|
This can be seen as a cache when an expiration can be set
|
||
|
|
by key to limit the lifetime of a ranking. You can create
|
||
|
|
a persistent ranking for setting up some "golden" domains.
|
||
|
|
|
||
|
|
== sample db 6 ==
|
||
|
|
|
||
|
|
1) "truecsi.org"
|
||
|
|
redis> GET "truecsi.org"
|
||
|
|
"1.00037650602"
|
||
|
|
redis>
|
||
|
|
|
||
|
|
= contact =
|
||
|
|
|
||
|
|
Update, extension, or new data-structure more than welcome.
|
||
|
|
|
||
|
|
https://www.github.com/adulau/
|
||
|
|
|