From 4c9db947d987d342d3436d594d85aff492405017 Mon Sep 17 00:00:00 2001 From: Aaron Kaplan Date: Wed, 25 Dec 2013 15:53:35 +0100 Subject: [PATCH 01/15] typos --- i-d/pdns-qof.txt | 6 +++--- i-d/pdns-qof.xml | 2 +- 2 files changed, 4 insertions(+), 4 deletions(-) diff --git a/i-d/pdns-qof.txt b/i-d/pdns-qof.txt index a17c51e..1f36fa5 100644 --- a/i-d/pdns-qof.txt +++ b/i-d/pdns-qof.txt @@ -157,7 +157,7 @@ Internet-Draft Passive DNS - Common Output Format December 2013 DNS Database from cache poisoning attacks [CACHEPOISONING]. Another limitiation that clients querying the database need to be aware of is that each query simply gets an snapshot-answer of the time of - querying. Clients MUST NOT rely on consistent answers. Not must + querying. Clients MUST NOT rely on consistent answers. Nor must they assume that answers must be identical across multiple Passive DNS Servers. @@ -178,8 +178,8 @@ Internet-Draft Passive DNS - Common Output Format December 2013 3.1. Overview and Example - The intent of this output format is to be easily parseable by - scripts. Every implementation MUST support the JSON output format. + The intent of this output format is to be easily parsable by scripts. + Every implementation MUST support the JSON output format. A sample output using the JSON format: diff --git a/i-d/pdns-qof.xml b/i-d/pdns-qof.xml index d031535..9b7303d 100644 --- a/i-d/pdns-qof.xml +++ b/i-d/pdns-qof.xml @@ -166,7 +166,7 @@ The document does not describe the protocol (e.g. WHOIS The formatting of the answer follows the JSON format. The order of the fields is not significant for the same resource type. That means, the same name tuple plus timing information identifies a unique answer per server.
- The intent of this output format is to be easily parseable by scripts. Every implementation MUST support the JSON output format. + The intent of this output format is to be easily parsable by scripts. Every implementation MUST support the JSON output format.
A sample output using the JSON format: Date: Wed, 25 Dec 2013 16:03:18 +0100 Subject: [PATCH 02/15] typos. ascii -> ASCII --- i-d/pdns-qof.xml | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/i-d/pdns-qof.xml b/i-d/pdns-qof.xml index 9b7303d..78cbcfd 100644 --- a/i-d/pdns-qof.xml +++ b/i-d/pdns-qof.xml @@ -196,10 +196,10 @@ The document does not describe the protocol (e.g. WHOISThis field returns the data of the queried resource. In general, this is to be interpreted as string. Depending on the rtype, this can be an IPv4 or IPv6 address, a domain name (as in the case of CNAMEs), an SPF record, etc. A client MUST be able to interpret any value which is legal as the right hand side in a DNS zone file RFC 1035 and RFC 1034. If the rdata came from an unknown DNS resource records, the server must follow the transparency principle as described in RFC 3597.
- This field returns the first time that the record / unique tuple (rrname, rrtype, rdata) has been seen by the passive DNS. The date is expressed in seconds (decimal ascii) since 1st of January 1970 (unix timestamp). The time zone MUST be UTC. + This field returns the first time that the record / unique tuple (rrname, rrtype, rdata) has been seen by the passive DNS. The date is expressed in seconds (decimal ASCII) since 1st of January 1970 (unix timestamp). The time zone MUST be UTC.
- This field returns the last time that the unique tuple (rrname, rrtype, rdata) record has been seen by the passive DNS. The date is expressed in seconds (decimal ascii) since 1st of January 1970 (unix timestamp). The time zone MUST be UTC. + This field returns the last time that the unique tuple (rrname, rrtype, rdata) record has been seen by the passive DNS. The date is expressed in seconds (decimal ASCII) since 1st of January 1970 (unix timestamp). The time zone MUST be UTC.
@@ -218,10 +218,10 @@ The document does not describe the protocol (e.g. WHOISThis field returns the sensor information where the record was seen. The sensor_id is an opaque byte string as defined by RFC 5001 in section 2.3.
- This field returns the first time that the unique tuple (rrname, rrtype, rdata) record has been seen via zone file import. The date is expressed in seconds (decimal ascii) since 1st of January 1970 (unix timestamp). The time zone MUST be UTC. + This field returns the first time that the unique tuple (rrname, rrtype, rdata) record has been seen via zone file import. The date is expressed in seconds (decimal ASCII) since 1st of January 1970 (unix timestamp). The time zone MUST be UTC.
- This field returns the last time that the unique tuple (rrname, rrtype, rdata) record has been seen via zone file import. The date is expressed in seconds (decimal ascii) since 1st of January 1970 (unix timestamp). The time zone MUST be UTC. + This field returns the last time that the unique tuple (rrname, rrtype, rdata) record has been seen via zone file import. The date is expressed in seconds (decimal ASCII) since 1st of January 1970 (unix timestamp). The time zone MUST be UTC.
From 7d58de44489ba3142c5ea0095103e443c54df75b Mon Sep 17 00:00:00 2001 From: Aaron Kaplan Date: Wed, 25 Dec 2013 16:32:05 +0100 Subject: [PATCH 03/15] more typos --- i-d/pdns-qof.xml | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/i-d/pdns-qof.xml b/i-d/pdns-qof.xml index 78cbcfd..0963320 100644 --- a/i-d/pdns-qof.xml +++ b/i-d/pdns-qof.xml @@ -196,10 +196,10 @@ The document does not describe the protocol (e.g. WHOISThis field returns the data of the queried resource. In general, this is to be interpreted as string. Depending on the rtype, this can be an IPv4 or IPv6 address, a domain name (as in the case of CNAMEs), an SPF record, etc. A client MUST be able to interpret any value which is legal as the right hand side in a DNS zone file RFC 1035 and RFC 1034. If the rdata came from an unknown DNS resource records, the server must follow the transparency principle as described in RFC 3597.
- This field returns the first time that the record / unique tuple (rrname, rrtype, rdata) has been seen by the passive DNS. The date is expressed in seconds (decimal ASCII) since 1st of January 1970 (unix timestamp). The time zone MUST be UTC. + This field returns the first time that the record / unique tuple (rrname, rrtype, rdata) has been seen by the passive DNS. The date is expressed in seconds (decimal ASCII) since 1st of January 1970 (Unix timestamp). The time zone MUST be UTC.
- This field returns the last time that the unique tuple (rrname, rrtype, rdata) record has been seen by the passive DNS. The date is expressed in seconds (decimal ASCII) since 1st of January 1970 (unix timestamp). The time zone MUST be UTC. + This field returns the last time that the unique tuple (rrname, rrtype, rdata) record has been seen by the passive DNS. The date is expressed in seconds (decimal ASCII) since 1st of January 1970 (Unix timestamp). The time zone MUST be UTC.
@@ -208,7 +208,7 @@ The document does not describe the protocol (e.g. WHOISSpecifies how many authoritative DNS answers were received at the Passive DNS Server's collectors with the set of answers (i.e. same data). The number of requests is expressed as a decimal value. Specifies the number of times this particular event denoted by the other type fields has been seen in the given time interval (between time_last and time_first). Decimal number.
-
+
The bailiwick is the best estimate of the apex of the zone where this data is authoritative. String.
@@ -218,10 +218,10 @@ The document does not describe the protocol (e.g. WHOISThis field returns the sensor information where the record was seen. The sensor_id is an opaque byte string as defined by RFC 5001 in section 2.3.
- This field returns the first time that the unique tuple (rrname, rrtype, rdata) record has been seen via zone file import. The date is expressed in seconds (decimal ASCII) since 1st of January 1970 (unix timestamp). The time zone MUST be UTC. + This field returns the first time that the unique tuple (rrname, rrtype, rdata) record has been seen via zone file import. The date is expressed in seconds (decimal ASCII) since 1st of January 1970 (Unix timestamp). The time zone MUST be UTC.
- This field returns the last time that the unique tuple (rrname, rrtype, rdata) record has been seen via zone file import. The date is expressed in seconds (decimal ASCII) since 1st of January 1970 (unix timestamp). The time zone MUST be UTC. + This field returns the last time that the unique tuple (rrname, rrtype, rdata) record has been seen via zone file import. The date is expressed in seconds (decimal ASCII) since 1st of January 1970 (Unix timestamp). The time zone MUST be UTC.
@@ -249,7 +249,7 @@ The document does not describe the protocol (e.g. WHOIS
- In some cases, Passive DNS output might contain confidential information and its access might be restricted. When an user is querying multiple Passive DNS and aggregating the data, the sensitivity of the data must be considered. + In some cases, Passive DNS output might contain confidential information and its access might be restricted. When a user is querying multiple Passive DNS and aggregating the data, the sensitivity of the data must be considered.
From d30d768b1904d7455fe93cf226fb99fcadca2f25 Mon Sep 17 00:00:00 2001 From: Aaron Kaplan Date: Wed, 25 Dec 2013 16:34:06 +0100 Subject: [PATCH 04/15] I believe that Henry now works at Farsight. Waiting for a confirmation --- i-d/pdns-qof.txt | 20 ++++++++++---------- i-d/pdns-qof.xml | 2 +- 2 files changed, 11 insertions(+), 11 deletions(-) diff --git a/i-d/pdns-qof.txt b/i-d/pdns-qof.txt index 1f36fa5..835c705 100644 --- a/i-d/pdns-qof.txt +++ b/i-d/pdns-qof.txt @@ -80,7 +80,7 @@ Table of Contents 3.2.5. time_last . . . . . . . . . . . . . . . . . . . . . . . 5 3.3. Optional Fields . . . . . . . . . . . . . . . . . . . . . . 5 3.3.1. count . . . . . . . . . . . . . . . . . . . . . . . . . 5 - 3.3.2. bailiwick . . . . . . . . . . . . . . . . . . . . . . . 5 + 3.3.2. Bailiwick . . . . . . . . . . . . . . . . . . . . . . . 5 3.4. Additional Fields . . . . . . . . . . . . . . . . . . . . . 5 3.4.1. sensor_id . . . . . . . . . . . . . . . . . . . . . . . 6 3.4.2. zone_time_first . . . . . . . . . . . . . . . . . . . . 6 @@ -240,14 +240,14 @@ Internet-Draft Passive DNS - Common Output Format December 2013 This field returns the first time that the record / unique tuple (rrname, rrtype, rdata) has been seen by the passive DNS. The date - is expressed in seconds (decimal ascii) since 1st of January 1970 - (unix timestamp). The time zone MUST be UTC. + is expressed in seconds (decimal ASCII) since 1st of January 1970 + (Unix timestamp). The time zone MUST be UTC. 3.2.5. time_last This field returns the last time that the unique tuple (rrname, rrtype, rdata) record has been seen by the passive DNS. The date is - expressed in seconds (decimal ascii) since 1st of January 1970 (unix + expressed in seconds (decimal ASCII) since 1st of January 1970 (Unix timestamp). The time zone MUST be UTC. 3.3. Optional Fields @@ -264,7 +264,7 @@ Internet-Draft Passive DNS - Common Output Format December 2013 other type fields has been seen in the given time interval (between time_last and time_first). Decimal number. -3.3.2. bailiwick +3.3.2. Bailiwick The bailiwick is the best estimate of the apex of the zone where this data is authoritative. String. @@ -291,15 +291,15 @@ Internet-Draft Passive DNS - Common Output Format December 2013 This field returns the first time that the unique tuple (rrname, rrtype, rdata) record has been seen via zone file import. The date - is expressed in seconds (decimal ascii) since 1st of January 1970 - (unix timestamp). The time zone MUST be UTC. + is expressed in seconds (decimal ASCII) since 1st of January 1970 + (Unix timestamp). The time zone MUST be UTC. 3.4.3. zone_time_last This field returns the last time that the unique tuple (rrname, rrtype, rdata) record has been seen via zone file import. The date - is expressed in seconds (decimal ascii) since 1st of January 1970 - (unix timestamp). The time zone MUST be UTC. + is expressed in seconds (decimal ASCII) since 1st of January 1970 + (Unix timestamp). The time zone MUST be UTC. 3.5. Additional Fields Registry @@ -322,7 +322,7 @@ Internet-Draft Passive DNS - Common Output Format December 2013 6. Security Considerations In some cases, Passive DNS output might contain confidential - information and its access might be restricted. When an user is + information and its access might be restricted. When a user is querying multiple Passive DNS and aggregating the data, the sensitivity of the data must be considered. diff --git a/i-d/pdns-qof.xml b/i-d/pdns-qof.xml index 0963320..09dd60d 100644 --- a/i-d/pdns-qof.xml +++ b/i-d/pdns-qof.xml @@ -108,7 +108,7 @@ - Cisco + Farsight Security, Inc.
1741 Brunswick Street, Suite 500 From 6c2a9645e244b59d88167f2511a9926c1fed213f Mon Sep 17 00:00:00 2001 From: Aaron Kaplan Date: Wed, 25 Dec 2013 16:34:37 +0100 Subject: [PATCH 05/15] update .txt file --- i-d/pdns-qof.txt | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/i-d/pdns-qof.txt b/i-d/pdns-qof.txt index 835c705..2ea30e0 100644 --- a/i-d/pdns-qof.txt +++ b/i-d/pdns-qof.txt @@ -6,9 +6,8 @@ Internet-Draft CIRCL Intended status: Informational A. Kaplan Expires: June 28, 2014 CERT.at P. Vixie - Farsight Security, Inc. H. Stern - Cisco + Farsight Security, Inc. December 25, 2013 @@ -49,6 +48,7 @@ Copyright Notice This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (http://trustee.ietf.org/license-info) in effect on the date of + publication of this document. Please review these documents @@ -57,7 +57,6 @@ Dulaunoy, et al. Expires June 28, 2014 [Page 1] Internet-Draft Passive DNS - Common Output Format December 2013 - publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Simplified BSD License text as described in Section 4.e of @@ -108,6 +107,7 @@ Table of Contents + Dulaunoy, et al. Expires June 28, 2014 [Page 2] Internet-Draft Passive DNS - Common Output Format December 2013 @@ -470,7 +470,7 @@ Internet-Draft Passive DNS - Common Output Format December 2013 Henry Stern - Cisco + Farsight Security, Inc. 1741 Brunswick Street, Suite 500 Halifax, Nova Scotia B3J 3X8 Canada From 9cc30eca27abdec6f5500237dd3ef3be892538da Mon Sep 17 00:00:00 2001 From: Aaron Kaplan Date: Wed, 25 Dec 2013 16:41:45 +0100 Subject: [PATCH 06/15] add Paul's address according to whois data --- i-d/pdns-qof.xml | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/i-d/pdns-qof.xml b/i-d/pdns-qof.xml index 09dd60d..22e3a0b 100644 --- a/i-d/pdns-qof.xml +++ b/i-d/pdns-qof.xml @@ -95,11 +95,11 @@ Farsight Security, Inc.
- - - - - + 11400 La Honda Road + Woodside + California + 94062 + U.S.A. paul@redbarn.org From 8413eb70776eb14db289a630e93d30aa8111c337 Mon Sep 17 00:00:00 2001 From: Aaron Kaplan Date: Wed, 25 Dec 2013 16:57:32 +0100 Subject: [PATCH 07/15] check meeting notes, cross out things which are done --- meeting/notes-20130402.txt | 29 +++++++++++++++++------------ 1 file changed, 17 insertions(+), 12 deletions(-) diff --git a/meeting/notes-20130402.txt b/meeting/notes-20130402.txt index 68061bd..0726c9c 100644 --- a/meeting/notes-20130402.txt +++ b/meeting/notes-20130402.txt @@ -1,24 +1,29 @@ -https://dnsdb-api.isc.org +old: https://dnsdb-api.isc.org +new: https://api.dnsdb.info -keep output format and query format separated in two different documents -we should not specify which rtypes MUST be supported. It is only an output format spec - rrtype should be mnemonic that is supported by IANA - rfc 3597 unknown record type -> reference this rfc for arbitrary rrtypes - same as for RDATA (if it can not format it correctly, it should be formated as in rfc3597 +[X] keep output format and query format separated in two different documents +[X] we should not specify which rtypes MUST be supported. It is only an output format spec +[X] rrtype should be mnemonic that is supported by IANA +[X] rfc 3597 unknown record type -> reference this rfc for arbitrary rrtypes +[X] same as for RDATA (if it can not format it correctly, it should be formated as in rfc3597 -optional: x-more-data-coming: 50 (for Cisco) +[x] optional: x-more-data-coming: 50 (for Cisco) --> nope... Cisco has a different (pre-recursor ) system. Not part of this draft. -we define the record format, any extensions should be specified in an RFC describing how the protocol works ---> specify the control packets in a separate RFC ---> Paul will ask Robert. +[ ] we define the record format, any extensions should be specified in an RFC describing how the protocol works +[ ] --> specify the control packets in a separate RFC +[ ] --> Paul will ask Robert. + +Discussion about Cisco's "more data coming" extension: fearful of breaking existing programs by adding a bookend Idea: if you can add &show_progress=1 to ReST GET request, then we can add the fields If a given implementation supports bookends, progress bars, then these have to be optional and they should be signalled via HTTP GET ¶meter Extension to the record format must be x-* (for exaple x-query-id) +--> we decided against that because of http://tools.ietf.org/search/rfc6648 +Instead we created a registry in the github wiki -Samples are OK! But move them to a separate section! +[ ] Samples are OK! But move them to a separate section! @@ -35,7 +40,7 @@ virustotal (?) -> later submission: - the dns-op still exists! So pdns-qof should be submitted to dns-op +[ ] the dns-op still exists! So pdns-qof should be submitted to dns-op From 46e06200011fdda53d3c4ce2001bf0a9ed1c3f9a Mon Sep 17 00:00:00 2001 From: Aaron Kaplan Date: Wed, 25 Dec 2013 17:03:21 +0100 Subject: [PATCH 08/15] clarify some things from the meeting notes --- meeting/notes-20130402.txt | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-) diff --git a/meeting/notes-20130402.txt b/meeting/notes-20130402.txt index 0726c9c..f390cfb 100644 --- a/meeting/notes-20130402.txt +++ b/meeting/notes-20130402.txt @@ -20,23 +20,22 @@ Idea: if you can add &show_progress=1 to ReST GET request, then we can add the f If a given implementation supports bookends, progress bars, then these have to be optional and they should be signalled via HTTP GET ¶meter Extension to the record format must be x-* (for exaple x-query-id) ---> we decided against that because of http://tools.ietf.org/search/rfc6648 -Instead we created a registry in the github wiki + --> we decided against that because of http://tools.ietf.org/search/rfc6648 + Instead we created a registry in the github wiki [ ] Samples are OK! But move them to a separate section! dnstap -> get mail from paul -in google protocol bufs -will tap any part of a nameserver. + in google protocol bufs + will tap any part of a nameserver (code). next steps: * private circulation with robert edmonds, bfk, auckland, cert.ee - -virustotal (?) -> later + * virustotal (?) -> later submission: From 5a6fa89768da9fa9d4f49cab6127c0645657ffc6 Mon Sep 17 00:00:00 2001 From: Aaron Kaplan Date: Wed, 25 Dec 2013 17:04:28 +0100 Subject: [PATCH 09/15] add my todo notes --- TODO.md | 11 +++++++++++ 1 file changed, 11 insertions(+) create mode 100644 TODO.md diff --git a/TODO.md b/TODO.md new file mode 100644 index 0000000..6ca4c6d --- /dev/null +++ b/TODO.md @@ -0,0 +1,11 @@ + +TODO +===== + + * formal grammar (ABNF) for the output format + * create query API RFC + * move samples to a separate section + * cross check out work with http://tools.ietf.org/id/draft-bortzmeyer-dnsop-dns-privacy-01.txt + * Submit to dns-op IETF WG + + From 8a24d5f0cd519de7c77d802deb9009831839ef86 Mon Sep 17 00:00:00 2001 From: Aaron Kaplan Date: Wed, 25 Dec 2013 17:13:20 +0100 Subject: [PATCH 10/15] follow up on TODO from meeting -> put Examples into it's own section --- i-d/pdns-qof.txt | 92 ++++++++++++++++++++++++------------------------ i-d/pdns-qof.xml | 4 ++- 2 files changed, 49 insertions(+), 47 deletions(-) diff --git a/i-d/pdns-qof.txt b/i-d/pdns-qof.txt index 2ea30e0..47bd411 100644 --- a/i-d/pdns-qof.txt +++ b/i-d/pdns-qof.txt @@ -70,21 +70,22 @@ Table of Contents 1.1. Requirements Language . . . . . . . . . . . . . . . . . . . 3 2. Limitation . . . . . . . . . . . . . . . . . . . . . . . . . . 3 3. Common Output Format . . . . . . . . . . . . . . . . . . . . . 4 - 3.1. Overview and Example . . . . . . . . . . . . . . . . . . . 4 - 3.2. Mandatory Fields . . . . . . . . . . . . . . . . . . . . . 4 - 3.2.1. rrname . . . . . . . . . . . . . . . . . . . . . . . . 4 - 3.2.2. rrtype . . . . . . . . . . . . . . . . . . . . . . . . 4 - 3.2.3. rdata . . . . . . . . . . . . . . . . . . . . . . . . . 5 - 3.2.4. time_first . . . . . . . . . . . . . . . . . . . . . . 5 - 3.2.5. time_last . . . . . . . . . . . . . . . . . . . . . . . 5 - 3.3. Optional Fields . . . . . . . . . . . . . . . . . . . . . . 5 - 3.3.1. count . . . . . . . . . . . . . . . . . . . . . . . . . 5 - 3.3.2. Bailiwick . . . . . . . . . . . . . . . . . . . . . . . 5 - 3.4. Additional Fields . . . . . . . . . . . . . . . . . . . . . 5 - 3.4.1. sensor_id . . . . . . . . . . . . . . . . . . . . . . . 6 - 3.4.2. zone_time_first . . . . . . . . . . . . . . . . . . . . 6 - 3.4.3. zone_time_last . . . . . . . . . . . . . . . . . . . . 6 - 3.5. Additional Fields Registry . . . . . . . . . . . . . . . . 6 + 3.1. Overview . . . . . . . . . . . . . . . . . . . . . . . . . 4 + 3.2. Example . . . . . . . . . . . . . . . . . . . . . . . . . . 4 + 3.3. Mandatory Fields . . . . . . . . . . . . . . . . . . . . . 4 + 3.3.1. rrname . . . . . . . . . . . . . . . . . . . . . . . . 4 + 3.3.2. rrtype . . . . . . . . . . . . . . . . . . . . . . . . 4 + 3.3.3. rdata . . . . . . . . . . . . . . . . . . . . . . . . . 5 + 3.3.4. time_first . . . . . . . . . . . . . . . . . . . . . . 5 + 3.3.5. time_last . . . . . . . . . . . . . . . . . . . . . . . 5 + 3.4. Optional Fields . . . . . . . . . . . . . . . . . . . . . . 5 + 3.4.1. count . . . . . . . . . . . . . . . . . . . . . . . . . 5 + 3.4.2. Bailiwick . . . . . . . . . . . . . . . . . . . . . . . 5 + 3.5. Additional Fields . . . . . . . . . . . . . . . . . . . . . 6 + 3.5.1. sensor_id . . . . . . . . . . . . . . . . . . . . . . . 6 + 3.5.2. zone_time_first . . . . . . . . . . . . . . . . . . . . 6 + 3.5.3. zone_time_last . . . . . . . . . . . . . . . . . . . . 6 + 3.6. Additional Fields Registry . . . . . . . . . . . . . . . . 6 4. Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . 6 5. IANA Considerations . . . . . . . . . . . . . . . . . . . . . . 6 6. Security Considerations . . . . . . . . . . . . . . . . . . . . 6 @@ -107,7 +108,6 @@ Table of Contents - Dulaunoy, et al. Expires June 28, 2014 [Page 2] Internet-Draft Passive DNS - Common Output Format December 2013 @@ -171,16 +171,18 @@ Internet-Draft Passive DNS - Common Output Format December 2013 3. Common Output Format +3.1. Overview + The formatting of the answer follows the JSON [RFC4627] format. The order of the fields is not significant for the same resource type. That means, the same name tuple plus timing information identifies a unique answer per server. -3.1. Overview and Example - The intent of this output format is to be easily parsable by scripts. Every implementation MUST support the JSON output format. +3.2. Example + A sample output using the JSON format: ... (list of )... @@ -191,18 +193,18 @@ Internet-Draft Passive DNS - Common Output Format December 2013 "time_last": "1386405372" } ... (separated by newline)... -3.2. Mandatory Fields +3.3. Mandatory Fields Implementation MUST support all the mandatory fields. The tuple (rrname,rrtype,rdata) will always be unique within one answer per server. -3.2.1. rrname +3.3.1. rrname This field returns the name of the queried resource. -3.2.2. rrtype +3.3.2. rrtype This field returns the resource record type as seen by the passive DNS. The key is rrtype and the value is in the interpreted record @@ -215,8 +217,6 @@ Internet-Draft Passive DNS - Common Output Format December 2013 AAAA, CNAME, PTR, SOA, TXT, DNAME, NS, SRV, RP, NAPTR, HINFO, A6. A client MUST be able to understand these textual rtype values. In addition, a client MUST be able to handle a decimal value (as - mentioned above) as answer. - @@ -225,7 +225,9 @@ Dulaunoy, et al. Expires June 28, 2014 [Page 4] Internet-Draft Passive DNS - Common Output Format December 2013 -3.2.3. rdata + mentioned above) as answer. + +3.3.3. rdata This field returns the data of the queried resource. In general, this is to be interpreted as string. Depending on the rtype, this @@ -236,25 +238,25 @@ Internet-Draft Passive DNS - Common Output Format December 2013 unknown DNS resource records, the server must follow the transparency principle as described in RFC 3597 [RFC3597]. -3.2.4. time_first +3.3.4. time_first This field returns the first time that the record / unique tuple (rrname, rrtype, rdata) has been seen by the passive DNS. The date is expressed in seconds (decimal ASCII) since 1st of January 1970 (Unix timestamp). The time zone MUST be UTC. -3.2.5. time_last +3.3.5. time_last This field returns the last time that the unique tuple (rrname, rrtype, rdata) record has been seen by the passive DNS. The date is expressed in seconds (decimal ASCII) since 1st of January 1970 (Unix timestamp). The time zone MUST be UTC. -3.3. Optional Fields +3.4. Optional Fields Implementations SHOULD support one or more field. -3.3.1. count +3.4.1. count Specifies how many authoritative DNS answers were received at the Passive DNS Server's collectors with the set of answers (i.e. same @@ -264,14 +266,12 @@ Internet-Draft Passive DNS - Common Output Format December 2013 other type fields has been seen in the given time interval (between time_last and time_first). Decimal number. -3.3.2. Bailiwick +3.4.2. Bailiwick The bailiwick is the best estimate of the apex of the zone where this data is authoritative. String. -3.4. Additional Fields - Implementations MAY support the following fields: @@ -281,27 +281,31 @@ Dulaunoy, et al. Expires June 28, 2014 [Page 5] Internet-Draft Passive DNS - Common Output Format December 2013 -3.4.1. sensor_id +3.5. Additional Fields + + Implementations MAY support the following fields: + +3.5.1. sensor_id This field returns the sensor information where the record was seen. The sensor_id is an opaque byte string as defined by RFC 5001 in section 2.3 [RFC5001]. -3.4.2. zone_time_first +3.5.2. zone_time_first This field returns the first time that the unique tuple (rrname, rrtype, rdata) record has been seen via zone file import. The date is expressed in seconds (decimal ASCII) since 1st of January 1970 (Unix timestamp). The time zone MUST be UTC. -3.4.3. zone_time_last +3.5.3. zone_time_last This field returns the last time that the unique tuple (rrname, rrtype, rdata) record has been seen via zone file import. The date is expressed in seconds (decimal ASCII) since 1st of January 1970 (Unix timestamp). The time zone MUST be UTC. -3.5. Additional Fields Registry +3.6. Additional Fields Registry In accordance with [RFC6648], designers of new passive DNS applications that would need additional fields can request and @@ -327,16 +331,14 @@ Internet-Draft Passive DNS - Common Output Format December 2013 sensitivity of the data must be considered. -7. References - - - Dulaunoy, et al. Expires June 28, 2014 [Page 6] Internet-Draft Passive DNS - Common Output Format December 2013 +7. References + 7.1. Normative References [RFC1034] Mockapetris, P., "Domain names - concepts and facilities", @@ -383,8 +385,6 @@ Internet-Draft Passive DNS - Common Output Format December 2013 agenda/attachment/rd4-papst-passive_dns.pdf>. [PDNSCIRCL] - "CIRCL Passive DNS", 2012, . - @@ -393,6 +393,8 @@ Dulaunoy, et al. Expires June 28, 2014 [Page 7] Internet-Draft Passive DNS - Common Output Format December 2013 + "CIRCL Passive DNS", 2012, . + [PDNSCLIENT] "Queries 5 major Passive DNS databases: BFK, CERTEE, DNSParse, ISC, and VirusTotal.", 2013, @@ -442,8 +444,6 @@ Authors' Addresses - - Dulaunoy, et al. Expires June 28, 2014 [Page 8] Internet-Draft Passive DNS - Common Output Format December 2013 @@ -462,7 +462,9 @@ Internet-Draft Passive DNS - Common Output Format December 2013 Paul Vixie Farsight Security, Inc. - + 11400 La Honda Road + Woodside, California 94062 + U.S.A. Phone: Email: paul@redbarn.org @@ -496,8 +498,6 @@ Internet-Draft Passive DNS - Common Output Format December 2013 - - Dulaunoy, et al. Expires June 28, 2014 [Page 9] diff --git a/i-d/pdns-qof.xml b/i-d/pdns-qof.xml index 22e3a0b..4fa3eb8 100644 --- a/i-d/pdns-qof.xml +++ b/i-d/pdns-qof.xml @@ -164,9 +164,11 @@ The document does not describe the protocol (e.g. WHOIS
+
The formatting of the answer follows the JSON format. The order of the fields is not significant for the same resource type. That means, the same name tuple plus timing information identifies a unique answer per server. -
The intent of this output format is to be easily parsable by scripts. Every implementation MUST support the JSON output format. +
+
A sample output using the JSON format: Date: Wed, 25 Dec 2013 17:31:42 +0100 Subject: [PATCH 11/15] add gitignore file --- i-d/.gitignore | 1 + 1 file changed, 1 insertion(+) create mode 100644 i-d/.gitignore diff --git a/i-d/.gitignore b/i-d/.gitignore new file mode 100644 index 0000000..f9b5ac1 --- /dev/null +++ b/i-d/.gitignore @@ -0,0 +1 @@ +pdns-qof.html From d37f41c4d9d131eee9f55fb284cecba36460391f Mon Sep 17 00:00:00 2001 From: Aaron Kaplan Date: Wed, 25 Dec 2013 17:32:53 +0100 Subject: [PATCH 12/15] add vim gitignore stuff --- i-d/.gitignore | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/i-d/.gitignore b/i-d/.gitignore index f9b5ac1..9918fa5 100644 --- a/i-d/.gitignore +++ b/i-d/.gitignore @@ -1 +1,7 @@ pdns-qof.html +[._]*.s[a-w][a-z] +[._]s[a-w][a-z] +*.un~ +Session.vim +.netrwhist +*~ From 51587c19807e73c6b5d2d49792c0cdd6f0ba8e87 Mon Sep 17 00:00:00 2001 From: Aaron Kaplan Date: Wed, 25 Dec 2013 17:51:21 +0100 Subject: [PATCH 13/15] write the country names --- i-d/pdns-qof.xml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/i-d/pdns-qof.xml b/i-d/pdns-qof.xml index 4fa3eb8..1b6019e 100644 --- a/i-d/pdns-qof.xml +++ b/i-d/pdns-qof.xml @@ -63,7 +63,7 @@ Luxembourg L-1611 - LU + Luxembourg (+352) 247 88444 alexandre.dulaunoy@circl.lu @@ -82,7 +82,7 @@ Vienna A-1010 - AT + Austria +43 1 5056416 78 kaplan@cert.at From 9467d8632be9a72de60a7f237b330f12ce68179b Mon Sep 17 00:00:00 2001 From: Aaron Kaplan Date: Wed, 25 Dec 2013 17:52:15 +0100 Subject: [PATCH 14/15] and make a new version of the txt --- i-d/pdns-qof.txt | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/i-d/pdns-qof.txt b/i-d/pdns-qof.txt index 47bd411..f0bf971 100644 --- a/i-d/pdns-qof.txt +++ b/i-d/pdns-qof.txt @@ -431,7 +431,7 @@ Authors' Addresses CIRCL 41, avenue de la gare Luxembourg, L-1611 - LU + Luxembourg Phone: (+352) 247 88444 Email: alexandre.dulaunoy@circl.lu @@ -453,7 +453,7 @@ Internet-Draft Passive DNS - Common Output Format December 2013 CERT.at Karlsplatz 1/2/9 Vienna, A-1010 - AT + Austria Phone: +43 1 5056416 78 Email: kaplan@cert.at From d52926d1c8e593b5f8bb383ad25821732a9a3cf3 Mon Sep 17 00:00:00 2001 From: Aaron Kaplan Date: Wed, 25 Dec 2013 17:56:24 +0100 Subject: [PATCH 15/15] hsterns' email address is still a problem --- TODO.md | 1 + i-d/pdns-qof.xml | 2 +- 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/TODO.md b/TODO.md index 6ca4c6d..dd3944b 100644 --- a/TODO.md +++ b/TODO.md @@ -2,6 +2,7 @@ TODO ===== + * get an updated address of Henry * formal grammar (ABNF) for the output format * create query API RFC * move samples to a separate section diff --git a/i-d/pdns-qof.xml b/i-d/pdns-qof.xml index 1b6019e..4229aa7 100644 --- a/i-d/pdns-qof.xml +++ b/i-d/pdns-qof.xml @@ -103,7 +103,7 @@ paul@redbarn.org - / + https://www.farsightsecurity.com/