Commit graph

79 commits

Author SHA1 Message Date
90fc808eec ValidTLD option added
A new default option has been introduced in DomainClassifier which
is the validTLD option (enable by default). Based on the assigned
list of TLD, the extraction of potential domains is filtered to the
IANA assigned list.

If you are extracting the data for non-assigned/internal TLDs, you can disable
the default option with  validTLD=False on the potentialdomain function.

The list of assigned TLDs is downloaded from IANA.
2014-08-31 14:46:13 +02:00
83fafea305 domain() to potentialdomain() updated 2014-08-22 08:31:29 +02:00
80dc03f59c potentialdomain() is now called at the init
To avoid confusion the method name domain() has
been renamed to potentialdomain().
2014-08-22 08:29:59 +02:00
b32ff4a776 domain() to potentialdomain() method name updated 2014-08-22 08:27:22 +02:00
c954da666c nameservers are now configurable 2014-08-22 07:26:15 +02:00
2f0afbba44 Version inline with pypi module 2014-08-20 11:00:10 +02:00
680c0f43da 0.2 2014-08-20 10:58:48 +02:00
d116cfc37b Exception handled for unknown origins 2014-08-20 10:51:55 +02:00
94482dcfbf Setup script for DomainClassifier 2014-08-18 21:56:11 +02:00
ef75b716bf PIP requirements.txt added 2014-08-15 16:39:27 +02:00
e967f91e8c debug removed 2013-06-14 10:34:08 +02:00
c637209267 Documentation updated for ipaddress() method 2013-06-14 10:20:35 +02:00
13c4bf22da ipaddress() method added
This method extracts valid IPv4 addresses from raw text. The validation
is done using the standard socket call. The extended parameter adds the
origin of the IP address via Cymru IP/ASN service.
2013-06-14 10:12:37 +02:00
a3f87b5135 Use Google default recursive server as default resolver. 2013-06-14 09:45:01 +02:00
cddf5b5c63 Documentation updated 2013-04-28 15:19:09 +02:00
5eca33529f Overview of the methods/flow in DomainClassifier
Overview of the methods/flow in DomainClassifier
2013-04-28 16:12:46 +03:00
2c6cc0940e Flow of operation in DomainClassifier expressed in GraphViz 2013-04-28 15:09:02 +02:00
0be08a31c6 Better error catching
- Timeout of DNS reduce to 1 sec
 - Socket exception catched
 - Ranking default value is now 1.0
2012-03-02 21:12:39 +01:00
738060ab57 Sampling ranking output added 2012-02-19 18:53:49 +01:00
5417fe4911 Domain ranking added to rank your domain 2012-02-19 18:49:04 +01:00
b46192f235 __origin now returns (ASN,CC) 2012-02-19 18:07:09 +01:00
ed9a25d1b7 exclude and include methods added
exclude or include domains from a regular expression. If validdomain
was called before, the exclude or include is on the validdomain list
only.
2012-01-31 14:20:51 +01:00
ed376c89e0 sample output 2012-01-24 23:03:09 +01:00
20c6c6a023 Basic pydoc description added 2012-01-24 22:53:20 +01:00
b5bfadb368 Update README to match new DomainClassifier 2012-01-24 11:58:00 +01:00
9f4ae469d9 Test extended to show the use of localizedomain 2012-01-24 11:49:07 +01:00
542c53dcca DomainClassifier Class extended
The class has been extended to add the localizedomain
method to geolocalize DNS records associated for an existing domain.
The localization rely on the Team Cymru ip2asn lookup via DNS.
2012-01-24 11:47:04 +01:00
e2d08d181f Minimal README added 2012-01-23 17:48:32 +01:00
9eb1e3e4ef First version of the domain extractor and classify
The class domainclassifer got two methods:

- domain() to extract all potential domains from a raw text
  The method returns a list.

- validdomain() returning all the existing domains based on their
  known DNS records sets like A,AAAA or CNAME records.
  The method returns a set. If the extended option is requested, it's a list
  of tuples containing the domain with their existing DNS records and
  their returned data.
2012-01-23 16:16:01 +01:00