We have written before about our ENTRADA research platform. ENTRADA is a Hadoop-based 'big data' system, which we use to store and analyse DNS data. The total number of DNS queries stored is huge: about 166 million a day.
Hidden within that mountain of data is information that can be used to make the internet a little better and a little more secure. Our link-up with the AbuseHUB is an example of how that can be done.
Abuse Information Exchange
The Abuse Information Exchange is a joint initiative by KPN, SIDN, Solcon, Tele2, XS4ALL, Zeelandnet and Ziggo/UPC. Its aim is to streamline the exchange of information about botnets and other forms of internet abuse in the Netherlands, with a view to facilitating effective countermeasures. Abuse reports are sent to a central system – the AbuseHUB, under SIDN's operational management – where they are gathered for correlation.
The AbuseHUB has been operating since June 2014. Data on botnet infections are submitted on a fully automated basis via 'reliable notifiers'. A reliable notifier is a contracted information provider that has been confirmed as a source of dependable, up-to-date information about botnet infections and other internet security issues. Reports from reliable notifiers are analysed and passed on to the relevant internet service providers, enabling them to take appropriate remedial action. The system operates on the basis of 'intelligent referral': ISPs receive prioritised information, with 'noise' (e.g. duplicate reports) filtered out.The Abuse Information Exchange also serves as a forum where members can share relevant expertise and know-how. By working together, the Exchange members are able to respond more effectively to botnet infections, thus increasing internet security.
SIDN Labs active as a reliable notifier
At SIDN Labs, we have developed an algorithm, which can be used to identify certain types of botnet from ENTRADA data. The research underpinning the algorithm was carried out as part of the ResRep’ project. ResRep is a contraction of 'Resolver Reputation' and, as that name suggests, the project involves assigning reputation scores to the DNS resolvers that interact with our authoritative name servers. The scores are based on the resolvers' behaviour. Reputation scoring is therefore a form of profiling or fingerprinting, which obviously has to be carried out with appropiate care. Detailed analysis of DNS behaviour enables us to identify certain traffic patterns as suspect. Unfortunately, we cannot reveal very much about the patterns we look out for, because placing such information in the public domain would obviously help abusers to avoid detection. Suffice it to say that, using ENTRADA, we can pinpoint when a given botnet becomes active and what IP address it is using. If that address is in the Netherlands, we report the detection to the AbuseHUB in our capacity as a reliable notifier. The ISP responsible for the IP address can then take immediate action. The process is of course fully automated, with scanning and reporting in progress twenty-four hours a day, seven days a week.
Speed means efficiency
Our system's main advantages are speed and comprehensive coverage. We can detect botnet activity almost as soon as it occurs. If they are able to discern such activity at all, other reliable notifiers need a significant period of time to do so. The ResRep feed to the AbuseHUB is therefore a good example of the capabilities of our ENTRADA system.If you have suggestions or ideas of your own about other ways ENTRADA could be used, please don't hesitate to share them with us.
