The main purpose of Scanhub is to provide an easy way to create a search engine out of raw network scans. Nmap is the most common and famous tool for such a job, but there are some new kids on the block that fill a different niche and Masscan is the first of the new breed of scanning tools that Scanhub now supports thanks to the help from @andrewsmhay. Masscan provides a fast way to scan the entire Internet on your own and with Scanhub you can make that data searchable for yourself and others. The only thing to note is that the XML from Masscan isn’t as full-featured as an Nmap scan, so you won’t be seeing as much summary information as you would if you uploaded an Nmap XML file (i.e. no breakdown of operating systems, domains, traceroutes and fewer banners). That being said, to get started with Scanhub + Masscan simply set your Masscan tool to output in XML format using -oX scan.xml and upload that file to your Scanhub repository.
As a way to test Scanhub and share how I use it, I will go through the process of setting up a new Scanhub to monitor the ZeuS Botnet Command & Control servers. I’ll show how to create a Scanhub, the Nmap command that will be used to scan the devices and at the end you’ll see how it looks like as an end-user!
Creating a Scanhub
To create a Scanhub you need to have a Shodan account (https://account.shodan.io) and a Scanhub subscription (https://scanhub.shodan.io/pricing). Once you’ve got those things setup, simply click on the big red “Create Scanhub” button And fill out the short form to create a Scanhub. Note that Scanhub names have to be unique! This means that there can’t be 2 Scanhubs of the same name to avoid confusion. This is how the form should look like after you click on the “Create new Scanhub” button:
Awesome, we’ve got a Scanhub up and running so we just need to feed it some data to start crunching. We could upload an XML file manually, but that doesn’t scale well and I want to automatically check the ZeuS trackers at least once a day. Fortunately, that’s very easy to do using bash, cron, cURL, Nmap and your Shodan API key (which is free). First things first: where to get the ZeuS C&C information from? There’s an awesome website that’s dedicated to tracking the ZeuS crimeware: https://zeustracker.abuse.ch/index.php
From there we can download an IP blocklist using cURL, which can then be fed into nmap. To download the file from your Linux shell, simply enter:
curl -o blocklist.txt "https://zeustracker.abuse.ch/blocklist.php?download=ipblocklist"
Now that we know which servers to scan, what does the Nmap command look like for this Scanhub? I want to gather as much information as possible including traceroutes, NSE scripts, operating system detection (“-A”) and all open ports (“-open”). To make sure I don’t miss servers that block Ping requests (ICMP), I will also choose to tell Nmap to ignore checking whether thehost is up or not and just assume that it is (“-P0”). The final command that I’ll be using is:
nmap -iL blocklist.txt --max-retries 1 --max-scan-delay 2 -oX zeus.xml -A -T5 -p- -sS --open -P0
Note that the results of the Nmap scan are saved as XML (“-oX”) and inside a file called “zeus.xml”.
Uploading to Scanhub
The final step is making the data searchable by uploading it to the Scanhub that was created earlier. To do this with your Linux terminal, you need to have a free Shodan API key. The data upload page for each Scanhub also includes basic instructions on how to do this using the API key, and it automatically includes your own key in the example. If you’re not sure what your API key is, simply visit your Shodan account page at https://account.shodan.io and you will see it at the top of the user information. Assuming that your key is called “MY_SHODAN_API_KEY” and your Scanhub is called “zeus-tracker” (note: that name was already taken for this example, so yours will be called differently) then you would enter the following command into your shell:
curl -F firstname.lastname@example.org "https://scanhub.shodan.io/repository/upload/zeus-tracker?key=MY_SHODAN_API_KEY"
And that’s it! Within a few seconds the results will show up on Scanhub and all the information that Nmap gathered (traceroutes, script output, operating systems, geolocation) will be searchable.
I’ll make another post about the results and how to scale this into analyzing a lot of malware-hosting servers, but for now I hope this has provided some insight into how my usual workflow is when using Scanhub. And if there’s something you would like to have me discuss or cover on the blog please let me know in the comments!