Telnet just doesn’t seem to die, and with millions of devices on the Internet still running it I don’t expect it to go away anytime soon. In fact, new products often use Telnet as an easy way to enable remote access to their device (see https://github.com/swisspol/GCDTelnetServer). To better keep track of which features the Telnet servers of the Internet support, the banners for Telnet now also include information about which options were negotiated during the initial connection:
In addition to the banner, you can now access the do, dont, will, wont options that the Telnet server supports. You’ll also be able to facet on these properties once the next API update comes out, which will make it easy to get a feel for how Telnet usage changes over time. Hopefully, more secure options will become easier to work with for the average developer and we’ll see fewer Telnet in the coming years.
Shodan collects a ton of data (1+ billion banners/ month) and it can be difficult to find the needle in the haystack. Sometimes the banner itself doesn’t tell you much about the device, for example a search for wind farms by Nordex is “jetty 2000”. Not exactly what you’d think when trying to locate wind farms. So most people end up searching Shodan using queries shared by other users and don’t really get to experience the fun in discovering new types of devices.
To help browse through the data that Shodan gathers I wrote a simple webapp called Shodan Explorer:
It randomly pulls an IP out of the stream of data that Shodan gathers, checks that the IP runs a webserver (HTTP) and then embeds the IP in an iframe so you can see it in your browser. Explorer is a simple, quick and hopefully fun way to check out all the weird stuff that’s on the Internet. If you end up finding something new please share the search result (Twitter, Blog, Shodan Search Directory, etc.) so we can start classifying all the unknown types of devices that are being crawled.
Note: the website has to be hosted on HTTP and not HTTPS since browsers don’t let you load mixed content on a HTTPS connection.
A lot of people prefer to avoid the terminal and instead want a user-friendly web interface to manage their server. So to determine which solution is the most popular Shodan has started to crawl the Internet for cPanel (2082, 2083), WHM (2086, 2087) and Webmin (10000)! The banners are all fairly standard HTTP(S) responses so I won’t show those, but lets take a look at who runs what and how they compare.
The majority of devices are located in the United States (117,000) followed by Canada (9,000) and the United Kingdom (7,000). So cPanel is definitely way more popular in the US than anywhere else. And it’s especially popular at the hosting provider Unified Layer, which is responsible for 21,000 of those US installations.
There are fewer Webmin instances on the Internet than cPanel, but the distribution of them is wider across the globe; i.e. it’s not as US-centric as cPanel. The US still leads with 38,000 instances, but not that much more compared to the following countries France (11,400) and Germany (11,000). In terms of hosting providers though there is once again a standout: OVH. Their organization accounts for roughly 10,000 of the installed Webmin instances.
Web Host Manager (WHM)
WHM is a bit different in that it is the software to manage cPanel accounts. This effectively gives us a measurement of which devices are being used by re-sellers. Unsurprisingly, the distribution of devices in terms of countries and organizations is very similar to cPanel. US with 97,000 devices, followed by Canada (9,000) and the UK (7,000) once again.
Call for Ports
Am I missing some ports or services? Is there something you’d like to see Shodan crawl? Then let me know and send me an email (email@example.com) or tweet at me (@achillean) with the ports that you’d like me to add!
SSH was one of the first protocols that I started crawling for 5 years ago because just connecting to the daemon already tells you what it’s running. I.e. you don’t have to send any data to SSH in order to get something interesting back. There have been some incremental improvements to add product and version detection but beyond that it’s stayed mostly the same.
Introducing the new, sexier SSH banner:
The crawlers now collect the key, key type, fingerprint, MAC and cipher used for each successful SSH connection! And alongside these changes the API has also been enhanced with 4 new facets for SSH:
I hope you enjoy the new banner and information that’s being gathered for SSH now! Let me know if there are other banners you’d like to see improved as well.
It’s now possible to see what information Shodan has available on a server from within Firefox thanks to the new Shodan add-on created by @PaulWebSec and @romainletendart! It’s a minimalistic yet powerful add-on to see what the website you’re visiting is exposing to the Internet. And the add-on will also tell you other information about the IP, for example who owns the IP space (organization), where it’s located and if possible the operating system it’s running. You can download the add-on from here:
And once you have it installed, you just need to click on the small Shodan icon to get an overview of what services that server is running on the Internet:
And what’s really cool is that the authors of the add-on made the code available at the following GitHub repository:
So if you have ideas, feedback or other suggestions on making it better just submit an issue or submit a pull request! And once again a big thank you to @PaulWebSec and @romainletendart for creating and sharing the add-on!
Have you ever needed to write a quick script to download data from Shodan? Or setup a cronjob to check what Shodan found on your network recently? How about getting a list of IPs out of the Shodan API? For the times where you’d like to have easy script-friendly access to Shodan there’s now a new command-line tool appropriately called shodan.
The shodan command-line interface (CLI) is packaged with the official Python library for Shodan, which means if you’re running the latest version of the library you already have access to the CLI. To install the new tool in Linux simply execute:
Or if you’re running an older version of the Shodan Python library and want to upgrade:
easy_install -U shodan
Once the tool is installed, you have to initialize the environment with your API key using shodan init:
shodan init YOUR_API_KEY
At the moment, the shodan CLI supports 6 commands. Note that for each command you can learn more about the options it supports by supplying the –help flag.
Returns the number of results for a search query.
$ shodan count microsoft iis 6.0 5310594
Search Shodan and download the results into a file where each line is a banner serialized in JSON as specified in https://developer.shodan.io/api/banner-specification
By default it will only download 1,000 results, if you want to download more look at the –limit flag.
For example, to download the latest 1,000 Microsoft-IIS 6.0 servers indexed by Shodan into a file called microsoft-data.json.gz use the following command:
This is the command that you should be using the most, since it lets you save your results and process them afterwards using the parse command. Because paging through results uses query credits, it makes sense to always store searches that you’re doing so you won’t need to use query credits for a search you already did in the past.
Initialize the shodan CLI. This is the first command you have to run for the tool to work, if you’re unsure about how to install the CLI please read the section above on installation.
shodan init YOUR_API_KEY
Returns your Internet-facing IP address.
$ shodan myip 22.214.171.124
Use parse to analyze a file that was generated using the download command. It lets you filter out the fields that you’re interested in, convert the JSON to a CSV and is friendly for pipe-ing to other scripts. For example, here’s the command to output the IP address, port and organization in CSV:
$ shodan parse --fields ip_str,port,org --separator , microsoft-data.json.gz
This command lets you search Shodan and view the results in a terminal-friendly way. By default it will display the IP, port, hostnames and data. You can use the –fields parameter to print whichever banner fields you’re interested in. For example, to search for Microsoft IIS 6.0 devices and print out their IP, port, organization and hostnames use the following command:
$ shodan search --fields ip_str,port,org,hostnames microsoft iis 6.0
I hope you’ve gotten a good idea of what the shodan CLI can do and how it might make your life easier. The tool is still in its early stages but it’s served me well so far. At this point, I’d love to get some feedback on things you’d like to see improved by submitting issues on the GitHub repository:
Coincidentally, that’s also the place where you can see the code for the tool in case you’re curious about the inner-workings. Please submit ideas for improvements and let me know via email (firstname.lastname@example.org) or Twitter whether this is useful to you!
As some of you have already seen, I’ve been working on a revamped version of Shodan. It already has some cool new enhancements over the current shodanhq.com website, such as support for CSV and JSON exports, prettier search listing, faster results, better integration with Shodan Exploits/ Maps and a lot of small tweaks to make life easier.
The latest feature that I’m adding is Shodan Reports:
To get a feel for what I’m talking about, please check out the following example report on Industrial Control Systems:
A report is a snapshot and overview of the search results at the time of report creation. At the moment, it creates a bunch of charts/ graphs for breakdowns on: location, organization, operating system, product, hostname and many more (see the developer documentation for a list of all facets). The purpose for reports is 3-fold:
- Pretty Overview That You Can Share
You should be able to get a basic feel for the devices in the search results just from looking at the report. And if you’re interested in the details, you can perform a live search to get a listing of actual results. Reports are meant to be accessible!
- Tracking Results Over Time
As mentioned earlier, reports are snapshots of the search results as Shodan sees them at the moment. You can create reports every few months to see how things are changing over time (this might actually be built into Shodan Reports as well)
The old shodanhq.com website has bookmarks (there’s a small star icon at the top of the search results) but it was rarely used and didn’t offer much that browser bookmarks didn’t. I’m hoping that these reports will provide a prettier bookmarking mechanism that’s also more useful!
Alright, you’re sold on reports and want to give it a shot yourself – here is how.
- Login to Shodan and perform a search. For example: Webcams
- Click on “Create Report” and give your report a title such as “Webcams of the World”
- You will be redirected to the page that will list all your reports
- Now just wait a few minutes for Shodan to generate your report, you will receive an email when it’s done! Once it’s ready, you can follow the link and you should see something like:
- And you can always find a list of all the reports you’ve generated by clicking on the chart icon in the top right corner
I created it because I needed a friendlier way to share search results and I got tired of manually generating my charts for blog posts 🙂 It’s very simple and straight-forward to create reports so give it a try and let me know what you think!