A few weeks ago I asked people on Twitter what sort of new ports/ services they’d like me to add to Shodan. I received a lot of awesome feedback which resulted in Shodan now crawling for more than 170 ports (!!!). One of those requests was for the FINS protocol created by Omron:
By the way, I’m always looking to add more ports to Shodan so if there’s something you’d like to see me add just let me know which port and protocol you’re interested in! Anyways, I did some quick Google-ing to learn more about this FINS protocol and I stumbled across the following advice in the official Omron documentation (PDF):
Just to reiterate: they’re saying that because their device (Omron PLC) isn’t a Windows-based operating system that makes it impenetrable to the standard hacking methods. And I’m not sure what they mean with the following sentence about not responding to “standard ethernet protocol commands”, since the FINS protocol in this case operates over UDP and/ or TCP. Either way, this is a good example of why many control systems can be found on the Internet. This document is a few years old now (released in 2009) so Omron as a company might’ve improved their stance on Internet-security, but control systems are a slow-moving world and this sort of mentality has lingered around for a long time.
So what about the initial request to add Omron FINS to Shodan? After reviewing the pcaps for Wireshark and trying to find a simulator, I hit a road block and stopped making progress. Fortunately, Stephen Hilt picked it up as a challenge and within a few days was able to create fully-working Nmap scripts for both TCP and UDP versions of the Omron FINS protocol. If you’re interested in doing ICS analysis with Nmap, that should be your goto location for getting started. Thanks to Stephen’s work, I was able to convert the NSEs into Python scripts for my crawler and it’s now possible to find Omron FINS devices on the Internet via Shodan:
The data is still flowing in so the results are on the lower-bound at the moment, but it’s been added to the list of services that Shodan permanently crawls for to keep track of how the exposure of these devices changes over time.
PS: If there is a port/ protocol that you’d like to see in Shodan please email me the information to firstname.lastname@example.org 🙂
Telnet just doesn’t seem to die, and with millions of devices on the Internet still running it I don’t expect it to go away anytime soon. In fact, new products often use Telnet as an easy way to enable remote access to their device (see https://github.com/swisspol/GCDTelnetServer). To better keep track of which features the Telnet servers of the Internet support, the banners for Telnet now also include information about which options were negotiated during the initial connection:
In addition to the banner, you can now access the do, dont, will, wont options that the Telnet server supports. You’ll also be able to facet on these properties once the next API update comes out, which will make it easy to get a feel for how Telnet usage changes over time. Hopefully, more secure options will become easier to work with for the average developer and we’ll see fewer Telnet in the coming years.
Shodan collects a ton of data (1+ billion banners/ month) and it can be difficult to find the needle in the haystack. Sometimes the banner itself doesn’t tell you much about the device, for example a search for wind farms by Nordex is “jetty 2000”. Not exactly what you’d think when trying to locate wind farms. So most people end up searching Shodan using queries shared by other users and don’t really get to experience the fun in discovering new types of devices.
To help browse through the data that Shodan gathers I wrote a simple webapp called Shodan Explorer:
It randomly pulls an IP out of the stream of data that Shodan gathers, checks that the IP runs a webserver (HTTP) and then embeds the IP in an iframe so you can see it in your browser. Explorer is a simple, quick and hopefully fun way to check out all the weird stuff that’s on the Internet. If you end up finding something new please share the search result (Twitter, Blog, Shodan Search Directory, etc.) so we can start classifying all the unknown types of devices that are being crawled.
Note: the website has to be hosted on HTTP and not HTTPS since browsers don’t let you load mixed content on a HTTPS connection.
A lot of people prefer to avoid the terminal and instead want a user-friendly web interface to manage their server. So to determine which solution is the most popular Shodan has started to crawl the Internet for cPanel (2082, 2083), WHM (2086, 2087) and Webmin (10000)! The banners are all fairly standard HTTP(S) responses so I won’t show those, but lets take a look at who runs what and how they compare.
The majority of devices are located in the United States (117,000) followed by Canada (9,000) and the United Kingdom (7,000). So cPanel is definitely way more popular in the US than anywhere else. And it’s especially popular at the hosting provider Unified Layer, which is responsible for 21,000 of those US installations.
There are fewer Webmin instances on the Internet than cPanel, but the distribution of them is wider across the globe; i.e. it’s not as US-centric as cPanel. The US still leads with 38,000 instances, but not that much more compared to the following countries France (11,400) and Germany (11,000). In terms of hosting providers though there is once again a standout: OVH. Their organization accounts for roughly 10,000 of the installed Webmin instances.
Web Host Manager (WHM)
WHM is a bit different in that it is the software to manage cPanel accounts. This effectively gives us a measurement of which devices are being used by re-sellers. Unsurprisingly, the distribution of devices in terms of countries and organizations is very similar to cPanel. US with 97,000 devices, followed by Canada (9,000) and the UK (7,000) once again.
Call for Ports
Am I missing some ports or services? Is there something you’d like to see Shodan crawl? Then let me know and send me an email (email@example.com) or tweet at me (@achillean) with the ports that you’d like me to add!
SSH was one of the first protocols that I started crawling for 5 years ago because just connecting to the daemon already tells you what it’s running. I.e. you don’t have to send any data to SSH in order to get something interesting back. There have been some incremental improvements to add product and version detection but beyond that it’s stayed mostly the same.
Introducing the new, sexier SSH banner:
The crawlers now collect the key, key type, fingerprint, MAC and cipher used for each successful SSH connection! And alongside these changes the API has also been enhanced with 4 new facets for SSH:
I hope you enjoy the new banner and information that’s being gathered for SSH now! Let me know if there are other banners you’d like to see improved as well.
It’s now possible to see what information Shodan has available on a server from within Firefox thanks to the new Shodan add-on created by @PaulWebSec and @romainletendart! It’s a minimalistic yet powerful add-on to see what the website you’re visiting is exposing to the Internet. And the add-on will also tell you other information about the IP, for example who owns the IP space (organization), where it’s located and if possible the operating system it’s running. You can download the add-on from here:
And once you have it installed, you just need to click on the small Shodan icon to get an overview of what services that server is running on the Internet:
And what’s really cool is that the authors of the add-on made the code available at the following GitHub repository:
So if you have ideas, feedback or other suggestions on making it better just submit an issue or submit a pull request! And once again a big thank you to @PaulWebSec and @romainletendart for creating and sharing the add-on!
Have you ever needed to write a quick script to download data from Shodan? Or setup a cronjob to check what Shodan found on your network recently? How about getting a list of IPs out of the Shodan API? For the times where you’d like to have easy script-friendly access to Shodan there’s now a new command-line tool appropriately called shodan.
The shodan command-line interface (CLI) is packaged with the official Python library for Shodan, which means if you’re running the latest version of the library you already have access to the CLI. To install the new tool in Linux simply execute:
Or if you’re running an older version of the Shodan Python library and want to upgrade:
easy_install -U shodan
Once the tool is installed, you have to initialize the environment with your API key using shodan init:
shodan init YOUR_API_KEY
At the moment, the shodan CLI supports 6 commands. Note that for each command you can learn more about the options it supports by supplying the –help flag.
Returns the number of results for a search query.
$ shodan count microsoft iis 6.0 5310594
Search Shodan and download the results into a file where each line is a banner serialized in JSON as specified in https://developer.shodan.io/api/banner-specification
By default it will only download 1,000 results, if you want to download more look at the –limit flag.
For example, to download the latest 1,000 Microsoft-IIS 6.0 servers indexed by Shodan into a file called microsoft-data.json.gz use the following command:
This is the command that you should be using the most, since it lets you save your results and process them afterwards using the parse command. Because paging through results uses query credits, it makes sense to always store searches that you’re doing so you won’t need to use query credits for a search you already did in the past.
Initialize the shodan CLI. This is the first command you have to run for the tool to work, if you’re unsure about how to install the CLI please read the section above on installation.
shodan init YOUR_API_KEY
Returns your Internet-facing IP address.
$ shodan myip 188.8.131.52
Use parse to analyze a file that was generated using the download command. It lets you filter out the fields that you’re interested in, convert the JSON to a CSV and is friendly for pipe-ing to other scripts. For example, here’s the command to output the IP address, port and organization in CSV:
$ shodan parse --fields ip_str,port,org --separator , microsoft-data.json.gz
This command lets you search Shodan and view the results in a terminal-friendly way. By default it will display the IP, port, hostnames and data. You can use the –fields parameter to print whichever banner fields you’re interested in. For example, to search for Microsoft IIS 6.0 devices and print out their IP, port, organization and hostnames use the following command:
$ shodan search --fields ip_str,port,org,hostnames microsoft iis 6.0
I hope you’ve gotten a good idea of what the shodan CLI can do and how it might make your life easier. The tool is still in its early stages but it’s served me well so far. At this point, I’d love to get some feedback on things you’d like to see improved by submitting issues on the GitHub repository:
Coincidentally, that’s also the place where you can see the code for the tool in case you’re curious about the inner-workings. Please submit ideas for improvements and let me know via email (firstname.lastname@example.org) or Twitter whether this is useful to you!