Shodan Plugin for Chrome

I’ve just released a new plugin for Chrome browser that lets you see what data Shodan has available for the website you’re currently visiting. This information includes the IP, hostnames (as determined via PTR lookups), location of the server (city, country), operating system, organization that owns the IP space as well as a list of open ports. As soon as you visit a website, it automatically asks the Shodan API (hence the api.shodan.io permission requirement) for all that information and then compiles it into a small pop-up that you can view when clicking on the Shodan icon next to the address bar:

shodan-chrome

The pop-up for the Shodan plugin is as simple as possible, which also means it doesn’t show the full banners for every port. There wasn’t enough room to fit everything in, especially for some of the hosts that have tons of ports open. I’m looking to find a way to show all of that information and more in future releases! For now, I’m happy to finally have a way to quickly see who’s hosting a website (the hostname often gives that away) and if they’re running some unusual services publicly. Oh, and if you see a dark blue square that means it can be opened by your browser and will open up a new tab with the IP:port if you click on it (in case you want to see what it shows when you access the webserver via its IP instead of hostname).

Go download the plugin and let me know what you think!

https://chrome.google.com/webstore/detail/shodan/jjalcfnidlmpjhdfepjhjbhnhkbgleap

Blast from the Past

As I flesh out more of the services that are offered on Shodan, I’ve started to look at some older protocols. To that end, I’ve added the following services:

All of these protocols have been deprecated due to security issues or superseded by better alternatives. Even though they’re probably not in the active minds of the modern sysadmin, these protocols are still alive on the Internet!

Systat

Displays information about the processes that are currently running on the system. Read More

  • Port: 11
  • Results: 2,969

USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
root 1 0.0 0.1 47540 3680 ? Ss Mar04 0:01 /sbin/init
root 2 0.0 0.0 0 0 ? S Mar04 0:00 [kthreadd]
root 3 0.0 0.0 0 0 ? S Mar04 0:00 [ksoftirqd/0]
root 5 0.0 0.0 0 0 ? S< Mar04 0:00 [kworker/0:0H]
root 7 0.0 0.0 0 0 ? S Mar04 0:00 [migration/0]
root 8 0.0 0.0 0 …

Note that the majority of the results don’t appear to actually be results of systat. Instead, it looks like the port has been re-purposed by a few people to run FTP, SSH and HTTP servers. This will also be true for netstat, where a lot of the results are from popular protocols running on a non-standard port.

Daytime

A simple protocol that returns the current date and time for the server.

  • Port: 13
  • Results: 92,539

Tuesday, March 30, 1993 14:14:55-GMT

Netstat

Shows all the currently active network connections on the device.

  • Port: 15
  • Results: 2,234

Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address Foreign Address State
tcp 0 0 *:cvspserver *:* LISTEN
tcp 0 0 *:amandaidx *:* LISTEN
tcp 0 0 *:printer *:* LISTEN
tcp 0 0 *:amidxtape *:* LISTEN
tcp 0 0 *:wbem-https …

Quote of the Day

As the name implies it shows a quote when a user connects to the service. Read More

  • Port: 17
  • Results: 40,574

“We want a few mad people now. See where the sane ones have landed us!”
George Bernard Shaw (1856-1950)

Finger

A service that lets you get information about user accounts on the server. Read More

  • Port: 79
  • Results: 59,699

Line User Host(s) Idle Location
* 66 vty 0 idle 00:00:00

There will be separate blog posts to look at the details of who’s still running these ancient services, but the data’s already been gathered and is searchable on Shodan. So please check to make sure your organization isn’t inadvertently using any of these protocols! I’ll be keeping track of these services over the coming months to determine whether these protocols are becoming more or less active and by how much.

Introducing Shodan Maps

If you’ve followed my blog posts and Twitter stream, you’ve probably seen images like this:

Global Backdoor Exposure

The above is a map of the globally exposed routers that contain a backdoor, and the way I’ve generated those images has until now been an internal tool. After making some changes to increase performance and make it more user-friendly, I’m officially releasing a new way to browse the Shodan search engine in the form of an add-on: Shodan Maps.

Shodan Maps

Shodan Maps provides a new and easy way to search for devices on the Internet and see the results on a map instead of a regular search listing. As you zoom into an area, it will narrow down your search results to only show devices that are within the viewable area. It essentially works and behaves the same way as every other map search service (Google Maps, Bing Maps etc.). It will currently display up to 1,000 results at a time on-screen as well as summary information about all the results (location-independent) such as top 5 services, organizations and countries. Note that it can take up to 30+ seconds sometimes to complete a search, depending on how far zoomed out you are and how many results there are in total.

There are also 3 different map styles that can be used depending on your preference. The default map style is “Satellite”, which is what I’ve been using most often and can be seen in the screenshot above. The next available map style is “Street View (Light)”:

Map Style: Light

And finally there’s “Street View (Dark)”:

Map Style: Dark

The map style can be changed anytime using the settings button next to the big read Search button. And when you click on one of the red dots you will see more information about the device, such as which services it’s running, who owns the IP space and anything else that is useful to know.

Host Information

I’ve found it very insightful to put results on a map and figure out patterns in the data, hopefully Shodan Maps can do the same for you!

Shodan Maps can be unlocked for a one-time payment of $19! No subscriptions, extra fees or anything like that and if you don’t love it let me know for a full refund.

Analyzing NTP Usage on the Internet with Shodan

NTP has been an interesting protocol for a while, I remember first hearing about the reflection attack 4 years ago when HD Moore unveiled some very clever research in how the “monlist” command could be used for various purposes. The gist is that a small packet can cause the NTP service to return a potentially large list of IPs, using the “monlist” command, that it has recently interacted with. And it looks like the protocol has made waves again recently when people started to actually use the “monlist” command in a reflection attack, effectively creating a DDoS. It’s interesting that it’s taken nearly 4 years to take it from HD Moore’s original research to publicly aware DDoS method. Have the attackers not needed a new method until recently, did their tools lag or have people simply not detected that this has been going on for a while? I’m not sure what the answer is and I’d be interested to hear whether others know more about this.

NTP Meet Shodan

To learn more about NTP and how widely deployed it is across the Internet, I’ve added the NTP service to the list of ports that Shodan surveys. So far, Shodan has identified more than 1 million devices that are running NTP on the Internet. And along with adding the service, I’m also making available a variety of new filters and facets. These are currently only accessible via the Shodan API (and any tools using the new API), but the filters will be rolled out to the main website very soon.

Filters

  • ntp.ip
    Search the initial list of IPs that the monlist command returned.
  • ntp.ip_count
    The number of IPs that the initial monlist command returned.
  • ntp.port
    The ports that the listed IPs used to connect to the NTP server.
  • ntp.more
    true or false – Whether or not there was more data to be read (i.e. more than the initial list of values).

Facets

  • ntp.ip
  • ntp.ip_count
  • ntp.port
  • ntp.more

NTP Is Everywhere

Somewhat unsurprisingly, the amount of NTP servers is fairly uniformly distributed across the globe:

NTP Around the World

The exact breakdown is as follows (facet on country:10):

United States 261924
South Korea 153381
Japan 75690
Russia 48508
China 30100
Germany 21659
France 17482
Great Britain 17027
Canada 15892
Italy 10340

It’s interesting that South Korea has a disproportionate amount of NTP servers, but otherwise I’d say the numbers and global distribution look fine.

Watching the Watchers

An interesting side-effect of the “monlist” command is that doing the survey on the entire Internet reveals patterns within the IP lists themselves. For example, faceting on ntp.ip reveals that there are a few standout IPs that get returned by a lot of NTP servers. These top IPs are most likely scanning the Internet for NTP. The majority of the top IPs have websites setup that indicate they’re doing research on the NTP DDoS reflection/ amplification attack, but the #1 IP that is crawling for NTP servers is from China and doesn’t have any research website available or indicative hostname. The IP is “58.215.177.51″ and historically it had an HTTP and FTP server running on it, though they aren’t available anymore (perhaps dynamic IP range). If anybody finds out more about that IP and whether or not it’s another research project, please let me know.

Supersize Me!

The default response packet for the “monlist” command returns up to 6 IPs. If the packet has the “more” flag set to true, then you can listen for more results until you have the entire list of IPs the NTP daemon stores. So how many of these NTP servers are very active and have more than just 6 IPs in their list? Lets first take a look at the general distribution of the number of IPs that are returned by NTP (facet on ntp.ip_count):

ntp-initial-iplist

The absolute numbers of the chart don’t reflect all the data that’s stored in Shodan because the facets are still being processed. And among all of the NTP servers found in Shodan that properly responded to the “monlist” command, this is how many had more than 6 IPs available to return (facet on ntp.more):

ntp-more

Note that for these charts NTP servers that returned errors weren’t included. If the servers with errors were included, then there would be ~13.5% with more data. I.e. out of all NTP servers on the Internet, including ones that didn’t respond properly to the “monlist” command for various reasons, ~13.5% of the devices returned more than 6 IP addresses.

Insider Info

The IPs returned by “monlist” aren’t limited to publicly accessible IP addresses, they can also include information about the local, private network! Below is a chart breaking down how many NTP devices at the moment are exposing 1 or more private IP address:

Private IPs Exposed via NTP

At the time of writing, there are nearly 10,000 NTP servers that list a private IP on the 10.0.0.0/8 netblock in their response, and another ~7,000 devices that list an address in the 192.168.0.0/16 IP range. Of note is also that a few devices (4,345)  even include their loopback IP in the response.

All the data is readily searchable via Shodan, so if you want to keep track of these stats and more just grab a free API key to get started. There’s also an updated Python library that exposes all of the facets, including the new NTP-related ones (stay tuned for more info on the API). There’s a lot more cool stuff that can be done with NTP and it’ll be exciting to see what further research into this area discovers!

Quick Statistics on the Router Backdoor on Port 32764

As soon as the backdoor information was posted on github I got curious about how common the issue would be, so I added port 32764 to the list of ports that I survey. I had set it up and moved on to work on other things, but the topic came up again recently so I thought I’d take a quick look at the data and see what can be learned.

A lot of devices respond on port 32764, but the majority of them are going to be firewalls, random web servers or other shenanigans that have nothing to do with the backdoor. To only get results that mattered, I searched not just for port 32764 but also for the strings “scmm” and “mmcs” (the same value just w/ different endianness). At the time of writing there are 6,401 devices in Shodan that match the criteria for routers with the potential backdoor vulnerability.

Global Backdoor Exposure

Global Backdoor Exposure

It’s interesting that the US doesn’t lead the number of public devices, since in general it’s at the top for online devices. Instead, we see that Great Britain leads the charge with 2,228 devices followed by Italy with 1315. Both of those are a bit surprising and probably indicate a bias in the routers that are purchased/ sold in those regions (more on that later).

Organizational Breakdown

Top Organizations

The largest ISPs in the respective countries (based on market capitalization) are leading the charge in backdoored devices. The organization with the most devices is Telecom Italia, so I took a deeper look into the data in Italy. To do so I wrote a quick Python script to go through the first 1,000 results in Italy, see if they have a HTTP service running as well and if they do I extract the product information from the “realm” section of the header:


from collections import defaultdict
from shodan import Shodan

stats = defaultdict(int)

api = Shodan('YOUR API KEY')

results = api.search('port:32764 country:it mmcs OR scmm', limit=1000)
for result in results['matches']:
 host = api.host(result['ip_str'])

if 80 in host['ports']:
 # Get the port 80 banner
 banner = None
 for tmp in host['data']:
 if tmp['port'] == 80:
 banner = tmp
 break

if not banner:
 continue

# Extract the realm info
 start = banner['data'].find('realm="')
 if start <= 0:
 continue
 start += len('realm="')
 end = banner['data'].find('"', start + 1)
 name = banner['data'][start:end]
 stats[name] += 1

for key,value in stats.items():
 print '%s,%s' % (key, value)

The results of this quick breakdown are below:

Italy Backdoor

I don’t know whether this is a general product bias in Italy or whether Telecom Italia prefers those brands of routers, but Netgear DG834G appears the most popular router in Italy that also has a built-in backdoor. Similar breakdowns can be done for the other countries using a modified version of the above script, which I might do later on (mostly for Great Britain).

Compared to routers that advertise their default credentials (62,883) or old Microsoft IIS servers (5.0 is at 312,334), the total number of routers with a publicly exposed backdoor is very small (6,401). It will be interesting to see whether the service is exposed on other devices in different ways or whether this is the complete picture for the vulnerability’s risk to the Internet. Either way, this is how the data presents the situation at the moment and as things change I will update the information/ charts!

I Know You Need New Toner

The first Shodan search that I remember people sharing like wildfire on  Twitter was for an HP LaserJet printer. In the early days of Shodan, before all the SCADA devices came to light, the majority of searches were for consumer devices such as printers, routers and webcams (the latter are still widely popular). Inspired by a cool new tool from @info_dox I decided to start crawling the Internet specifically for printers using the Printer Job Language protocol on port 9100. Obviously there aren’t as many of these devices out there as web servers, but within a few hours I had gathered enough to start analyzing the data and see what sort of cool stuff I can learn.

1. Global Map of Public Printers

To give you an idea for where these printers are located I’ve created the map below, where each red dot indicates an instance of a printer that is connected to the Internet and allows connections from anywhere in the world. The breakdown by country is:

  1. United States: 2692
  2. South Korea: 494
  3. Taiwan: 336
  4. Canada: 266
  5. Germany: 203

The surprise standouts in that list are South Korea and Taiwan, I expected the devices to follow the general distribution as for other services (i.e. United States > China > Mexico > Russia > Germany).

Global Map of Public Printers

2. Overexposed Universities

Not surprisingly Universities have a lot of printers, but they appear to be more exposed than one would anticipate. Out of the top 100 organizations running public printers, 58 of them are universities and another 4 are academic institutions. That means roughly 2/3 of all publicly exposed printers so far operate on an academic network.

Top 10 Organizations with Public Printers

The above explains why Taiwan is so high on the list of countries that have publicly available printers: the Taiwan Academic Network and its Information Center have nearly 100 printers online. It looks like they’re single-handedly putting Taiwan on the map! It’s also interesting that there are so many devices on Korea Telecom’s network, I don’t have a good explanation for that. If anybody has any explanation for why there are so many in South Korea and on Comcast, please let me know.

3. Need Toner?

An interesting side-effect of the crawling is that I can determine via Shodan which organizations are low on toner and are likely due for a refresh. Many printers advertise in their banner whether they’re running low and the user needs to order new toner. Note that no authentication is required to obtain this information. So here I present the top 10 organizations that will be needing new toner very soon :)

Top 10 Organizations Needing New Toner

At the time of writing the top position was shared between the University of California San Francisco and University of Pennsylvania with 4 printers needing toner each. They were narrowly followed by University of California Santa Cruz with 3 printers. There is a very long tail of organizations that need to replace the ink on 2 printers, with a total of 137 printers that need a replacement.

I will be watching these numbers develop over time (using a Google Spreadsheet), and I could theoretically also determine when the toner has been replaced which I might do in a follow-up post. Until then, enjoy the data and let me know if you discover anything interesting!

Shodan Add-On for Google Spreadsheets

I like to periodically do Shodan searches on a variety of services and see what sort of organizations run them the most, which ports are popular or see which version of Apache dominates the market (“2.2.3″ is leading at the moment for Apache while “6.0″ is the preferred choice for Microsoft IIS). For the most part that involves writing a script in Python to access the API, output a CSV and then import that into Google Spreadsheet to create pretty charts. I finally got tired of having to import CSV and decided that it’s time to integrate the ability to search Shodan into Google Spreadsheet. And it turns out that’s fairly easy to do! They have a simple Javascript API to create new macros and it took just a few minutes to get an initial prototype working. If you want to skip ahead, you can check out the code and instructions at https://gist.github.com/achillean/8367958

There are 3 new macros that the Shodan add-on for Google Spreadsheets provides:

SHODAN_COUNT

SHODAN_COUNT lets you get the total number of results for a given search query. For example:

=SHODAN_COUNT(“microsoft-iis/6.0″)

The above macro will fill the current cell with the total number of results of web servers running on Microsoft-IIS 6.0. The code that provides this macro is below:


/**
 * Search the Shodan database using the given query. Returns the number of matches.
 */
function SHODAN_COUNT(query) {
 var API_KEY = 'YOUR API KEY';

 var url = 'https://api.shodan.io/shodan/host/count?key=' + API_KEY + '&query=' + query;
 var response = UrlFetchApp.fetch(url);
 var data = Utilities.jsonParse(response.getContentText());

 return data.total;
};

SHODAN_FACET_KEYS and SHODAN_FACET_VALUES

These 2 macros are very useful, but they’re slightly awkward to use. I couldn’t find a way for a macro to create 2 columns/ rows so I had to separate the operation into 2 macros instead of just 1.

A facet provides you with aggregate information about a property. In practical terms, this is what gives you the Top 5 Countries, Most Popular Web Servers and other similar breakdowns. For example, try running a search such as:

=SHODAN_FACET_KEYS(“mongodb”, “domain”, 20)

This will provide you with a breakdown of the Top 20 Domains running MongoDB, though the above will only return the name and not the number for each domain. To do that you will need to use the other macro called SHODAN_FACET_VALUES:

=SHODAN_FACET_VALUES(“mongodb”, “domain”, 20)

The parameters for both macros should always be identical. If you enter the first macro into one cell, and the 2nd macro into the cell below then you will end up with 2 rows that contain a breakdown which you can then easily convert into a chart.

Below is the relevant code that provides the 2 macros:

/**
 * Return the names of the facet values for a given search. Optionally, you can also provide a total number of
 * facet results that should be returned. Use this method alongside SHODAN_FACET_VALUES to get the corresponding
 * values for a facet name. Most filters can be used as facets.
 */
function SHODAN_FACET_KEYS(query, facet, count) {
 var API_KEY = 'YOUR API KEY';

 if (!count) {
 count = 10;
 }

 var url = 'https://api.shodan.io/shodan/host/count?key=' + API_KEY + '&query=' + query + '&facets=' + facet + ':' + count;
 var response = UrlFetchApp.fetch(url);
 var data = Utilities.jsonParse(response.getContentText());

 var values = [];
 for (var i = 0; i < data.facets[facet].length; i++) {
 values.push(data.facets[facet][i].value);
 }

 return values;
};

/**
 * Return the values of the facet values for a given search. Optionally, you can also provide a total number of
 * facet results that should be returned. Use this method alongside SHODAN_FACET_KEYS to get the corresponding
 * names for a facet value. Most filters can be used as facets.
 */
function SHODAN_FACET_VALUES(query, facet, count) {
 var API_KEY = 'YOUR API KEY';

 if (!count) {
 count = 10;
 }

 var url = 'https://api.shodan.io/shodan/host/count?key=' + API_KEY + '&query=' + query + '&facets=' + facet + ':' + count;
 var response = UrlFetchApp.fetch(url);
 var data = Utilities.jsonParse(response.getContentText());

 var values = [];
 for (var i = 0; i < data.facets[facet].length; i++) {
 values.push(data.facets[facet][i].count);
 }

 return values;
};

Installation

  1. Create a Google Spreadsheet that you want to use for Shodan research
  2. Click on “Tools > Script Editor” in the menu
  3. A new tab will be opened
  4. Click on “Spreadsheet” on the left side
  5. Copy the code located at: https://gist.github.com/achillean/8367958
  6. Paste the code into the script editor
  7. Click on “File > Save” or “CTRL + S” to save the code
  8. Done! You can now use the Shodan macros inside your Shodan research spreadsheet

I hope these macros will be useful to others, and if you have ideas for new macros or other integrations that you’d like to see please let me know!

PS: I recommend trying out the new Google Sheets, it’s much nicer to work with.