McGrew Security: GooSweep

GooSweep

Some notes

Welcome fellow IFIP attendees! Feel free to email me (wesleymcgrew@gmail.com) if you attended my talk and have any questions or just want to comment on things.

The YaSweep variant of this (to help alleviate the problems with the Google SOAP API being deprecated) is available for download here.

The only difference in usage is an additional command line option to specify appid.

Introduction

GooSweep is a pen-test tool for information-gathering that uses the Google Search Engine to find information on IP addresses and hostnames on a target network. The original purpose of GooSweep was to perform host-discovery in a stealthy manner by finding publicly accessible web logs, however, in some situations it can give clues about browsing habits, user and service enumeration, password policy, and much more.

Features

GooSweep differs from other "Google Hacking" tools in that it is not intended as a vulnerability sweep, looking for known-vulnerable scripts and apps with "inurl:"-style queries. This tool performs simpler queries of IP addresses and host names on a subnet and displays the results in a way that a penetration tester or systems administrator can quickly see at a glance how much information about the target network is publicly accessible. While the hosts are displayed with graphs showing relative popularity on Google, the actual search results are the sort of thing that need to be parsed by a person. Preferably one with a brain. Some things you might find in the results are:

Hits to web sites - For whatever reason, a lot of web sites like to (or don't realize that they are doing it) publish statistics about their traffic, even so detailed as to include the IP addresses of visitors.
Mailing list posts - From list archives, often with full headers. Users, workstation IP addresses, mail servers, etc.
Guestbook entries, Forum posts, other Misc. web stuff
Site-specific documentation - Instructions for employees on how to log on, default passwords, password policy, etc.

Then again, you might not. It might miss your most important server, or find some old information that's not relevant anymore. That's up to you to sort out. Some other nice things about GooSweep:

Stealth - GooSweep is a good tool to run across a subnet first to discover active hosts and other information without interacting with the target network.
Report Generation - Generates HTML reports with a graph showing relative popularity and links to the query results. Also generates comma delimited output for use in your own scripts, spreadsheets, or databases.
- Sample HTML output
- Sample comma delimited output
"Burst" mode - The Google API limits you to 1000 queries a day, which may not be sufficient for scanning large networks (or perhaps you want to save some queries for another program). This lets you do a specified number of queries, and then sleep until the next day and continue.

Requirements

GooSweep has been tested on Python 2.4.1 with pygoogle 0.6 (along with the few things it depends on).

It has been reported that it does not work in Windows under Cygwin. There seems to be a problem between Cygwin's python package and SOAPpy (one of pygoogle's dependencies). I'm not certain how to resolve this, however I have confirmed that GooSweep does work with the native windows version of Python available from python.org, after installing fpconst, SOAPpy, and pygoogle.

You will also need a Google API license, which you can learn more about here. They're free.

Once you get ahold of a Google API license, you'll want to put the key somewhere that pygoogle can find it. The easiest is to just have it in ".googlekey" in your home directory, but other options are listed in pygoogle documentation.

GooSweep will chew through hundreds of your API queries, of which you are only alloted 1,000 a day, so keep that in mind.

Downloads