Bulk extractor - Revision history

Simsong: /* Download */

2013-04-17T13:44:44Z

‎Download

Simsong: /* Output Feature Files */

2013-04-17T13:42:50Z

‎Output Feature Files

Simsong: /* Academic Publications */

2013-04-09T16:32:52Z

‎Academic Publications

Simsong: /* Academic Publications */

2013-04-09T16:29:31Z

‎Academic Publications

Simsong at 16:28, 9 April 2013

2013-04-09T16:28:06Z

Show changes

Simsong: /* Overview */

2013-03-23T16:36:49Z

‎Overview

Jessek: Updated current version number

2012-11-09T14:08:05Z

Updated current version number

Simsong at 03:45, 12 April 2012

2012-04-12T03:45:59Z

Simsong: updated download location

2012-04-10T00:26:21Z

updated download location

Dykstra: Summary and download info

2011-06-21T13:29:34Z

Summary and download info

@@ Line 45: / Line 45: @@
 == Download ==
-The current version of '''bulk_extractor''' is 1.3. It can be downloaded from http://digitalcorpora.org/downloads/bulk_extractor/
+The current version of '''bulk_extractor''' is 1.3.
 == Bibliography ==

@@ Line 8: / Line 8: @@
 ==Output Feature Files==
-bulk_extractor now creates an output directory that has the following layout:
+bulk_extractor now creates an output directory that includes:
-;alerts.txt
+* '''ccn.txt''' -- Credit card numbers
-:Processing errors.
+* '''ccn_track2.txt''' -- Credit card “track 2″ information
-;ccn.txt
+* '''domain.txt''' -- Internet domains found on the drive, including dotted-quad addresses found in text.
-:Credit card numbers
+* '''email.txt''' -- Email addresses
-;ccn_track2.txt
+* '''ether.txt''' -- Ethernet MAC addresses found through IP packet carving of swap files and compressed system hibernation files and file fragments.
-:Credit card “track 2″ informaiton, which has previously been found in some bank card fraud cases.
+* '''exif.txt''' -- EXIFs from JPEGs and video segments. This feature file contains all of the EXIF fields, expanded as XML records.
-;domain.txt
+* '''find.txt''' -- The results of specific regular expression search requests.
-:Internet domains found on the drive, including dotted-quad addresses found in text.
+* '''ip.txt''' -- IP addresses found through IP packet carving.
-;email.txt
+* '''telephone.txt''' --- US and international telephone numbers.
-:Email addresses.
+* '''url.txt''' --- URLs, typically found in browser caches, email messages, and pre-compiled into executables.
-;ether.txt
+* '''url_searches.txt''' --- A histogram of terms used in Internet searches from services such as Google, Bing, Yahoo, and others.
-;Ethernet MAC addresses found through IP packet carving of swap files and compressed system hibernation files and file fragments.
+* '''wordlist.txt''' --- :A list of all “words” extracted from the disk, useful for password cracking.
-;exif.txt
+* '''wordlist_*.txt''' --- The wordlist with duplicates removed, formatted in a form that can be easily imported into a popular password-cracking program.
-:EXIFs from JPEGs and video segments. This feature file contains all of the EXIF fields, expanded as XML records.
+* '''zip.txt''' --- A file containing information regarding every ZIP file component found on the media. This is exceptionally useful as ZIP files contain internal structure and ZIP is increasingly the compound file format of choice for a variety of products such as Microsoft Office
-;find.txt
-:The results of specific regular expression search requests.
-;ip.txt
-:IP addresses found through IP packet carving.
-;rfc822.txt
-:Email message headers including Date:, Subject: and Message-ID: fields.
-;tcp.txt
-:TCP flow information found through IP packet carving.
-;telephone.txt
-:US and international telephone numbers.
-;url.txt
-:URLs, typically found in browser caches, email messages, and pre-compiled into executables.
-;url_searches.txt
-:A histogram of terms used in Internet searches from services such as Google, Bing, Yahoo, and others.
-;url_services.txt
-:A histogram of the domain name portion of all the URLs found on the media.
-;wordlist.txt
 :A list of all “words” extracted from the disk, useful for password cracking.
-;wordlist_*.txt
-:The wordlist with duplicates removed, formatted in a form that can be easily imported into a popular password-cracking program.
-;zip.txt
-:A file containing information regarding every ZIP file component found on the media. This is exceptionally useful as ZIP files contain internal structure and ZIP is increasingly the compound file format of choice for a variety of products such as Microsoft Office
 For each of the above, two additional files may be created:
-;*_stopped.txt
+* '''*_stopped.txt''' --- bulk_extractor supports a stop list, or a list of items that do not need to be brought to the user’s attention. However rather than simply suppressing this information, which might cause something critical to be hidden, stopped entries are stored in the stopped files.
-:bulk_extractor supports a stop list, or a list of items that do not need to be brought to the user’s attention. However rather than simply suppressing this information, which might cause something critical to be hidden, stopped entries are stored in the stopped files.
+* '''*_histogram.txt''' --- bulk_extractor can also create histograms of features. This is important, as experience has shown that email addresses, domain names, URLs, and other information that appear more frequently on a hard drive or in a cell phone’s memory can be used to rapidly create a pattern of life report.
-;*_histogram.txt
-:bulk_extractor can also create histograms of features. This is important, as experience has shown that email addresses, domain names, URLs, and other informaiton that appear more frequently on a hard drive or in a cell phone’s memory can be used to rapidly create a pattern of life report.
 Bulk extractor also creates a file that captures the provenance of the run:

@@ Line 76: / Line 76: @@
 # Beverly, Robert, Simson Garfinkel and Greg Cardwell, [http://simson.net/clips/academic/2011.DFRWS.ipcarving.pdf "Forensic Carving of Network Packets and Associated Data Structures"], DFRWS 2011, Aug. 1-3, 2011, New Orleans, LA. BEST PAPER AWARD (Acceptance rate: 23%, 14/62)
 #Garfinkel, S., [http://simson.net/clips/academic/2006.DFRWS.pdf Forensic Feature Extraction and Cross-Drive Analysis,]The 6th Annual Digital Forensic Research Workshop Lafayette, Indiana, August 14-16, 2006. (Acceptance rate: 43%, 16/37)
 ===Tutorials===
 # [http://simson.net/ref/2012/2012-08-08%20bulk_extractor%20Tutorial.pdf Using bulk_extractor for digital forensics triage and cross-drive analysis], DFRWS 2012

@@ Line 75: / Line 75: @@
 # Garfinkel, Simson Simson, [http://simson.net/clips/academic/2013.COSE.bulk_extractor.pdf Digital media triage with bulk data analysis and bulk_extractor]. Computers and Security 32: 56-72 (2013)
 # Beverly, Robert, Simson Garfinkel and Greg Cardwell, [http://simson.net/clips/academic/2011.DFRWS.ipcarving.pdf "Forensic Carving of Network Packets and Associated Data Structures"], DFRWS 2011, Aug. 1-3, 2011, New Orleans, LA. BEST PAPER AWARD (Acceptance rate: 23%, 14/62)
 ===Tutorials===
 # [http://simson.net/ref/2012/2012-08-08%20bulk_extractor%20Tutorial.pdf Using bulk_extractor for digital forensics triage and cross-drive analysis], DFRWS 2012

@@ Line 1: / Line 1: @@
 == Overview ==
-'''bulk_extractor''' is a C++ program that scans a disk image, a file, or a directory of files and extracts useful information without parsing the file system or file system structures. The results are stored in [[feature files]] that can be easily inspected, parsed, or processed with automated tools. '''bulk_extractor''' also created a histograms of features that it finds, as features that are more common tend to be more important.
+'''bulk_extractor''' is a C++ program that scans a disk image, a file, or a directory of files and extracts useful information without parsing the file system or file system structures. The results can be easily inspected, parsed, or processed with automated tools. '''bulk_extractor''' also created a histograms of features that it finds, as features that are more common tend to be more important. The program can be used for law enforcement, defense, intelligence, and cyber-investigation applications.
 bulk_extractor is distinguished from other forensic tools by its speed and thoroughness. Because it ignores file system structure, bulk_extractor can process different parts of the disk in parallel. In practice, the program splits the disk up into 16MiByte pages and processes one page on each available core. This means that 24-core machines process a disk roughly 24 times faster than a 1-core machine. bulk_extractor is also thorough. That’s because bulk_extractor automatically detects, decompresses, and recursively re-processes compressed data that is compressed with a variety of algorithms. Our testing has shown that there is a significant amount of compressed data in the unallocated regions of file systems that is missed by most forensic tools that are commonly in use today.

@@ Line 69: / Line 69: @@
 == Download ==
-The current version of '''bulk_extractor''' is 1.0. It can be downloaded from https://github.com/simsong/bulk_extractor
+The current version of '''bulk_extractor''' is 1.3. It can be downloaded from https://github.com/simsong/bulk_extractor
 ==Sample Output==

← Older revision		Revision as of 13:29, 21 June 2011
Line 1:		Line 1:
		+	= Overview =
		+	'''bulk_extractor''' is a C++ program that scans a disk image, a file, or a directory of files and extracts useful information without parsing the file system or file system structures. The results are stored in feature files that can be easily inspected, parsed, or processed with automated tools. '''bulk_extractor''' also created a histograms of features that it finds, as features that are more common tend to be more important.
		+
		+	== Download ==
		+	The current version of '''bulk_extractor''' is 1.0. It can be downloaded from http://afflib.org/downloads/
		+
	==Sample Output==		==Sample Output==
	Running on 2.4Ghz iMac with MacOS 10.5.8 on the nps-2009-realistic.aff disk image, bulk extractor version 0.0.10 took 21816 seconds (6 hours, 3 minutes) and produced an [[Media:Nps-2009-realistic.extract.txt\|output with 14,160 lines]].		Running on 2.4Ghz iMac with MacOS 10.5.8 on the nps-2009-realistic.aff disk image, bulk extractor version 0.0.10 took 21816 seconds (6 hours, 3 minutes) and produced an [[Media:Nps-2009-realistic.extract.txt\|output with 14,160 lines]].