Difference between pages "NSF DUE-0919593" and "Prefetch"

From ForensicsWiki
(Difference between pages)
Jump to: navigation, search
 
(External Links)
 
Line 1: Line 1:
This page includes links to digital forensics resources produced under NSF DUE-0919593, "Creating Realistic Forensic Corpora for Undergraduate Education and Research"
+
{{Expand}}
 +
Windows Prefetch files, introduced in [[Windows|Windows XP]], are designed to speed up the application startup process. Prefetch files contain the name of the executable, a Unicode list of DLLs used by that executable, a count of how many times the executable has been run, and a timestamp indicating the last time the program was run. Although Prefetch is present in Windows 2003, by default it is only enabled for boot prefetching. The feature is also found in [[Windows|Windows Vista]], where it has been augmented with [[SuperFetch]], [[ReadyBoot]], and [[ReadyBoost]]. For SSD drives Prefetch is disabled by default [http://blogs.msdn.com/b/e7/archive/2009/05/05/support-and-q-a-for-solid-state-drives-and.aspx].
  
'''EDUCATIONAL DATA SETS'''
+
The Prefetch files are stored in the directory:
 +
<pre>
 +
%SystemRoot%\Prefetch
 +
</pre>
  
1. 2009-M57 "Patents" scenario
+
The following files can be found in the Prefetch directory:
 +
* <tt>*.pf</tt>, which are Prefetch files;
 +
* <tt>Ag*.db</tt> and <tt>Ag*.db.trx</tt>, which are [[SuperFetch]] files;
 +
* <tt>Layout.ini</tt>;
 +
* <tt>PfPre_*.db</tt>;
 +
* <tt>PfSvPerfStats.bin</tt>
  
This scenario involves a small company called M57 which was engaged in prior art searches for patents. The fictional company is contacted by the local police in November 2009 after a person purchases a computer from Craigslist and discovers "kitty porn" on the computer. The police trace the computer back to the M57 company.
+
A Prefetch file contains the name of the application, a dash, and then an eight character hash of the location from which that application was run, and a <tt>.pf</tt> extension. The filenames should be all uppercase except for the extension. E.g. a filename for [[md5deep]] would look like: <tt>MD5DEEP.EXE-4F89AB0C.pf</tt>. If an application is run from two different locations on the drive (i.e. the user runs <tt>C:\md5deep.exe</tt> and then <tt>C:\Apps\Hashing\md5deep.exe</tt>), there will be two different prefetch files in the Prefetch folder. According to MSDN up to 128 Prefetch files can be stored in the Prefetch directory [http://blogs.msdn.com/ryanmy/archive/2005/05/25/421882.aspx].
  
The scenario actually involves three separate criminal activities:
+
== File format ==
      1 - Exfiltration of proprietary information by an M57 employee.
+
Each Prefetch file has a 4-byte signature (at offset 4) "SCCA" (or in hexadecimal notation 0x53 0x43 0x43 0x41). The signature is assumed to be preceded by a 4-byte format version indicator:
      2 - Stealing of M57's property and selling it on Craigslist.
+
* 17 (0x00000011) for [[Windows XP]] and [[Windows 2003]]
      3 - The possession of "kitty porn" photos by an M57 employee.
+
* 23 (0x00000017) for [[Windows Vista]], [[Windows 2008]], [[Windows 7]] and [[Windows 2012]] (note Windows 2012 has not been confirmed)
 +
* 26 (0x0000001a) for [[Windows 8|Windows 8.1]] (note this could be Windows 8 as well but has not been confirmed)
  
This is an involved scenario which has the following information available to students trying to "solve" the case:
+
For more information about the file format see: [[Windows Prefetch File Format]]
      1 - Disk image of the computer that was sold on Craigs List
+
      2 - Disk images of the firm's five computers when the police show up.
+
      3 - Disk images of the four USB drives that were found on-site belonging to M57 employees
+
      4 - The RAM image of each computer just before the disk was imaged.
+
  
There are approximately 2-4 weeks of use on each computer.
+
== Metadata ==
 +
The Prefetch file contains various metadata.
 +
* The executable's name, up to 29 characters.
 +
* The run count, or number of times the application has been run.
 +
* Volume related information, like volume path and volume serial number.
 +
* The size of the Prefetch file (sometimes referred to as end of file (EOF)).
 +
* The files and directories that were used doing the application's start-up.
  
2. Nitroba University Harassment Scenario
+
=== Timestamps ===
 +
The Prefetch file contains 2 types of timestamps
 +
* The time when the application was last ran (executed). Version 26 of the Prefetch format maintains 7 previous last run times.
 +
* The volume creation time (part of the volume information) of the volume the Prefetch file was created on.
  
This scenario involves a harassment case at the fictional Nitroba University.
+
The file system creation time of the Prefetch file indicates the first time the application was executed. Both the file system modification time of the Prefetch file and the embedded last run time indicate the last time the application was executed.
  
Nitroba's IT department has received an email from Lily Tuckrige, a teacher in the Chemistry Department.  Tuckrige has been receiving harassing emails and she suspects that they are being sent by a student in her class Chemistry 109, which she is teaching this summer.  The email was received at Tuckridge's personal email account, lilytuckrige@yahoo.com. She took a screenshot of the web browser and sent it in.
+
== Prefetch hash ==
 +
There are multiple known hashing functions to be used for prefetch file filename hashing, namely:
 +
* SCCA XP hash function; used on Windows XP and Windows 2003
 +
* SCCA Vista hash function; used on Windows Vista
 +
* SCCA 2008 hash function; used on Windows 2008, Windows 7, (possibly: Windows 2012) and Windows 8 (including 8.1)
  
The system administrator who received the complaint wrote back to Tuckridge that Nitroba needed the full headers of the email message. Tuckridge responded by clicking the "Full message headers" button in Yahoo Mail and sent in another screen shot, this one with mail headers.
+
=== SCCA XP hash function ===
 +
A Python implementation of the SCCA XP hash function:
  
The mail header shows that the mail message originated from the IP address 140.247.62.34, which is a Nitroba student dorm room. Three women share the dorm room. Nitroba provides an Ethernet connection in every dorm room but not Wi-Fi access, so one of the women's friends installed a Wi-Fi router in the room. There is no password on the Wi-Fi.
+
<pre>
 +
def ssca_xp_hash_function(filename):
 +
    hash_value = 0
 +
    for character in filename:
 +
        hash_value = ((hash_value * 37) + ord(character)) % 0x100000000
 +
        hash_value = (hash_value * 314159269) % 0x100000000
 +
        if hash_value > 0x80000000:
 +
            hash_value = 0x100000000 - hash_value
  
Because several email messages appear to come from the IP address, Nitroba decides to place a network sniffer on the ethernet port. All of the packets are logged. On Monday 7/21 Tuckridge received another harassing email. But this time instead of receiving it directly, the perpetrator sent it through a web-based service called
+
    return (abs(hash_value) % 1000000007) % 0x100000000
"willselfdestruct.com."  The website briefly shows the message to Tuckridge, and then the website reports that the "Message Has Been Destroyed."
+
</pre>
  
Students are provided with the screen shots, the packets that were collected from the Ethernet tap, and the Chem 109 roster. Their job is to determine if one of the students in the class was responsible for the harassing email and to provide clear, conclusive evidence to support your conclusion.
+
=== SCCA Vista hash function ===
 +
A Python implementation of the SCCA Vista hash function:
  
'''RESEARCH DATA SETS'''
+
<pre>
 +
def ssca_vista_hash_function(filename):
 +
    hash_value = 314159
 +
    for character in filename:
 +
        hash_value = ((hash_value * 37) + ord(character)) % 0x100000000
 +
    return hash_value
 +
</pre>
  
We are also making available an enlarged "research data set" which contains a wealth of information that can be used by students interested in RAM, Network, or Disk Forensics.
+
=== SCCA 2008 hash function ===
 +
A Python implementation of the SCCA 2008 hash function:
  
The research data set was created at the same time as the 2009-M57 Patents dataset but contains substantially more information:
+
<pre>
  * All of the IP packets in and out of the M57 test network.
+
def ssca_2008_hash_function(filename):
  * Daily disk images and RAM captures of each computer on the network.
+
    hash_value = 314159
 +
    filename_index = 0
 +
    filename_length = len(filename)
 +
    while filename_index + 8 < filename_length:
 +
        character_value = ord(filename[filename_index + 1]) * 37
 +
        character_value += ord(filename[filename_index + 2])
 +
        character_value *= 37
 +
        character_value += ord(filename[filename_index + 3])
 +
        character_value *= 37
 +
        character_value += ord(filename[filename_index + 4])
 +
        character_value *= 37
 +
        character_value += ord(filename[filename_index + 5])
 +
        character_value *= 37
 +
        character_value += ord(filename[filename_index + 6])
 +
        character_value *= 37
 +
        character_value += ord(filename[filename_index]) * 442596621
 +
        character_value += ord(filename[filename_index + 7])
 +
        hash_value = ((character_value - (hash_value * 803794207)) % 0x100000000)
 +
        filename_index += 8
  
This data is not needed to "solve" the scenario, but it might be interesting for students that are:
+
    while filename_index < filename_length:  
 +
      hash_value = (((37 * hash_value) + ord(filename[filename_index])) % 0x100000000)
 +
      filename_index += 1
  
  * Interested in learning about RAM analysis and needs a source of RAM images.
+
    return hash_value
  * Interested in network forensics and wants packets.
+
</pre>
  * Interested in writing software that does "disk differencing" or can detect the installation of malware.
+
  * Wants examples of how a Windows registry is modified over time with use.
+
  
'''OBTAINING THE DATA'''
+
== Registry Keys ==
 +
<pre>
 +
Key: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\Memory Management\PrefetchParameters
 +
</pre>
  
You can obtain our data at the following addresses:
+
The EnablePrefetcher Registry value can be used to disable prefetch.
  
The M57 Corpus:
+
== See Also ==
  * http://torrent.ibiblio.org/doc/187/torrents  (bit torrent form)
+
* [[Prefetch XML]]
  * http://domex.nps.edu/corp/scenarios/2009-m57/  (individual files)
+
* [[ReadyBoot]]
 +
* [[SuperFetch]]
 +
* [[Windows]]
 +
* [[Windows Prefetch File Format]]
  
Please download from the iBiblio bittorrent server if possible. There are a number of torrents available for your convenience. If you examine the manifests, you will notice that the files overlap (some disk images appearing in more than one torrent).
+
== External Links ==
 +
* [http://msdn.microsoft.com/msdnmag/issues/01/12/XPKernel/default.aspx More detail from Microsoft]
 +
* [http://en.wikipedia.org/wiki/Prefetcher Wikipedia Prefetcher]
 +
* [http://msdn.microsoft.com/en-us/library/ms940847(v=winembedded.5).aspx MSDN: Disabling Prefetch]
 +
* [http://www.microsoft.com/whdc/driver/kernel/XP_kernel.mspx Kernel Enhancements for Windows XP], by [[Microsoft]], January 13, 2003 (Microsoft's description of Prefetch when Windows XP was introduced)
 +
* [http://blogs.msdn.com/b/ryanmy/archive/2005/05/25/421882.aspx Misinformation and the The Prefetch Flag], MSDN Blogs, May 25, 2005
 +
* [http://windowsir.blogspot.ch/2005/07/prefetch-file-metadata.html Prefetch file metadata], by [[Harlan Carvey]], July 13, 2005
 +
* [http://windowsir.blogspot.ch/2006/04/prefetch-files-revisited.html Prefetch files, revisited], by [[Harlan Carvey]], April 13, 2006
 +
* [http://blogs.msdn.com/b/e7/archive/2009/05/05/support-and-q-a-for-solid-state-drives-and.aspx Support and Q&A for Solid-State Drives], by Steven Sinofsky, May 5, 2009
 +
* [http://computer-forensics.sans.org/blog/2009/08/05/de-mystifying-defrag-identifying-when-defrag-has-been-used-for-anti-forensics-part-1-windows-xp/ De-mystifying Defrag: Identifying When Defrag Has Been Used for Anti-Forensics (Part 1 - Windows XP)], by [[Chad Tilbury]], August 5, 2009
 +
* [http://www.swiftforensics.com/2010/04/the-windows-prefetchfile.html Windows Prefetch File (old blog entry from 42 LLC)], by [[Yogesh Khatri]], April 14, 2010
 +
* [http://msdn.microsoft.com/en-us/library/windows/hardware/dn653317(v=vs.85).aspx Windows PC Accelerators], by Microsoft, October 8, 2010
 +
* [http://www.dfinews.com/articles/2010/12/decoding-prefetch-files-forensic-purposes-part-1 Decoding Prefetch Files for Forensic Purposes: Part 1], by [[Mark Wade]], December 8, 2010
 +
* [http://crucialsecurityblog.harris.com/2011/04/11/prefetch-files-at-face-value/ Prefetch Files at Face Value], by [[Mark Wade]], April 11, 2011
 +
* [http://kitrap08.blogspot.hk/2011/07/windows-logical-prefetcher.html Windows Logical Prefetcher], TTS blog, July 30, 2011 (article is in Russian)
 +
* [http://labit.in/pliki-prefetch-w-windows/ Prefetch i niedokładny licznik] by Paweł Hałdrzyński, August 20, 2011 (article in Polish; uncertain about the year of publication)
 +
* [http://windowsir.blogspot.ch/2012/03/prefetch-analysis-revisited.html Prefetch Analysis, Revisited], by [[Harlan Carvey]], March 8, 2012
 +
* [http://windowsir.blogspot.ch/2012/03/prefetch-analysis-revisitedagain.html Prefetch Analysis, Revisited...Again...], by [[Harlan Carvey]], March 15, 2012
 +
* [http://www.hexacorn.com/blog/2012/06/13/prefetch-hash-calculator-a-hash-lookup-table-xpvistaw7w2k3w2k8/ Prefetch Hash Calculator + a hash lookup table xp/vista/w7/w2k3/w2k8], Hexacorn blog, June 13, 2012
 +
* [http://www.hexacorn.com/blog/2012/10/29/prefetch-file-names-and-unc-paths/ Prefetch file names and UNC paths], Hexacorn blog, October 29, 2012
 +
* [http://journeyintoir.blogspot.ch/2012/12/ntosboot-prefetch-file.html NTOSBOOT Prefetch File], by [[Corey Harrell]], December 5, 2012
 +
* [http://www.invoke-ir.com/2013/09/whats-new-in-prefetch-for-windows-8.html What's New in the Prefetch for Windows 8??], by Jared Atkinson, September 21, 2013
 +
* [http://www.swiftforensics.com/2013/10/windows-prefetch-pf-files.html?m=1 Windows Prefetch (.PF) files], by [[Yogesh Khatri]], October 21, 2013
 +
* [http://resources.infosecinstitute.com/windows-systems-artifacts-digital-forensics-part-iii-prefetch-files/ Windows Systems and Artifacts in Digital Forensics: Part III: Prefetch Files], by Ivan Dimov, November 21, 2013
 +
* [http://i.imgur.com/riuljsK.jpg Prefetch 101 - a Windows 8 Prefetch file walkthrough], by Jared Atkinson, 2014
  
Each torrent will place files into:
+
== Tools ==
  
          [YOUR_LOCAL_DIRECTORY]/torrent_name/corp/scenarios/2009_m57/
+
=== Commercial ===
  
Please seed if possible! The "police materials" torrent references only those materials that would be captured in a raid (e.g. the final day of the scenario).
+
=== Free - Non Open Source ===
 +
* [http://www.woanware.co.uk/forensics/prefetchforensics.html PrefetchForensics], PrefetchForensics is an application to extract information from Windows Prefetch files
 +
* [http://redwolfcomputerforensics.com/index.php?option=com_content&task=view&id=42&Itemid=55 Prefetch-Parser], Parse the prefetch files and display information
 +
* [http://www.mitec.cz/wfa.html Windows File Analyzer] - Parses Prefetch files, thumbnail databases, shortcuts, index.dat files, and the recycle bin
 +
* [http://www.tzworks.net/prototype_page.php?proto_id=1 Windows Prefetch Parser (pf)], Free tool that can be run on Windows, Linux or Mac OS-X
  
 +
=== Open Source ===
 +
* [https://code.google.com/p/prefetch-tool/ prefetch-tool], Script to extract information from windows prefetch folder
 +
* [http://bitbucket.cassidiancybersecurity.com/prefetch-parser prefetch-parser], Standalone Python tools that parses Windows prefetch files and extracts all known and forensically relevant artefacts contained.
  
The 2008-Nitroba corpus:
+
[[Category:Windows]]
  * http://domex.nps.edu/corp/scenarios/2008-nitroba/
+

Revision as of 02:22, 24 April 2014

Information icon.png

Please help to improve this article by expanding it.
Further information might be found on the discussion page.

Windows Prefetch files, introduced in Windows XP, are designed to speed up the application startup process. Prefetch files contain the name of the executable, a Unicode list of DLLs used by that executable, a count of how many times the executable has been run, and a timestamp indicating the last time the program was run. Although Prefetch is present in Windows 2003, by default it is only enabled for boot prefetching. The feature is also found in Windows Vista, where it has been augmented with SuperFetch, ReadyBoot, and ReadyBoost. For SSD drives Prefetch is disabled by default [1].

The Prefetch files are stored in the directory:

%SystemRoot%\Prefetch

The following files can be found in the Prefetch directory:

  • *.pf, which are Prefetch files;
  • Ag*.db and Ag*.db.trx, which are SuperFetch files;
  • Layout.ini;
  • PfPre_*.db;
  • PfSvPerfStats.bin

A Prefetch file contains the name of the application, a dash, and then an eight character hash of the location from which that application was run, and a .pf extension. The filenames should be all uppercase except for the extension. E.g. a filename for md5deep would look like: MD5DEEP.EXE-4F89AB0C.pf. If an application is run from two different locations on the drive (i.e. the user runs C:\md5deep.exe and then C:\Apps\Hashing\md5deep.exe), there will be two different prefetch files in the Prefetch folder. According to MSDN up to 128 Prefetch files can be stored in the Prefetch directory [2].

File format

Each Prefetch file has a 4-byte signature (at offset 4) "SCCA" (or in hexadecimal notation 0x53 0x43 0x43 0x41). The signature is assumed to be preceded by a 4-byte format version indicator:

For more information about the file format see: Windows Prefetch File Format

Metadata

The Prefetch file contains various metadata.

  • The executable's name, up to 29 characters.
  • The run count, or number of times the application has been run.
  • Volume related information, like volume path and volume serial number.
  • The size of the Prefetch file (sometimes referred to as end of file (EOF)).
  • The files and directories that were used doing the application's start-up.

Timestamps

The Prefetch file contains 2 types of timestamps

  • The time when the application was last ran (executed). Version 26 of the Prefetch format maintains 7 previous last run times.
  • The volume creation time (part of the volume information) of the volume the Prefetch file was created on.

The file system creation time of the Prefetch file indicates the first time the application was executed. Both the file system modification time of the Prefetch file and the embedded last run time indicate the last time the application was executed.

Prefetch hash

There are multiple known hashing functions to be used for prefetch file filename hashing, namely:

  • SCCA XP hash function; used on Windows XP and Windows 2003
  • SCCA Vista hash function; used on Windows Vista
  • SCCA 2008 hash function; used on Windows 2008, Windows 7, (possibly: Windows 2012) and Windows 8 (including 8.1)

SCCA XP hash function

A Python implementation of the SCCA XP hash function:

def ssca_xp_hash_function(filename):
    hash_value = 0
    for character in filename:
        hash_value = ((hash_value * 37) + ord(character)) % 0x100000000
        hash_value = (hash_value * 314159269) % 0x100000000
        if hash_value > 0x80000000:
            hash_value = 0x100000000 - hash_value

    return (abs(hash_value) % 1000000007) % 0x100000000

SCCA Vista hash function

A Python implementation of the SCCA Vista hash function:

def ssca_vista_hash_function(filename):
    hash_value = 314159
    for character in filename:
        hash_value = ((hash_value * 37) + ord(character)) % 0x100000000
    return hash_value

SCCA 2008 hash function

A Python implementation of the SCCA 2008 hash function:

def ssca_2008_hash_function(filename):
    hash_value = 314159
    filename_index = 0
    filename_length = len(filename)
    while filename_index + 8 < filename_length:
        character_value = ord(filename[filename_index + 1]) * 37
        character_value += ord(filename[filename_index + 2])
        character_value *= 37
        character_value += ord(filename[filename_index + 3])
        character_value *= 37
        character_value += ord(filename[filename_index + 4])
        character_value *= 37
        character_value += ord(filename[filename_index + 5])
        character_value *= 37
        character_value += ord(filename[filename_index + 6])
        character_value *= 37
        character_value += ord(filename[filename_index]) * 442596621
        character_value += ord(filename[filename_index + 7])
        hash_value = ((character_value - (hash_value * 803794207)) % 0x100000000)
        filename_index += 8

    while filename_index < filename_length: 
       hash_value = (((37 * hash_value) + ord(filename[filename_index])) % 0x100000000)
       filename_index += 1

    return hash_value 

Registry Keys

Key: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\Memory Management\PrefetchParameters

The EnablePrefetcher Registry value can be used to disable prefetch.

See Also

External Links

Tools

Commercial

Free - Non Open Source

Open Source

  • prefetch-tool, Script to extract information from windows prefetch folder
  • prefetch-parser, Standalone Python tools that parses Windows prefetch files and extracts all known and forensically relevant artefacts contained.