Difference between pages "Prefetch" and "Gzip"

From ForensicsWiki
(Difference between pages)
Jump to: navigation, search
 
 
Line 1: Line 1:
{{Expand}}
+
{{expand}}
Windows Prefetch files, introduced in [[Windows|Windows XP]], are designed to speed up the application startup process. Prefetch files contain the name of the executable, a Unicode list of DLLs used by that executable, a count of how many times the executable has been run, and a timestamp indicating the last time the program was run. Although Prefetch is present in Windows 2003, by default it is only enabled for boot prefetching. The feature is also found in [[Windows|Windows Vista]], where it has been augmented with [[SuperFetch]], [[ReadyBoot]], and [[ReadyBoost]]. For SSD drives Prefetch is disabled by default [http://blogs.msdn.com/b/e7/archive/2009/05/05/support-and-q-a-for-solid-state-drives-and.aspx].
+
 
+
Up to 128 Prefetch files are stored in the <tt>%SystemRoot%\Prefetch</tt> directory [http://blogs.msdn.com/ryanmy/archive/2005/05/25/421882.aspx]. Each file in that directory should contain the name of the application, a dash, and then an eight character hash of the location from which that application was run, and a <tt>.pf</tt> extension. The filenames should be all uppercase except for the extension. The format of hashes is not known. A sample filename for [[md5deep]] would look like: <tt>MD5DEEP.EXE-4F89AB0C.pf</tt>. If an application is run from two different locations on the drive (i.e. the user runs <tt>C:\md5deep.exe</tt> and then <tt>C:\Apps\Hashing\md5deep.exe</tt>), there will be two different prefetch files in the Prefetch folder.
+
  
 
== File format ==
 
== File format ==
Each Prefetch file has a 4-byte signature (at offset 4) "SCCA" (or in hexadecimal notation 0x53 0x43 0x43 0x41). The signature is assumed to be preceded by a 4-byte format version indicator:
+
The gzip file (.gz) format consists of:
* 17 (0x00000011) for [[Windows XP]] and [[Windows 2003]]
+
* a file header
* 23 (0x00000017) for [[Windows Vista]], [[Windows 2008]], [[Windows 7]] and [[Windows 2012]] (note Windows 2012 has not been confirmed)
+
* optional headers
* 26 (0x0000001a) for [[Windows 8|Windows 8.1]] (note this could be Windows 8 as well but has not been confirmed)
+
** extra fields
 +
** original file name
 +
** comment
 +
** header checksum
 +
* a body, containing a DEFLATE-compressed payload
 +
* a file footer
  
For more information about the file format see: [[Windows Prefetch File Format]]
+
=== File header ===
 +
The file header is 10 bytes in size and contains:
 +
{| class="wikitable"
 +
! align="left"| Offset
 +
! Size
 +
! Value
 +
! Description
 +
|-
 +
| 0
 +
| 2
 +
| 0x1f 0x8b
 +
| Signature (or identification byte 1 and 2)
 +
|-
 +
| 2
 +
| 1
 +
|
 +
| Compression Method
 +
|-
 +
| 3
 +
| 1
 +
|
 +
| Flags
 +
|-
 +
| 4
 +
| 4
 +
|
 +
| Last modification time <br> Contains a POSIX timestamp.
 +
|-
 +
| 8
 +
| 1
 +
|
 +
| Extra flags
 +
|-
 +
| 9
 +
| 1
 +
|
 +
| Operating system <br> Value that indicates on which operating system the gzip file was created.
 +
|}
  
== Metadata ==
+
==== Compression method ====
The Prefetch file contains various metadata.
+
* The executable's name, up to 29 characters.
+
* The run count, or number of times the application has been run.
+
* Volume related information, like volume path and volume serial number.
+
* The size of the Prefetch file (sometimes referred to as end of file (EOF)).
+
* The files and directories that were used doing the application's start-up.
+
  
=== Timestamps ===
+
{| class="wikitable"
The Prefetch file contains 2 types of timestamps
+
! align="left"| Value
* The time when the application was last ran (executed). Version 26 of the Prefetch format maintains 7 previous last run times.
+
! Identifier
* The volume creation time (part of the volume information) of the volume the Prefetch file was created on.
+
! Description
 +
|-
 +
| 0 - 7
 +
|
 +
| Reserved
 +
|-
 +
| 8
 +
| "deflate"
 +
| zlib compressed data
 +
|}
  
The creation date of the Prefetch file indicates the first time the application was executed. Both the modification date of the Prefetch file and the embedded Last Run timestamp indicate the last time the application was executed.
+
==== Flags ====
  
== Prefetch hash ==
+
{| class="wikitable"
There are multiple known hashing functions to be used for prefetch file filename hashing, namely:
+
! align="left"| Value
* SCCA XP hash function; used on Windows XP and Windows 2003
+
! Identifier
* SCCA Vista hash function; used on Windows Vista
+
! Description
* SCCA 2008 hash function; used on Windows 2008, Windows 7, (possibly: Windows 2012) and Windows 8 (including 8.1)
+
|-
 +
| 0x01
 +
| FTEXT
 +
| If set the uncompressed data needs to be treated as text instead of binary data. <br> This flag hints end-of-line conversion for cross-platform text files but does not enforce it.
 +
|-
 +
| 0x02
 +
| FHCRC
 +
| The file contains a header checksum (CRC-16)
 +
|-
 +
| 0x04
 +
| FEXTRA
 +
| The file contains extra fields
 +
|-
 +
| 0x08
 +
| FNAME
 +
| The file contains an original file name string
 +
|-
 +
| 0x10
 +
| FCOMMENT
 +
| The file contains comment
 +
|-
 +
| 0x20
 +
|
 +
| Reserved
 +
|-
 +
| 0x40
 +
|
 +
| Reserved
 +
|-
 +
| 0x80
 +
|
 +
| Reserved
 +
|}
  
=== SCCA XP hash function ===
+
<b>Note:</b> The FHCRC bit was never set by versions of gzip up to 1.2.4, even though it was documented with a different meaning in gzip 1.2.4.
A Python implementation of the SCCA XP hash function:
+
  
<pre>
+
==== Extra flags ====
def ssca_xp_hash_function(filename):
+
If compression method is 8 the following extra flags can be defined:
    hash_value = 0
+
{| class="wikitable"
    for character in filename:
+
! align="left"| Value
        hash_value = ((hash_value * 37) + ord(character)) % 0x100000000
+
! Identifier
        hash_value = (hash_value * 314159269) % 0x100000000
+
! Description
        if hash_value > 0x80000000:
+
|-
            hash_value = 0x100000000 - hash_value
+
| 0x02
 +
|
 +
| compressor used maximum compression, slowest algorithm
 +
|-
 +
| 0x04
 +
|
 +
| compressor used fastest algorithm
 +
|}
  
    return (abs(hash_value) % 1000000007) % 0x100000000
+
==== Operating System ====
</pre>
+
{| class="wikitable"
 +
! align="left"| Value
 +
! Identifier
 +
! Description
 +
|-
 +
| 0
 +
|
 +
| FAT filesystem (MS-DOS, OS/2, NT/Win32)
 +
|-
 +
| 1
 +
|
 +
| Amiga
 +
|-
 +
| 2
 +
|
 +
| VMS (or OpenVMS)
 +
|-
 +
| 3
 +
|
 +
| Unix
 +
|-
 +
| 4
 +
|
 +
| VM/CMS
 +
|-
 +
| 5
 +
|
 +
| Atari TOS
 +
|-
 +
| 6
 +
|
 +
| HPFS filesystem (OS/2, NT)
 +
|-
 +
| 7
 +
|
 +
| Macintosh
 +
|-
 +
| 8
 +
|
 +
| Z-System
 +
|-
 +
| 9
 +
|
 +
| CP/M
 +
|-
 +
| 10
 +
|
 +
| TOPS-20
 +
|-
 +
| 11
 +
|
 +
| NTFS filesystem (NT)
 +
|-
 +
| 12
 +
|
 +
| QDOS
 +
|-
 +
| 13
 +
|
 +
| Acorn RISCOS
 +
|-
 +
| 255
 +
|
 +
| unknown
 +
|}
  
=== SCCA Vista hash function ===
+
=== Optional headers ===
A Python implementation of the SCCA Vista hash function:
+
==== Extra fields ====
 +
<b>TODO: add description</b>
  
<pre>
+
The extra field are variable of size and contains:
def ssca_vista_hash_function(filename):
+
{| class="wikitable"
    hash_value = 314159
+
! align="left"| Offset
    for character in filename:
+
! Size
        hash_value = ((hash_value * 37) + ord(character)) % 0x100000000
+
! Value
    return hash_value
+
! Description
</pre>
+
|-
 +
| 0
 +
| 2
 +
|
 +
| Extra field data size <br> Value in bytes.
 +
|-
 +
| 2
 +
| ...
 +
|
 +
| Extra field data
 +
|}
  
=== SCCA 2008 hash function ===
+
==== Original file name ====
A Python implementation of the SCCA 2008 hash function:
+
This is the original name of the file being compressed, with any directory components removed, and, if the file being compressed is on a file system with case insensitive names, forced to lower case.
  
<pre>
+
Contains an ISO 8859-1 (LATIN-1) string with end-of-string character.
def ssca_2008_hash_function(filename):
+
    hash_value = 314159
+
    filename_index = 0
+
    filename_length = len(filename)
+
    while filename_index + 8 < filename_length:
+
        character_value = ord(filename[filename_index + 1]) * 37
+
        character_value += ord(filename[filename_index + 2])
+
        character_value *= 37
+
        character_value += ord(filename[filename_index + 3])
+
        character_value *= 37
+
        character_value += ord(filename[filename_index + 4])
+
        character_value *= 37
+
        character_value += ord(filename[filename_index + 5])
+
        character_value *= 37
+
        character_value += ord(filename[filename_index + 6])
+
        character_value *= 37
+
        character_value += ord(filename[filename_index]) * 442596621
+
        character_value += ord(filename[filename_index + 7])
+
        hash_value = ((character_value - (hash_value * 803794207)) % 0x100000000)
+
        filename_index += 8
+
  
    while filename_index < filename_length:
+
==== Comment ====
      hash_value = (((37 * hash_value) + ord(filename[filename_index])) % 0x100000000)
+
Contains an ISO 8859-1 (LATIN-1) string with end-of-string character. Line breaks should be denoted by a single line feed character.
      filename_index += 1
+
  
    return hash_value
+
==== Header checksum ====
</pre>
+
The header checksum contain a CRC-16 that consists of the two least significant bytes of the CRC-32 for all bytes of the gzip header up to and not including the CRC-16.
  
== Registry Keys ==
+
=== File footer ===
<pre>
+
The file footer is 8 bytes in size and contains:
Key: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\Memory Management\PrefetchParameters
+
{| class="wikitable"
</pre>
+
! align="left"| Offset
 
+
! Size
The EnablePrefetcher Registry value can be used to disable prefetch.
+
! Value
 
+
! Description
== See Also ==
+
|-
* [[Windows Prefetch File Format]]
+
| 0
* [[SuperFetch]]
+
| 4
* [[Prefetch XML]]
+
|
* [[Windows]]
+
| Checksum (CRC-32)
 +
|-
 +
| 4
 +
| 4
 +
|
 +
| Uncompressed data size <br> Value in bytes.
 +
|}
  
 
== External Links ==
 
== External Links ==
* [http://msdn.microsoft.com/msdnmag/issues/01/12/XPKernel/default.aspx More detail from Microsoft]
 
* [http://en.wikipedia.org/wiki/Prefetcher Wikipedia Prefetcher]
 
* [http://msdn.microsoft.com/en-us/library/ms940847(v=winembedded.5).aspx MSDN: Disabling Prefetch]
 
* [http://www.microsoft.com/whdc/driver/kernel/XP_kernel.mspx Kernel Enhancements for Windows XP], by [[Microsoft]], January 13, 2003 (Microsoft's description of Prefetch when Windows XP was introduced)
 
* [http://blogs.msdn.com/b/ryanmy/archive/2005/05/25/421882.aspx Misinformation and the The Prefetch Flag], MSDN Blogs, May 25, 2005
 
* [http://windowsir.blogspot.ch/2005/07/prefetch-file-metadata.html Prefetch file metadata], by [[Harlan Carvey]], July 13, 2005
 
* [http://windowsir.blogspot.ch/2006/04/prefetch-files-revisited.html Prefetch files, revisited], by [[Harlan Carvey]], April 13, 2006
 
* [http://blogs.msdn.com/b/e7/archive/2009/05/05/support-and-q-a-for-solid-state-drives-and.aspx Support and Q&A for Solid-State Drives], by Steven Sinofsky, May 5, 2009
 
* [http://computer-forensics.sans.org/blog/2009/08/05/de-mystifying-defrag-identifying-when-defrag-has-been-used-for-anti-forensics-part-1-windows-xp/ De-mystifying Defrag: Identifying When Defrag Has Been Used for Anti-Forensics (Part 1 - Windows XP)], by [[Chad Tilbury]], August 5, 2009
 
* [http://www.swiftforensics.com/2010/04/the-windows-prefetchfile.html Windows Prefetch File (old blog entry from 42 LLC)], by [[Yogesh Khatri]], April 14, 2010
 
* [http://www.dfinews.com/articles/2010/12/decoding-prefetch-files-forensic-purposes-part-1 Decoding Prefetch Files for Forensic Purposes: Part 1], by [[Mark Wade]], December 8, 2010
 
* [http://crucialsecurityblog.harris.com/2011/04/11/prefetch-files-at-face-value/ Prefetch Files at Face Value], by [[Mark Wade]], April 11, 2011
 
* [http://kitrap08.blogspot.hk/2011/07/windows-logical-prefetcher.html Windows Logical Prefetcher], TTS blog, July 30, 2011 (article is in Russian)
 
* [http://labit.in/pliki-prefetch-w-windows/ Prefetch i niedokładny licznik] by Paweł Hałdrzyński, August 20, 2011 (article in Polish; uncertain about the year of publication)
 
* [http://windowsir.blogspot.ch/2012/03/prefetch-analysis-revisited.html Prefetch Analysis, Revisited], by [[Harlan Carvey]], March 8, 2012
 
* [http://windowsir.blogspot.ch/2012/03/prefetch-analysis-revisitedagain.html Prefetch Analysis, Revisited...Again...], by [[Harlan Carvey]], March 15, 2012
 
* [http://www.hexacorn.com/blog/2012/06/13/prefetch-hash-calculator-a-hash-lookup-table-xpvistaw7w2k3w2k8/ Prefetch Hash Calculator + a hash lookup table xp/vista/w7/w2k3/w2k8], Hexacorn blog, June 13, 2012
 
* [http://www.hexacorn.com/blog/2012/10/29/prefetch-file-names-and-unc-paths/ Prefetch file names and UNC paths], Hexacorn blog, October 29, 2012
 
* [http://journeyintoir.blogspot.ch/2012/12/ntosboot-prefetch-file.html NTOSBOOT Prefetch File], by [[Corey Harrell]], December 5, 2012
 
* [http://www.invoke-ir.com/2013/09/whats-new-in-prefetch-for-windows-8.html What's New in the Prefetch for Windows 8??], by Jared Atkinson, September 21, 2013
 
* [http://www.swiftforensics.com/2013/10/windows-prefetch-pf-files.html?m=1 Windows Prefetch (.PF) files], by [[Yogesh Khatri]], October 21, 2013
 
 
== Tools ==
 
 
=== Commercial ===
 
  
=== Free - Non Open Source ===
+
* [http://www.gzip.org/format.txt The gzip file format], by the [http://www.gzip.org/ gzip project]
* [http://www.woanware.co.uk/forensics/prefetchforensics.html PrefetchForensics], PrefetchForensics is an application to extract information from Windows Prefetch files
+
* [http://www.gzip.org/algorithm.txt The gzip compression algorithm], by the [http://www.gzip.org/ gzip project]
* [http://redwolfcomputerforensics.com/index.php?option=com_content&task=view&id=42&Itemid=55 Prefetch-Parser], Parse the prefetch files and display information
+
* [http://tools.ietf.org/html/rfc1952 RFC1952: GZIP file format specification version 4.3], by [[IETF]]
* [http://www.mitec.cz/wfa.html Windows File Analyzer] - Parses Prefetch files, thumbnail databases, shortcuts, index.dat files, and the recycle bin
+
* [http://en.wikipedia.org/wiki/Gzip Wikipedia: gzip]
* [http://www.tzworks.net/prototype_page.php?proto_id=1 Windows Prefetch Parser (pf)], Free tool that can be run on Windows, Linux or Mac OS-X
+
  
=== Open Source ===
+
[[Category:File Formats]]
* [https://code.google.com/p/prefetch-tool/ prefetch-tool], Script to extract information from windows prefetch folder
+

Revision as of 03:05, 28 November 2013

Information icon.png

Please help to improve this article by expanding it.
Further information might be found on the discussion page.

File format

The gzip file (.gz) format consists of:

  • a file header
  • optional headers
    • extra fields
    • original file name
    • comment
    • header checksum
  • a body, containing a DEFLATE-compressed payload
  • a file footer

File header

The file header is 10 bytes in size and contains:

Offset Size Value Description
0 2 0x1f 0x8b Signature (or identification byte 1 and 2)
2 1 Compression Method
3 1 Flags
4 4 Last modification time
Contains a POSIX timestamp.
8 1 Extra flags
9 1 Operating system
Value that indicates on which operating system the gzip file was created.

Compression method

Value Identifier Description
0 - 7 Reserved
8 "deflate" zlib compressed data

Flags

Value Identifier Description
0x01 FTEXT If set the uncompressed data needs to be treated as text instead of binary data.
This flag hints end-of-line conversion for cross-platform text files but does not enforce it.
0x02 FHCRC The file contains a header checksum (CRC-16)
0x04 FEXTRA The file contains extra fields
0x08 FNAME The file contains an original file name string
0x10 FCOMMENT The file contains comment
0x20 Reserved
0x40 Reserved
0x80 Reserved

Note: The FHCRC bit was never set by versions of gzip up to 1.2.4, even though it was documented with a different meaning in gzip 1.2.4.

Extra flags

If compression method is 8 the following extra flags can be defined:

Value Identifier Description
0x02 compressor used maximum compression, slowest algorithm
0x04 compressor used fastest algorithm

Operating System

Value Identifier Description
0 FAT filesystem (MS-DOS, OS/2, NT/Win32)
1 Amiga
2 VMS (or OpenVMS)
3 Unix
4 VM/CMS
5 Atari TOS
6 HPFS filesystem (OS/2, NT)
7 Macintosh
8 Z-System
9 CP/M
10 TOPS-20
11 NTFS filesystem (NT)
12 QDOS
13 Acorn RISCOS
255 unknown

Optional headers

Extra fields

TODO: add description

The extra field are variable of size and contains:

Offset Size Value Description
0 2 Extra field data size
Value in bytes.
2 ... Extra field data

Original file name

This is the original name of the file being compressed, with any directory components removed, and, if the file being compressed is on a file system with case insensitive names, forced to lower case.

Contains an ISO 8859-1 (LATIN-1) string with end-of-string character.

Comment

Contains an ISO 8859-1 (LATIN-1) string with end-of-string character. Line breaks should be denoted by a single line feed character.

Header checksum

The header checksum contain a CRC-16 that consists of the two least significant bytes of the CRC-32 for all bytes of the gzip header up to and not including the CRC-16.

File footer

The file footer is 8 bytes in size and contains:

Offset Size Value Description
0 4 Checksum (CRC-32)
4 4 Uncompressed data size
Value in bytes.

External Links