Difference between pages "Windows Prefetch File Format" and "SuperFetch"

From ForensicsWiki
(Difference between pages)
Jump to: navigation, search
 
 
Line 1: Line 1:
{{expand}}
+
{{Expand}}
  
A Windows Prefetch file consists of one file header and multiple file sections with different content. Not all content has an obvious forensic value.
+
SuperFetch is a performance enhancement introduced in [[Microsoft]] [[Windows|Windows Vista]] to reduce the time necessary to launch applications. SuperFetch works with the memory manager service in Windows to analyze memory usage patterns over time to determine the optimal memory content for a given user for a date or time of day. This differs from the [[Prefetch]] technique used in Microsoft Windows XP, which preloads data into memory without analyzing usage patterns.
  
As far as have been possible to ascertain, there is no public description of the format. The description below has been synthesised from examination
+
From [http://msdn.microsoft.com/en-us/library/windows/hardware/dn653317(v=vs.85).aspx]: SuperFetch prioritizes the following kinds of pages to remain in memory:
of multiple prefetch files.
+
* Pages of applications that are used most frequently overall.
 +
* Pages of applications that are commonly used when resuming:
 +
** After extensive hibernation (for example, first thing in the morning).
 +
** After shorter periods of sleep or hibernation (for example, after lunch).  
  
== Characteristics ==
+
If SuperFetch detects that the system drive is a fast solid-state drive (SSD) (as measured by Windows Experience Index Disk score), then SuperFetch turns off [[ReadyBoot]].
Integer values are stored in little-endian.
+
  
Strings are stored as [http://en.wikipedia.org/wiki/UTF-16/UCS-2 UTF-16 little-endian] without a byte-order-mark (BOM).
 
  
Timestamps are stored as [http://msdn2.microsoft.com/en-us/library/ms724284.aspx Windows FILETIME] in UTC.
+
== Configuration ==
  
== File header ==
+
Because SuperFetch appears to leave a system with no available memory, some users turn it off to create the appearance of having more free memory. The feature can be configured by changing the <tt>HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\Memory Management\PrefetchParameters\EnableSuperfetch</tt> [[Registry]] key [http://www.codinghorror.com/blog/archives/000688.html]. A value of zero disables SuperFetch, one enables it for booting only, two for applications, and three for both applications and boot. This setting can also be changed using the Services console, <tt>services.msc</tt> [http://tiredblogger.wordpress.com/2007/03/27/superfetch-not-so-super-for-gaming/].
  
{| class="wikitable"
+
== File Formats ==
|-
+
! Field
+
! Offset
+
! Length
+
! Type
+
! Notes
+
|-
+
| H1
+
| 0x0000
+
| 4
+
| DWORD
+
| Format version (see format version section below)
+
|-
+
| H2
+
| 0x0004
+
| 4
+
| DWORD
+
| Signature 'SCCA' (or in hexadecimal representation 0x53 0x43 0x43 0x4)
+
|-
+
| H3
+
| 0x0008
+
| 4
+
| DWORD?
+
| Unknown - Values observed: 0x0F - Windows XP, 0x11 - Windows 7, Windows 8.1
+
|-
+
| H4
+
| 0x000C
+
| 4
+
| DWORD
+
| Prefetch file size (or length) (sometimes referred to as End of File (EOF)).
+
|-
+
| H5
+
|0x0010
+
| 60
+
| USTR
+
| The name of the (original) executable as a Unicode (UTF-16 litte-endian string), up to 29 characters and terminated by an end-of-string character (U+0000). This name should correspond with the one in the prefetch file filename.
+
|-
+
| H6
+
|0x004C
+
|4
+
|DWORD
+
|The prefetch hash. This hash value should correspond with the one in the prefetch file filename.
+
|-
+
| H7
+
|0x0050
+
|4
+
|?
+
| Unknown (flags)? Values observed: 0 for almost all prefetch files (XP); 1 for NTOSBOOT-B00DFAAD.pf (XP)
+
|-
+
|}
+
  
=== Format version ===
+
Data for SuperFetch is gathered by the <tt>%SystemRoot%\System32\Sysmain.dll</tt>, part of the Service Host process, <tt>%SystemRoot%\System32\Svchost.exe</tt>, and stored in a series of files in the <tt>%SystemRoot%\Prefetch</tt> directory [http://www.microsoft.com/technet/technetmag/issues/2007/03/VistaKernel/]. These files appear to start with the prefix <tt>Ag</tt> and have a <tt>.db</tt> extension. The format of these files is not fully known, there is available unofficial partial specification [http://blog.rewolf.pl/blog/?p=214] and open source (GPL) dumper for .db files [http://code.google.com/p/rewolf-superfetch-dumper/]. Some information can be gleaned from these files by searching for [[Unicode]] [[strings]] in them.
  
{| class="wikitable"
+
The SuperFetch feature is seeded with some basic usage patterns when the operating system is installed [http://channel9.msdn.com/showpost.aspx?postid=242429].
|-
+
! Value
+
! Windows version
+
|-
+
| 17 (0x11)
+
| Windows XP, Windows 2003
+
|-
+
| 23 (0x17)
+
| Windows Vista, Windows 7
+
|-
+
| 26 (0x1a)
+
| Windows 8.1 (note this could be Windows 8 as well but has not been confirmed)
+
|-
+
|}
+
 
+
=== File information - version 17 ===
+
 
+
The following part of the file header is version dependent. It is sometime considered part of the file header. Below the structure for format version 17.
+
 
+
{| class="wikitable"
+
|-
+
! Field
+
! Offset
+
! Length
+
! Type
+
! Notes
+
|-
+
| H8
+
| 0x0054
+
| 4
+
| DWORD
+
| The offset to section A. The offset is relative from the start of the file.
+
|-
+
| H9
+
| 0x0058
+
| 4
+
| DWORD
+
| The number of entries in section A.
+
|-
+
| H10
+
| 0x005C
+
| 4
+
| DWORD
+
| The offset to section B. The offset is relative from the start of the file.
+
|-
+
| H11
+
| 0x0060
+
| 4
+
| DWORD
+
| The number of entries in section B.
+
|-
+
| H12
+
| 0x0064
+
| 4
+
| DWORD
+
| The offset to section C. The offset is relative from the start of the file.
+
|-
+
| H13
+
| 0x0068
+
| 4
+
| DWORD
+
| Length of section C.
+
|-
+
| H14
+
| 0x006C
+
| 4
+
| DWORD
+
| Offset to section D. The offset is relative from the start of the file.
+
|-
+
| H15
+
| 0x0070
+
| 4
+
| DWORD
+
| The number of entries in section D.
+
|-
+
| H16
+
| 0x0074
+
| 4
+
| DWORD
+
| Unknown ? (Previously opted: Length of section D)
+
|-
+
| H17
+
| 0x0078
+
| 8
+
| FTIME
+
| Latest execution time (or run time) of executable (FILETIME)
+
|-
+
| H18
+
| 0x0080
+
| 16
+
| ?
+
| Unknown ? Possibly structured as 4 DWORD. Observed values: /0x00000000 0x00000000 0x00000000 0x00000000/, /0x47868c00 0x00000000 0x47860c00 0x00000000/ (don't exclude the possibility here that this is remnant data)
+
|-
+
| H19
+
| 0x0090
+
| 4
+
| DWORD
+
| Execution counter (or run count)
+
|-
+
| H20
+
| 0x0094
+
| 4
+
| DWORD?
+
| Unknown ? Observed values: 1, 2, 3, 4, 5, 6 (XP)
+
|-
+
|}
+
 
+
It's worth noting that the name of a carved prefetch file can be restored using the information in field H5 and H6, and its size can be determined by field H4.
+
 
+
== Section A ==
+
This section contains an array with 20 byte (version 17) or 32 byte (version 23 and 26) entry records.
+
 
+
The actual format and usage of these entry records is currently not known.
+
 
+
== Section B ==
+
This section contains an array with 12 byte (version 17, 23 and 26) entry records.
+
 
+
The actual format and usage of these entry records is currently not known.
+
 
+
== Section C ==
+
This section contains an array of UTF-16 little-endian formatted strings with end-of-string characters (U+0000).
+
 
+
At the end of the section there seems to be alignment padding that can contain remnant values.
+
 
+
== Section D - Volume information (block) ==
+
 
+
Section D contains one or more subsections. The number is (most likely) determined by the DWORD at file offset 0x0070. Each subsection refers to directories on an identified volume.
+
 
+
In this section, all offsets are assumed to be counted from the start of the D section.
+
 
+
=== Volume information - version 17 ===
+
The following values are version dependent. Below the structure for format version 17.
+
 
+
{| class="wikitable"
+
|-
+
! Field
+
! Offset
+
! Length
+
! Type
+
! Notes
+
|-
+
| DH1
+
| +0x0000
+
| 4
+
| DWORD
+
| Offset to volume string (Unicode, terminated by U+0000)
+
|-
+
| DH2
+
| +0x0004
+
| 4
+
| DWORD
+
| Length of volume string (nr of characters, including terminating U+0000)
+
|-
+
| DH3
+
| +0x0008
+
| 8
+
| FILETIME
+
| Volume creation time.
+
|-
+
| DH4
+
| +0x0010
+
| 4
+
| DWORD
+
| Volume serial number of volume indicated by volume string
+
|-
+
| DH5
+
| +0x0014
+
| 4
+
| DWORD
+
| ? Offset to section DHS1
+
|-
+
| DH6
+
| +0x0018
+
| 4
+
| DWORD
+
| ? Length of section DHS1 (in bytes)
+
|-
+
| DH7
+
| +0x001C
+
| 4
+
| DWORD
+
| ? Offset to section DHS2
+
|-
+
| DH8
+
| +0x0020
+
| 4
+
| DWORD
+
| ? Nr of strings in section DHS2
+
|-
+
| ?
+
| +0x0024
+
| ?
+
| ?
+
| ? additional 28 bytes (includes one timestamp?)
+
|}
+
 
+
If all the executables and libraries referenced in the C section are from one single disk volume, there will be only one section in the D section. If multiple volumes are referenced by section C, section D will contain multiple sections.  (A simple way to force this situation is to copy, say, NOTEPAD.EXE to a USB drive, and start it from that volume. The corresponding prefetch file will have one D header referring to, e.g. \DEVICE\HARDDISK1\DP(1)0-0+4 (the USB drive), and one to, e.g. \DEVICE\HARDDISKVOLUME1\ (where the .DLLs and other support files were found).
+
  
 
== See Also ==
 
== See Also ==
 
* [[Prefetch]]
 
* [[Prefetch]]
 +
* [[ReadyBoost]]
 +
* [[ReadyBoot]]
 +
* [[Windows SuperFetch Format|SuperFetch Format]]
 +
* [[Windows]]
  
 
== External Links ==
 
== External Links ==
* [https://googledrive.com/host/0B3fBvzttpiiSbl9XZGZzQ05hZkU/Windows%20Prefetch%20File%20(PF)%20format.pdf Windows Prefetch File (PF) format], by the [[libssca|libssca project]]
+
* [http://en.wikipedia.org/wiki/Windows_Vista_I/O_technologies#SuperFetch Wikipedia: Windows Vista I/O technologies - SuperFetch]
 +
* [http://channel9.msdn.com/showpost.aspx?postid=242429 Channel 9 Interview with Michael Fortin of Microsoft on SuperFetch]
 +
* [http://www.informationweek.com/news/showArticle.jhtml?articleID=196902178 Microsoft Predicts The Future With Vista's SuperFetch] from Information Week
 +
* [http://jessekornblum.com/presentations/dodcc08-2.pdf DC3 Presentation: My You Look SuperFetching], by Jesse Kornblum
 +
 
 +
== Tools ==
 +
=== Open Source ===
 +
* [https://code.google.com/p/rewolf-superfetch-dumper/ rewolf-superfetch-dumper]
  
[[Category:File Formats]]
+
[[Category:Windows]]

Revision as of 01:48, 24 April 2014

Information icon.png

Please help to improve this article by expanding it.
Further information might be found on the discussion page.

SuperFetch is a performance enhancement introduced in Microsoft Windows Vista to reduce the time necessary to launch applications. SuperFetch works with the memory manager service in Windows to analyze memory usage patterns over time to determine the optimal memory content for a given user for a date or time of day. This differs from the Prefetch technique used in Microsoft Windows XP, which preloads data into memory without analyzing usage patterns.

From [1]: SuperFetch prioritizes the following kinds of pages to remain in memory:

  • Pages of applications that are used most frequently overall.
  • Pages of applications that are commonly used when resuming:
    • After extensive hibernation (for example, first thing in the morning).
    • After shorter periods of sleep or hibernation (for example, after lunch).

If SuperFetch detects that the system drive is a fast solid-state drive (SSD) (as measured by Windows Experience Index Disk score), then SuperFetch turns off ReadyBoot.


Configuration

Because SuperFetch appears to leave a system with no available memory, some users turn it off to create the appearance of having more free memory. The feature can be configured by changing the HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\Memory Management\PrefetchParameters\EnableSuperfetch Registry key [2]. A value of zero disables SuperFetch, one enables it for booting only, two for applications, and three for both applications and boot. This setting can also be changed using the Services console, services.msc [3].

File Formats

Data for SuperFetch is gathered by the %SystemRoot%\System32\Sysmain.dll, part of the Service Host process, %SystemRoot%\System32\Svchost.exe, and stored in a series of files in the %SystemRoot%\Prefetch directory [4]. These files appear to start with the prefix Ag and have a .db extension. The format of these files is not fully known, there is available unofficial partial specification [5] and open source (GPL) dumper for .db files [6]. Some information can be gleaned from these files by searching for Unicode strings in them.

The SuperFetch feature is seeded with some basic usage patterns when the operating system is installed [7].

See Also

External Links

Tools

Open Source