Difference between pages "SuperFetch" and "Executable"

From ForensicsWiki
(Difference between pages)
Jump to: navigation, search
(Scenarios)
 
 
Line 1: Line 1:
{{Expand}}
+
{{expand}}
  
SuperFetch is a performance enhancement introduced in [[Microsoft]] [[Windows|Windows Vista]] to reduce the time necessary to launch applications. SuperFetch works with the memory manager service in Windows to analyze memory usage patterns over time to determine the optimal memory content for a given user for a date or time of day. This differs from the [[Prefetch]] technique used in Microsoft Windows XP, which preloads data into memory without analyzing usage patterns.
+
An executable file is used to perform tasks according to encoded instructions. Executable files are sometimes also referred to as binaries which technically can be considered a sub class of executable files.
  
From [http://msdn.microsoft.com/en-us/library/windows/hardware/dn653317(v=vs.85).aspx]: SuperFetch prioritizes the following kinds of pages to remain in memory:
+
There are multiple families of executable files:
* Pages of applications that are used most frequently overall.  
+
* Scripts; e.g. shell scripts, batch scripts (.bat)
* Pages of applications that are commonly used when resuming:  
+
* DOS, Windows executable files (.exe) which can be of various formats like: MZ, PE/COFF, NE
** After extensive hibernation (for example, first thing in the morning).
+
** EFI fat binary; roughly a 48-byte header and 2x MZ-PE/COFF
** After shorter periods of sleep or hibernation (for example, after lunch).
+
* ELF
 +
* Mach-O
  
Prefetched pages are added to the system’s standby page list, which has been reorganized and redesigned to retain useful data in memory over longer periods of time.
+
== External Links ==
 +
* [http://en.wikipedia.org/wiki/Executable Wikipedia: Executable]
 +
* [ftp://ftp.cs.wisc.edu/paradyn/papers/Rosenblum10prov.pdf Extracting Compiler Provenance from Program Binaries], by Nathan E. Rosenblum, Barton P. Miller, Xiaojin Zhu, June 2010
 +
* [http://duartes.org/gustavo/blog/post/journey-to-the-stack/ Journey to the Stack, Part I], by Gustavo Duarte, March 10, 2014
  
If SuperFetch detects that the system drive is a fast SSD (as measured by Windows Experience Index Disk score), then SuperFetch turns off [[ReadyBoot]], [[ReadyBoost]], and the SuperFetch service itself.
+
=== ELF ===
 +
* [http://robinhoksbergen.com/papers/howto_elf.html Manually Creating an ELF Executable], by Robin Hoksbergen
  
To calculate the Windows Experience Index Disk score run:
+
=== MZ, PE/COFF ===
<pre>
+
* [http://en.wikipedia.org/wiki/Portable_Executable Wikipedia: Portable Executable]
winsat formal
+
* [http://msdn.microsoft.com/en-us/windows/hardware/gg463119.aspx Microsoft PE and COFF Specification]
</pre>
+
* [http://msdn.microsoft.com/en-us/magazine/ms809762.aspx Peering Inside the PE: A Tour of the Win32 Portable Executable File Format], by Matt Pietrek, March 1994
 +
* [http://www.microsoft.com/msj/0797/hood0797.aspx Under the Hood], by Matt Pietrek, July 1997
 +
* [http://msdn.microsoft.com/en-us/magazine/cc301805.aspx An In-Depth Look into the Win32 Portable Executable File Format], by Matt Pietrek, February 2002
 +
* [https://googledrive.com/host/0B3fBvzttpiiSd1dKQVU0WGVESlU/Executable%20(EXE)%20file%20format.pdf MZ, PE-COFF executable file format (EXE)], by the [[libexe|libexe project]], October 2011
 +
* [http://ho.ax/posts/2012/02/carving-up-efi-fat-binaries/ Carving up EFI fat binaries], by snare, February 24, 2012
 +
* [http://seclists.org/fulldisclosure/2013/Oct/157 The Internal of Reloc .text], Full Disclosure Mailing list, October 21, 2013
  
== Components ==
+
=== DBG, PDB ===
=== Robust performance ===
+
* [http://en.wikipedia.org/wiki/Program_database Wikipedia: Program database]
Robust performance (or robustness) is a component of SuperFetch to watch for specific file I/O access that might harm system performance by populating the standby lists with unneeded data.
+
* [http://www.debuginfo.com/articles/debuginfomatch.html Matching Debug Information], by debuginfo.com
 +
* [http://support.microsoft.com/kb/121366 Description of the .PDB files and of the .DBG files], by [[Microsoft]]
 +
* [http://msdn.microsoft.com/en-us/library/ff553493(v=vs.85).aspx Public and Private Symbols], by [[Microsoft]]
 +
* [http://msdn.microsoft.com/en-us/library/windows/desktop/ms679293(v=vs.85).aspx DbgHelp Structures], by [[Microsoft]]
 +
* [http://web.archive.org/web/20070915060650/http://www.x86.org/ftp/manuals/tools/sym.pdf Internet Archive: Microsoft Symbol and Type Information], by [[Microsoft]]
 +
* [http://pierrelib.pagesperso-orange.fr/exec_formats/MS_Symbol_Type_v1.0.pdf Microsoft Symbol and Type Information]
 +
* [https://code.google.com/p/pdbparse/wiki/StreamDescriptions Stream Descriptions], [https://code.google.com/p/pdbparse/ pdbparse project]
 +
* [http://sourceforge.net/p/mingw-w64/code/HEAD/tree/experimental/tools/libmsdebug/ libmsdebug], by the [[MinGW|MinGW project]]
 +
* [http://moyix.blogspot.com/2007/10/types-stream.html The Types Stream], by [[Brendan Dolan-Gavitt]], October 4, 2007
  
== Scenarios ==
+
=== Minidump ===
SuperFetch distinguishes between different scenarios to accurately measure performance.
+
* [http://msdn.microsoft.com/en-us/library/windows/desktop/ms680378(v=vs.85).aspx MSDN: MINIDUMP_HEADER structure]
 +
* [https://code.google.com/p/google-breakpad/source/browse/trunk/src/google_breakpad/common/minidump_format.h minidump_format.h], by [[Google]], 2006
 +
* [http://moyix.blogspot.com/2008/05/parsing-windows-minidumps.html Parsing Windows Minidumps], by [[Brendan Dolan-Gavitt]], May 7, 2008
 +
* [http://web.archive.org/web/20110814041817/http://www.stackhash.com/blog/post/Format-of-a-minidump-(mdmp)-file.aspx Format of a minidump (mdmp) file], Internet Archive: StackHash blog, May 16, 2011
  
=== Cold scenario ===
+
=== Mach-O ===
In a cold scenario, the test applications are not already in memory when the test begins. Cold scenarios measure performance either after a state transition, such as boot or resume from hibernation, or after another application claims most of the available memory, such as after launching and quitting a game.  
+
* [http://en.wikipedia.org/wiki/Mach-O Wikipedia: Mach-O]
  
=== Warm scenario ===
+
=== Packers ===
In a warm scenario, some or all the scenario contents are in memory before measurement. This usually means that the test has run at least once during this logon session.
+
* [http://www.woodmann.com/crackz/Packers.htm Packers & Unpackers]
  
=== Performance scenarios ===
+
== Tools ==
Performance scenarios defined by the Windows Performance Recorder (WPR):
+
* General: Records general performance while the computer is running.
+
* On/Off - Boot: Records performance while the computer is booting.
+
* On/Off – Fast Startup: Records performance during a fast startup.
+
* On/Off - Shutdown: Records performance while shutting the computer down.
+
* On/Off - RebootCycle: Records performance during the entire cycle while the computer is rebooting.
+
* On/Off - Standby/Resume: Records performance when the computer is placed on standby and then resumed.
+
* On/Off - Hibernate/Resume: Records performance when the computer is placed in hibernation and then resumed.
+
  
Where "On/Off" likely refers to a cold scenario.
+
=== MZ, PE/COFF ===
 +
* [https://code.google.com/p/pefile/ pefile], multi-platform Python module to read and work with Portable Executable (aka PE) files
  
== Configuration ==
+
=== PDB ===
 
+
* [https://code.google.com/p/pdbparse/ pdbparse], Open-source parser for Microsoft debug symbols (PDB files)
Because SuperFetch appears to leave a system with no available memory, some users turn it off to create the appearance of having more free memory. The feature can be configured by changing the [[Registry]] value [http://www.codinghorror.com/blog/archives/000688.html]:
+
<pre>
+
Key: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\Memory Management\PrefetchParameters
+
Value: EnableSuperfetch
+
</pre>
+
 
+
A value of zero disables SuperFetch, one enables it for booting only, two for applications, and three for both applications and boot. This setting can also be changed using the Services console, <tt>services.msc</tt> [http://tiredblogger.wordpress.com/2007/03/27/superfetch-not-so-super-for-gaming/].
+
 
+
== File Formats ==
+
 
+
Data for SuperFetch is gathered by the <tt>%SystemRoot%\System32\Sysmain.dll</tt>, part of the Service Host process, <tt>%SystemRoot%\System32\Svchost.exe</tt>, and stored in a series of files in the <tt>%SystemRoot%\Prefetch</tt> directory [http://www.microsoft.com/technet/technetmag/issues/2007/03/VistaKernel/]. These files appear to start with the prefix <tt>Ag</tt> and have a <tt>.db</tt> extension. Note that there are likely more SuperFetch database files named differently, presumably all using the .db extension.
+
 
+
The format of the SuperFetch database files is not fully known, there is available unofficial partial specification [http://blog.rewolf.pl/blog/?p=214] and open source (GPL) dumper for .db files [http://code.google.com/p/rewolf-superfetch-dumper/]. For more information see [[Windows SuperFetch Format|SuperFetch Format]].
+
 
+
The SuperFetch feature is seeded with some basic usage patterns when the operating system is installed [http://channel9.msdn.com/showpost.aspx?postid=242429].
+
 
+
The SuperFetch service is managed by the File Information FS MiniFilter service. It appears that most of the SuperFetch database files are updated (written) when the service is shut down. AgAppLaunch.db is also written when the service starts.
+
 
+
== See Also ==
+
* [[Prefetch]]
+
* [[ReadyBoost]]
+
* [[ReadyBoot]]
+
* [[Windows SuperFetch Format|SuperFetch Format]]
+
* [[Windows]]
+
 
+
== External Links ==
+
* [http://technet.microsoft.com/en-us/magazine/2007.03.vistakernel.aspx Inside the Windows Vista Kernel: Part 2], by [[Mark Russinovich]], March 2007
+
* [http://download.microsoft.com/download/7/E/7/7E7662CF-CBEA-470B-A97E-CE7CE0D98DC2/Win7Perf.docx Performance Testing Guide for Windows], by [[Microsoft]], August 18, 2009 
+
* [http://msdn.microsoft.com/en-us/library/windows/hardware/hh162965.aspx Performance Scenarios], by [[Microsoft]], October 20, 2013
+
* [http://en.wikipedia.org/wiki/Windows_Vista_I/O_technologies#SuperFetch Wikipedia: Windows Vista I/O technologies - SuperFetch]
+
* [http://channel9.msdn.com/showpost.aspx?postid=242429 Channel 9 Interview with Michael Fortin of Microsoft on SuperFetch]
+
* [http://www.informationweek.com/news/showArticle.jhtml?articleID=196902178 Microsoft Predicts The Future With Vista's SuperFetch] from Information Week
+
* [http://jessekornblum.com/presentations/dodcc08-2.pdf DC3 Presentation: My You Look SuperFetching], by Jesse Kornblum
+
 
+
== Tools ==
+
=== Open Source ===
+
* [https://code.google.com/p/rewolf-superfetch-dumper/ rewolf-superfetch-dumper]
+
  
[[Category:Windows]]
+
=== Minidump ===
 +
* [http://support.microsoft.com/kb/315271 Dumpchk.exe], by [[Microsoft]]
 +
* [http://amnesia.gtisc.gatech.edu/~moyix/minidump.py minidump.py], by [[Brendan Dolan-Gavitt]]

Revision as of 11:20, 28 June 2014

Information icon.png

Please help to improve this article by expanding it.
Further information might be found on the discussion page.

An executable file is used to perform tasks according to encoded instructions. Executable files are sometimes also referred to as binaries which technically can be considered a sub class of executable files.

There are multiple families of executable files:

  • Scripts; e.g. shell scripts, batch scripts (.bat)
  • DOS, Windows executable files (.exe) which can be of various formats like: MZ, PE/COFF, NE
    • EFI fat binary; roughly a 48-byte header and 2x MZ-PE/COFF
  • ELF
  • Mach-O

External Links

ELF

MZ, PE/COFF

DBG, PDB

Minidump

Mach-O

Packers

Tools

MZ, PE/COFF

  • pefile, multi-platform Python module to read and work with Portable Executable (aka PE) files

PDB

  • pdbparse, Open-source parser for Microsoft debug symbols (PDB files)

Minidump