Difference between pages "Log2timeline" and "SuperFetch"

From ForensicsWiki
(Difference between pages)
Jump to: navigation, search
(Description)
 
(Components)
 
Line 1: Line 1:
==log2timeline==
+
{{Expand}}
  
log2timeline is designed as a framework for artifact timeline creation and analysis. The main purpose is to provide a single tool to parse various log files and artifacts found on suspect systems (and supporting systems, such as network equipment) and produce a body file that can be used to create a timeline, using tools such as mactime from TSK, for forensic investigators.
+
SuperFetch is a performance enhancement introduced in [[Microsoft]] [[Windows|Windows Vista]] to reduce the time necessary to launch applications. SuperFetch works with the memory manager service in Windows to analyze memory usage patterns over time to determine the optimal memory content for a given user for a date or time of day. This differs from the [[Prefetch]] technique used in Microsoft Windows XP, which preloads data into memory without analyzing usage patterns.
  
The tool is written in Perl for Linux but has been tested using Mac OS X (10.5.7 and 10.5.8). Parts of it should work natively in Windows as well (with ActiveState Perl installed).
+
From [http://msdn.microsoft.com/en-us/library/windows/hardware/dn653317(v=vs.85).aspx]: SuperFetch prioritizes the following kinds of pages to remain in memory:
 +
* Pages of applications that are used most frequently overall.
 +
* Pages of applications that are commonly used when resuming:
 +
** After extensive hibernation (for example, first thing in the morning).
 +
** After shorter periods of sleep or hibernation (for example, after lunch).  
  
==Description==
+
If SuperFetch detects that the system drive is a fast SSD (as measured by Windows Experience Index Disk score), then SuperFetch turns off [[ReadyBoot]], [[ReadyBoost]], and the SuperFetch service itself.
log2timeline takes a log file (or a directory) and parses it to produce a body file that can be imported into other tools for timeline analysis. The tool has both a modular based approach to the input file as well as the output file. The default behavior of the current version is to export the timeline in a body format readable by TSK's (The SleuthKit) [http://wiki.sleuthkit.org/index.php?title=Body_file mactime] (although this can be easily changed). log2timeline is build as a series of scripts, this one being the front-end, which uses other scripts to actually parse the log files (called modules). The tool is build to be easily extended for anyone that wants to create a new module.
+
  
The tool contains (current version of 0.51 nightly build (20102608)) three front-ends:
+
== Components ==
* '''log2timeline''' - The main front-end.  A tool capable of parsing a single log file/directory pointed to the tool using a selected input module.
+
=== Robust performance ===
* '''timescanner''' - A recursive front-end capable of parsing a directory passed to the tool and recursively go through each and every file/dir and try to parse it with every or selected input modules (to provide an automatic method of creating a super timeline).
+
Robust performance (or robustness) is a component of SuperFetch to watch for specific file I/O access that might harm system performance by populating the standby lists with unneeded data.
* '''glog2timeline''' - A simple GUI front-end, with similar capabilities as log2timeline (the main front-end)
+
  
==Currently Supported Input Modules==
+
== Scenarios ==
 +
SuperFetch distinguishes between different scenarios to accurately measure performance.
  
The currently supported input modules (as of version 0.51 nightly build (20102608)) are:
+
=== Cold scenario ===
 +
In a cold scenario, the test applications are not already in memory when the test begins. Cold scenarios measure performance either after a state transition, such as boot or resume from hibernation, or after another application claims most of the available memory, such as after launching and quitting a game.  
  
* '''apache2_access''' - Parse the content of a Apache2 access log file
+
=== Warm scenario ===
* '''apache2_error''' - Parse the content of a Apache2 error log file
+
In a warm scenario, some or all the scenario contents are in memory before measurement. This usually means that the test has run at least once during this logon session.
* '''chrome''' - Parse the content of a Chrome history file
+
* '''evt''' - Parse the content of a Windows 2k/XP/2k3 Event Log
+
* '''evtx''' - Parse the content of a Windows Event Log File (EVTX)
+
* '''exif''' - Extract metadata information from files using ExifTool
+
* '''ff_bookmark''' - Parse the content of a Firefox bookmark file
+
* '''firefox2''' - Parse the content of a Firefox 2 browser history
+
* '''firefox3''' - Parse the content of a Firefox 3 history file
+
* '''iehistory''' - Parse the content of an index.dat file containg IE history
+
* '''iis''' - Parse the content of a IIS W3C log file
+
* '''isatxt''' - Parse the content of a ISA text export log file
+
* '''mactime''' - Parse the content of a body file in the mactime format
+
* '''mcafee''' - Parse the content of a log file
+
* '''opera''' - Parse the content of an Opera's global history file
+
* '''oxml''' - Parse the content of an OpenXML document (Office 2007 documents)
+
* '''pcap''' - Parse the content of a PCAP file
+
* '''pdf''' - Parse some of the available PDF document metadata
+
* '''prefetch''' - Parse the content of the Prefetch directory
+
* '''recycler''' - Parse the content of the recycle bin directory
+
* '''restore''' - Parse the content of the restore point directory
+
* '''setupapi''' - Parse the content of the SetupAPI log file in Windows XP
+
* '''sol''' - Parse the content of a .sol (LSO) or a Flash cookie file
+
* '''squid''' - Parse the content of a Squid access log (http_emulate off)
+
* '''syslog''' - Parse the content of a Linux Syslog log file
+
* '''tln''' - Parse the content of a body file in the TLN format
+
* '''userassist''' - Parses the NTUSER.DAT registry file
+
* '''volatility''' - Parse the content of a Volatility output files (psscan2, sockscan2, ...)
+
* '''win_link''' - Parse the content of a Windows shortcut file (or a link file)
+
* '''wmiprov''' - Parse the content of the wmiprov log file
+
* '''xpfirewall''' - Parse the content of a XP Firewall log
+
  
==Links==
+
== Configuration ==
; [http://log2timeline.net log2timeline web site]
+
 
; [http://www.sans.org/reading_room/whitepapers/logging/mastering-super-timeline-log2timeline_33438 SANS GCFA Gold paper about the tool]
+
Because SuperFetch appears to leave a system with no available memory, some users turn it off to create the appearance of having more free memory. The feature can be configured by changing the [[Registry]] value [http://www.codinghorror.com/blog/archives/000688.html]:
; [http://blogs.sans.org/computer-forensics/2010/03/19/digital-forensic-sifting-super-timeline-analysis-and-creation/ A quick run on how to create a super timeline]
+
<pre>
; [http://blog.kiddaland.net/2009/08/log2timeline-artifact-timeline-analysis-part-i/ A blog post introducing the tool]
+
Key: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\Memory Management\PrefetchParameters
; [https://blogs.sans.org/computer-forensics/2009/08/13/artifact-timeline-creation-and-analysis-tool-release-log2timeline/ Part 1 of the SANS Forensic blog post about the tool]
+
Value: EnableSuperfetch
; [https://blogs.sans.org/computer-forensics/2009/08/14/artifact-timeline-creation-and-analysis-part-2/ Part 2 of the SANS forensic blog post about the tool]
+
</pre>
 +
 
 +
A value of zero disables SuperFetch, one enables it for booting only, two for applications, and three for both applications and boot. This setting can also be changed using the Services console, <tt>services.msc</tt> [http://tiredblogger.wordpress.com/2007/03/27/superfetch-not-so-super-for-gaming/].
 +
 
 +
== File Formats ==
 +
 
 +
Data for SuperFetch is gathered by the <tt>%SystemRoot%\System32\Sysmain.dll</tt>, part of the Service Host process, <tt>%SystemRoot%\System32\Svchost.exe</tt>, and stored in a series of files in the <tt>%SystemRoot%\Prefetch</tt> directory [http://www.microsoft.com/technet/technetmag/issues/2007/03/VistaKernel/]. These files appear to start with the prefix <tt>Ag</tt> and have a <tt>.db</tt> extension. Note that there are likely more SuperFetch database files named differently, presumably all using the .db extension.
 +
 
 +
The format of the SuperFetch database files is not fully known, there is available unofficial partial specification [http://blog.rewolf.pl/blog/?p=214] and open source (GPL) dumper for .db files [http://code.google.com/p/rewolf-superfetch-dumper/]. For more information see [[Windows SuperFetch Format|SuperFetch Format]].
 +
 
 +
The SuperFetch feature is seeded with some basic usage patterns when the operating system is installed [http://channel9.msdn.com/showpost.aspx?postid=242429].
 +
 
 +
The SuperFetch service is managed by the File Information FS MiniFilter service. It appears that most of the SuperFetch database files are updated (written) when the service is shut down. AgAppLaunch.db is also written when the service starts.
 +
 
 +
== See Also ==
 +
* [[Prefetch]]
 +
* [[ReadyBoost]]
 +
* [[ReadyBoot]]
 +
* [[Windows SuperFetch Format|SuperFetch Format]]
 +
* [[Windows]]
 +
 
 +
== External Links ==
 +
* [http://technet.microsoft.com/en-us/magazine/2007.03.vistakernel.aspx Inside the Windows Vista Kernel: Part 2], by [[Mark Russinovich]], March 2007
 +
* [http://download.microsoft.com/download/7/E/7/7E7662CF-CBEA-470B-A97E-CE7CE0D98DC2/Win7Perf.docx Performance Testing Guide for Windows], by [[Microsoft]], August 18, 2009 
 +
* [http://en.wikipedia.org/wiki/Windows_Vista_I/O_technologies#SuperFetch Wikipedia: Windows Vista I/O technologies - SuperFetch]
 +
* [http://channel9.msdn.com/showpost.aspx?postid=242429 Channel 9 Interview with Michael Fortin of Microsoft on SuperFetch]
 +
* [http://www.informationweek.com/news/showArticle.jhtml?articleID=196902178 Microsoft Predicts The Future With Vista's SuperFetch] from Information Week
 +
* [http://jessekornblum.com/presentations/dodcc08-2.pdf DC3 Presentation: My You Look SuperFetching], by Jesse Kornblum
 +
 
 +
== Tools ==
 +
=== Open Source ===
 +
* [https://code.google.com/p/rewolf-superfetch-dumper/ rewolf-superfetch-dumper]
 +
 
 +
[[Category:Windows]]

Revision as of 07:10, 28 June 2014

Information icon.png

Please help to improve this article by expanding it.
Further information might be found on the discussion page.

SuperFetch is a performance enhancement introduced in Microsoft Windows Vista to reduce the time necessary to launch applications. SuperFetch works with the memory manager service in Windows to analyze memory usage patterns over time to determine the optimal memory content for a given user for a date or time of day. This differs from the Prefetch technique used in Microsoft Windows XP, which preloads data into memory without analyzing usage patterns.

From [1]: SuperFetch prioritizes the following kinds of pages to remain in memory:

  • Pages of applications that are used most frequently overall.
  • Pages of applications that are commonly used when resuming:
    • After extensive hibernation (for example, first thing in the morning).
    • After shorter periods of sleep or hibernation (for example, after lunch).

If SuperFetch detects that the system drive is a fast SSD (as measured by Windows Experience Index Disk score), then SuperFetch turns off ReadyBoot, ReadyBoost, and the SuperFetch service itself.

Components

Robust performance

Robust performance (or robustness) is a component of SuperFetch to watch for specific file I/O access that might harm system performance by populating the standby lists with unneeded data.

Scenarios

SuperFetch distinguishes between different scenarios to accurately measure performance.

Cold scenario

In a cold scenario, the test applications are not already in memory when the test begins. Cold scenarios measure performance either after a state transition, such as boot or resume from hibernation, or after another application claims most of the available memory, such as after launching and quitting a game.

Warm scenario

In a warm scenario, some or all the scenario contents are in memory before measurement. This usually means that the test has run at least once during this logon session.

Configuration

Because SuperFetch appears to leave a system with no available memory, some users turn it off to create the appearance of having more free memory. The feature can be configured by changing the Registry value [2]:

Key: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\Memory Management\PrefetchParameters
Value: EnableSuperfetch

A value of zero disables SuperFetch, one enables it for booting only, two for applications, and three for both applications and boot. This setting can also be changed using the Services console, services.msc [3].

File Formats

Data for SuperFetch is gathered by the %SystemRoot%\System32\Sysmain.dll, part of the Service Host process, %SystemRoot%\System32\Svchost.exe, and stored in a series of files in the %SystemRoot%\Prefetch directory [4]. These files appear to start with the prefix Ag and have a .db extension. Note that there are likely more SuperFetch database files named differently, presumably all using the .db extension.

The format of the SuperFetch database files is not fully known, there is available unofficial partial specification [5] and open source (GPL) dumper for .db files [6]. For more information see SuperFetch Format.

The SuperFetch feature is seeded with some basic usage patterns when the operating system is installed [7].

The SuperFetch service is managed by the File Information FS MiniFilter service. It appears that most of the SuperFetch database files are updated (written) when the service is shut down. AgAppLaunch.db is also written when the service starts.

See Also

External Links

Tools

Open Source