Difference between pages "Prefetch" and "Forensic Live CD issues"

From ForensicsWiki
(Difference between pages)
Jump to: navigation, search
(File format)
 
(Problems)
 
Line 1: Line 1:
{{Expand}}
+
== The problem ==
Windows Prefetch files, introduced in [[Windows|Windows XP]], are designed to speed up the application startup process. Prefetch files contain the name of the executable, a Unicode list of DLLs used by that executable, a count of how many times the executable has been run, and a timestamp indicating the last time the program was run. Although Prefetch is present in Windows 2003, by default it is only enabled for boot prefetching. The feature is also found in [[Windows|Windows Vista]], where it has been augmented with [[SuperFetch]], [[ReadyBoot]], and [[ReadyBoost]]. For SSD drives Prefetch is disabled by default [http://blogs.msdn.com/b/e7/archive/2009/05/05/support-and-q-a-for-solid-state-drives-and.aspx].
+
  
Up to 128 Prefetch files are stored in the <tt>%SystemRoot%\Prefetch</tt> directory [http://blogs.msdn.com/ryanmy/archive/2005/05/25/421882.aspx]. Each file in that directory should contain the name of the application, a dash, and then an eight character hash of the location from which that application was run, and a <tt>.pf</tt> extension. The filenames should be all uppercase except for the extension. The format of hashes is not known. A sample filename for [[md5deep]] would look like: <tt>MD5DEEP.EXE-4F89AB0C.pf</tt>. If an application is run from two different locations on the drive (i.e. the user runs <tt>C:\md5deep.exe</tt> and then <tt>C:\Apps\Hashing\md5deep.exe</tt>), there will be two different prefetch files in the Prefetch folder.
+
[[Live CD|Forensic Live CDs]] are widely used during computer forensic investigations. Currently, many vendors of such Live CD distributions spread false claims that their distributions "do not touch anything", "write protect everything" and so on. Unfortunately, community-developed distributions are no exception here. Finally, it turns out that many Linux-based forensic Live CDs are not tested properly and there are no suitable test cases published.
  
 +
== Another side of the problem ==
  
== File format ==
+
Another side of the problem of insufficient testing of forensic Live CDs is that many users do not know what happens "under the hood" of the provided operating system and cannot adequately test them.
Each Prefetch file has a 4-byte signature (at offset 4) "SCCA" (or in hexadecimal notation 0x53 0x43 0x43 0x41). The signature is assumed to be preceded by a 4-byte format version indicator:
+
* 0x00000011 for [[Windows XP]] and [[Windows 2003]]
+
* 0x00000017 for [[Windows Vista]] and [[Windows 7]]
+
* 0x0000001a for [[Windows 8|Windows 8.1]]
+
  
For more information about the file format see: [[Windows Prefetch File Format]]
+
=== Example ===
  
== Timestamps ==
+
For example, [http://forensiccop.blogspot.com/2009/10/forensic-cop-journal-13-2009.html ''Forensic Cop Journal'' (Volume 1(3), Oct 2009)] describes a test case when an Ext3 file system was mounted using "-o ro" mount flag as a way to write protect the data. The article says that all tests were successful (i.e. no data modification was found after unmounting the file system), but it is known that damaged (i.e not properly unmounted) Ext3 file systems cannot be write protected using only "-o ro" mount flags (write access will be enabled during file system recovery).
  
Both the [[NTFS]] timestamps for a Prefetch file and the timestamp embedded in each Prefetch file contain valuable information. The timestamp embedded within the Prefetch file is a 64-bit (QWORD) [http://msdn2.microsoft.com/en-us/library/ms724284.aspx FILETIME] object The creation date of the file indicates the first time the application was executed. Both the modification date of the file and the embedded timestamp indicate the last time the application was executed.
+
And the question is: will many users test damaged Ext3 file system (together with testing the clean one) when validating their favourite forensic Live CD distribution? My answer is "no", because many users are unaware of such traits.
  
Windows will store timestamps according to Windows [http://msdn.microsoft.com/en-us/library/ms724290%28VS.85%29.aspx epoch].
+
== Problems ==
  
==== Creation Time ====
+
Each problem is followed by a list of distributions affected (currently this list is not up-to-date).
The creation time does not have a static offset on any Windows platform. The location of the creation time can be found using the offset 0x8 + length of Volume path offset. See section Volume for more information.
+
  
==== Last Run Time ====
+
=== Journaling file system updates ===
A timestamp of when the application was last ran is embedded into the Prefetch file.
+
The offset from the beginning of the file to the "Last Run Time" is located:
+
* at offset 0x78 on Windows XP and Windows 2003.
+
* at offset 0x80 on Windows Vista and Windows 7.
+
  
== MetaData ==
+
When mounting (and unmounting) several journaling file systems with only "-o ro" mount flag a different number of data writes may occur. Here is a list of such file systems:
==== Header ====
+
In each Prefetch file, the size of the header is stored and can be found at offset 0x54 on [[Windows|Windows XP]], [[Windows|Windows Vista]], and [[Windows|Windows 7]]. The header size for [[Windows|Windows XP]] is 0x98 (152) and 0xf0 (240) on Windows Vista and Windows 7.
+
  
The Prefetch file will embed the application's name into the header at offset 0x10.
+
{| class="wikitable" border="1"
 +
|-
 +
!  File system
 +
!  When data writes happen
 +
!  Notes
 +
|-
 +
|  Ext3
 +
|  File system requires journal recovery
 +
|  To disable recovery: use "noload" flag, or use "ro,loop" flags, or use "ext2" file system type
 +
|-
 +
|  Ext4
 +
|  File system requires journal recovery
 +
|  To disable recovery: use "noload" flag, or use "ro,loop" flags, or use "ext2" file system type
 +
|-
 +
|  ReiserFS
 +
|  File system has unfinished transactions
 +
|  "nolog" flag does not work (see ''man mount''). To disable journal updates: use "ro,loop" flags
 +
|-
 +
|  XFS
 +
|  Always (when unmounting)
 +
|  "norecovery" flag does not help (fixed in recent 2.6 kernels). To disable data writes: use "ro,loop" flags.
 +
|}
  
==== Run Count ====
+
Incorrect mount flags can be used to mount file systems on evidentiary media during the boot process or during the file system preview process. As described above, this may result in data writes to evidentiary media. For example, several Ubuntu-based forensic Live CD distributions mount and recover damaged Ext3/4 file systems on fixed media (e.g. hard drives) during execution of [http://en.wikipedia.org/wiki/Initrd ''initrd''] scripts (these scripts mount every supported file system type on every supported media type using only "-o ro" flag in order to find a root file system image).
The run count, or number of times the application has been run, is a 4-byte (DWORD) value located at offset 0x90 from the beginning of the file on [[Windows|Windows XP]]. On [[Windows|Windows Vista]] and [[Windows|Windows 7]], the run time can be found at 0x98.
+
  
==== Volume ====
+
[[Image:ext3 recovery.png|thumb|right|[[Helix3]]: damaged Ext3 recovery during the boot]]
Volume related information, volume path and volume serial number, are embedded into the Prefetch file. The precise offset for this information is the same for each Prefetch file and Windows operating system. In the header at offset 0x6c, the location of the volume path is stored. The location is a 4-bytes (DWORD) value.
+
  
At the location given from offset 0x6c, a 4-byte value is stored which is the number of bytes from current offset (location from offset 0x6c) to the beginning of the volume path string. The location from the offset 0x6c, for ease of reading, will be called the "volume path offset." The volume path is embedded as an [http://en.wikipedia.org/wiki/UTF-16/UCS-2 UTF-16] encoded string.
+
List of distributions that recover Ext3 (and sometimes Ext4) file systems during the boot:
  
The length of the volume path string is a 4-byte value is located at volume path offset + 0x4.
+
{| class="wikitable" border="1"
 +
|-
 +
!  Distribution
 +
!  Version
 +
|-
 +
|  Helix3
 +
|  2009R1
 +
|-
 +
|  SMART Linux (Ubuntu)
 +
|  2010-01-20
 +
|-
 +
|  FCCU GNU/Linux Forensic Boot CD
 +
|  12.1
 +
|-
 +
|  SPADA
 +
|  4
 +
|-
 +
|  DEFT Linux
 +
|  7
 +
|}
  
The volume [http://en.wikipedia.org/wiki/Volume_serial_number serial number] is a 4-byte value that identifies a media storage. A serial number does not have a consistent offset within a Prefetch between Windows operating systems. The 4-byte value can be found eight (8) bytes from the creation time location. The [http://en.wikipedia.org/wiki/Vol_%28command%29 vol] command on Windows can verify the volume serial number.
+
=== Orphan inodes deletion ===
  
==== End of File ====
+
When mounting Ext3/4 file systems all orphan inodes are removed, even if "-o ro" mount flag was specified. Currently, there is no specific mount flag to disable orphan inodes deletion. The only solution here is to use "-o ro,loop" flags.
The end of file (EOF) for each Prefetch file is located at offset 0xc. The location of EOF also denotes the size of the Prefetch file.
+
  
==== Files ====
+
=== Root file system spoofing ===
  
Embedded within each Prefetch file are files and directories that were used doing the application's startup. The Prefetch file separates both filenames and directories into two different location in the file. Each string is encoded as a [http://en.wikipedia.org/wiki/UTF-16/UCS-2 UTF-16] string. Windows operating system uses UTF-16 encoding.
+
''See also: [[Early userspace | early userspace]]''
  
The offset to the first set of filenames are at 0x64. The size of the first set of filenames can be found at offset 0x68. Both offsets are consistent between Windows XP, Windows Vista and Windows 7.
+
Most Ubuntu-based forensic Live CD distributions use Casper (a set of scripts used to complete initialization process during early stage of boot). Casper is responsible for searching for a root file system (typically, an image of live environment) on all supported devices (because a bootloader does not pass any information about device used for booting to the kernel), mounting it and executing ''/sbin/init'' program on a mounted root file system that will continue the boot process. Unfortunately, Casper was not designed to meet computer forensics requirements and is responsible for damaged Ext3/4 file systems recovery during the boot (see above) and root file system spoofing.
  
In the bottom section of the Prefetch file are UTF-16 strings of directories. At the time of this writing (7/2011), the precise offset and size of the directory listing is unknown. The distance between the end of the Volume Path string and the beginning of the directory strings is given. An approach to finding the offset to the beginning of the directories listing is to obtain the distance value and the offset when the Volume Path string ends (after the NULL bytes). The distance value is at volume path offset + 0x18 (24). The distance is a 4-byte (DWORD) value. The end of second set of strings will complete the Prefetch file. The size of the directory listing is calculated by subtracting the start position of the directory listing from the end of file position.
+
[[Image:Grml.png|thumb|right|[[grml]] mounted root file system from the [[hard drive]]]]
  
== Registry Keys ==
+
Currently, Casper may select fake root file system image on evidentiary media (e.g. [[Hard Drive|HDD]]), because there are no authenticity checks performed (except optional UUID check for a possible live file system), and this fake root file system image may be used to execute malicious code during the boot with root privileges. Knoppix-based forensic Live CD distributions are vulnerable to the same attack.
<pre>
+
Key: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Session Manager\Memory Management\PrefetchParameters
+
</pre>
+
  
The EnablePrefetcher Registry value can be used to disable prefetch.
+
List of Ubuntu-based distributions that allow root file system spoofing:
  
== See Also ==
+
{| class="wikitable" border="1"
* [[Windows Prefetch File Format]]
+
|-
* [[SuperFetch]]
+
!  Distribution
* [[Prefetch XML]]
+
!  Version
* [[Windows]]
+
|-
 +
|  Helix3
 +
|  2009R1
 +
|-
 +
|  Helix3 Pro
 +
|  2009R3
 +
|-
 +
|  CAINE
 +
|  1.5
 +
|-
 +
|  DEFT Linux
 +
|  5
 +
|-
 +
|  Raptor
 +
|  2.0
 +
|-
 +
|  BackTrack
 +
|  4
 +
|-
 +
|  SMART Linux (Ubuntu)
 +
|  2010-01-20
 +
|-
 +
|  FCCU GNU/Linux Forensic Boot CD
 +
|  12.1
 +
|}
  
== External Links ==
+
Vulnerable Knoppix-based distributions include: SPADA, LinEn Boot CD, BitFlare.
* [http://www.microsoft.com/whdc/driver/kernel/XP_kernel.mspx#ECLAC Microsoft's description of Prefetch when Windows XP was introduced]
+
* [http://msdn.microsoft.com/msdnmag/issues/01/12/XPKernel/default.aspx More detail from Microsoft]
+
* [http://en.wikipedia.org/wiki/Prefetcher Wikipedia Prefetcher]
+
* [http://msdn.microsoft.com/en-us/library/ms940847(v=winembedded.5).aspx MSDN: Disabling Prefetch]
+
* [http://blogs.msdn.com/b/e7/archive/2009/05/05/support-and-q-a-for-solid-state-drives-and.aspx Support and Q&A for Solid-State Drives], by Steven Sinofsky, May 5, 2009
+
* [http://computer-forensics.sans.org/blog/2009/08/05/de-mystifying-defrag-identifying-when-defrag-has-been-used-for-anti-forensics-part-1-windows-xp/ De-mystifying Defrag: Identifying When Defrag Has Been Used for Anti-Forensics (Part 1 - Windows XP)], by [[Chad Tilbury]], August 5, 2009
+
* [http://www.dfinews.com/articles/2010/12/decoding-prefetch-files-forensic-purposes-part-1 Decoding Prefetch Files for Forensic Purposes: Part 1], by [[Mark Wade]], December 8, 2010
+
* [http://crucialsecurityblog.harris.com/2011/04/11/prefetch-files-at-face-value/ Prefetch Files at Face Value], by [[Mark Wade]], April 11, 2011
+
* [http://windowsir.blogspot.ch/2012/03/prefetch-analysis-revisited.html Prefetch Analysis, Revisited], by [[Harlan Carvey]], March 8, 2012
+
* [http://journeyintoir.blogspot.ch/2012/12/ntosboot-prefetch-file.html NTOSBOOT Prefetch File], by [[Corey Harrell]], December 5, 2012
+
  
== Tools ==
+
[http://anti-forensics.ru/ Anti-Forensics.Ru project] [http://digitalcorpora.org/corp/aor/drives/ released several ISO 9660 images] used to test various Linux Live CD distributions for root file system spoofing (description for all images is [http://anti-forensics.ru/casper/ here]).
  
=== Commercial ===
+
=== Swap space activation ===
  
=== Free - Non Open Source ===
+
''Feel free to add information about swap space activation during the boot in some distributions''
* [http://www.woanware.co.uk/forensics/prefetchforensics.html PrefetchForensics], PrefetchForensics is an application to extract information from Windows Prefetch files
+
* [http://redwolfcomputerforensics.com/index.php?option=com_content&task=view&id=42&Itemid=55 Prefetch-Parser], Parse the prefetch files and display information
+
* [http://www.mitec.cz/wfa.html Windows File Analyzer] - Parses Prefetch files, thumbnail databases, shortcuts, index.dat files, and the recycle bin
+
* [http://www.tzworks.net/prototype_page.php?proto_id=1 Windows Prefetch Parser (pf)], Free tool that can be run on Windows, Linux or Mac OS-X
+
  
=== Open Source ===
+
=== Incorrect mount policy ===
* [https://code.google.com/p/prefetch-tool/ prefetch-tool], Script to extract information from windows prefetch folder
+
 
 +
==== rebuildfstab and scanpartitions scripts ====
 +
 
 +
Several forensic Linux Live CD distributions (Helix3 2009R1, Helix3 Pro 2009R3, old versions of CAINE, old versions of grml) use rebuildfstab and scanpartition scripts to create entries for attached devices in ''/etc/fstab''. Some versions of these scripts use wrong wildcards while searching for available block devices (''/dev/?d?'' instead of ''/dev/?d*''), this results in missing several "exotic" devices (like /dev/sdad, /dev/sdad1, etc) and in data writes when mounting them (because fstab lacks of read-only mount options for these devices).
 +
 
 +
=== Incorrect write-blocking approach ===
 +
 
 +
Some forensic Linux Live CD distributions rely on [[hdparm]] and [[blockdev]] programs to mount file systems in read-only mode (by setting the underlying block device to read-only mode). Unfortunately, setting a block device to read-only mode does not guarantee that [http://oss.sgi.com/archives/xfs/2009-07/msg00213.html no write commands will be passed to the drive]. There were several other bugs related to writing on a read-only device in the past (like [https://lkml.org/lkml/2007/2/6/1 Ext3/4 orphan inodes deletion]). At present, kernel code still disregards read-only mode set on block devices in many places (it should be noted that setting a block device to read-only mode will efficiently write-protect the drive from programs running in userspace, while kernel and its modules still can write anything to the block device, regardless of the read-only mode).
 +
 
 +
=== TRIM aka discard command ===
 +
 
 +
== External links ==
 +
 
 +
* [http://www.computer-forensics-lab.org/pdf/Linux_for_computer_forensic_investigators_2.pdf Linux for computer forensic investigators: problems of booting trusted operating system]
 +
* [http://www.computer-forensics-lab.org/pdf/Linux_for_computer_forensic_investigators.pdf Linux for computer forensic investigators: «pitfalls» of mounting file systems]
 +
 
 +
[[Category:Live CD]]

Revision as of 17:59, 15 June 2014

The problem

Forensic Live CDs are widely used during computer forensic investigations. Currently, many vendors of such Live CD distributions spread false claims that their distributions "do not touch anything", "write protect everything" and so on. Unfortunately, community-developed distributions are no exception here. Finally, it turns out that many Linux-based forensic Live CDs are not tested properly and there are no suitable test cases published.

Another side of the problem

Another side of the problem of insufficient testing of forensic Live CDs is that many users do not know what happens "under the hood" of the provided operating system and cannot adequately test them.

Example

For example, Forensic Cop Journal (Volume 1(3), Oct 2009) describes a test case when an Ext3 file system was mounted using "-o ro" mount flag as a way to write protect the data. The article says that all tests were successful (i.e. no data modification was found after unmounting the file system), but it is known that damaged (i.e not properly unmounted) Ext3 file systems cannot be write protected using only "-o ro" mount flags (write access will be enabled during file system recovery).

And the question is: will many users test damaged Ext3 file system (together with testing the clean one) when validating their favourite forensic Live CD distribution? My answer is "no", because many users are unaware of such traits.

Problems

Each problem is followed by a list of distributions affected (currently this list is not up-to-date).

Journaling file system updates

When mounting (and unmounting) several journaling file systems with only "-o ro" mount flag a different number of data writes may occur. Here is a list of such file systems:

File system When data writes happen Notes
Ext3 File system requires journal recovery To disable recovery: use "noload" flag, or use "ro,loop" flags, or use "ext2" file system type
Ext4 File system requires journal recovery To disable recovery: use "noload" flag, or use "ro,loop" flags, or use "ext2" file system type
ReiserFS File system has unfinished transactions "nolog" flag does not work (see man mount). To disable journal updates: use "ro,loop" flags
XFS Always (when unmounting) "norecovery" flag does not help (fixed in recent 2.6 kernels). To disable data writes: use "ro,loop" flags.

Incorrect mount flags can be used to mount file systems on evidentiary media during the boot process or during the file system preview process. As described above, this may result in data writes to evidentiary media. For example, several Ubuntu-based forensic Live CD distributions mount and recover damaged Ext3/4 file systems on fixed media (e.g. hard drives) during execution of initrd scripts (these scripts mount every supported file system type on every supported media type using only "-o ro" flag in order to find a root file system image).

Helix3: damaged Ext3 recovery during the boot

List of distributions that recover Ext3 (and sometimes Ext4) file systems during the boot:

Distribution Version
Helix3 2009R1
SMART Linux (Ubuntu) 2010-01-20
FCCU GNU/Linux Forensic Boot CD 12.1
SPADA 4
DEFT Linux 7

Orphan inodes deletion

When mounting Ext3/4 file systems all orphan inodes are removed, even if "-o ro" mount flag was specified. Currently, there is no specific mount flag to disable orphan inodes deletion. The only solution here is to use "-o ro,loop" flags.

Root file system spoofing

See also: early userspace

Most Ubuntu-based forensic Live CD distributions use Casper (a set of scripts used to complete initialization process during early stage of boot). Casper is responsible for searching for a root file system (typically, an image of live environment) on all supported devices (because a bootloader does not pass any information about device used for booting to the kernel), mounting it and executing /sbin/init program on a mounted root file system that will continue the boot process. Unfortunately, Casper was not designed to meet computer forensics requirements and is responsible for damaged Ext3/4 file systems recovery during the boot (see above) and root file system spoofing.

grml mounted root file system from the hard drive

Currently, Casper may select fake root file system image on evidentiary media (e.g. HDD), because there are no authenticity checks performed (except optional UUID check for a possible live file system), and this fake root file system image may be used to execute malicious code during the boot with root privileges. Knoppix-based forensic Live CD distributions are vulnerable to the same attack.

List of Ubuntu-based distributions that allow root file system spoofing:

Distribution Version
Helix3 2009R1
Helix3 Pro 2009R3
CAINE 1.5
DEFT Linux 5
Raptor 2.0
BackTrack 4
SMART Linux (Ubuntu) 2010-01-20
FCCU GNU/Linux Forensic Boot CD 12.1

Vulnerable Knoppix-based distributions include: SPADA, LinEn Boot CD, BitFlare.

Anti-Forensics.Ru project released several ISO 9660 images used to test various Linux Live CD distributions for root file system spoofing (description for all images is here).

Swap space activation

Feel free to add information about swap space activation during the boot in some distributions

Incorrect mount policy

rebuildfstab and scanpartitions scripts

Several forensic Linux Live CD distributions (Helix3 2009R1, Helix3 Pro 2009R3, old versions of CAINE, old versions of grml) use rebuildfstab and scanpartition scripts to create entries for attached devices in /etc/fstab. Some versions of these scripts use wrong wildcards while searching for available block devices (/dev/?d? instead of /dev/?d*), this results in missing several "exotic" devices (like /dev/sdad, /dev/sdad1, etc) and in data writes when mounting them (because fstab lacks of read-only mount options for these devices).

Incorrect write-blocking approach

Some forensic Linux Live CD distributions rely on hdparm and blockdev programs to mount file systems in read-only mode (by setting the underlying block device to read-only mode). Unfortunately, setting a block device to read-only mode does not guarantee that no write commands will be passed to the drive. There were several other bugs related to writing on a read-only device in the past (like Ext3/4 orphan inodes deletion). At present, kernel code still disregards read-only mode set on block devices in many places (it should be noted that setting a block device to read-only mode will efficiently write-protect the drive from programs running in userspace, while kernel and its modules still can write anything to the block device, regardless of the read-only mode).

TRIM aka discard command

External links