Difference between pages "Forensic Live CD issues" and "Advanced Format"

From ForensicsWiki
(Difference between pages)
Jump to: navigation, search
m (Incorrect automount policy for removable media)
 
(The Problem: Death of LBA 63)
 
Line 1: Line 1:
== The problem ==
+
=The Technology=
 +
Hard drive manufacturers have moved to a new standard: 4KB (4,096 bytes) sectors, replacing 512B sectors. This is a good thing; it means that the signal-to-noise ratio improves, and less space is needed for error correction. Long-term improvements in speed, density, and overall capacity. Western Digital has started releasing drives with 4KB sectors under the name "Advanced Format" (not to be confused with the [[Advanced Forensics Format]]).
  
[[Tools#Forensics_Live_CDs | Forensic Linux Live CD distributions]] are widely used during computer forensic investigations. Currently, many vendors of such Live CD distributions spread false claims that their distributions "do not touch anything", "write protect everything" and so on. Community-developed distributions are not exception here, unfortunately. Finally, it turns out that many forensic Linux Live CD distributions are not tested properly and there are no suitable test cases developed.
+
=The Standard=
 +
ATA 7 (T13/D1532, INCITS 397-2005) introduced Long Physical Sector (LPS) and Long Logical Sector (LLS) feature sets. Drives with large sector sizes shall report the actual physical/logical size in words 106 and 117-118 of the ATA IDENTIFY data.
  
== Another side of the problem ==
+
Some Western Digital drives with "Advanced Format" reportedly do not provide the information about physical sector size (see [[#External_Links|External Links]]).
  
Another side of the problem of insufficient testing of forensic Live CD distributions is that many users do not know what happens "under the hood" of such distributions and cannot adequately test them.
+
=The Problem: Death of LBA 63=
 +
Operating systems written before the transition, particularly XP, have trouble with the new drives. XP makes an assumption about where the format should start (LBA 63), but this doesn't work well with the translation software that maps from logical 512B blocks to physical 4K blocks.
  
=== Example ===
+
The nutshell is that XP should not be used to format these drives, and some assumptions made by tools and users need to be corrected. For analysis purposes, note that you can't assume that an NTFS partition starts at LBA 63. If you are used to using, for example, the Sleuthkit command "fls -o 63 <image>", this may need to change. Hopefully more information about these drives will come forth as time progresses.
  
For example, [http://forensiccop.blogspot.com/2009/10/forensic-cop-journal-13-2009.html ''Forensic Cop Journal'' (Volume 1(3), Oct 2009)] describes a test case when an Ext3 file system was mounted using "-o ro" mount flag as a way to write protect the data. The article says that all tests were successful (i.e. no data modification was found after unmounting the file system), but it is known that damaged (i.e not properly unmounted) Ext3 file systems cannot be write protected using only "-o ro" mount flags (write access will be enabled during file system recovery).
+
LBA 63 was dying anyway. Windows Vista and Windows 7 both align to LBA 2048 by default. This change happened before the Advanced Format 512e drives hit the marketplace.
  
And the question is: will many users test damaged Ext3 file system (together with testing the clean one) when validating their favourite forensic Live CD distribution? My answer is "no", because many users are unaware of such traits.
+
=The Solution=
 +
To format one of these drives properly for Windows XP, use the following utility (this applies only to drives from Western Digital):
  
== Problems ==
+
[http://www.wdc.com/en/products/advancedformat/ Western Digital Advanced Drive Format Utility]
  
Here is a list of common problems of forensic Linux Live CD distributions that can be used by developers and users for testing purposes. Each problem is followed by an up to date list of distributions affected.
+
=External Links=
 
+
*[http://www.anandtech.com/storage/showdoc.aspx?i=3691 A Good Overview at AnandTech]
=== Journaling file systems updates ===
+
*[http://www.wdc.com/wdproducts/library/WhitePapers/ENG/2579-771430.pdf PDF White Paper]
 
+
*[http://www.tomshardware.com/reviews/green-terabyte-1tb,2078-2.html A Tom's Hardware Review of the WD Caviar Green Drives]
When mounting (and unmounting) several journaling file systems with only "-o ro" mount flag a different number of data writes may occur. Here is a list of such file systems:
+
*[https://ata.wiki.kernel.org/index.php/ATA_4_KiB_sector_issues ATA 4 KiB sector issues (good summary from Linux ATA wiki)]
 
+
*[http://lwn.net/Articles/377895/ 4K-sector drives and Linux (LWN.net)]
{| class="wikitable" border="1"
+
*[http://sourceforge.net/apps/trac/smartmontools/ticket/62 WD6400AARS-00Y5B1 does not provide sector size info (smartmontools ticket)]
|-
+
!  File system
+
!  When data writes happen
+
!  Notes
+
|-
+
|  Ext3
+
|  File system requires journal recovery
+
|  To disable recovery: use "noload" flag, or use "ro,loop" flags, or use "ext2" file system type
+
|-
+
|  Ext4
+
|  File system requires journal recovery
+
|  To disable recovery: use "noload" flag, or use "ro,loop" flags, or use "ext2" file system type
+
|-
+
|  ReiserFS
+
|  File system has unfinished transactions
+
|  "nolog" flag does not work (see ''man mount''). To disable journal updates: use "ro,loop" flags
+
|-
+
|  XFS
+
|  Always
+
|  "norecovery" flag does not help. To disable data writes: use "ro,loop" flags. The bug was fixed in recent 2.6 kernels.
+
|}
+
 
+
Incorrect mount flags can be used to mount a file system on evidentiary media during the boot process or during the file system preview process. As described above, this may result in data writes to evidentiary media. For example, several Ubuntu-based forensic Linux Live CD distributions mount Ext3/4 file systems on fixed media (e.g. hard drives) during execution of [http://en.wikipedia.org/wiki/Initrd ''initrd''] scripts (these scripts mount every supported file system type on every supported media type using only "-o ro" flag in order to find a root file system image).
+
 
+
[[Image:ext3 recovery.png|thumb|right|[[Helix3]]: damaged Ext3 recovery during the boot]]
+
 
+
List of distributions that recover Ext3 (and sometimes Ext4) file systems during the boot:
+
 
+
{| class="wikitable" border="1"
+
|-
+
!  Distribution
+
!  Version
+
|-
+
|  Helix3
+
|  2009R1
+
|-
+
|  SMART Linux (Ubuntu)
+
|  2010-01-20
+
|-
+
|  FCCU GNU/Linux Forensic Boot CD
+
|  12.1
+
|-
+
|  SPADA
+
|  4
+
|}
+
 
+
=== Root file system spoofing ===
+
 
+
Most Ubuntu-based forensic Live CD distributions use Casper (set of scripts used to complete initialization process during early stage of boot). Casper is responsible for searching for a root file system (typically, an image of live environment) on all supported devices (because a bootloader does not pass any information about device used for booting to the kernel), mounting it and executing ''/sbin/init'' program on a mounted root file system that will continue the boot process. Unfortunately, Casper was not designed to meet computer forensics requirements and is responsible for damaged Ext3/4 file systems recovery during the boot (see above) and root file system spoofing.
+
 
+
[[Image:Grml.png|thumb|right|[[grml]] mounted root file system from the [[hard drive]]]]
+
 
+
Currently, Casper may select fake root file system image on evidentiary media (e.g. [[HDD]]), because there are no authenticity checks performed (except optional UUID check for a possible live file system), and this fake root file system image may be used to execute malicious code during the boot with root privileges. Knoppix-based forensic Live CD distributions are vulnerable to the same attack.
+
 
+
List of Ubuntu-based distributions that allow root file system spoofing:
+
 
+
{| class="wikitable" border="1"
+
|-
+
!  Distribution
+
!  Version
+
!  Notes
+
|-
+
|  Helix3
+
|  2009R1
+
|
+
|-
+
|  Helix3 Pro
+
|  2009R3
+
|
+
|-
+
|  CAINE
+
|  1.5
+
|
+
|-
+
|  DEFT Linux
+
|  5
+
|
+
|-
+
|  Raptor
+
|  20091026
+
|
+
|-
+
|  grml
+
|  2009.10
+
|  Actually, [[grml]] uses live-initramfs scripts (Casper fork)
+
|-
+
|  BackTrack
+
|  4
+
|
+
|-
+
|  SMART Linux (Ubuntu)
+
|  2010-01-20
+
|
+
|-
+
|  FCCU GNU/Linux Forensic Boot CD
+
|  12.1
+
|
+
|}
+
 
+
Vulnerable Knoppix-based distributions include: SPADA, LinEn boot CD, BitFlare.
+
 
+
=== Swap space activation ===
+
 
+
=== Incorrect automount policy ===
+
 
+
=== Incorrect write-blocking approach ===
+
 
+
== See also ==
+
 
+
* [http://www.computer-forensics-lab.org/pdf/Linux_for_computer_forensic_investigators_2.pdf Linux for computer forensic investigators: problems of booting trusted operating system]
+
* [http://www.computer-forensics-lab.org/pdf/Linux_for_computer_forensic_investigators.pdf Linux for computer forensic investigators: «pitfalls» of mounting file systems]
+

Latest revision as of 20:26, 15 June 2011

The Technology

Hard drive manufacturers have moved to a new standard: 4KB (4,096 bytes) sectors, replacing 512B sectors. This is a good thing; it means that the signal-to-noise ratio improves, and less space is needed for error correction. Long-term improvements in speed, density, and overall capacity. Western Digital has started releasing drives with 4KB sectors under the name "Advanced Format" (not to be confused with the Advanced Forensics Format).

The Standard

ATA 7 (T13/D1532, INCITS 397-2005) introduced Long Physical Sector (LPS) and Long Logical Sector (LLS) feature sets. Drives with large sector sizes shall report the actual physical/logical size in words 106 and 117-118 of the ATA IDENTIFY data.

Some Western Digital drives with "Advanced Format" reportedly do not provide the information about physical sector size (see External Links).

The Problem: Death of LBA 63

Operating systems written before the transition, particularly XP, have trouble with the new drives. XP makes an assumption about where the format should start (LBA 63), but this doesn't work well with the translation software that maps from logical 512B blocks to physical 4K blocks.

The nutshell is that XP should not be used to format these drives, and some assumptions made by tools and users need to be corrected. For analysis purposes, note that you can't assume that an NTFS partition starts at LBA 63. If you are used to using, for example, the Sleuthkit command "fls -o 63 <image>", this may need to change. Hopefully more information about these drives will come forth as time progresses.

LBA 63 was dying anyway. Windows Vista and Windows 7 both align to LBA 2048 by default. This change happened before the Advanced Format 512e drives hit the marketplace.

The Solution

To format one of these drives properly for Windows XP, use the following utility (this applies only to drives from Western Digital):

Western Digital Advanced Drive Format Utility

External Links