Difference between pages "Solid State Drive (SSD) Forensics" and "Gzip"

From ForensicsWiki
(Difference between pages)
Jump to: navigation, search
m
 
 
Line 1: Line 1:
Solid State Drives pose a variety of interesting challenges for computer forensics. Most SSD devices are based on flash memory. Flash has two properties that complicate its use in computer storage systems:
+
{{expand}}
# Unlike normal hard drives that can be written in a single pass, flash memory is arranged in pages that must first be erased before it can be written.
+
# Each flash page consists of multiple blocks. Typically block size is 512 bytes and page size is 2KiB, 4KiB, or larger.
+
# Each page can be erased and rewritten a limited number of times---typically 1000 to 10,000. (Hard drive sectors, in contrast, can be rewritten millions of times or more.)
+
  
To overcome these problems, SSD manufacturers have created a system for ''wear leveling''---that is, spreading the writes to flash out among different sectors. Wear leveling is typically done with a ''flash translation layer'' that maps ''logical sectors'' (or LBAs) to ''physical pages.''  Most FTLs are contained within the SSD device and are not accessible to end users.
+
== File format ==
 +
The gzip file (.gz) format consists of:
 +
* a file header
 +
* optional headers
 +
** extra fields
 +
** original file name
 +
** comment
 +
** header checksum
 +
* a body, containing a DEFLATE-compressed payload
 +
* a file footer
  
==Bibliography==
+
=== File header ===
<bibtex>
+
The file header is 10 bytes in size and contains:
@inproceedings{wei2011,
+
{| class="wikitable"
  author = {Michael Wei and Laura M. Grupp and Frederick M. Spada and Steven Swanson},
+
! align="left"| Offset
  title = {Reliably Erasing Data from Flash-Based Solid State Drives},
+
! Size
  booktitle={FAST 2011},
+
! Value
  year = 2011,
+
! Description
  keywords = {erasing flash security ssd},
+
|-
  added-at = {2011-02-22T09:22:03.000+0100},
+
| 0
  url={http://cseweb.ucsd.edu/users/m3wei/assets/pdf/FMS-2010-Secure-Erase.pdf},
+
| 2
  biburl = {http://www.bibsonomy.org/bibtex/27c408ad559fc19f829717f485707a909/schmidt2}
+
| 0x1f 0x8b
}
+
| Signature (or identification byte 1 and 2)
</bibtex>
+
|-
<bibtex>
+
| 2
@article{bell2011,
+
| 1
author="Graeme B. Bell and Richard Boddington",
+
|
title="Solid State Drives: The Beginning of the End for Current Practice in Digital Forensic Recovery?",
+
| Compression Method
journal="Journal of Digital Forensics, Security and Law",
+
|-
volume=5,
+
| 3
issue=3,
+
| 1
url={http://www.jdfsl.org/subscriptions/JDFSL-V5N3-Bell.pdf}
+
|
}
+
| Flags
</bibtex>
+
|-
 +
| 4
 +
| 4
 +
|
 +
| Last modification time <br> Contains a POSIX timestamp.
 +
|-
 +
| 8
 +
| 1
 +
|
 +
| Extra flags
 +
|-
 +
| 9
 +
| 1
 +
|
 +
| Operating system <br> Value that indicates on which operating system the gzip file was created.
 +
|}
  
==Scott Moulton's Shmoocon 2008 Presentation ==
+
==== Compression method ====
Scott Moulton had a presentation at Shmoocon regarding SSD drives vs. Hard Drives.
+
* [http://www.youtube.com/watch?v=l4hbdZFWGog SSD Flash Hard Drives - Shmoocon 2008 - Part 1]
+
* [http://www.youtube.com/watch?v=mglEnIPnzjo SSD Flash Hard Drives - Shmoocon 2008 - Part 2]
+
* [http://www.youtube.com/watch?v=3psy_d-pyNg SSD Flash Hard Drives - Shmoocon 2008 - Part 3]
+
* [http://www.youtube.com/watch?v=pKeZvhDd5c4 SSD Flash Hard Drives - Shmoocon 2008 - Part 4]
+
* [http://www.youtube.com/watch?v=9XMBdDypSO4 SSD Flash Hard Drives - Shmoocon 2008 - Part 5]
+
* [http://www.youtube.com/watch?v=LY36SWbfQg0 SSD Flash Hard Drives - Shmoocon 2008 - Part 6]
+
  
== SSD, wear-leveling and Windows 7 ==
+
{| class="wikitable"
 +
! align="left"| Value
 +
! Identifier
 +
! Description
 +
|-
 +
| 0 - 7
 +
|
 +
| Reserved
 +
|-
 +
| 8
 +
| "deflate"
 +
| zlib compressed data
 +
|}
  
* [http://www.snia.org/events/storage-developer2009/presentations/thursday/NealChristiansen_ATA_TrimDeleteNotification_Windows7.pdf ATA Trim / Delete Notification Support in Windows 7]
+
==== Flags ====
 +
 
 +
{| class="wikitable"
 +
! align="left"| Value
 +
! Identifier
 +
! Description
 +
|-
 +
| 0x01
 +
| FTEXT
 +
| If set the uncompressed data needs to be treated as text instead of binary data. <br> This flag hints end-of-line conversion for cross-platform text files but does not enforce it.
 +
|-
 +
| 0x02
 +
| FHCRC
 +
| The file contains a header checksum (CRC-16)
 +
|-
 +
| 0x04
 +
| FEXTRA
 +
| The file contains extra fields
 +
|-
 +
| 0x08
 +
| FNAME
 +
| The file contains an original file name string
 +
|-
 +
| 0x10
 +
| FCOMMENT
 +
| The file contains comment
 +
|-
 +
| 0x20
 +
|
 +
| Reserved
 +
|-
 +
| 0x40
 +
|
 +
| Reserved
 +
|-
 +
| 0x80
 +
|
 +
| Reserved
 +
|}
 +
 
 +
<b>Note:</b> The FHCRC bit was never set by versions of gzip up to 1.2.4, even though it was documented with a different meaning in gzip 1.2.4.
 +
 
 +
==== Extra flags ====
 +
If compression method is 8 the following extra flags can be defined:
 +
{| class="wikitable"
 +
! align="left"| Value
 +
! Identifier
 +
! Description
 +
|-
 +
| 0x02
 +
|
 +
| compressor used maximum compression, slowest algorithm
 +
|-
 +
| 0x04
 +
|
 +
| compressor used fastest algorithm
 +
|}
 +
 
 +
==== Operating System ====
 +
{| class="wikitable"
 +
! align="left"| Value
 +
! Identifier
 +
! Description
 +
|-
 +
| 0
 +
|
 +
| FAT filesystem (MS-DOS, OS/2, NT/Win32)
 +
|-
 +
| 1
 +
|
 +
| Amiga
 +
|-
 +
| 2
 +
|
 +
| VMS (or OpenVMS)
 +
|-
 +
| 3
 +
|
 +
| Unix
 +
|-
 +
| 4
 +
|
 +
| VM/CMS
 +
|-
 +
| 5
 +
|
 +
| Atari TOS
 +
|-
 +
| 6
 +
|
 +
| HPFS filesystem (OS/2, NT)
 +
|-
 +
| 7
 +
|
 +
| Macintosh
 +
|-
 +
| 8
 +
|
 +
| Z-System
 +
|-
 +
| 9
 +
|
 +
| CP/M
 +
|-
 +
| 10
 +
|
 +
| TOPS-20
 +
|-
 +
| 11
 +
|
 +
| NTFS filesystem (NT)
 +
|-
 +
| 12
 +
|
 +
| QDOS
 +
|-
 +
| 13
 +
|
 +
| Acorn RISCOS
 +
|-
 +
| 255
 +
|
 +
| unknown
 +
|}
 +
 
 +
=== Optional headers ===
 +
==== Extra fields ====
 +
<b>TODO: add description</b>
 +
 
 +
The extra field are variable of size and contains:
 +
{| class="wikitable"
 +
! align="left"| Offset
 +
! Size
 +
! Value
 +
! Description
 +
|-
 +
| 0
 +
| 2
 +
|
 +
| Extra field data size <br> Value in bytes.
 +
|-
 +
| 2
 +
| ...
 +
|
 +
| Extra field data
 +
|}
 +
 
 +
==== Original file name ====
 +
This is the original name of the file being compressed, with any directory components removed, and, if the file being compressed is on a file system with case insensitive names, forced to lower case.
 +
 
 +
Contains an ISO 8859-1 (LATIN-1) string with end-of-string character.
 +
 
 +
==== Comment ====
 +
Contains an ISO 8859-1 (LATIN-1) string with end-of-string character. Line breaks should be denoted by a single line feed character.
 +
 
 +
==== Header checksum ====
 +
The header checksum contain a CRC-16 that consists of the two least significant bytes of the CRC-32 for all bytes of the gzip header up to and not including the CRC-16.
 +
 
 +
=== File footer ===
 +
The file footer is 8 bytes in size and contains:
 +
{| class="wikitable"
 +
! align="left"| Offset
 +
! Size
 +
! Value
 +
! Description
 +
|-
 +
| 0
 +
| 4
 +
|
 +
| Checksum (CRC-32)
 +
|-
 +
| 4
 +
| 4
 +
|
 +
| Uncompressed data size <br> Value in bytes.
 +
|}
 +
 
 +
== External Links ==
 +
 
 +
* [http://www.gzip.org/format.txt The gzip file format], by the [http://www.gzip.org/ gzip project]
 +
* [http://www.gzip.org/algorithm.txt The gzip compression algorithm], by the [http://www.gzip.org/ gzip project]
 +
* [http://tools.ietf.org/html/rfc1952 RFC1952: GZIP file format specification version 4.3], by [[IETF]]
 +
* [http://en.wikipedia.org/wiki/Gzip Wikipedia: gzip]
 +
 
 +
[[Category:File Formats]]

Revision as of 03:05, 28 November 2013

Information icon.png

Please help to improve this article by expanding it.
Further information might be found on the discussion page.

File format

The gzip file (.gz) format consists of:

  • a file header
  • optional headers
    • extra fields
    • original file name
    • comment
    • header checksum
  • a body, containing a DEFLATE-compressed payload
  • a file footer

File header

The file header is 10 bytes in size and contains:

Offset Size Value Description
0 2 0x1f 0x8b Signature (or identification byte 1 and 2)
2 1 Compression Method
3 1 Flags
4 4 Last modification time
Contains a POSIX timestamp.
8 1 Extra flags
9 1 Operating system
Value that indicates on which operating system the gzip file was created.

Compression method

Value Identifier Description
0 - 7 Reserved
8 "deflate" zlib compressed data

Flags

Value Identifier Description
0x01 FTEXT If set the uncompressed data needs to be treated as text instead of binary data.
This flag hints end-of-line conversion for cross-platform text files but does not enforce it.
0x02 FHCRC The file contains a header checksum (CRC-16)
0x04 FEXTRA The file contains extra fields
0x08 FNAME The file contains an original file name string
0x10 FCOMMENT The file contains comment
0x20 Reserved
0x40 Reserved
0x80 Reserved

Note: The FHCRC bit was never set by versions of gzip up to 1.2.4, even though it was documented with a different meaning in gzip 1.2.4.

Extra flags

If compression method is 8 the following extra flags can be defined:

Value Identifier Description
0x02 compressor used maximum compression, slowest algorithm
0x04 compressor used fastest algorithm

Operating System

Value Identifier Description
0 FAT filesystem (MS-DOS, OS/2, NT/Win32)
1 Amiga
2 VMS (or OpenVMS)
3 Unix
4 VM/CMS
5 Atari TOS
6 HPFS filesystem (OS/2, NT)
7 Macintosh
8 Z-System
9 CP/M
10 TOPS-20
11 NTFS filesystem (NT)
12 QDOS
13 Acorn RISCOS
255 unknown

Optional headers

Extra fields

TODO: add description

The extra field are variable of size and contains:

Offset Size Value Description
0 2 Extra field data size
Value in bytes.
2 ... Extra field data

Original file name

This is the original name of the file being compressed, with any directory components removed, and, if the file being compressed is on a file system with case insensitive names, forced to lower case.

Contains an ISO 8859-1 (LATIN-1) string with end-of-string character.

Comment

Contains an ISO 8859-1 (LATIN-1) string with end-of-string character. Line breaks should be denoted by a single line feed character.

Header checksum

The header checksum contain a CRC-16 that consists of the two least significant bytes of the CRC-32 for all bytes of the gzip header up to and not including the CRC-16.

File footer

The file footer is 8 bytes in size and contains:

Offset Size Value Description
0 4 Checksum (CRC-32)
4 4 Uncompressed data size
Value in bytes.

External Links