Difference between revisions of "Forensic file formats"

From Forensics Wiki
Jump to: navigation, search
m
Line 1: Line 1:
Many computer forensic programs, especially the all-in-one suites, use their own file formats to store information. This page lists many of those formats. Note that this page represents a subset of all of the [[File_Formats|known file formats]].
+
Many computer forensic programs, especially the all-in-one suites, use their own file formats to store information. This page lists many of those formats. Note that this page represents a subset of all of the [[:Category:Forensics File Format]].
 
+
 
+
  
 
; [[AFF]]
 
; [[AFF]]
 
Full details of the format and a working implementation can be downloaded from http://www.afflib.org/
 
Full details of the format and a working implementation can be downloaded from http://www.afflib.org/
 
  
 
; [[EnCase]]
 
; [[EnCase]]
 
Perhaps the de facto standard for forensic analyses in law
 
Perhaps the de facto standard for forensic analyses in law
enforcement, Guidance Software's EnCase Forensic uses
+
enforcement, Guidance Software's [[EnCase]] Forensic uses
 
a proprietary format for images, reportedly based on ASR Data's
 
a proprietary format for images, reportedly based on ASR Data's
 
Expert Witness Compression Format.  EnCase's Evidence File
 
Expert Witness Compression Format.  EnCase's Evidence File
Line 15: Line 12:
 
of an acquired disk, prefixed with a "Case Info" header,
 
of an acquired disk, prefixed with a "Case Info" header,
 
interlaced with CRCs for every block of 64 sectors (32 KB), and
 
interlaced with CRCs for every block of 64 sectors (32 KB), and
followed by a footer containing an MD5 hash for the entire
+
followed by a footer containing an [[MD5]] hash for the entire
 
bitstream.  Contained in the header are the date and time of
 
bitstream.  Contained in the header are the date and time of
 
acquisition, an examiner's name, notes on the acquisition, and an
 
acquisition, an examiner's name, notes on the acquisition, and an
Line 21: Line 18:
  
 
Not only is the format is compressible, it is also searchable.
 
Not only is the format is compressible, it is also searchable.
Compression is block-based~\cite{pyflagformat}, and ``jump tables''
+
Compression is block-based, and jump tables and "file pointers" are maintained in the format's header or
and "file pointers" are maintained in the format's header or
+
between blocks "to enhance speed". Disk images
between blocks "to enhance speed."  Disk images
+
 
can be split into multiple files (e.g., for archival to CD or
 
can be split into multiple files (e.g., for archival to CD or
 
DVD).
 
DVD).
Line 33: Line 29:
 
remains closed.
 
remains closed.
  
 +
; [[FTK Imager]] ([[FTK]]'s) File Formats
  
; FTK Imager ([[FTK]]'s) File Formats
+
A popular alternative to [[EnCase]], AccessData's Forensic Toolkit ([[FTK]])
 
+
supports storage of disk images in EnCase's or [[SMART]]'s file format,
A popular alternative to EnCase, AccessData's Forensic Toolkit (FTK)
+
as well as in raw ([[dd]]) format.  With Isobuster technology built in, [[FTK Imager]] Images CD's to a ISO/CUE file combination.  This also includes multi and open session CDs.  
supports storage of disk images in EnCase's or SMART's file format,
+
as well as in raw(dd)format.  With Isobuster technology built in, FTK Imager Images CD's to a ISO/CUE file combination.  This also includes multi and open session CDs.  
+
  
 
; [[gfzip]] (generic forensic zip) file format
 
; [[gfzip]] (generic forensic zip) file format
  
 
Gfzip aims to provide an open file format for 'forensic complete' 'compressed' and 'signed' disk image data files.
 
Gfzip aims to provide an open file format for 'forensic complete' 'compressed' and 'signed' disk image data files.
Uncompressed disk images can be used the same way dd images are, as gfzip uses a data first footer last design.
+
Uncompressed disk images can be used the same way [[dd]] images are, as gfzip uses a data first footer last design.
Gfzip uses multi level sha256 digest based integrity guards instead of sha1 or the deprecated md5 algoritm.
+
Gfzip uses multi level [[SHA256]] digest based integrity guards instead of [[SHA1]] or the deprecated [[MD5]] algoritm.
 
User supplied meta data is embedded in a meta data section within the file.  
 
User supplied meta data is embedded in a meta data section within the file.  
 
A very important feature that gfzip focuses on extensively is the use of signed data and meta data sections using x509 certificates.
 
A very important feature that gfzip focuses on extensively is the use of signed data and meta data sections using x509 certificates.
 
  
 
; [[ILook Investigator]]'s IDIF, IRBF, and IEIF Formats
 
; [[ILook Investigator]]'s IDIF, IRBF, and IEIF Formats
  
 
ILook Investigator v8 and its disk-imaging
 
ILook Investigator v8 and its disk-imaging
counterpart, IXimager, offer three proprietary, authenticated image
+
counterpart, [[IXimager]], offer three proprietary, authenticated image
 
formats: compressed (IDIF), non-compressed (IRBF), and encrypted
 
formats: compressed (IDIF), non-compressed (IRBF), and encrypted
 
(IEIF). Although few technical details are disclosed publicly,
 
(IEIF). Although few technical details are disclosed publicly,
 
IXimager's online documentation provides some
 
IXimager's online documentation provides some
insights: IDIF "includes protective mechanisms to detect changes
+
insights: IDIF "includes protective mechanisms to detect changes
 
from the source image entity to the output form" and supports
 
from the source image entity to the output form" and supports
 
"logging of user actions within the confines of that event;"  IRBF
 
"logging of user actions within the confines of that event;"  IRBF
Line 65: Line 59:
 
tools, IXimager allows for the transformation of each of these
 
tools, IXimager allows for the transformation of each of these
 
formats into raw format.
 
formats into raw format.
 
  
 
; [[ProDiscover]] Family's ProDiscover Image File Format
 
; [[ProDiscover]] Family's ProDiscover Image File Format
  
Used by [[Technology Pathways]]' [[ProDiscover]] Family of security tools, the ProDiscover Image File format consists of five parts: a 16-byte Image
+
Used by [[Technology Pathways]] [[ProDiscover]] Family of security tools, the ProDiscover Image File format consists of five parts: a 16-byte Image File Header, which includes a signature and version number for an
File Header, which includes a signature and version number for an
+
 
image; a 681-byte Image Data Header, which contains user-provided
 
image; a 681-byte Image Data Header, which contains user-provided
 
metadata about the image; Image Data, which comprises a single block
 
metadata about the image; Image Data, which comprises a single block
Line 79: Line 71:
  
 
Though fairly well documented, the format is not extensible.
 
Though fairly well documented, the format is not extensible.
 
  
 
; [[PyFlag]]'s [[sgzip]] Format
 
; [[PyFlag]]'s [[sgzip]] Format
  
Supported by PyFlag, a "Forensic and Log
+
Supported by [[PyFlag]], a "Forensic and Log
 
Analysis GUI" begun as a project in the Australian Department of
 
Analysis GUI" begun as a project in the Australian Department of
 
Defence, sgzip is a seekable variant of the gzip format.  By
 
Defence, sgzip is a seekable variant of the gzip format.  By
 
compressing blocks (of 32KB, by default) individually, sgzip allows
 
compressing blocks (of 32KB, by default) individually, sgzip allows
 
disk images to be searched for keywords without being fully
 
disk images to be searched for keywords without being fully
decompressed.  The format does not associate metadata with images. {In addition to its own sgzip format, PyFlag can also read and write the Expert Witness Compression Format.  
+
decompressed.  The format does not associate metadata with images. In addition to its own sgzip format, PyFlag can also read and write the Expert Witness Compression Format.
 
+
  
 
; [[Rapid Action Imaging Device]] (RAID)'s Format
 
; [[Rapid Action Imaging Device]] (RAID)'s Format
Line 97: Line 87:
 
[sic] integrity checking" and is to be designed to
 
[sic] integrity checking" and is to be designed to
 
create an identical copy in raw format of one disk on another.  The copy can then
 
create an identical copy in raw format of one disk on another.  The copy can then
"be inserted into a forensic workstation."  
+
"be inserted into a forensic workstation".
 
+
  
 
; [[Safeback]]'s Format
 
; [[Safeback]]'s Format
Line 108: Line 97:
 
few technical details are disclosed publicly, SafeBack's authors
 
few technical details are disclosed publicly, SafeBack's authors
 
claim that the software "safeguards the internally stored SHA256
 
claim that the software "safeguards the internally stored SHA256
values."
+
values".
 
+
  
 
; [[SDi32]]'s Format
 
; [[SDi32]]'s Format
Line 115: Line 103:
 
Imaging software designed to be used with write-blocking hardware,
 
Imaging software designed to be used with write-blocking hardware,
 
Vogon International's SDi32 is capable of making identical copies
 
Vogon International's SDi32 is capable of making identical copies
of disks to tape, disk, or file, with optional CRC32 and MD5
+
of disks to tape, disk, or file, with optional CRC32 and [[MD5]]
 
fingerprints.  The copies are stored in raw format.
 
fingerprints.  The copies are stored in raw format.
  
Line 128: Line 116:
 
can be stored as a single file or in multiple segment files, each of
 
can be stored as a single file or in multiple segment files, each of
 
which consist of a standard 13-byte header followed by a series of
 
which consist of a standard 13-byte header followed by a series of
sections, each of type "header," "volume," "table," "next,"
+
sections, each of type "header", "volume", "table", "next",
or "done." Each section includes its type string, a 64-bit offset
+
or "done". Each section includes its type string, a 64-bit offset
 
to the next section, its 64-bit size, padding, and a CRC, in
 
to the next section, its 64-bit size, padding, and a CRC, in
 
addition to actual data or comments, if applicable. Although the
 
addition to actual data or comments, if applicable. Although the
 
format's "header" section supports free-form notes, an image can
 
format's "header" section supports free-form notes, an image can
 
have only one such section (in its first segment file only).
 
have only one such section (in its first segment file only).

Revision as of 05:43, 28 September 2008

Many computer forensic programs, especially the all-in-one suites, use their own file formats to store information. This page lists many of those formats. Note that this page represents a subset of all of the Category:Forensics File Format.

AFF

Full details of the format and a working implementation can be downloaded from http://www.afflib.org/

EnCase

Perhaps the de facto standard for forensic analyses in law enforcement, Guidance Software's EnCase Forensic uses a proprietary format for images, reportedly based on ASR Data's Expert Witness Compression Format. EnCase's Evidence File (.E01) format contains a physical bitstream of an acquired disk, prefixed with a "Case Info" header, interlaced with CRCs for every block of 64 sectors (32 KB), and followed by a footer containing an MD5 hash for the entire bitstream. Contained in the header are the date and time of acquisition, an examiner's name, notes on the acquisition, and an optional password; the header concludes with its own CRC.

Not only is the format is compressible, it is also searchable. Compression is block-based, and jump tables and "file pointers" are maintained in the format's header or between blocks "to enhance speed". Disk images can be split into multiple files (e.g., for archival to CD or DVD).

But files in this format can be no larger than 2 GB. The format also restricts the type and quantity of metadata that can be associated with an image. And, though some vendors have reverse-engineered the format for compatibility's sake, the format remains closed.

FTK Imager (FTK's) File Formats

A popular alternative to EnCase, AccessData's Forensic Toolkit (FTK) supports storage of disk images in EnCase's or SMART's file format, as well as in raw (dd) format. With Isobuster technology built in, FTK Imager Images CD's to a ISO/CUE file combination. This also includes multi and open session CDs.

gfzip (generic forensic zip) file format

Gfzip aims to provide an open file format for 'forensic complete' 'compressed' and 'signed' disk image data files. Uncompressed disk images can be used the same way dd images are, as gfzip uses a data first footer last design. Gfzip uses multi level SHA256 digest based integrity guards instead of SHA1 or the deprecated MD5 algoritm. User supplied meta data is embedded in a meta data section within the file. A very important feature that gfzip focuses on extensively is the use of signed data and meta data sections using x509 certificates.

ILook Investigator's IDIF, IRBF, and IEIF Formats

ILook Investigator v8 and its disk-imaging counterpart, IXimager, offer three proprietary, authenticated image formats: compressed (IDIF), non-compressed (IRBF), and encrypted (IEIF). Although few technical details are disclosed publicly, IXimager's online documentation provides some insights: IDIF "includes protective mechanisms to detect changes from the source image entity to the output form" and supports "logging of user actions within the confines of that event;" IRBF is similar to IDIF except that disk images are left uncompressed; IEIF, meanwhile, encrypts said images.

For compatibility with ILook Investigator v7 and other forensic tools, IXimager allows for the transformation of each of these formats into raw format.

ProDiscover Family's ProDiscover Image File Format

Used by Technology Pathways ProDiscover Family of security tools, the ProDiscover Image File format consists of five parts: a 16-byte Image File Header, which includes a signature and version number for an image; a 681-byte Image Data Header, which contains user-provided metadata about the image; Image Data, which comprises a single block of uncompressed data or an array of blocks of compressed data; an Array of Compressed Blocks sizes (if the Image Data is, in fact, compressed); and I/O Log Errors describing any problems during the image's acquisition.

Though fairly well documented, the format is not extensible.

PyFlag's sgzip Format

Supported by PyFlag, a "Forensic and Log Analysis GUI" begun as a project in the Australian Department of Defence, sgzip is a seekable variant of the gzip format. By compressing blocks (of 32KB, by default) individually, sgzip allows disk images to be searched for keywords without being fully decompressed. The format does not associate metadata with images. In addition to its own sgzip format, PyFlag can also read and write the Expert Witness Compression Format.

Rapid Action Imaging Device (RAID)'s Format

Though relatively little technical detail is publicly available, DIBS USA's Rapid Action Imaging Device (RAID) offers "built in [sic] integrity checking" and is to be designed to create an identical copy in raw format of one disk on another. The copy can then "be inserted into a forensic workstation".

Safeback's Format

SafeBack, a DOS-based utility designed to create exact copies of entire disks or partitions, offers a "self-authenticating" format for images, whereby SHA256 hashes are stored along with data to ensure the latter's integrity. Although few technical details are disclosed publicly, SafeBack's authors claim that the software "safeguards the internally stored SHA256 values".

SDi32's Format

Imaging software designed to be used with write-blocking hardware, Vogon International's SDi32 is capable of making identical copies of disks to tape, disk, or file, with optional CRC32 and MD5 fingerprints. The copies are stored in raw format.


SMART's Formats

SMART, a software utility for Linux designed by the original authors of Expert Witness (now sold under the name of EnCase), can store disk images as pure bitstreams (compressed or uncompressed) and also in ASR Data's Expert Witness Compression Format. Images stored in the latter format can be stored as a single file or in multiple segment files, each of which consist of a standard 13-byte header followed by a series of sections, each of type "header", "volume", "table", "next", or "done". Each section includes its type string, a 64-bit offset to the next section, its 64-bit size, padding, and a CRC, in addition to actual data or comments, if applicable. Although the format's "header" section supports free-form notes, an image can have only one such section (in its first segment file only).