Difference between pages "Libewf" and "Encase image file format"

From ForensicsWiki
(Difference between pages)
Jump to: navigation, search
(External Links)
 
(External Links)
 
Line 1: Line 1:
{{Infobox_Software |
+
The Encase image file format is used by [[EnCase]] used to store various types of digital evidence e.g.
  name = libewf |
+
* disk image (physical bitstream of an acquired disk)
  maintainer = [[Joachim Metz]], [[David Loveall]] |
+
* volume image
  os = [[Linux]], [[FreeBSD]], [[NetBSD]], [[OpenBSD]], [[Mac OS X]], [[Windows]] |
+
* memory
  genre = {{Disk imaging}} |
+
* logical files
  license = {{LGPL}} |
+
  website = [http://code.google.com/p/libewf/ code.google.com/p/libewf/] |
+
}}
+
  
'''Libewf''' is a library to access the [[Encase image file format|Expert Witness Compression Format (EWF)]].
+
== History ==
 +
Expert Witness (for Windows) was the original name for EnCase (dating back to 1998). More info about this can be found on the Internet Archive [http://web.archive.org/web/19980504153628/http://guidancesoftware.com/] including a demo of the original software [http://web.archive.org/web/19980504153759/http://guidancesoftware.com/data/ewsetup.exe].
  
== Features ==
+
(presumably) the product was renamed because it intruded the Expert Wittness trademark held by ASR Data [http://www.asrdata.com/wp-content/themes/asr/pdf/ruling.pdf].
Read or write supported EWF formats:
+
* [[SMART]] .s01 (EWF-S01)
+
* [[EnCase]] .E01 (EWF-E01) and .Ex01 (EWF2-Ex01)
+
  
Read-only supported EWF formats:
+
The Encase image file format therefore is also referred to as the Expert Witness (Compression) Format.
* Logical Evidence File (LEF) .L01 (EWF-L01) and .Lx01 (EWF2-Lx01)
+
  
Other features:
 
* empty-block compression
 
* read/write access using delta (or shadow) files
 
* write resume
 
  
== Tools ==
+
Currently there are 2 versions of the format:
The '''libewf''' package contains the following tools:
+
* version 1 is (reportedly) based on [[ASR Data's Expert Witness Compression Format]]
* '''ewfacquire''', which writes storage media data from devices and files to EWF files.
+
* version 2 was introduced in EnCase 7, for which a format specification (at least non-encrypted Ex01) is available, but requires registration.
* '''ewfacquirestream''', which writes data from stdin to EWF files.
+
* '''ewfdebug'''; experimental tool does nothing at the moment.
+
* '''ewfexport''', which exports storage media data in EWF files to (split) RAW format or a specific version of EWF files.
+
* '''ewfinfo''', which shows the metadata in EWF files.
+
* '''ewfmount''', which FUSE mounts EWF files.
+
* '''ewfrecover'''; special variant of ewfexport to create a new set of EWF files from a corrupt set.
+
* '''ewfverify''', which verifies the storage media data in EWF files.
+
  
The '''libewf''' package also contains the following bindings:
+
The libewf project indicates that the January 2012 version of the version 2 format specification, besides Lx01 not being specified, is sufficient to read non-encrypted Ex01 files.
* '''ewf.net''', bindings for .Net
+
* '''pyewf''', bindings for Python contributed by [[David Collett]] in 2008
+
  
=== Contributions ===
 
Tools that have been contributed to the project are provided as separate tools on the sourceforge libewf project site. These are:
 
* '''mount_ewf.py''', which allows the storage media data in a EWF files to be mounted, contributed by [[David Loveall]] in 2007.
 
* '''libewf-java''', Java (JNA) bindings were contributed by [[Bradley Schatz]] in 2009.
 
* '''delphi imdisk proxy''', Borland Delphi imdisk proxy, as an alternative to mount_ewf.py for Windows, contributed by [[Brendan Berney]] in 2010.
 
* '''jlibewf''', native Java EWF reader contributed by [[Bruce Allen]] in 2010.
 
* '''libewfcs''', native C# EWF reader contributed by [[Bruce Allen]] in 2011.
 
  
A menu based interface for ewfacquirestream called pyEWF, contributed by [[Dennis Schreiber]], was originally also available on the uitwisselplatform project site. However this is currently no longer maintained and was not moved to the sourceforge project size. The uitwisselplatform no longer exists. The name pyewf was reused for the libewf Python bindings created by [[David Collett]] which is now included in the libewf package.
+
Although the format specification is not complete, at the moment Guidance Software is working on an update. This will not include:
 +
* encrypted Ex01
 +
* Lx01
  
=== Examples ===
 
  
Imaging a device on a Unix-based system:
+
So in contrast to other claims [http://tech.groups.yahoo.com/group/linux_forensics/message/3555] both versions of the EWF file format are:
<pre>
+
* proprietary
ewfacquire /dev/sda
+
* partially open specification
</pre>
+
  
Imaging a device on a Windows system:
+
For more information about these definitions see: [[File formats]]
<pre>
+
ewfacquire \\.\PhysicalDrive0
+
</pre>
+
  
Converting a RAW into an EWF image
+
== Version 1 ==
<pre>
+
The media data can be stored in multiple evidence files, which are called segment files.
ewfacquire myfile.raw
+
Each segment file consist of multiple sections, which has a distinct section start definition containing a section type.
</pre>
+
Up to EnCase 5 the segment file were limited to 2 GiB, due to the internal 31-bit file offset representation. This limitation was lifted by adding a base offset value in EnCase 6.
  
or:
 
<pre>
 
ewfacquire -c best -m fixed -t myfile -S 1T -u [-q] myfile.raw
 
</pre>
 
  
or  
+
EnCase allows to store the data compressed either using a fast or best level of the deflate compression method.
 +
EnCase 7 no longer distinguishes between fast or best compression and just provides for either uncompressed or compressed.
  
<pre>
 
cat split.raw.??? | ewfacquirestream
 
cat myfile.??? | ewfacquirestream  -c best -m fixed -t myfile -S 1T
 
  
</pre>
+
Besides digital evidence the evidence files, or segment files, contain a header containing case information.
 +
The case information which entails date and time of acquisition, an examiner's name, notes on the acquisition, and an optional password.
 +
* In EnCase 3 the case information header is stored in the "header" section, which is defined twice within the file and contain the same information.
 +
* As of EnCase 4 an additional "header2" section was added. The "header" section now appears only once, but the new "header2" section twice.
  
Converting an optical disc (split) RAW into an EWF image (libewf 20110109 or later)
 
<pre>
 
ewfacquire -T optical.cue optical.iso
 
</pre>
 
  
Converting an EWF into another EWF format or a (split) RAW image
+
The format adds error detection by storing the data with checksums (Adler32), for both the metadata as the data blocks, which are by default 64 x 512 byte sectors (32 KiB).
<pre>
+
As of EnCase 5 the number of sectors per block (chunk) can vary.
ewfexport image.E01
+
EnCase 3F introduced an "error2" section that it uses to record the location and number of bad sector chunks. The way it handles the sections it can't read is that those areas are filled with zero.
</pre>
+
Then EnCase displays to the user the areas that could not be read when the image was acquired. The granularity of unreadable chunks appears to be 32K.
 +
As of EnCase 5 the granularity of unreadable chunks can vary.
  
Exporting files from a logical image (L01)
 
<pre>
 
ewfexport image.L01
 
</pre>
 
  
FUSE mounting an EWF image (libewf 20110828 or later)
+
EnCase 3 can store a one-way hash of the data. For a bitstream it does so by calculating e.g. a MD5 hash of the original media data and adds a hash section to the last of the segment file.
<pre>
+
As of EnCase 6 the option to store a SHA1 hash was added.
ewfmount image.E01 mount_point
+
</pre>
+
  
FUSE mounting a logical image (L01) (libewf 20111016 or later)
 
<pre>
 
ewfmount -f files image.L01 mount_point
 
</pre>
 
  
Verify an single image with results to the screen
+
EnCase 5 and later have the option to store '''single files''' into the EnCase Logical Evidence File (LEF) or EWF-L01.
<pre>
+
This format changed slightly in EnCase 6 and 7.
ewfverify image.E01
+
</pre>
+
  
From a linux shell, verify a group of images in subdirectories of the current directory creating a simple log file per image.
+
== Version 2 ==
<pre>
+
find . -name \*.E01 -printf '%f %p\n' | xargs printf "ewfverify -l \$(basename -s .E01 %s).ewfverify.out  %s\n" | sh
+
</pre>
+
  
or
+
In EnCase 7 the EWF format was succeeded by the EnCase Evidence File Format Version 2 (EWF2-EX01 and EWF2-LX01).
 +
EWF2-EX01 is at it's lower levels a different format then EWF-E01 and provides support for:
 +
* bzip2 compression
 +
* direct encryption (AES-256) of the section data
  
<pre>
+
The same features are added to the new logical evidence file format (EWF2-LX01) with the exception of encryption.
find . -name '*.E01' | while read F
+
The actual encryption method and corresponding key derivation are, currently, not open.
do
+
  echo ewfverify -l "$(basename -s .E01 $F).ewfverify.out" "$F"
+
done
+
</pre>
+
  
== History ==
+
EWF2-EX01, EWF2-LX01 are not backwards compatible with previous EnCase products.
  
Libewf was created by [[Joachim Metz]] in 2006, while working for [http://en.hoffmannbv.nl/ Hoffmann Investigations].
+
== See Also ==
  
Libewf is a rewrite of earlier work on the EnCase 4 file format by [[Michael Cohen]] part of [[PyFlag]] and the [[:File:ASR Data's Expert Witness Compression Format.pdf|Expert Witness Compression Format]] Specification by [[Andrew Rosen]]. It has been updated to read and write EnCase version 1 to 7 .E01 files, EnCase 5 to 7 .L01 files, EnCase 7 .Ex01 and .Lx01 files and SMART .s01 files. Libewf has initiated an Extended EWF (EWF-X) specifications to bypass limitations on the format imposed by the EnCase .E01 format.
+
* [[ASR Data's Expert Witness Compression Format]]
 +
* [[EnCase]]
  
In 2007 [[David Loveall]] contributed mount_ewf.py to the libewf project. This application allows a [[fuse]] based mount of the storage media data in the EWF files to be mounted. Due to repeated issues with Python and the fuse Python-bindings on [[Mac OS X]] part of the functionality of these scripts has been rewritten into '''ewfmount'''.
+
== External Links ==
  
As of version 20120715 support for EWF version 2 (.Ex01 and .Lx01) was added.
+
* [http://www.cfreds.nist.gov/v2/Basic_Mac_Image.html Sample image in EnCase, iLook, and dd format] - From the [[Computer Forensic Reference Data Sets]] Project, the E01 sample image dates from January 2005
 +
* [https://googledrive.com/host/0B3fBvzttpiiSMTdoaVExWWNsRjg/Expert%20Witness%20Compression%20Format%20(EWF).pdf Expert Witness Compression Format (EWF)], by the [[libewf|libewf project]], March 2006
 +
* [http://encase-enterprise-blog.guidancesoftware.com/2012/01/2nd-generation-encase-evidence-file.html 2nd Generation EnCase Evidence File Technical Specification now Available], Guidance Software, January 2012
 +
* Requires registration: [http://www.guidancesoftware.com/DocumentRegistration.aspx?did=1000018246 EnCase Evidence File Format Version 2], Guidance Software, January 2012
 +
* [http://www.guidancesoftware.com/Document.aspx?did=1000019161 ex01 Example], by [[Guidance Software]], March 2012
 +
* [https://googledrive.com/host/0B3fBvzttpiiSMTdoaVExWWNsRjg/Expert%20Witness%20Compression%20Format%202%20(EWF2).pdf Expert Witness Compression Format (EWF) version 2], by the [[libewf|libewf project]], July 2012
  
== External Links ==
+
[[Category:Forensics File Formats]]
 
+
* [https://code.google.com/p/libewf/ Project site]
+
* [https://code.google.com/p/libewf/wiki/Building Building libewf and tools from source]
+
* [https://code.google.com/p/libewf/wiki/Mounting Mounting a set of EWF file(s)]
+
* [http://libewf.sourceforge.net Old project site]
+

Latest revision as of 00:36, 15 July 2013

The Encase image file format is used by EnCase used to store various types of digital evidence e.g.

  • disk image (physical bitstream of an acquired disk)
  • volume image
  • memory
  • logical files

History

Expert Witness (for Windows) was the original name for EnCase (dating back to 1998). More info about this can be found on the Internet Archive [1] including a demo of the original software [2].

(presumably) the product was renamed because it intruded the Expert Wittness trademark held by ASR Data [3].

The Encase image file format therefore is also referred to as the Expert Witness (Compression) Format.


Currently there are 2 versions of the format:

  • version 1 is (reportedly) based on ASR Data's Expert Witness Compression Format
  • version 2 was introduced in EnCase 7, for which a format specification (at least non-encrypted Ex01) is available, but requires registration.

The libewf project indicates that the January 2012 version of the version 2 format specification, besides Lx01 not being specified, is sufficient to read non-encrypted Ex01 files.


Although the format specification is not complete, at the moment Guidance Software is working on an update. This will not include:

  • encrypted Ex01
  • Lx01


So in contrast to other claims [4] both versions of the EWF file format are:

  • proprietary
  • partially open specification

For more information about these definitions see: File formats

Version 1

The media data can be stored in multiple evidence files, which are called segment files. Each segment file consist of multiple sections, which has a distinct section start definition containing a section type. Up to EnCase 5 the segment file were limited to 2 GiB, due to the internal 31-bit file offset representation. This limitation was lifted by adding a base offset value in EnCase 6.


EnCase allows to store the data compressed either using a fast or best level of the deflate compression method. EnCase 7 no longer distinguishes between fast or best compression and just provides for either uncompressed or compressed.


Besides digital evidence the evidence files, or segment files, contain a header containing case information. The case information which entails date and time of acquisition, an examiner's name, notes on the acquisition, and an optional password.

  • In EnCase 3 the case information header is stored in the "header" section, which is defined twice within the file and contain the same information.
  • As of EnCase 4 an additional "header2" section was added. The "header" section now appears only once, but the new "header2" section twice.


The format adds error detection by storing the data with checksums (Adler32), for both the metadata as the data blocks, which are by default 64 x 512 byte sectors (32 KiB). As of EnCase 5 the number of sectors per block (chunk) can vary. EnCase 3F introduced an "error2" section that it uses to record the location and number of bad sector chunks. The way it handles the sections it can't read is that those areas are filled with zero. Then EnCase displays to the user the areas that could not be read when the image was acquired. The granularity of unreadable chunks appears to be 32K. As of EnCase 5 the granularity of unreadable chunks can vary.


EnCase 3 can store a one-way hash of the data. For a bitstream it does so by calculating e.g. a MD5 hash of the original media data and adds a hash section to the last of the segment file. As of EnCase 6 the option to store a SHA1 hash was added.


EnCase 5 and later have the option to store single files into the EnCase Logical Evidence File (LEF) or EWF-L01. This format changed slightly in EnCase 6 and 7.

Version 2

In EnCase 7 the EWF format was succeeded by the EnCase Evidence File Format Version 2 (EWF2-EX01 and EWF2-LX01). EWF2-EX01 is at it's lower levels a different format then EWF-E01 and provides support for:

  • bzip2 compression
  • direct encryption (AES-256) of the section data

The same features are added to the new logical evidence file format (EWF2-LX01) with the exception of encryption. The actual encryption method and corresponding key derivation are, currently, not open.

EWF2-EX01, EWF2-LX01 are not backwards compatible with previous EnCase products.

See Also

External Links