Difference between pages "Ddrescue" and "Encase image file format"

From ForensicsWiki
(Difference between pages)
Jump to: navigation, search
m (syntax fail.. ;-))
 
(Added link to CFRDS)
 
Line 1: Line 1:
{{Infobox_Software |
+
[[EnCase]] uses a proprietary format for images which is reportedly based on ASR Data's Expert Witness Compression Format. (Source?) The evidence files, or E01 files, contain a physical bitstream of an acquired disk, prefixed with a '"Case Info" header, interlaced with CRCs for every block of 64 sectors~(32 KB), and followed by a footer containing an MD5 hash for the entire bitstream. Contained in the header are the date and time of acquisition, an examiner's name, notes on the acquisition, and an optional password; the header concludes with its own CRC.
  name = ddrescure |
+
  maintainer = [[Antonio Diaz Diaz]]|
+
  os = {{Linux}}|
+
  genre = {{Disk imaging}} |
+
  license = {{GPL}} |
+
  website = [http://www.gnu.org/software/ddrescue/ddrescue.html http://www.gnu.org/software/ddrescue/ddrescue.html] |
+
}}
+
  
'''ddrescue''' is a raw disk imaging tool that "copies data from one file or block device to another, trying hard to rescue data in case of read errors."  The application is developed as part of the GNU project and has written with UNIX/Linux in mind.
+
Encase can store media data into multiple evidence files, which are called segment files. Each segment file consist of multiple sections. Each section consist of a section start definition. This contains a section type.
  
'''ddrescue''' and '''[[dd_rescue]]''' are completely different programs which share no development between them.  The two projects are not related in any way except that they both attempt to enhance the standard [[dd]] tool and coincidentally chose similar names for their new programs.
+
At least from Encase 3 the case info header is contained in the "header" section, which is defined twice within the file and contain the same information.
  
From the [[ddrescue]] info pages:
+
With Encase 4 an additional "header2" section was added. The "header" section now appears only once, but the new "header2" section twice.
<blockquote>
+
GNU ddrescue is a data recovery tool. It copies data from one file or block device (hard disc, cdrom, etc) to another, trying hard to rescue data in case of read errors.<br><br>
+
  
Ddrescue does not truncate the output file if not asked to. So, every time you run it on the same output file, it tries to fill in the gaps.<br><br>
+
Version 3 of The Encase F introduced an "error2" sections that it uses to record the location and number of bad sector chunks. The way it handles the sections it can't read is that those areas are filled with zero. Then Encase displays to the user the areas that could not be read when the image was acquired. The granularity of unreadable chunks appears to be 32K.
  
The basic operation of ddrescue is fully automatic. That is, you don't have to wait for an error, stop the program, read the log, run it in reverse mode, etc.<br><br>
+
Within Encase 5 the amount of sectors per block (chunk) can vary.
  
If you use the logfile feature of ddrescue, the data is rescued very efficiently (only the needed blocks are read). Also you can interrupt the rescue at any time and resume it later at the same point.<br><br>
+
Encase from at least in version 3, 4 and 5 can hash the data of the media it acquires.
 +
It does this by calculating a MD5 hash of the original media data and adds a hash section
 +
to the last of the segment files.
  
Automatic merging of backups: If you have two or more damaged copies of a file, cdrom, etc, and run ddrescue on all of them, one at a time, with the same output file, you will probably obtain a complete and error-free file. This is so because the probability of having damaged areas at the same places on different input files is very low. Using
+
== See Also ==
the logfile, only the needed blocks are read from the second and successive copies.
+
</blockquote>
+
  
== Installation ==
+
[[EnCase]]
  
=== Bootable CD ===
+
== External Links ==  
ddrescue is available on bootable rescue cds such as SystemRescueCd http://www.sysresccd.org/Main_Page.
+
=== Debian and Ubuntu ===
+
The package 'ddrescue' in Debian and Ubuntu is actually [[dd_rescue]], another dd-like program which does not maintain a recovery log.  The correct package is gddrescue.
+
  
Debian
+
* A great deal of information about the format has been documented by the [https://www.uitwisselplatform.nl/projects/libewf/ libewf project], including some of the [https://www.uitwisselplatform.nl/docman/view.php/53/62/ewf.pdf E01 file format specifications].
<blockquote>
+
* [http://www.cfreds.nist.gov/v2/Basic_Mac_Image.html Sample image in EnCase, iLook, and dd format] - From the [[Computer Forensic Reference Data Sets]] Project
aptitude install gddrescue
+
</blockquote>
+
Ubuntu
+
<blockquote>
+
sudo apt-get install gddrescue
+
</blockquote>
+
=== Gentoo ===
+
<blockquote>
+
emerge ddrescue
+
</blockquote>
+
== Partition recovery ==
+
  
=== Kernel 2.6.3+ & ddrescue 1.4+ ===
+
[[Category:Forensics File Format]]
'ddrescue --direct' will open the input with the O_DIRECT option for uncached reads. 'raw devices' are not needed on newer kernels. For older kernels see below.
+
 
+
First you copy as much data as possible, without retrying or splitting sectors:
+
<blockquote>
+
ddrescue --no-split /dev/hda1 imagefile logfile
+
</blockquote>
+
 
+
Now let it retry previous errors 3 times, using uncached reads:
+
<blockquote>
+
ddrescue --direct --max-retries=3 /dev/hda1 imagefile logfile
+
</blockquote>
+
 
+
If that fails you can try again but retrimmed, so it tries to reread full sectors:
+
<blockquote>
+
ddrescue --direct --retrim  --max-retries=3 /dev/hda1 imagefile logfile
+
</blockquote>
+
 
+
You can now use ddrescue (or normal dd) to copy the imagefile to a new partition on a new disk. Use the appropriate filesystem checkers (fsck, CHKDSK) to try to fix errors caused by the bad blocks. Be sure to keep the imagefile around. Just in case the filesystem is severely broken, and datacarving tools like testdisk need to to be used on the original image.
+
 
+
=== Before linux kernel 2.6.3 / 2.4.x ===
+
In 2.6.3 the 'raw device' has been marked obsolete. On later kernels ddrescue will use O_DIRECT on the input to do uncached reads.
+
 
+
First you copy as much data as possible, without retrying or splitting sectors:
+
<blockquote>
+
ddrescue --no-split /dev/hda1 imagefile logfile
+
</blockquote>
+
 
+
Now change over to raw device access. Let it retry previous errors 3 times, don't read past last block in logfile:
+
<blockquote>
+
modprobe raw<br>
+
raw /dev/raw/raw1 /dev/hda1<br>
+
ddrescue --max-retries=3 --complete-only /dev/raw/raw1 imagefile logfile
+
</blockquote>
+
 
+
If that fails you can try again (still using raw) but retrimmed, so it tries to reread full sectors:
+
<blockquote>
+
ddrescue --retrim --max-retries=3 --complete-only /dev/raw/raw1 imagefile logfile
+
</blockquote>
+
 
+
You can now use ddrescue (or normal dd) to copy the imagefile to a new partition on a new disk. Use the appropriate filesystem checkers (fsck, CHKDSK) to try to fix errors caused by the bad blocks. Be sure to keep the imagefile around. Just in case the filesystem is severely broken, and datacarving tools like testdisk need to to be used on the original image.
+
 
+
At the end you may want to unbind the raw device:
+
<blockquote>
+
raw /dev/raw/raw1 0 0
+
</blockquote>
+
 
+
== Examples ==
+
 
+
These two examples are taken directly from the [[ddrescue]] info pages.
+
 
+
Example 1: Rescue an ext2 partition in /dev/hda2 to /dev/hdb2
+
 
+
'''Please Note:''' This will overwrite ALL data on the partition you are copying to. If you do not want to do that, rather create an image of the partition to be rescued.
+
<blockquote>
+
ddrescue -r3 /dev/hda2 /dev/hdb2 logfile<br>
+
e2fsck -v -f /dev/hdb2<br>
+
mount -t ext2 -o ro /dev/hdb2 /mnt<br>
+
</blockquote>
+
 
+
Example 2: Rescue a CD-ROM in /dev/cdrom
+
<blockquote>
+
ddrescue -b 2048 /dev/cdrom cdimage logfile
+
</blockquote>
+
write cdimage to a blank CD-ROM
+
 
+
 
+
This example is derived from the ddrescue manual.
+
 
+
Example 3: Rescue an entire hard disk /dev/sda to another disk /dev/sdb
+
 
+
copy the error free areas first
+
ddrescue -n /dev/sda /dev/sdb rescue.log
+
attempt to recover any bad sectors
+
ddrescue -r 1 /dev/sda /dev/sdb rescue.log
+
 
+
== Options ==
+
 
+
-h, --help
+
    display this help and exit
+
-V, --version
+
    output version information and exit
+
-b, --block-size=<bytes>
+
    hardware block size of input device [512]  
+
-B, --binary-prefixes
+
    show binary multipliers in numbers [default SI]
+
-c, --cluster-size=<blocks>
+
    hardware blocks to copy at a time [128]
+
-C, --complete-only
+
    do not read new data beyond logfile limits
+
-d, --direct
+
    use direct disc access for input file
+
-D, --synchronous
+
    use synchronous writes for output file
+
-e, --max-errors=<n>
+
    maximum number of error areas allowed
+
-F, --fill=<types>
+
    fill given type areas with infile data (?*/-+)
+
-g, --generate-logfile
+
    generate approximate logfile from partial copy
+
-i, --input-position=<pos>
+
    starting position in input file [0]
+
-n, --no-split
+
    do not try to split or retry error areas
+
-o, --output-position=<pos>
+
    starting position in output file [ipos]
+
-q, --quiet
+
    quiet operation
+
-r, --max-retries=<n>
+
    exit after given retries (-1=infinity) [0]
+
-R, --retrim
+
    mark all error areas as non-trimmed
+
-s, --max-size=<bytes>
+
    maximum size of data to be copied
+
-S, --sparse
+
    use sparse writes for output file
+
-t, --truncate
+
    truncate output file
+
-v, --verbose
+
    verbose operation
+
 
+
Numbers may be followed by a multiplier: b = blocks, k = kB = 10^3 = 1000, Ki = KiB = 2^10 = 1024, M = 10^6, Mi = 2^20, G = 10^9, Gi = 2^30, etc...
+
 
+
 
+
== Cygwin ==
+
 
+
As of release 1.4-rc1, it can be compiled directly in [[Cygwin]] [http://en.wikipedia.org/wiki/Out_of_the_box Out of the Box]. Precompiled packages are available in the [http://cygwin.com/packages/ Cygwin distribution]. This makes it usable natively on [[Windows]] systems.
+
 
+
== See also ==
+
 
+
* [[aimage]]
+
* [[Blackbag]]
+
* [[dcfldd]]
+
* [[dd]]
+
* [[dd_rescue]]
+
* [[sdd]]
+
 
+
== Other Resources ==
+
[http://pfuender.net/?p=80|Useful code-snippets for DDrescue]
+

Revision as of 08:16, 9 April 2007

EnCase uses a proprietary format for images which is reportedly based on ASR Data's Expert Witness Compression Format. (Source?) The evidence files, or E01 files, contain a physical bitstream of an acquired disk, prefixed with a '"Case Info" header, interlaced with CRCs for every block of 64 sectors~(32 KB), and followed by a footer containing an MD5 hash for the entire bitstream. Contained in the header are the date and time of acquisition, an examiner's name, notes on the acquisition, and an optional password; the header concludes with its own CRC.

Encase can store media data into multiple evidence files, which are called segment files. Each segment file consist of multiple sections. Each section consist of a section start definition. This contains a section type.

At least from Encase 3 the case info header is contained in the "header" section, which is defined twice within the file and contain the same information.

With Encase 4 an additional "header2" section was added. The "header" section now appears only once, but the new "header2" section twice.

Version 3 of The Encase F introduced an "error2" sections that it uses to record the location and number of bad sector chunks. The way it handles the sections it can't read is that those areas are filled with zero. Then Encase displays to the user the areas that could not be read when the image was acquired. The granularity of unreadable chunks appears to be 32K.

Within Encase 5 the amount of sectors per block (chunk) can vary.

Encase from at least in version 3, 4 and 5 can hash the data of the media it acquires. It does this by calculating a MD5 hash of the original media data and adds a hash section to the last of the segment files.

See Also

EnCase

External Links