Difference between pages "File Format Identification" and "Data Compass"

From ForensicsWiki
(Difference between pages)
Jump to: navigation, search
m (Bibliography)
 
(What can Data Compass do?)
 
Line 1: Line 1:
File Format Identification is the process of figuring out the format of a sequence of bytes. Operating systems typically do this by file extension or by embedded MIME information. Forensic applications need to identify file types by content.
+
== Overview ==
  
=Tools=
+
According to our pioneer 3+1 Data Recovery process, the stage following the drive restoration will be data recovery using Data Compass.
==libmagic==
+
* Written in C.
+
* Rules in /usr/share/file/magic and compiled at runtime.
+
* Powers the Unix “file” command, but you can also call the library directly from a C program.
+
* http://sourceforge.net/projects/libmagic
+
  
==DROID==
+
After bringing the failed drive back to life, as a data recovery professional, you know now you need to recover the damaged file system by using file system recovery software; and maybe you know also you need to do a disk imaging in order to work from an accurate, stable hard drive image.
* Writen in Java
+
* Developed by National Archives of the United Kingdom.
+
* http://droid.sourceforge.net
+
  
==TrID==
+
Unfortunately, traditional disk imaging tools and methods are designed for and work on good HDDs only, not the patient HDDs that are unstable or inaccessible because of media defects and instable head, which are common challenges of Stage 2 in practice. Even more, with those traditional imaging tools, the time involved and the ordinary user-level repeated-read access to the media bring a risk of damaging the disk and head, making data lost irretrievable.
* XML config file
+
* Closed source; free for non-commercial use
+
* http://mark0.net/soft-trid-e.html
+
  
==Stellent/Oracle Outside-In==
+
But now there's a better way: The disk probing equipment included in the Data Compass suite bypassing the disk-level problems such as multiple bad sectors, damaged surfaces, malfunctioning head assembly, or corrupted servo info, In the meantime you can use the default software or any other defined software you have been familiar with (R-studio, Winhex, any) to perform file recovery. Through the Data Compass, problem drives will become intact hard drives and ready for file recovery attempts..
* Proprietary but free demo.
+
* http://www.oracle.com/technology/products/content-management/oit/oit_all.html
+
  
[[Category:Tools]]
 
  
=Bibliography=
+
== What can Data Compass do? ==
  
;2001
 
  
Current research papers on the file format identification problem. Most of these papers concern themselves with identifying file format of a few file sectors, rather than an entire file.  '''Please note that this bibliography is in chronological order!'''
+
Data recovery from physically damaged HDDs is what Data Compass designed for.
  
* Mason McDaniel, [[Media:Mcdaniel01.pdf|Automatic File Type Detection Algorithm]], Masters Thesis, James Madison University,2001
+
    * Data recovery from HDDs with severe multiple BAD sectors, which appear because of platter surface scratch or malfunction or instability of the magnetic head assembly (MHA).
  
; 2003
+
    * Data recovery from HDDs that start to produce "clicking" sounds, which may be caused by corruption of sector servo labels or a MHA malfunction. If some heads or surfaces are damaged it is possible (before installation of MHA replacement) to create a copy of data using the remaining good surfaces or drive heads.
  
* [http://www2.computer.org/portal/web/csdl/abs/proceedings/hicss/2003/1874/09/187490332a.pdf Content Based File Type Detection Algorithms], Mason McDaniel and M. Hossain Heydari, 36th Annual Hawaii International Conference on System Sciences (HICSS'03) - Track 9, 2003.
+
    * Availability of tools for logical analysis of FAT and NTFS file systems in the software complex allows data recovery in cases, when a drive is functional and only logical data structure is corrupted.
  
; 2005
+
    * When used with malfunctioning drives, Data Compass complex often allows selective extraction of data necessary to your customers without reading all data from a drive ("recover data by file" without creating a complete disk image) saving a lot of time. In some cases, when drive malfunctions cause constant self-damage (like scratches on disks or instable MHA) these are the only means to accomplish this task. With the ShadowDisk technology adopted, users need not to worry about the drive degradation problem.
  
* [http://www1.cs.columbia.edu/ids/publications/FilePrintPaper-revised.pdf Fileprints: identifying file types by n-gram analysis], LiWei-Jen, Wang Ke, Stolfo SJ, Herzog B..,  IProceeding of the 2005 IEEE workshop on information assurance; 2005 [http://www.itoc.usma.edu/workshop/2005/Papers/Follow%20ups/FilePrintPresentation-final.pdf [slides]]
+
== Related links ==
  
* [http://www.micsymposium.org/mics_2005/papers/paper7.pdf File Type Detection Technology], Douglas J. Hickok, Daine Richard Lesniak, Michael C. Rowe, 2005 Midwest Instruction and Computing Symposium.
+
[http://www.salvationdata.com Official Webiste]
 
+
; 2006
+
 
+
* [http://ieeexplore.ieee.org/iel5/10992/34632/01652088.pdf  File type identification of data fragments by their binary structure. ], Karresand Martin, Shahmehri Nahid. Proceedings of the IEEE workshop on information assurance; 2006. p. 140–7. [http://www.itoc.usma.edu/workshop/2006/Program/Presentations/IAW2006-07-3.pdf [slides]]
+
 
+
* [http://www.mantechcfia.com/SlidingWindowMeasurementforFileTypeIdentification.pdf Sliding Window Measurement for File Type Identification], Gregory A. Hall, Ph.D., Computer Forensics and Intrusion Analysis Group, ManTech Security and Mission Assurance, 2006
+
 
+
* FORSIGS; Forensic Signature Analysis of the Hard Drive for Multimedia File Fingerprints, John Haggerty and Mark Taylor, IFIP TC11 International Information Security Conference, 2006, Sandton, South Africa.
+
 
+
* Oscar -- Using Byte Pairs to Find File Type and Camera Make of Data Fragments, Martin Karresand , Nahid Shahmehri
+
Annual Workshop on Digital Forensics and Incident Analysis ( 2006 : Pontypridd, Wales, UK ) , s. 85 - 94, London, UK : Springer-Verlag, 2006
+
 
+
; 2007
+
 
+
* Karresand M., Shahmehri N., [http://dx.doi.org/10.1007/0-387-33406-8_35 Oscar: File Type Identification of Binary Data in Disk Clusters and RAM Pages], Proceedings of IFIP International Information Security Conference: Security and Privacy in Dynamic Environments (SEC2006), Springer, ISBN 0-387-33405-x, pp 413-424, May 22 - 24, Karlstad, Sweden. 
+
 
+
* "Identification and Localization of Data Types within Large-Scale File Systems," Robert F. Erbacher and John Mulholland,, Proceedings of the 2nd International Workshop on Systematic Approaches to Digital Forensic Engineering, Seattle, WA, April 2007,
+
 
+
* [https://www.cerias.purdue.edu/tools_and_resources/bibtex_archive/archive/2007-19.pdf Using Artificial Neural Networks for Forensic File Type Identification], Ryan M. Harris, Master's Thesis, Purdue University, May 2007
+
 
+
* [http://www.dfrws.org/2008/proceedings/p14-calhoun.pdf Predicting the Types of File Fragments], William Calhoun, Drue Coles, DFRWS 2008 [http://www.dfrws.org/2008/proceedings/p14-calhoun_pres.pdf [slides]]
+
 
+
* [http://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=04545366 SÁDI – Statistical Analysis for Data type Identification], Sarah J. Moody and Robert F. Erbacher, 3rd International Workshop on Systematic Approaches to Digital Forensic Engineering, Third International Workshop on Systematic Approaches to Digital Forensic Engineering, 2008]
+
 
+
[[Category:Bibliographies]]
+

Revision as of 07:35, 16 July 2009

Overview

According to our pioneer 3+1 Data Recovery process, the stage following the drive restoration will be data recovery using Data Compass.

After bringing the failed drive back to life, as a data recovery professional, you know now you need to recover the damaged file system by using file system recovery software; and maybe you know also you need to do a disk imaging in order to work from an accurate, stable hard drive image.

Unfortunately, traditional disk imaging tools and methods are designed for and work on good HDDs only, not the patient HDDs that are unstable or inaccessible because of media defects and instable head, which are common challenges of Stage 2 in practice. Even more, with those traditional imaging tools, the time involved and the ordinary user-level repeated-read access to the media bring a risk of damaging the disk and head, making data lost irretrievable.

But now there's a better way: The disk probing equipment included in the Data Compass suite bypassing the disk-level problems such as multiple bad sectors, damaged surfaces, malfunctioning head assembly, or corrupted servo info, In the meantime you can use the default software or any other defined software you have been familiar with (R-studio, Winhex, any) to perform file recovery. Through the Data Compass, problem drives will become intact hard drives and ready for file recovery attempts..


What can Data Compass do?

Data recovery from physically damaged HDDs is what Data Compass designed for.

   * Data recovery from HDDs with severe multiple BAD sectors, which appear because of platter surface scratch or malfunction or instability of the magnetic head assembly (MHA).
   * Data recovery from HDDs that start to produce "clicking" sounds, which may be caused by corruption of sector servo labels or a MHA malfunction. If some heads or surfaces are damaged it is possible (before installation of MHA replacement) to create a copy of data using the remaining good surfaces or drive heads.
   * Availability of tools for logical analysis of FAT and NTFS file systems in the software complex allows data recovery in cases, when a drive is functional and only logical data structure is corrupted.
   * When used with malfunctioning drives, Data Compass complex often allows selective extraction of data necessary to your customers without reading all data from a drive ("recover data by file" without creating a complete disk image) saving a lot of time. In some cases, when drive malfunctions cause constant self-damage (like scratches on disks or instable MHA) these are the only means to accomplish this task. With the ShadowDisk technology adopted, users need not to worry about the drive degradation problem.

Related links

Official Webiste