Difference between pages "Tools:Data Recovery" and "Internet Explorer History File Format"

From ForensicsWiki
(Difference between pages)
Jump to: navigation, search
(Data Recovery)
 
(URL Records)
 
Line 1: Line 1:
= Partition Recovery =
+
{{Expand}}
 +
[[Internet Explorer]] stores the web browsing history in a file called <tt>index.dat</tt>. The file contains multiple records.
  
*[http://www.ptdd.com/index.htm Partition Table Doctor]
+
== File Locations ==
: Recover deleted or lost partitions (FAT16/FAT32/NTFS/NTFS5/EXT2/EXT3/SWAP).
+
  
*[http://www.diskinternals.com/ntfs-recovery/ NTFS Recovery]
+
Internet Explorer history files keep a record of URLs that the browser has visited, cookies that were created by these sites, and any temporary internet files that were downloaded by the site visit. As a result, Internet Explorer history files are kept in several locations. Regardless of the information stored in the file, the file is named index.dat.
: DiskInternals NTFS Recovery is a fully automatic utility that recovers data from damaged or formatted disks.
+
  
*[http://www.stud.uni-hannover.de/user/76201/gpart/ gpart]
+
On Windows 95/98 these files were located in the following locations:
: Gpart is a tool which tries to guess the primary partition table of a PC-type hard disk in case the primary partition table in sector 0 is damaged, incorrect or deleted.
+
<tt>%systemdir%\Temporary Internet Files\Content.ie5
 +
%systemdir%\Cookies
 +
%systemdir%\History\History.ie5</tt>
  
*[http://www.cgsecurity.org/wiki/TestDisk TestDisk]
+
On Windows 2000/XP the file locations have changed:
: [[TestDisk]] is an OpenSource software and is licensed under the GNU Public License (GPL).  
+
<tt>%systemdir%\Documents and Settings\%username%\Local Settings\Temporary Internet Files\Content.ie5
 +
%systemdir%\Documents and Settings\%username%\Cookies
 +
%systemdir%\Documents and Settings\%username%\Local Settings\History\history.ie5</tt>
  
*[http://www.stellarinfo.com/partition-recovery.htm Partition Recovery Software]
+
Internet Explorer also keeps daily, weekly, and monthly history logs that will be located in subfolders of %systemdir%\Documents and Settings\%username%\Local Settings\History\history.ie5. The folders will be named <tt>MSHist<two-digit number><starting four-digit year><starting two-digit month><starting two-digit day><ending four-digit year><ending two-digit month><ending two-digit day></tt>. For example, the folder containing data from March 26, 2008 to March 27, 2008 might be named <tt>MSHist012008032620080327</tt>.
: Partition Recovery software for NTFS & FAT system that examines lost windows partition of damaged and corrupted hard drive.
+
  
== See Also ==
+
== File Header ==
 +
Every version of Internet Explorer since Internet Explorer 5 has used the same structure for the file header and the individual records.  Internet Explorer history files begin with:
 +
43 6c 69 65 6e 74 20 55 72 6c 43 61 63 68 65 20 4d 4d 46 20 56 65 72 20 35 2e 32
 +
Which represents the ascii string "Client UrlCache MMF Ver 5.2"
  
* [http://support.microsoft.com/?kbid=166997 Using Norton Disk Edit to Backup Your Master Boot Record]
+
The next field in the file header starts at byte offset 28 and is a four byte representation of the file size. The number will be stored in [[endianness | little-endian]] format so the numbers must actually be reversed to calculate the value.
  
== Notes ==
+
Also of interest in the file header is the location of the cache directories.  In the URL records the cache directories are given as a number, with one representing the first cache directory, two representing the second and so on.  The names of the cache directories are kept at byte offset 64 in the file.  Each directory entry is 12 bytes long of which the first eight bytes contain the directory name.
  
* "fdisk /mbr" restores the boot code in the [[Master Boot Record]], but not the partition itself. On newer versions of Windows you should use fixmbr, bootrec, mbrfix, or [[MBRWizard]]. You can also extract a copy of the specific standard MBR code from tools like bootrec.exe and diskpart.exe in Windows (from various offsets) and copy it to disk with dd (Use bs=446 count=1). For Windows XP SP2 c:\%WINDIR%\System32\diskpart.exe the MBR code is found between offset 1b818h and 1ba17h.
+
== Record Formats ==
  
= Data Recovery =
+
=== URL Records ===
The term "Data Recovery" is frequently used to mean forensic recovery, but the term really should be used for recovering data from damaged media.
+
  
*[http://www.salvationdata.com/data-recovery-equipment/hd-doctor.htm HD Doctor Suite]
+
These records indicate URIs that were actually requested. They contain the location and additional data like the web server's HTTP response. They begin with the header, in hexadecimal:
: HD Doctor Suite is a set of professional tools used to fix firmware problem
+
  
*[http://www.salvationdata.com SalvationDATA]
+
<pre>55 52 4C 20</pre>
: Claims to have a program that can read the "bad blocks" of Maxtor drives with proprietary commands.
+
This corresponds to the string <tt>URL</tt> followed by a space.
  
*[http://www.toolsthatwork.com/bringback.htm BringBack]
+
The definition for the structure in C99 format:
: BringBack offers easy to use, inexpensive, and highly successful data recovery for Windows and Linux (ext2) operating systems and digital images stored on memory cards, etc.
+
  
*[http://www.runtime.org/raid.htm RAID Reconstructor]
+
<pre>typedef struct _URL_RECORD_HEADER {
: Runtime Software's RAID Reconstructor will reconstruct RAID Level 0 (Striping) and RAID Level 5 drives.
+
  /* 000 */ char        Signature[4];
 +
  /* 004 */ uint32_t    Length;
 +
  /* 008 */ FILETIME    LastModified;
 +
  /* 010 */ FILETIME    LastAccessed;
 +
  /* 018 */ FATTIME    Expires;
 +
  /* 01c */
 +
  // Not finished yet
 +
} URL_RECORD_HEADER;</pre>
  
* [http://www.e-rol.com/en/ e-ROL]
+
<pre>
: Erol allows you to recover through the internet files erased by mistake. Recover your files online for free.
+
typedef struct _FILETIME {
 +
  /* 000 */ uint32_t    lower;
 +
  /* 004 */ uint32_t    upper;
 +
} FILETIME;</pre>
  
* [http://www.recuva.com/ Recuva]
+
<pre>
: Recuva is a freeware Windows tool that will recover accidentally deleted files.
+
typedef struct _FATTIME {
 +
  /* 000 */ uint16_t    date;
 +
  /* 002 */ uint16_t    time;
 +
} FATTIME;</pre>
  
* [http://www.snapfiles.com/get/restoration.html Restoration]
+
The Length field is represented by four bytes that give the number of 128 byte blocks that make up the URL record. Therefore, a length of <pre>05 00 00 00</pre> would indicate five blocks (because the number is stored in little-endian format) of 128 bytes for a total record length of 640 bytes.
: Restoration is a freeware Windows software that will allow you to recover deleted files
+
  
* [http://www.undelete-plus.com/ Undelete Plus]
+
The actual interpretation of the "LastModified" and "LastAccessed" fields depends on the type of history file in which the record is contained. As a matter of fact, Internet Explorer uses three different types of history files, namely Daily History, Weekly History, and Main History. Other "index.dat" files are used to store cached copies of visited pages and cookies.
: Undelete Plus is a free deleted file recovery tool that works for all versions of Windows (95-Vista), FAT12/16/32, NTFS and NTFS5 filesystems and can perform recovery on various solid state devices.
+
The information concerning how to intepret the dates of these different files can be found on Capt. Steve Bunting's web page at the University of Delaware Computer Forensics Lab (http://128.175.24.251/forensics/default.htm).
 +
Please be aware that most free and/or open source index.dat parsing programs, as well as quite a few commercial forensic tools, are not able to correctly interpret the above dates. More specifically, they interpret all the time and dates as if the records were contained into a Daily History file regardless of the actual type of the file they are stored in.
  
* [http://www.data-recovery-software.net/ R-Studio]
+
=== REDR Records ===
: R-Studio is a data recovery software suite that can recover files from FAT(12-32), NTFS, NTFS 5, HFS/HFS+, FFS, UFS/UFS2 (*BSD, Solaris), Ext2/Ext3 (Linux) and so on.
+
REDR records are very simple records. They simply indicate that the browser was redirected to another site. REDR records always start with the string REDR (0x52 45  44 52).  The next four bytes are the size of the record in little endian format.  The size will indicate the number 128 byte blocks.
  
* [http://www.stellarinfo.com/ Stellar Phoenix]
+
At offset 8 from the start of the REDR record is an unknown data field. It has been confirmed that this is not a date field.
: Data recovery software services & tools to recover lost data from hard drive.
+
  
* [http://www.deepspar.com/ DeepSpar Disk Imager]
+
16 bytes into the REDR record is the URL that was visited in a null-terminated string.  After the URL, the REDR record appears to be padded with zeros until the end of the 128 byte block.
: DeepSpar Disk Imager is a dedicated disk imaging device built to handle disk-level problems and to recover bad sectors on a hard drive.
+
  
* [http://digital-assembly.com/products/adroit-photo-recovery/ Adroit Photo Recovery]
+
=== HASH Records ===
: Adroit Photo Recovery is a photo recovery tool that uses validated carving and is able to recover fragmented photos. Adroit Photo Recovery is able
+
: to recover high definition RAW images from Canon, Nikon etc.
+
  
See also [[Data Recovery Stories]]
+
=== LEAK Records ===
  
=Carving=
+
== External Links ==
*[http://www.datalifter.com/products.htm DataLifter® - File Extractor Pro]
+
: Data carving runs on multiple threads to make use of modern processors
+
  
*[http://www.simplecarver.com/ Simple Carver Suite]
+
* [http://www.cqure.net/wp/?page_id=18 IEHist program for reading index.dat files]
: Simple Carver Suite is a collection of unique tools designed for a number of purposes including data recovery, forensic computing and eDiscovery. The suite was originally designed for data recovery and has since expanded to include unique file decoding, file identification and file classification.  
+
* [http://www.milincorporated.com/a3_index.dat.html What is in Index.dat files]
 +
* [http://www.foundstone.com/us/pdf/wp_index_dat.pdf Detailed analysis of index.dat file format]
 +
* [http://downloads.sourceforge.net/sourceforge/libmsiecf/MSIE_Cache_File_format.pdf MSIE Cache File (index.dat) format specification]
  
*[http://foremost.sourceforge.net/ Foremost]
+
[[Category:File Formats]]
: Foremost is a console program to recover files based on their headers, footers, and internal data structures.
+
 
+
*[http://www.digitalforensicssolutions.com/Scalpel/ Scalpel]
+
: Scalpel is a fast file carver that reads a database of header and footer definitions and extracts matching files from a set of image files or raw device files. Scalpel is filesystem-independent and will carve files from FATx, NTFS, ext2/3, or raw partitions.
+
 
+
*[[EnCase]]
+
: EnCase comes with some enScripts that will do carving.
+
 
+
*[[CarvFs]]
+
: A virtual file system (fuse) implementation that can provide carving tools with the possibility to do recursive multi tool zero-storage carving (also called in-place carving). Patches and scripts for scalpel and foremost are provided. Works on raw and encase images.
+
 
+
*[[LibCarvPath]]
+
: A shared library that allows carving tools to use zero-storage carving on carvfs virtual files.
+
 
+
*[http://www.cgsecurity.org/wiki/PhotoRec PhotoRec]
+
: PhotoRec is file data recovery software designed to recover lost files including video, documents and archives from Hard Disks and CDRom and lost pictures (thus, its 'Photo Recovery' name) from digital camera memory.
+
 
+
*[http://www.datarescue.com/photorescue/ PhotoRescue]
+
: Datarescue PhotoRescue Advanced is picture and photo data recovery solution made by the creators of IDA Pro. PhotoRescue will undelete, unerase and recover pictures and files lost on corrupted, erased or damaged compact flash (CF) cards, SD Cards, Memory Sticks, SmartMedia and XD cards.
+
 
+
* [https://www.uitwisselplatform.nl/projects/revit RevIt]
+
: RevIt (Revive It) is an experimental carving tool, initially developed for the DFRWS 2006 carving challenge. It uses 'file structure based carving'. Note that RevIt currently is a work in progress.
+
 
+
* [http://jbj.rapanden.dk/magicrescue/ Magic Rescue]
+
: Magic Rescue is a file carving tool that uses "magic bytes" in a file contents to recover data.
+
 
+
* [[FTK]]
+
: FTK2 includes some file carvers
+
 
+
*[[Adroit Photo Forensics]]
+
: Adroit Photo Forensics supports data carving of popular image formats. Also supports fragmented carving using [[File_Carving:SmartCarving|SmartCarving]] and [[File_Carving:GuidedCarving|GuidedCarving]].
+

Revision as of 02:59, 6 June 2009

Information icon.png

Please help to improve this article by expanding it.
Further information might be found on the discussion page.

Internet Explorer stores the web browsing history in a file called index.dat. The file contains multiple records.

File Locations

Internet Explorer history files keep a record of URLs that the browser has visited, cookies that were created by these sites, and any temporary internet files that were downloaded by the site visit. As a result, Internet Explorer history files are kept in several locations. Regardless of the information stored in the file, the file is named index.dat.

On Windows 95/98 these files were located in the following locations: %systemdir%\Temporary Internet Files\Content.ie5 %systemdir%\Cookies %systemdir%\History\History.ie5

On Windows 2000/XP the file locations have changed: %systemdir%\Documents and Settings\%username%\Local Settings\Temporary Internet Files\Content.ie5 %systemdir%\Documents and Settings\%username%\Cookies %systemdir%\Documents and Settings\%username%\Local Settings\History\history.ie5

Internet Explorer also keeps daily, weekly, and monthly history logs that will be located in subfolders of %systemdir%\Documents and Settings\%username%\Local Settings\History\history.ie5. The folders will be named MSHist<two-digit number><starting four-digit year><starting two-digit month><starting two-digit day><ending four-digit year><ending two-digit month><ending two-digit day>. For example, the folder containing data from March 26, 2008 to March 27, 2008 might be named MSHist012008032620080327.

File Header

Every version of Internet Explorer since Internet Explorer 5 has used the same structure for the file header and the individual records. Internet Explorer history files begin with:

43 6c 69 65 6e 74 20 55 72 6c 43 61 63 68 65 20 4d 4d 46 20 56 65 72 20 35 2e 32

Which represents the ascii string "Client UrlCache MMF Ver 5.2"

The next field in the file header starts at byte offset 28 and is a four byte representation of the file size. The number will be stored in little-endian format so the numbers must actually be reversed to calculate the value.

Also of interest in the file header is the location of the cache directories. In the URL records the cache directories are given as a number, with one representing the first cache directory, two representing the second and so on. The names of the cache directories are kept at byte offset 64 in the file. Each directory entry is 12 bytes long of which the first eight bytes contain the directory name.

Record Formats

URL Records

These records indicate URIs that were actually requested. They contain the location and additional data like the web server's HTTP response. They begin with the header, in hexadecimal:

55 52 4C 20

This corresponds to the string URL followed by a space.

The definition for the structure in C99 format:

typedef struct _URL_RECORD_HEADER {
  /* 000 */ char        Signature[4];
  /* 004 */ uint32_t    Length;
  /* 008 */ FILETIME    LastModified;
  /* 010 */ FILETIME    LastAccessed;
  /* 018 */ FATTIME     Expires;
  /* 01c */ 
  // Not finished yet
} URL_RECORD_HEADER;
typedef struct _FILETIME {
  /* 000 */ uint32_t    lower;
  /* 004 */ uint32_t    upper;
} FILETIME;
typedef struct _FATTIME {
  /* 000 */ uint16_t    date;
  /* 002 */ uint16_t    time;
} FATTIME;
The Length field is represented by four bytes that give the number of 128 byte blocks that make up the URL record. Therefore, a length of
05 00 00 00
would indicate five blocks (because the number is stored in little-endian format) of 128 bytes for a total record length of 640 bytes.

The actual interpretation of the "LastModified" and "LastAccessed" fields depends on the type of history file in which the record is contained. As a matter of fact, Internet Explorer uses three different types of history files, namely Daily History, Weekly History, and Main History. Other "index.dat" files are used to store cached copies of visited pages and cookies. The information concerning how to intepret the dates of these different files can be found on Capt. Steve Bunting's web page at the University of Delaware Computer Forensics Lab (http://128.175.24.251/forensics/default.htm). Please be aware that most free and/or open source index.dat parsing programs, as well as quite a few commercial forensic tools, are not able to correctly interpret the above dates. More specifically, they interpret all the time and dates as if the records were contained into a Daily History file regardless of the actual type of the file they are stored in.

REDR Records

REDR records are very simple records. They simply indicate that the browser was redirected to another site. REDR records always start with the string REDR (0x52 45 44 52). The next four bytes are the size of the record in little endian format. The size will indicate the number 128 byte blocks.

At offset 8 from the start of the REDR record is an unknown data field. It has been confirmed that this is not a date field.

16 bytes into the REDR record is the URL that was visited in a null-terminated string. After the URL, the REDR record appears to be padded with zeros until the end of the 128 byte block.

HASH Records

LEAK Records

External Links