Difference between pages "Windows Registry" and "File Carving"

From Forensics Wiki
(Difference between pages)
Jump to: navigation, search
(Bibliography)
 
m (See also)
 
Line 1: Line 1:
==File Locations==
+
'''Carving''' is the practice of searching an input for files or other kinds of objects based on content, rather than on metadata. File carving is a powerful tool for recovering files and fragments of files when directory entries are corrupt or missing, as may be the case with old files that have been deleted or when performing an analysis on damaged media. Memory carving is a useful tool for analyzing physical and virtual memory dumps when the memory structures are unknown or have been overwritten.
The Windows Registry is stored in multiple files.
+
  
===Windows NT 4 ===
 
In Windows NT 4 (and later) the Registry is stored in the [[Windows NT Registry File (REGF)]] format.
 
  
Basically the following Registry hives are stored in the corresponding files:
+
=File Carving=
* HKEY_USERS: \Documents and Setting\User Profile\NTUSER.DAT
+
* HKEY_USERS\DEFAULT: C:\Windows\system32\config\default
+
* HKEY_LOCAL_MACHINE\SAM: C:\Windows\system32\config\SAM
+
* HKEY_LOCAL_MACHINE\SECURITY: C:\Windows\system32\config\SECURITY
+
* HKEY_LOCAL_MACHINE\SOFTWARE: C:\Windows\system32\config\software
+
* HKEY_LOCAL_MACHINE\SYSTEM: C:\Windows\system32\config\system
+
  
===Windows 98/ME===
+
Most file carvers operate by looking for file headers and/or footers, and then "carving out" the blocks between these two boundaries. [[Semantic Carving]] performs carving based on an analysis of the contents of the proposed files.  
* \Windows\user.dat
+
* \Windows\system.dat
+
* \Windows\profiles\user profile\user.dat
+
  
== Keys ==
+
File carving should be done on a [[disk image]], rather than on the original disk.
  
=== Run/RunOnce ===
+
File carving tools are listed on the [[Tools:Data_Recovery]] wiki page.
System-wide:
+
<pre>
+
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Run
+
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\RunOnce
+
</pre>
+
  
Per user:
+
Many carving programs have an option to only look at or near sector boundaries where headers are found. However, searching the entire input can find files that have been embedded into other files, such as [[JPEG]]s being embedded into [[Microsoft]] [[DOC|Word documents]]. This may be considered an advantage or a disadvantage, depending on the circumstances.
<pre>
+
HKEY_CURRENT_USER\SOFTWARE\Microsoft\Windows\CurrentVersion\Run
+
HKEY_CURRENT_USER\SOFTWARE\Microsoft\Windows\CurrentVersion\RunOnce
+
</pre>
+
  
== Special cases ==
+
Today most file carving programs will only recover files that are contiguous on the media.
The Windows Registry has several special case scenarios, mainly concerning key and value name, that are easy to fail to account for:
+
* special characters key and value names
+
* duplicate key and value names
+
* the names when stored in extended ASCII (ANSI string) use a codepage that is dependent on the system settings
+
  
=== special characters key and value names ===
+
== File Carving Taxonomy==
Both key and values names are case insensitive. The \ character is used as the key separator. Note
+
[[Simson Garfinkel]] and [[Joachim Metz]] have proposed the following file carving taxonomy:
that the \ character can be used in value names. The / character is used in both key and value names.
+
Some examples of which are:
+
<pre>
+
Key: HKEY_LOCAL_MACHINE\SYSTEM\ControlSet002\Services\NetBT\Parameters\
+
Value: Size/Small/Medium/Large
+
</pre>
+
  
<pre>
+
;Carving
Key: HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\Control\Terminal Server\VIDEO\disc\
+
:General term for extracting data (files) out of undifferentiated blocks (raw data), like "carving" a sculpture out of soap stone.
Value: \Device\Video0
+
</pre>
+
  
<pre>
+
;Block Based Carving
Key:
+
:Any carving method (algorithm) that analyzes the input on block-by-block basis to determine if a block is part of a possible output file. This method assumes that each block can only be part of a single file (or embedded file).
HKEY_LOCAL_MACHINE\SYSTEM\ControlSet002\services\xmlprov\Parameters\SchemaGroups\User\http://www.microsoft.com/provisioning/eaptlsuserpropertiesv1\
+
Value: SchemaFile
+
</pre>
+
  
=== codepaged ASCII strings ===
+
;Characteristic Based Carving
 +
:Any carving method (algorithm) that analyzes the input on characteristic basis (for example, entropy) to determine if the input is part of a possible output file.
  
Value with name "ëigenaardig" created on Windows XP codepage 1252.
+
;Header/Footer Carving
 +
:A method for carving files out of raw data using a distinct header (start of file marker) and footer (end of file marker).
  
<pre>
+
;Header/Maximum (file) size Carving
value key data:
+
:A method for carving files out of raw data using a distinct header (start of file marker) and a maximum (file) size. This approach works because many file formats (e.g. JPEG, MP3) do not care if additional junk is appended to the end of a valid file.
00000000: 76 6b 0b 00 46 00 00 00  20 98 1a 00 01 00 00 00  vk..F...  .......
+
00000010: 01 00 69 6e eb 69 67 65  6e 61 61 72 64 69 67 00  ..in.ige naardig.
+
00000020: 55 4e 49 43                                        UNIC
+
  
value key signature                    : vk
+
;Header/Embedded Length Carving
value key value name size              : 11
+
:A method for carving files out of raw data using a distinct header and a file length (size) which is embedded in the file format
value key data size                     : 0x00000046 (70)
+
value key data offset                  : 0x001a9820
+
value key data type                    : 1 (REG_SZ) String
+
value key flags                        : 0x0001
+
        Value name is an ASCII string
+
  
value key unknown1                      : 0x6e69 (28265)
+
;File structure based Carving
value key value name                    : ëigenaardig
+
:A method for carving files out of raw data using a certain level of knowledge of the internal structure of file types. Garfinkel called this approach "Semantic Carving" in his DFRWS2006 carving challenge submission, while Metz and Mora called the approach "Deep Carving."
value key value name hash              : 0xb78835ee
+
value key padding:
+
00000000: 00 55 4e 49 43                                    .UNIC
+
</pre>
+
  
As you can see the name is stored in extended ASCII (ANSI) using codepage 1252.
+
;Semantic Carving
 +
:A method for carving files based on a linguistic analysis of the file's content. For example, a semantic carver might conclude that six blocks of french in the middle of a long HTML file written in English is a fragment left from a previous allocated file, and not from the English-language HTML file.
  
==Tools==
+
;Carving with Validation
===Open Source===
+
:A method for carving files out of raw data where the carved files are validated using a file type specific validator.
* [https://www.pinguin.lu/index.php Forensic Registry EDitor (fred)] - "Forensic Registry EDitor (fred) is a cross-platform M$ registry hive editor" by [[Daniel Gillen]]
+
* [http://projects.sentinelchicken.org/data/doc/reglookup/regfi/ libregfi] - The regfi library is a read-only NT registry library which serves as the main engine behind the reglookup tool
+
* [http://projects.sentinelchicken.org/reglookup/ reglookup] — "small command line utility for reading and querying Windows NT-based registries."
+
* [http://sourceforge.net/projects/regviewer/ regviewer] — a tool for looking at the registry.
+
* [[Regripper|RegRipper]] — "the fastest, easiest, and best tool for registry analysis in forensics examinations."
+
* [http://search.cpan.org/~jmacfarla/Parse-Win32Registry-0.51/lib/Parse/Win32Registry.pm Parse::Win32Registry] Perl module.
+
* [http://www.williballenthin.com/registry/index.html python-registry] Python module.
+
* [http://code.google.com/p/registrydecoder/ Registry Decoder] offline analysis component, by [[Andrew Case]]
+
* [http://code.google.com/p/registrydecoder/ RegDecoderLive] live hive acquisition component, by [[Andrew Case]]
+
* [[libregf]] - Library and tools to access the Windows NT Registry File (REGF) format
+
* [[Registryasxml]] - Tool to import/export registry sections as XML
+
  
===Freeware===
+
;Fragment Recovery Carving
* [http://www.tzworks.net/prototype_page.php?proto_id=3 Yet Another Registry Utility (yaru)] Free tool that can be run on Windows, Linux or Mac OS-X. If run in admin mode, allows viewing of registry hives on live system.
+
:A carving method in which two or more fragments are reassembled to form the original file or object. Garfinkel previously called this approach "Split Carving."
  
* [http://www.tzworks.net/prototype_page.php?proto_id=14 Windows ShellBag Parser] Free tool that can be run on Windows, Linux or Mac OS-X.
+
== File Carving challenges and test images ==
  
* [http://tzworks.net/prototype_page.php?proto_id=19 ''cafae''] - Computer Account Forensic Artifact Extractor.  Free tool that can be run on Windows, Linux or Mac OS-X to parse ntuser.dat hives.
+
[http://www.dfrws.org/2006/challenge/]
 +
File Carving Challenge - [[Digital Forensic Research Workshop|DFRWS]] 2006
  
===Commercial===
+
[http://dftt.sourceforge.net/test6/index.html]
* [http://www.abexo.com/free-registry-cleaner.htm Abexo Free Regisry Cleaner]
+
FAT Undelete Test #1 - Digital Forensics Tool Testing Image (dftt #6)
* [http://www.auslogics.com/registry-defrag Auslogics Registry Defrag]
+
* [http://lastbit.com/arv/ Alien Registry Viewer]
+
* [http://www.larshederer.homepage.t-online.de/erunt/index.htm NT Registry Optimizer]
+
* [http://www.registry-clean.net/free-registry-defrag.htm iExpert Software-Free Registry Defrag]
+
* [http://arsenalrecon.com/apps Registry Recon]
+
* [http://paullee.ru/regundel Registry Undelete (russian)]
+
* [http://mitec.cz/wrr.html Windows Registry Recovery]
+
* [http://registrytool.com/ Registry Tool]
+
  
==Bibliography==
+
[http://dftt.sourceforge.net/test7/index.html]
* [http://www.dfrws.org/2009/proceedings/p69-zhu.pdf Using ShellBag Information to Reconstruct User Activities.], Yuandong Zhu*, Pavel Gladyshev, Joshua James, DFRWS 2009
+
NTFS Undelete (and leap year) Test #1 - Digital Forensics Tool Testing Image (dftt #7)
* Recovering Deleted Data From the Windows Registry. Timothy Morgan, DFRWS 2008 [http://www.dfrws.org/2008/proceedings/p33-morgan.pdf [paper]] [http://www.dfrws.org/2008/proceedings/p33-morgan_pres.pdf [slides]]
+
* [http://dfrws.org/2008/proceedings/p26-dolan-gavitt.pdf Forensic Analysis of the Windows Registry in Memory], Brendan Dolan-Gavitt, DFRWS 2008  [http://dfrws.org/2008/proceedings/p26-dolan-gavitt_pres.pdf [slides]]
+
* [http://eptuners.com/forensics/A%20Windows%20Registry%20Quick%20Reference.pdf A Windows Registry Quick-Reference], Derrick Farmer, Burlington, VT.
+
  
* [http://www.sciencedirect.com/science?_ob=ArticleURL&_udi=B7CW4-4GX1J3B-1&_user=3326500&_rdoc=1&_fmt=&_orig=search&_sort=d&view=c&_acct=C000060280&_version=1&_urlVersion=0&_userid=3326500&md5=ab887593e7be6d5257696707886978f1 The Windows Registry as a forensic resource], Digital Investigation, Volume 2, Issue 3, September 2005, Pages 201--205.
+
[http://dftt.sourceforge.net/test11/index.html]
 +
Basic Data Carving Test - fat32 (by Nick Mikus) - Digital Forensics Tool Testing Image (dftt #11)
  
* [http://www.forensicfocus.com/downloads/forensic-analysis-windows-registry.pdf Forensic Analysis of the Windows Registry], Lih Wern Wong , School of Computer and Information Science, Edith Cowan University
+
[http://dftt.sourceforge.net/test12/index.html]
 +
Basic Data Carving Test - ext2 (by Nick Mikus) - Digital Forensics Tool Testing Image (dftt #12)
  
* [http://www.sentinelchicken.com/research/registry_format/ The Windows NT Registry File Format], Timothy D. Morgan
+
==File Carving Bibliography==
  
==See Also==
+
Mikus, Nicholas A. "An analysis of disc carving techniques," Master's Thesis, Naval Postgraduate School. March 2005. http://handle.dtic.mil/100.2/ADA432468
* [http://windowsir.blogspot.com/search/label/Registry Windows Incident Response Articles on Registry]
+
* [http://www.answers.com/topic/win-registry Windows Registry Information]
+
* [http://en.wikipedia.org/wiki/Windows_Registry Wikipedia Article on Windows Registry]
+
[[Category:Bibliographies]]
+
* [http://moyix.blogspot.com/search/label/registry Push the Red Button] — Articles on Registry
+
* [http://tech.groups.yahoo.com/group/win4n6/ Windows Forensics Mailing List]
+
* [http://samba.org/~jelmer/kregedit/ kregedit] - a KDE utility for viewing and editing registry files.
+
* [http://www.bindview.com/Services/RAZOR/Utilities/Unix_Linux/ntreg_readme.cfm ntreg] a file system driver for linux, which understands the NT registry file format.
+
* [http://www.beginningtoseethelight.org/ntsecurity/ Security Accounts Manager]
+
  
* http://www.opensourceforensics.org/tools/unix.html - Open Source Forensic Tools on Brian Carrier's website.
+
Garfinkel, S., "Carving Contiguous and Fragmented Files with Fast Object Validation", Digital Forensics Workshop (DFRWS 2007), Pittsburgh, PA, August 2007.  http://www.simson.net/clips/academic/2007.DFRWS.pdf
  
[[Category:Windows Analysis]]
+
== See also ==
 +
* [[Tools:Data_Recovery#Carving | FIle Carving Tools]]
 +
* [[File Carving Bibliography]]
 +
 
 +
=Memory Carving=

Revision as of 22:56, 20 October 2008

Carving is the practice of searching an input for files or other kinds of objects based on content, rather than on metadata. File carving is a powerful tool for recovering files and fragments of files when directory entries are corrupt or missing, as may be the case with old files that have been deleted or when performing an analysis on damaged media. Memory carving is a useful tool for analyzing physical and virtual memory dumps when the memory structures are unknown or have been overwritten.


Contents

File Carving

Most file carvers operate by looking for file headers and/or footers, and then "carving out" the blocks between these two boundaries. Semantic Carving performs carving based on an analysis of the contents of the proposed files.

File carving should be done on a disk image, rather than on the original disk.

File carving tools are listed on the Tools:Data_Recovery wiki page.

Many carving programs have an option to only look at or near sector boundaries where headers are found. However, searching the entire input can find files that have been embedded into other files, such as JPEGs being embedded into Microsoft Word documents. This may be considered an advantage or a disadvantage, depending on the circumstances.

Today most file carving programs will only recover files that are contiguous on the media.

File Carving Taxonomy

Simson Garfinkel and Joachim Metz have proposed the following file carving taxonomy:

Carving
General term for extracting data (files) out of undifferentiated blocks (raw data), like "carving" a sculpture out of soap stone.
Block Based Carving
Any carving method (algorithm) that analyzes the input on block-by-block basis to determine if a block is part of a possible output file. This method assumes that each block can only be part of a single file (or embedded file).
Characteristic Based Carving
Any carving method (algorithm) that analyzes the input on characteristic basis (for example, entropy) to determine if the input is part of a possible output file.
Header/Footer Carving
A method for carving files out of raw data using a distinct header (start of file marker) and footer (end of file marker).
Header/Maximum (file) size Carving
A method for carving files out of raw data using a distinct header (start of file marker) and a maximum (file) size. This approach works because many file formats (e.g. JPEG, MP3) do not care if additional junk is appended to the end of a valid file.
Header/Embedded Length Carving
A method for carving files out of raw data using a distinct header and a file length (size) which is embedded in the file format
File structure based Carving
A method for carving files out of raw data using a certain level of knowledge of the internal structure of file types. Garfinkel called this approach "Semantic Carving" in his DFRWS2006 carving challenge submission, while Metz and Mora called the approach "Deep Carving."
Semantic Carving
A method for carving files based on a linguistic analysis of the file's content. For example, a semantic carver might conclude that six blocks of french in the middle of a long HTML file written in English is a fragment left from a previous allocated file, and not from the English-language HTML file.
Carving with Validation
A method for carving files out of raw data where the carved files are validated using a file type specific validator.
Fragment Recovery Carving
A carving method in which two or more fragments are reassembled to form the original file or object. Garfinkel previously called this approach "Split Carving."

File Carving challenges and test images

[1] File Carving Challenge - DFRWS 2006

[2] FAT Undelete Test #1 - Digital Forensics Tool Testing Image (dftt #6)

[3] NTFS Undelete (and leap year) Test #1 - Digital Forensics Tool Testing Image (dftt #7)

[4] Basic Data Carving Test - fat32 (by Nick Mikus) - Digital Forensics Tool Testing Image (dftt #11)

[5] Basic Data Carving Test - ext2 (by Nick Mikus) - Digital Forensics Tool Testing Image (dftt #12)

File Carving Bibliography

Mikus, Nicholas A. "An analysis of disc carving techniques," Master's Thesis, Naval Postgraduate School. March 2005. http://handle.dtic.mil/100.2/ADA432468

Garfinkel, S., "Carving Contiguous and Fragmented Files with Fast Object Validation", Digital Forensics Workshop (DFRWS 2007), Pittsburgh, PA, August 2007. http://www.simson.net/clips/academic/2007.DFRWS.pdf

See also

Memory Carving