Difference between pages "Windows Registry" and "SQLite database format"

From ForensicsWiki
(Difference between pages)
Jump to: navigation, search
(Bibliography)
 
(Use Cases)
 
Line 1: Line 1:
==File Locations==
+
{{expand}}
The Windows Registry is stored in multiple files.
+
  
===Windows NT 4 ===
+
SQLite databases are used by many programs including several forensics tools, e.g. [[Autopsy]] 3.
In Windows NT 4 (and later) the Registry is stored in the [[Windows NT Registry File (REGF)]] format.
+
SQLite 3 is current and older SQLite packages cannot use sqlite3 databases so use sqlite3 tools.
  
Basically the following Registry hives are stored in the corresponding files:
+
== SQLite3 ==
* HKEY_USERS: \Documents and Setting\User Profile\NTUSER.DAT
+
* HKEY_USERS\DEFAULT: C:\Windows\system32\config\default
+
* HKEY_LOCAL_MACHINE\SAM: C:\Windows\system32\config\SAM
+
* HKEY_LOCAL_MACHINE\SECURITY: C:\Windows\system32\config\SECURITY
+
* HKEY_LOCAL_MACHINE\SOFTWARE: C:\Windows\system32\config\software
+
* HKEY_LOCAL_MACHINE\SYSTEM: C:\Windows\system32\config\system
+
  
===Windows 98/ME===
+
SQLite version 3 uses a page-based storage where the pages are used for various types of data e.g. there are:
* \Windows\user.dat
+
* lock-byte pages
* \Windows\system.dat
+
* freelist pages
* \Windows\profiles\user profile\user.dat
+
** freelist trunk pages
 +
** freelist leaf pages
 +
* B-tree pages
 +
** table B-tree interior pages
 +
** table B-tree leaf pages
 +
** index B-tree interior pages
 +
** index B-tree leaf pages
 +
* payload overflow pages
 +
* pointer map pages
  
== Keys ==
+
=== Write-Ahead Log (WAL) ===
 +
The default method by which SQLite implements atomic commit and rollback is a rollback journal. In version 3.7.0 a "Write-Ahead Log" option was added.
  
=== Run/RunOnce ===
+
== Temporary sqlite files ==
System-wide:
+
Seen in e.g.
 
<pre>
 
<pre>
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Run
+
/Users/%USERNAME%/AppData/Local/Temp/etilqs_%RANDOM%
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\RunOnce
+
 
</pre>
 
</pre>
  
Per user:
+
Where "etilqs" is "sqlite" in reverse
<pre>
+
HKEY_CURRENT_USER\SOFTWARE\Microsoft\Windows\CurrentVersion\Run
+
HKEY_CURRENT_USER\SOFTWARE\Microsoft\Windows\CurrentVersion\RunOnce
+
</pre>
+
 
+
== Special cases ==
+
The Windows Registry has several special case scenarios, mainly concerning key and value name, that are easy to fail to account for:
+
* special characters key and value names
+
* duplicate key and value names
+
* the names when stored in extended ASCII (ANSI string) use a codepage that is dependent on the system settings
+
 
+
=== special characters key and value names ===
+
Both key and values names are case insensitive. The \ character is used as the key separator. Note
+
that the \ character can be used in value names. The / character is used in both key and value names.
+
Some examples of which are:
+
<pre>
+
Key: HKEY_LOCAL_MACHINE\SYSTEM\ControlSet002\Services\NetBT\Parameters\
+
Value: Size/Small/Medium/Large
+
</pre>
+
 
+
<pre>
+
Key: HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\Control\Terminal Server\VIDEO\disc\
+
Value: \Device\Video0
+
</pre>
+
 
+
<pre>
+
Key:
+
HKEY_LOCAL_MACHINE\SYSTEM\ControlSet002\services\xmlprov\Parameters\SchemaGroups\User\http://www.microsoft.com/provisioning/eaptlsuserpropertiesv1\
+
Value: SchemaFile
+
</pre>
+
 
+
=== codepaged ASCII strings ===
+
 
+
Value with name "ëigenaardig" created on Windows XP codepage 1252.
+
 
+
<pre>
+
value key data:
+
00000000: 76 6b 0b 00 46 00 00 00  20 98 1a 00 01 00 00 00  vk..F...  .......
+
00000010: 01 00 69 6e eb 69 67 65  6e 61 61 72 64 69 67 00  ..in.ige naardig.
+
00000020: 55 4e 49 43                                        UNIC
+
 
+
value key signature                    : vk
+
value key value name size              : 11
+
value key data size                    : 0x00000046 (70)
+
value key data offset                  : 0x001a9820
+
value key data type                    : 1 (REG_SZ) String
+
value key flags                        : 0x0001
+
        Value name is an ASCII string
+
 
+
value key unknown1                      : 0x6e69 (28265)
+
value key value name                    : ëigenaardig
+
value key value name hash              : 0xb78835ee
+
value key padding:
+
00000000: 00 55 4e 49 43                                    .UNIC
+
</pre>
+
 
+
As you can see the name is stored in extended ASCII (ANSI) using codepage 1252.
+
 
+
==Tools==
+
===Open Source===
+
* [https://www.pinguin.lu/index.php Forensic Registry EDitor (fred)] - "Forensic Registry EDitor (fred) is a cross-platform M$ registry hive editor" by [[Daniel Gillen]]
+
* [http://projects.sentinelchicken.org/data/doc/reglookup/regfi/ libregfi] - The regfi library is a read-only NT registry library which serves as the main engine behind the reglookup tool
+
* [http://projects.sentinelchicken.org/reglookup/ reglookup] — "small command line utility for reading and querying Windows NT-based registries."
+
* [http://sourceforge.net/projects/regviewer/ regviewer] — a tool for looking at the registry.
+
* [[Regripper|RegRipper]] — "the fastest, easiest, and best tool for registry analysis in forensics examinations."
+
* [http://search.cpan.org/~jmacfarla/Parse-Win32Registry-0.51/lib/Parse/Win32Registry.pm Parse::Win32Registry] Perl module.
+
* [http://www.williballenthin.com/registry/index.html python-registry] Python module.
+
* [http://code.google.com/p/registrydecoder/ Registry Decoder] offline analysis component, by [[Andrew Case]]
+
* [http://code.google.com/p/registrydecoder/ RegDecoderLive] live hive acquisition component, by [[Andrew Case]]
+
* [[libregf]] - Library and tools to access the Windows NT Registry File (REGF) format
+
* [[Registryasxml]] - Tool to import/export registry sections as XML
+
 
+
===Freeware===
+
* [http://www.tzworks.net/prototype_page.php?proto_id=3 Yet Another Registry Utility (yaru)] Free tool that can be run on Windows, Linux or Mac OS-X. If run in admin mode, allows viewing of registry hives on live system.
+
 
+
* [http://www.tzworks.net/prototype_page.php?proto_id=14 Windows ShellBag Parser] Free tool that can be run on Windows, Linux or Mac OS-X.
+
 
+
* [http://tzworks.net/prototype_page.php?proto_id=19 ''cafae''] - Computer Account Forensic Artifact Extractor.  Free tool that can be run on Windows, Linux or Mac OS-X to parse ntuser.dat hives.
+
 
+
===Commercial===
+
* [http://www.abexo.com/free-registry-cleaner.htm Abexo Free Regisry Cleaner]
+
* [http://www.auslogics.com/registry-defrag Auslogics Registry Defrag]
+
* [http://lastbit.com/arv/ Alien Registry Viewer]
+
* [http://www.larshederer.homepage.t-online.de/erunt/index.htm NT Registry Optimizer]
+
* [http://www.registry-clean.net/free-registry-defrag.htm iExpert Software-Free Registry Defrag]
+
* [http://arsenalrecon.com/apps Registry Recon]
+
* [http://paullee.ru/regundel Registry Undelete (russian)]
+
* [http://mitec.cz/wrr.html Windows Registry Recovery]
+
* [http://registrytool.com/ Registry Tool]
+
 
+
==Bibliography==
+
* [http://www.dfrws.org/2009/proceedings/p69-zhu.pdf Using ShellBag Information to Reconstruct User Activities.], Yuandong Zhu*, Pavel Gladyshev, Joshua James, DFRWS 2009
+
* Recovering Deleted Data From the Windows Registry. Timothy Morgan, DFRWS 2008 [http://www.dfrws.org/2008/proceedings/p33-morgan.pdf [paper]] [http://www.dfrws.org/2008/proceedings/p33-morgan_pres.pdf [slides]]
+
* [http://dfrws.org/2008/proceedings/p26-dolan-gavitt.pdf Forensic Analysis of the Windows Registry in Memory], Brendan Dolan-Gavitt, DFRWS 2008  [http://dfrws.org/2008/proceedings/p26-dolan-gavitt_pres.pdf [slides]]
+
* [http://eptuners.com/forensics/A%20Windows%20Registry%20Quick%20Reference.pdf A Windows Registry Quick-Reference], Derrick Farmer, Burlington, VT.
+
 
+
* [http://www.sciencedirect.com/science?_ob=ArticleURL&_udi=B7CW4-4GX1J3B-1&_user=3326500&_rdoc=1&_fmt=&_orig=search&_sort=d&view=c&_acct=C000060280&_version=1&_urlVersion=0&_userid=3326500&md5=ab887593e7be6d5257696707886978f1 The Windows Registry as a forensic resource], Digital Investigation, Volume 2, Issue 3, September 2005, Pages 201--205.
+
  
* [http://www.forensicfocus.com/downloads/forensic-analysis-windows-registry.pdf Forensic Analysis of the Windows Registry], Lih Wern Wong , School of Computer and Information Science, Edith Cowan University
+
== Use Cases ==
 +
=== Web Browser Data ===
 +
[[Mozilla Firefox]] and [[Google Chrome]] both use SQLite version 3 databases for user data such as history, downloaded files.
  
* [http://www.sentinelchicken.com/research/registry_format/ The Windows NT Registry File Format], Timothy D. Morgan
+
=== Mobile OS ===
 +
[[Google Android]] and [[Apple iOS]] use SQLite3 databases for many system applications. Phone data including calls, messages, and credentials are all stored in SQLite3.
  
==See Also==
+
== External Links ==
* [http://windowsir.blogspot.com/search/label/Registry Windows Incident Response Articles on Registry]
+
* [http://sqlite.org/fileformat2.html The SQLite Database File Format], by the [[SQLite|SQLite project]]
* [http://www.answers.com/topic/win-registry Windows Registry Information]
+
* [http://sqlite.org/wal.html Write-Ahead Logging], by the [[SQLite|SQLite project]]
* [http://en.wikipedia.org/wiki/Windows_Registry Wikipedia Article on Windows Registry]
+
* [http://forensicsfromthesausagefactory.blogspot.com/2011/04/carving-sqlite-databases-from.html Carving SQLite databases from unallocated clusters], by Richard Drinkwater, April 27, 2011
[[Category:Bibliographies]]
+
* [http://linuxsleuthing.blogspot.ch/2013/09/recovering-data-from-deleted-sqlite.html Recovering Data from Deleted SQLite Records: Redux], by [[John Lehr]], September 13, 2013
* [http://moyix.blogspot.com/search/label/registry Push the Red Button] — Articles on Registry
+
* [http://tech.groups.yahoo.com/group/win4n6/ Windows Forensics Mailing List]
+
* [http://samba.org/~jelmer/kregedit/ kregedit] - a KDE utility for viewing and editing registry files.
+
* [http://www.bindview.com/Services/RAZOR/Utilities/Unix_Linux/ntreg_readme.cfm ntreg] a file system driver for linux, which understands the NT registry file format.
+
* [http://www.beginningtoseethelight.org/ntsecurity/ Security Accounts Manager]
+
  
* http://www.opensourceforensics.org/tools/unix.html - Open Source Forensic Tools on Brian Carrier's website.
+
== Tools ==
 +
* [[SQLite]]
 +
* [[SQLite Forensic Reporter]]
  
[[Category:Windows Analysis]]
+
[[Category:File Formats]]

Revision as of 11:15, 26 June 2014

Information icon.png

Please help to improve this article by expanding it.
Further information might be found on the discussion page.

SQLite databases are used by many programs including several forensics tools, e.g. Autopsy 3. SQLite 3 is current and older SQLite packages cannot use sqlite3 databases so use sqlite3 tools.

SQLite3

SQLite version 3 uses a page-based storage where the pages are used for various types of data e.g. there are:

  • lock-byte pages
  • freelist pages
    • freelist trunk pages
    • freelist leaf pages
  • B-tree pages
    • table B-tree interior pages
    • table B-tree leaf pages
    • index B-tree interior pages
    • index B-tree leaf pages
  • payload overflow pages
  • pointer map pages

Write-Ahead Log (WAL)

The default method by which SQLite implements atomic commit and rollback is a rollback journal. In version 3.7.0 a "Write-Ahead Log" option was added.

Temporary sqlite files

Seen in e.g.

/Users/%USERNAME%/AppData/Local/Temp/etilqs_%RANDOM%

Where "etilqs" is "sqlite" in reverse

Use Cases

Web Browser Data

Mozilla Firefox and Google Chrome both use SQLite version 3 databases for user data such as history, downloaded files.

Mobile OS

Google Android and Apple iOS use SQLite3 databases for many system applications. Phone data including calls, messages, and credentials are all stored in SQLite3.

External Links

Tools