Difference between pages "Tools:Visualization" and "Windows Registry"

From ForensicsWiki
(Difference between pages)
Jump to: navigation, search
m (See Also)
 
(Bibliography)
 
Line 1: Line 1:
Although not strictly for forensic purposes, '''visualization tools''' such as the ones discussed here can be very useful for visualizing large data sets. As forensic practitioners need to process more and more data, it is likely that some of the techniques implemented by these tools will need to be adopted.
+
==File Locations==
 +
The Windows Registry is stored in multiple files.
  
==Programming Languages and Developer Toolkits==
+
===Windows NT 4 ===
If you are building forensic tools, you probably want to start with one of these:
+
In Windows NT 4 (and later) the Registry is stored in the [[Windows NT Registry File (REGF)]] format.
; Java and Swing
+
: Advantage: Portable and lots of good documentation out there.
+
: Disadvantage: Programs are a bit verbose, and only offers about 1/2 the performance of C
+
  
; Python with tkinter
+
Basically the following Registry hives are stored in the corresponding files:
: Advantage: Portable
+
* HKEY_USERS: \Documents and Setting\User Profile\NTUSER.DAT
: Disadvantage: Python is one of the slowest modern languages around.
+
* HKEY_USERS\DEFAULT: C:\Windows\system32\config\default
 +
* HKEY_LOCAL_MACHINE\SAM: C:\Windows\system32\config\SAM
 +
* HKEY_LOCAL_MACHINE\SECURITY: C:\Windows\system32\config\SECURITY
 +
* HKEY_LOCAL_MACHINE\SOFTWARE: C:\Windows\system32\config\software
 +
* HKEY_LOCAL_MACHINE\SYSTEM: C:\Windows\system32\config\system
  
; Python with wxWidgets
+
===Windows 98/ME===
: Advantage: Portable and a better development environment than tkiner
+
* \Windows\user.dat
: Disadvantage: wxWidgets is not installed by default, so you'll need to get it installed. Not as well documented as Tkinter
+
* \Windows\system.dat
 +
* \Windows\profiles\user profile\user.dat
  
; [http://processing.org processing.org]
+
== Keys ==
: Advantage: Programming language specifically developed for visualization; compiles to java byte code
+
: Disadvantage: Very oddball
+
  
; JavaFX - Java's version of Flash
+
=== Run/RunOnce ===
 +
System-wide:
 +
<pre>
 +
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Run
 +
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\RunOnce
 +
</pre>
  
; Flash
+
Per user:
 +
<pre>
 +
HKEY_CURRENT_USER\SOFTWARE\Microsoft\Windows\CurrentVersion\Run
 +
HKEY_CURRENT_USER\SOFTWARE\Microsoft\Windows\CurrentVersion\RunOnce
 +
</pre>
  
== Applications ==
+
== Special cases ==
Most of these are scriptable.
+
The Windows Registry has several special case scenarios, mainly concerning key and value name, that are easy to fail to account for:
 +
* special characters key and value names
 +
* duplicate key and value names
 +
* the names when stored in extended ASCII (ANSI string) use a codepage that is dependent on the system settings
 +
 
 +
=== special characters key and value names ===
 +
Both key and values names are case insensitive. The \ character is used as the key separator. Note
 +
that the \ character can be used in value names. The / character is used in both key and value names.
 +
Some examples of which are:
 +
<pre>
 +
Key: HKEY_LOCAL_MACHINE\SYSTEM\ControlSet002\Services\NetBT\Parameters\
 +
Value: Size/Small/Medium/Large
 +
</pre>
 +
 
 +
<pre>
 +
Key: HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\Control\Terminal Server\VIDEO\disc\
 +
Value: \Device\Video0
 +
</pre>
 +
 
 +
<pre>
 +
Key:
 +
HKEY_LOCAL_MACHINE\SYSTEM\ControlSet002\services\xmlprov\Parameters\SchemaGroups\User\http://www.microsoft.com/provisioning/eaptlsuserpropertiesv1\
 +
Value: SchemaFile
 +
</pre>
 +
 
 +
=== codepaged ASCII strings ===
 +
 
 +
Value with name "ëigenaardig" created on Windows XP codepage 1252.
 +
 
 +
<pre>
 +
value key data:
 +
00000000: 76 6b 0b 00 46 00 00 00  20 98 1a 00 01 00 00 00  vk..F...  .......
 +
00000010: 01 00 69 6e eb 69 67 65  6e 61 61 72 64 69 67 00  ..in.ige naardig.
 +
00000020: 55 4e 49 43                                        UNIC
 +
 
 +
value key signature                    : vk
 +
value key value name size              : 11
 +
value key data size                    : 0x00000046 (70)
 +
value key data offset                  : 0x001a9820
 +
value key data type                    : 1 (REG_SZ) String
 +
value key flags                        : 0x0001
 +
        Value name is an ASCII string
 +
 
 +
value key unknown1                      : 0x6e69 (28265)
 +
value key value name                    : ëigenaardig
 +
value key value name hash              : 0xb78835ee
 +
value key padding:
 +
00000000: 00 55 4e 49 43                                    .UNIC
 +
</pre>
 +
 
 +
As you can see the name is stored in extended ASCII (ANSI) using codepage 1252.
 +
 
 +
==Tools==
 
===Open Source===
 
===Open Source===
====Data Plotting====
+
* [https://www.pinguin.lu/index.php Forensic Registry EDitor (fred)] - "Forensic Registry EDitor (fred) is a cross-platform M$ registry hive editor" by [[Daniel Gillen]]
* http://ploticus.sourceforge.net
+
* [http://projects.sentinelchicken.org/data/doc/reglookup/regfi/ libregfi] - The regfi library is a read-only NT registry library which serves as the main engine behind the reglookup tool
* http://www.gnuplot.info/
+
* [http://projects.sentinelchicken.org/reglookup/ reglookup] — "small command line utility for reading and querying Windows NT-based registries."
====Graph and Network Visualization====
+
* [http://sourceforge.net/projects/regviewer/ regviewer] — a tool for looking at the registry.
* [http://code.google.com/p/boomslang/ Boomslang] --- A thin layer of python over matplotlib that makes it easier to do common plots.
+
* [[Regripper|RegRipper]] — "the fastest, easiest, and best tool for registry analysis in forensics examinations."
* [http://www.graphviz.org/ Graphviz] - Originally developed by the [http://public.research.att.com/areas/visualization/ AT&T Information Visualization Gorup], designed for drawing connected graphs of nodes and edges. Neato is a similar system but does layout based on a spring model. Can produce output as [[PostScript]], [[PNG]], [[GIF]], or as an annotated graph file with the locations of all of the objects &mdash; ideal for drawing in a GUI. Runs from the command line on [[Unix]], [[Windows]] and [[Mac]], although there is also a [http://www.pixelglow.com/graphviz/ MacOS GUI version].
+
* [http://search.cpan.org/~jmacfarla/Parse-Win32Registry-0.51/lib/Parse/Win32Registry.pm Parse::Win32Registry] Perl module.
* [http://graphexploration.cond.org/ Guess: The Graph Exploration System] - Originally developed at HP, this is a large Jython/Java-based system that you can use for building your own applications. Distributed under GPL.
+
* [http://www.williballenthin.com/registry/index.html python-registry] Python module.
* [http://www.codeplex.com/nodexl NodeXL] - Free/open excel add-in extends the spreadsheet with network metrics and visualizations
+
* [http://code.google.com/p/registrydecoder/ Registry Decoder] offline analysis component, by [[Andrew Case]]
* [http://sourceforge.net/projects/ivc/ InfoVis Cyberinfrastructure] - Another graph drawing system written in Java.
+
* [http://code.google.com/p/registrydecoder/ RegDecoderLive] live hive acquisition component, by [[Andrew Case]]
* [http://jung.sourceforge.net/ Java Universal Network/Graph Framework (JUNG)] - Graphing, [[data mining]], [[social network]] analysis, and other stuff.
+
* [[libregf]] - Library and tools to access the Windows NT Registry File (REGF) format
* [http://www.andrew.cmu.edu/user/krack/krackplot.shtml Krackplot] - "KrackPlot is a program for network visualization designed for social network analysts."
+
* [[Registryasxml]] - Tool to import/export registry sections as XML
* [http://bioinformatics.icmb.utexas.edu/lgl/ Large Graph Layout (LGL)] - A bioinformatics system from University of Texas. They really mean Large.
+
* [http://www.sfu.ca/~richards/Multinet/Pages/multinet.htm MultiNet] - A data analysis package for drawing conventional data and graph data.  
+
* [http://www.analytictech.com/netdraw.htm NetDraw] - "a free program written by Steve Borgatti for visualizing both 1-mode and 2-mode social network data."
+
* [http://web.mit.edu/bshi/Public/nv2d/ NetVis 2D] - Another graph visualization and layout tool written in Java.
+
* [http://www.opendx.org/ OpenDX] - Based on [[IBM]]'s Visualization Data Explorer, runs on [[Unix]]/X11/Motif.
+
* [http://vlado.fmf.uni-lj.si/pub/networks/pajek/ Pajek] - Windows program for drawing large networks.
+
* [http://sourceforge.net/projects/sonia/ Social Network Image Animator (SoNIA)] - Originally developed at Stanford. Written in Java. Makes movies.
+
* [http://www.informatik.uni-bremen.de/uDrawGraph/en/uDrawGraph/uDrawGraph.html uDrawGraph]
+
* [http://wilma.sourceforge.net/ WilmaScope] - Real-time animations of dynamic graph structures. Written in Java. Sophisticated force model with strings and attraction.
+
* [http://www.caida.org/tools/visualization/walrus/ Walrus] - A 3-d graph network exploration tool. Employs 3D hyperbolic displays and layout based on a user-supplied spanning tree.
+
  
=== Commercial Graphic Applications and Tools===
+
===Freeware===
 +
* [http://www.tzworks.net/prototype_page.php?proto_id=3 Yet Another Registry Utility (yaru)] Free tool that can be run on Windows, Linux or Mac OS-X. If run in admin mode, allows viewing of registry hives on live system.
  
* [http://www.aisee.com/ aiSee Graph Layout Software] - Supports 15 layout algorithms, recursive graph nesting, and easy printing. Runs on [[Windows]], [[Linux]], [[Solaris]], [[NetBSD]], and [[MacOS]]. 30-day trial and free registered versions available. Academic pricing available.
+
* [http://www.tzworks.net/prototype_page.php?proto_id=14 Windows ShellBag Parser] Free tool that can be run on Windows, Linux or Mac OS-X.
*  [http://www.geomantics.com/ Geomantics] - Geographical, Visualization and Graphics software. Runs on [[Windows]].
+
* [http://www.kylebank.com/ Graphis 2D and 3D graphing software] - Runs on [[Windows]]. Free 30-day evaluation copy available.
+
* [http://www.openviz.com/ OpenViz] and  [http://www.powerviz.com/ PowerViz] - Both from Advanced Visual Systems, super high-end visualization toolkits. $$$$
+
* [http://www.tomsawyer.com/ Tom Sawyer Software] Analysis, Visualizaiton, and Layout programs. - Heavy support for drawing graphs. Beautiful gallery. ActiveX, Java, C++ and .NET editions.
+
* [http://www.netminer.com/ NetMiner] - A comprehensive tool for Social Network Analysis. Runs on Windows, with a Linux version under development. $35 for "Express" student version, $250 for "Professional" student version, $950 for "Normal" "Professional" version.
+
* [http://www.analytictech.com/ucinet.htm UCINET] - A comprehensive package for the analysis of social network data as well as other 1-mode and 2-mode data.
+
* [http://www.clarifiednetworks.com/logster Logster] - an ultra-easy software tool to visualize Apache-style logs on a world map.
+
* [http://www.clarifiednetworks.com/Clarified%20Analyzer Clarified Analyzer] - Visualizes Network Traffic and allows to drill down from visualizations to the packet level.
+
  
== Visualization Toolkits and Libraries ==
+
* [http://tzworks.net/prototype_page.php?proto_id=19 ''cafae''] - Computer Account Forensic Artifact ExtractorFree tool that can be run on Windows, Linux or Mac OS-X to parse ntuser.dat hives.
===C/C++===
+
* [http://public.kitware.com/VTK/ The Visualization Toolkit] - C++ multi-platform with interfaces available for Tcl/Tk, Java and Python. Professional support provided by [http://www.kitware.com/ Kitware].
+
* [http://kdirstat.sourceforge.net/ KDirStat], an open source implementation of [http://www.cs.umd.edu/hcil/treemap-history/index.shtml Treemaps] written in C. (Treemaps are a visualization technique developed at the University of Maryland for visualizing large amounts of multi-dimensional data.) You can find a copy of it in [http://www.derlien.com/ Disk Inventory X] and
+
===Java===
+
* [http://csbi.sourceforge.net/index.html Graph Interface Library (GINY)] - Java
+
* [http://hypergraph.sourceforge.net/ HyperGraph] - Hyperbolic trees, in Java. Check out the home page. Try clicking on the logo...
+
* [http://ivtk.sourceforge.net/ InfoViz Toolkit] - Java, originally developed at [[INRA]].
+
* [https://jdigraph.dev.java.net/ Jdigrah] - Java Directed Graphs.
+
* [http://jgrapht.sourceforge.net/ JGraphT] - A Java visualization kit designed to be simple and extensible.  
+
* [http://prefuse.sourceforge.net/ Perfuse] - A Java-based toolkit for building interactive information visualization applications
+
* [http://www.ssec.wisc.edu/~billh/visad.html#intro VisAD] - A Java component library for interactive and collaborative visualization.
+
* [http://www.softwaresecretweapons.com/jspwiki/Wiki.jsp?page=LinguineMaps Linguine Maps] - An open-source Java-based system for visualizing software call maps.
+
* [http://zvtm.sourceforge.net/index.html Zoomable Visual Transformation Machine] - Java. Originally started at Xerox Research Europe.
+
* [http://openmap.bbn.com/ OpenMap] A Java-based Geographical Information System framework, from [[BBN]].
+
  
===Unclassified===
+
===Commercial===
* [http://gravisto.fim.uni-passau.de/ Gravisto: Graph Visualization Toolkit] - An editor and toolkit for developing graph visualization algorithms.  
+
* [http://www.abexo.com/free-registry-cleaner.htm Abexo Free Regisry Cleaner]
* [http://www.gnu.frb.br:8080/rox Rox Graph Theory Framework] - An open-source plug-in framework for graph theory visualization.
+
* [http://www.auslogics.com/registry-defrag Auslogics Registry Defrag]
* [http://touchgraph.sourceforge.net/ TouchGraph] - Library for building graph-based interfaces.
+
* [http://lastbit.com/arv/ Alien Registry Viewer]
 +
* [http://www.larshederer.homepage.t-online.de/erunt/index.htm NT Registry Optimizer]
 +
* [http://www.registry-clean.net/free-registry-defrag.htm iExpert Software-Free Registry Defrag]
 +
* [http://arsenalrecon.com/apps Registry Recon]
 +
* [http://paullee.ru/regundel Registry Undelete (russian)]
 +
* [http://mitec.cz/wrr.html Windows Registry Recovery]
 +
* [http://registrytool.com/ Registry Tool]
 +
 
 +
==Bibliography==
 +
* [http://www.dfrws.org/2009/proceedings/p69-zhu.pdf Using ShellBag Information to Reconstruct User Activities.], Yuandong Zhu*, Pavel Gladyshev, Joshua James, DFRWS 2009
 +
* Recovering Deleted Data From the Windows Registry. Timothy Morgan, DFRWS 2008 [http://www.dfrws.org/2008/proceedings/p33-morgan.pdf [paper]] [http://www.dfrws.org/2008/proceedings/p33-morgan_pres.pdf [slides]]
 +
* [http://dfrws.org/2008/proceedings/p26-dolan-gavitt.pdf Forensic Analysis of the Windows Registry in Memory], Brendan Dolan-Gavitt, DFRWS 2008  [http://dfrws.org/2008/proceedings/p26-dolan-gavitt_pres.pdf [slides]]
 +
* [http://eptuners.com/forensics/A%20Windows%20Registry%20Quick%20Reference.pdf A Windows Registry Quick-Reference], Derrick Farmer, Burlington, VT.
 +
 
 +
* [http://www.sciencedirect.com/science?_ob=ArticleURL&_udi=B7CW4-4GX1J3B-1&_user=3326500&_rdoc=1&_fmt=&_orig=search&_sort=d&view=c&_acct=C000060280&_version=1&_urlVersion=0&_userid=3326500&md5=ab887593e7be6d5257696707886978f1 The Windows Registry as a forensic resource], Digital Investigation, Volume 2, Issue 3, September 2005, Pages 201--205.
 +
 
 +
* [http://www.forensicfocus.com/downloads/forensic-analysis-windows-registry.pdf Forensic Analysis of the Windows Registry], Lih Wern Wong , School of Computer and Information Science, Edith Cowan University
 +
 
 +
* [http://www.sentinelchicken.com/research/registry_format/ The Windows NT Registry File Format], Timothy D. Morgan
  
==Journals and Conferences==
 
* [http://www.palgrave-journals.com/ivs/index.html Information Visualization Journal]
 
* [http://rw4.cs.uni-sb.de/~diehl/softvis/seminar/index.php?goto=seminar ACM Symposium on Software Visualization]
 
==Research Groups==
 
===Berkeley===
 
* [http://bailando.sims.berkeley.edu/infovis.html Bailando Visualization]
 
* [http://vis.berkeley.edu/ Berkeley Visualization Lab]
 
===Brown===
 
* [http://www.cs.brown.edu/people/rt/gd.html Roberto Tamassia's resources on Graph Drawing]
 
===Stanford===
 
* [http://window.stanford.edu/projects/rivet/ Rivet Project] (Visualization complex systems)
 
===UNM===
 
* [http://www.msi.umn.edu/user_support/scivis/scivis-list.html Scientific Visualization at the Supercomputing Institute]
 
===Wattenberg===
 
* [http://www.bewitched.com/ Bewitched], a one-man research group.
 
 
==See Also==
 
==See Also==
* [http://www-static.cc.gatech.edu/gvu/ii/resources/infovis.html GVU's Information Visualization Resources link farm]
+
* [http://windowsir.blogspot.com/search/label/Registry Windows Incident Response Articles on Registry]
* [http://directory.google.com/Top/Science/Math/Combinatorics/Software/Graph_Drawing/ Google Directory of Graph Drawing Software]
+
* [http://www.answers.com/topic/win-registry Windows Registry Information]
* [http://directory.fsf.org/science/visual/ GNU Free Software directory of scientific visualization software]
+
* [http://en.wikipedia.org/wiki/Windows_Registry Wikipedia Article on Windows Registry]
* [http://www.manageability.org/blog/stuff/open-source-graph-network-visualization-in-java/view Open Source Graph Network Visualization in Java]
+
[[Category:Bibliographies]]
* [http://www.insna.org/INSNA/soft_inf.html INSNA's web page of Computer Programs for Social Network Analysis]
+
* [http://moyix.blogspot.com/search/label/registry Push the Red Button] — Articles on Registry
* http://www.dmoz.org/Science/Math/Combinatorics/Software/Graph_Drawing/
+
* [http://tech.groups.yahoo.com/group/win4n6/ Windows Forensics Mailing List]
 +
* [http://samba.org/~jelmer/kregedit/ kregedit] - a KDE utility for viewing and editing registry files.
 +
* [http://www.bindview.com/Services/RAZOR/Utilities/Unix_Linux/ntreg_readme.cfm ntreg] a file system driver for linux, which understands the NT registry file format.
 +
* [http://www.beginningtoseethelight.org/ntsecurity/ Security Accounts Manager]
 +
 
 +
* http://www.opensourceforensics.org/tools/unix.html - Open Source Forensic Tools on Brian Carrier's website.
  
CAIDA has 15+ years of work visualizing Internet topologies. You may find their tools to be useful:
+
[[Category:Windows Analysis]]
* http://www.caida.org/tools/visualization/
+
* http://www.caida.org/publications/visualizations/
+
* http://www.caida.org/tools/visualization/walrus/gallery1/
+
* http://www.caida.org/research/topology/as_core_network/
+

Revision as of 02:00, 13 March 2013

File Locations

The Windows Registry is stored in multiple files.

Windows NT 4

In Windows NT 4 (and later) the Registry is stored in the Windows NT Registry File (REGF) format.

Basically the following Registry hives are stored in the corresponding files:

  • HKEY_USERS: \Documents and Setting\User Profile\NTUSER.DAT
  • HKEY_USERS\DEFAULT: C:\Windows\system32\config\default
  • HKEY_LOCAL_MACHINE\SAM: C:\Windows\system32\config\SAM
  • HKEY_LOCAL_MACHINE\SECURITY: C:\Windows\system32\config\SECURITY
  • HKEY_LOCAL_MACHINE\SOFTWARE: C:\Windows\system32\config\software
  • HKEY_LOCAL_MACHINE\SYSTEM: C:\Windows\system32\config\system

Windows 98/ME

  • \Windows\user.dat
  • \Windows\system.dat
  • \Windows\profiles\user profile\user.dat

Keys

Run/RunOnce

System-wide:

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Run
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\RunOnce

Per user:

HKEY_CURRENT_USER\SOFTWARE\Microsoft\Windows\CurrentVersion\Run
HKEY_CURRENT_USER\SOFTWARE\Microsoft\Windows\CurrentVersion\RunOnce

Special cases

The Windows Registry has several special case scenarios, mainly concerning key and value name, that are easy to fail to account for:

  • special characters key and value names
  • duplicate key and value names
  • the names when stored in extended ASCII (ANSI string) use a codepage that is dependent on the system settings

special characters key and value names

Both key and values names are case insensitive. The \ character is used as the key separator. Note that the \ character can be used in value names. The / character is used in both key and value names. Some examples of which are:

Key: HKEY_LOCAL_MACHINE\SYSTEM\ControlSet002\Services\NetBT\Parameters\
Value: Size/Small/Medium/Large
Key: HKEY_LOCAL_MACHINE\SYSTEM\ControlSet001\Control\Terminal Server\VIDEO\disc\
Value: \Device\Video0
Key:
HKEY_LOCAL_MACHINE\SYSTEM\ControlSet002\services\xmlprov\Parameters\SchemaGroups\User\http://www.microsoft.com/provisioning/eaptlsuserpropertiesv1\
Value: SchemaFile

codepaged ASCII strings

Value with name "ëigenaardig" created on Windows XP codepage 1252.

value key data:
00000000: 76 6b 0b 00 46 00 00 00  20 98 1a 00 01 00 00 00   vk..F...  .......
00000010: 01 00 69 6e eb 69 67 65  6e 61 61 72 64 69 67 00   ..in.ige naardig.
00000020: 55 4e 49 43                                        UNIC

value key signature                     : vk
value key value name size               : 11
value key data size                     : 0x00000046 (70)
value key data offset                   : 0x001a9820
value key data type                     : 1 (REG_SZ) String
value key flags                         : 0x0001
        Value name is an ASCII string

value key unknown1                      : 0x6e69 (28265)
value key value name                    : ëigenaardig
value key value name hash               : 0xb78835ee
value key padding:
00000000: 00 55 4e 49 43                                     .UNIC

As you can see the name is stored in extended ASCII (ANSI) using codepage 1252.

Tools

Open Source

Freeware

  • cafae - Computer Account Forensic Artifact Extractor. Free tool that can be run on Windows, Linux or Mac OS-X to parse ntuser.dat hives.

Commercial

Bibliography

See Also