Difference between pages "Tools:Visualization" and "Forensic Disk Differencing"

From ForensicsWiki
(Difference between pages)
Jump to: navigation, search
m (See Also)
 
m (See Also)
 
Line 1: Line 1:
Although not strictly for forensic purposes, '''visualization tools''' such as the ones discussed here can be very useful for visualizing large data sets. As forensic practitioners need to process more and more data, it is likely that some of the techniques implemented by these tools will need to be adopted.
+
Forensic Disk Differencing is the process of taking two or more disk images from the same computer and determining what changes in the first disk image might have resulted in the changes that are observed in the second. One common use of differencing is to determine what an attacker did during a break-in. To be used for this purpose, it is necessary to have a forensic disk image of the computer before the break-in and after the break-in.
  
==Programming Languages and Developer Toolkits==
+
==Differencing Tools==
If you are building forensic tools, you probably want to start with one of these:
+
===idifference.py===
; Java and Swing
+
idifference.py is part of the [[Digital Forensics XML]] Python Toolkit distributed with [[fiwalk]]. This tool will compare two different disk images and report changes in files between the first and the second. It also produces a timeline of changes.
: Advantage: Portable and lots of good documentation out there.
+
: Disadvantage: Programs are a bit verbose, and only offers about 1/2 the performance of C
+
  
; Python with tkinter
+
For example, using the '''nps-2009-canon2''' series of disk images:
: Advantage: Portable
+
: Disadvantage: Python is one of the slowest modern languages around.
+
  
; Python with wxWidgets
+
<pre>
: Advantage: Portable and a better development environment than tkiner
+
$ python idifference.py /nps-2009-canon2-gen2.raw nps-2009-canon2-gen3.raw
: Disadvantage: wxWidgets is not installed by default, so you'll need to get it installed. Not as well documented as Tkinter
+
>>> Reading nps-2009-canon2-gen2.raw
 +
>>> Reading nps-2009-canon2-gen3.raw
  
; [http://processing.org processing.org]
+
Disk image:/corp/drives/nps/nps-2009-canon2/nps-2009-canon2-gen3.raw
: Advantage: Programming language specifically developed for visualization; compiles to java byte code
+
: Disadvantage: Very oddball
+
  
; JavaFX - Java's version of Flash
+
New Files:
  
; Flash
+
2008-12-23 14:26:12 1315993 DCIM/100CANON/IMG_0041.JPG
  
== Applications ==
+
Deleted Files:  
Most of these are scriptable.
+
===Open Source===
+
====Data Plotting====
+
* http://ploticus.sourceforge.net
+
* http://www.gnuplot.info/
+
====Graph and Network Visualization====
+
* [http://code.google.com/p/boomslang/ Boomslang] --- A thin layer of python over matplotlib that makes it easier to do common plots.
+
* [http://www.graphviz.org/ Graphviz] - Originally developed by the [http://public.research.att.com/areas/visualization/ AT&T Information Visualization Gorup], designed for drawing connected graphs of nodes and edges. Neato is a similar system but does layout based on a spring model. Can produce output as [[PostScript]], [[PNG]], [[GIF]], or as an annotated graph file with the locations of all of the objects &mdash; ideal for drawing in a GUI. Runs from the command line on [[Unix]], [[Windows]] and [[Mac]], although there is also a [http://www.pixelglow.com/graphviz/ MacOS GUI version].
+
* [http://graphexploration.cond.org/ Guess: The Graph Exploration System] - Originally developed at HP, this is a large Jython/Java-based system that you can use for building your own applications. Distributed under GPL.
+
* [http://www.codeplex.com/nodexl NodeXL] - Free/open excel add-in extends the spreadsheet with network metrics and visualizations
+
* [http://sourceforge.net/projects/ivc/ InfoVis Cyberinfrastructure] - Another graph drawing system written in Java.
+
* [http://jung.sourceforge.net/ Java Universal Network/Graph Framework (JUNG)] - Graphing, [[data mining]], [[social network]] analysis, and other stuff.
+
* [http://www.andrew.cmu.edu/user/krack/krackplot.shtml Krackplot] - "KrackPlot is a program for network visualization designed for social network analysts."
+
* [http://bioinformatics.icmb.utexas.edu/lgl/ Large Graph Layout (LGL)] - A bioinformatics system from University of Texas. They really mean Large.
+
* [http://www.sfu.ca/~richards/Multinet/Pages/multinet.htm MultiNet] - A data analysis package for drawing conventional data and graph data.
+
* [http://www.analytictech.com/netdraw.htm NetDraw] - "a free program written by Steve Borgatti for visualizing both 1-mode and 2-mode social network data."
+
* [http://web.mit.edu/bshi/Public/nv2d/ NetVis 2D] - Another graph visualization and layout tool written in Java.
+
* [http://www.opendx.org/ OpenDX] - Based on [[IBM]]'s Visualization Data Explorer, runs on [[Unix]]/X11/Motif.
+
* [http://vlado.fmf.uni-lj.si/pub/networks/pajek/ Pajek] - Windows program for drawing large networks.
+
* [http://sourceforge.net/projects/sonia/ Social Network Image Animator (SoNIA)] - Originally developed at Stanford. Written in Java. Makes movies.
+
* [http://www.informatik.uni-bremen.de/uDrawGraph/en/uDrawGraph/uDrawGraph.html uDrawGraph]
+
* [http://wilma.sourceforge.net/ WilmaScope] - Real-time animations of dynamic graph structures. Written in Java. Sophisticated force model with strings and attraction.
+
* [http://www.caida.org/tools/visualization/walrus/ Walrus] - A 3-d graph network exploration tool. Employs 3D hyperbolic displays and layout based on a user-supplied spanning tree.
+
  
=== Commercial Graphic Applications and Tools===
+
2008-12-23 14:12:38 855935 DCIM/100CANON/IMG_0001.JPG
 +
2008-12-23 14:22:38 1347778 DCIM/100CANON/IMG_0037.JPG
  
* [http://www.aisee.com/ aiSee Graph Layout Software] - Supports 15 layout algorithms, recursive graph nesting, and easy printing. Runs on [[Windows]], [[Linux]], [[Solaris]], [[NetBSD]], and [[MacOS]]. 30-day trial and free registered versions available. Academic pricing available.
+
Files with modified content (but size unchanged):  
*  [http://www.geomantics.com/ Geomantics] - Geographical, Visualization and Graphics software. Runs on [[Windows]].
+
* [http://www.kylebank.com/ Graphis 2D and 3D graphing software] - Runs on [[Windows]]. Free 30-day evaluation copy available.
+
* [http://www.openviz.com/ OpenViz] and  [http://www.powerviz.com/ PowerViz] - Both from Advanced Visual Systems, super high-end visualization toolkits. $$$$
+
* [http://www.tomsawyer.com/ Tom Sawyer Software] Analysis, Visualizaiton, and Layout programs. - Heavy support for drawing graphs. Beautiful gallery. ActiveX, Java, C++ and .NET editions.
+
* [http://www.netminer.com/ NetMiner] - A comprehensive tool for Social Network Analysis. Runs on Windows, with a Linux version under development. $35 for "Express" student version, $250 for "Professional" student version, $950 for "Normal" "Professional" version.
+
* [http://www.analytictech.com/ucinet.htm UCINET] - A comprehensive package for the analysis of social network data as well as other 1-mode and 2-mode data.
+
* [http://www.clarifiednetworks.com/logster Logster] - an ultra-easy software tool to visualize Apache-style logs on a world map.
+
* [http://www.clarifiednetworks.com/Clarified%20Analyzer Clarified Analyzer] - Visualizes Network Traffic and allows to drill down from visualizations to the packet level.
+
  
== Visualization Toolkits and Libraries ==
+
Files with changed file properties:  
===C/C++===
+
* [http://public.kitware.com/VTK/ The Visualization Toolkit] - C++ multi-platform with interfaces available for Tcl/Tk, Java and Python. Professional support provided by [http://www.kitware.com/ Kitware].
+
* [http://kdirstat.sourceforge.net/ KDirStat], an open source implementation of [http://www.cs.umd.edu/hcil/treemap-history/index.shtml Treemaps] written in C. (Treemaps are a visualization technique developed at the University of Maryland for visualizing large amounts of multi-dimensional data.)  You can find a copy of it in [http://www.derlien.com/ Disk Inventory X] and
+
===Java===
+
* [http://csbi.sourceforge.net/index.html Graph Interface Library (GINY)] - Java
+
* [http://hypergraph.sourceforge.net/ HyperGraph] - Hyperbolic trees, in Java. Check out the home page. Try clicking on the logo...
+
* [http://ivtk.sourceforge.net/ InfoViz Toolkit] - Java, originally developed at [[INRA]].
+
* [https://jdigraph.dev.java.net/ Jdigrah] - Java Directed Graphs.
+
* [http://jgrapht.sourceforge.net/ JGraphT] - A Java visualization kit designed to be simple and extensible.
+
* [http://prefuse.sourceforge.net/ Perfuse] - A Java-based toolkit for building interactive information visualization applications
+
* [http://www.ssec.wisc.edu/~billh/visad.html#intro VisAD] - A Java component library for interactive and collaborative visualization.
+
* [http://www.softwaresecretweapons.com/jspwiki/Wiki.jsp?page=LinguineMaps Linguine Maps] - An open-source Java-based system for visualizing software call maps.
+
* [http://zvtm.sourceforge.net/index.html Zoomable Visual Transformation Machine] - Java. Originally started at Xerox Research Europe.
+
* [http://openmap.bbn.com/ OpenMap] A Java-based Geographical Information System framework, from [[BBN]].
+
  
===Unclassified===
+
DCIM/CANONMSC/M0100.CTG SHA1 changed 69b30c352ee802f49b1ea25325af9fa05c3ffca1 -> baa42c03a917b01b212fb7e538e5deb525995f31
* [http://gravisto.fim.uni-passau.de/ Gravisto: Graph Visualization Toolkit] - An editor and toolkit for developing graph visualization algorithms.
+
DCIM/CANONMSC/M0100.CTG crtime changed to 1230070924 -> 1230071142
* [http://www.gnu.frb.br:8080/rox Rox Graph Theory Framework] - An open-source plug-in framework for graph theory visualization.
+
DCIM/CANONMSC/M0100.CTG mtime changed to 1230070924 -> 1230071142
* [http://touchgraph.sourceforge.net/ TouchGraph] - Library for building graph-based interfaces.
+
DCIM/CANONMSC/M0100.CTG resized 180 -> 188
  
==Journals and Conferences==
+
Timeline
* [http://www.palgrave-journals.com/ivs/index.html Information Visualization Journal]
+
* [http://rw4.cs.uni-sb.de/~diehl/softvis/seminar/index.php?goto=seminar ACM Symposium on Software Visualization]
+
==Research Groups==
+
===Berkeley===
+
* [http://bailando.sims.berkeley.edu/infovis.html Bailando Visualization]
+
* [http://vis.berkeley.edu/ Berkeley Visualization Lab]
+
===Brown===
+
* [http://www.cs.brown.edu/people/rt/gd.html Roberto Tamassia's resources on Graph Drawing]
+
===Stanford===
+
* [http://window.stanford.edu/projects/rivet/ Rivet Project] (Visualization complex systems)
+
===UNM===
+
* [http://www.msi.umn.edu/user_support/scivis/scivis-list.html Scientific Visualization at the Supercomputing Institute]
+
===Wattenberg===
+
* [http://www.bewitched.com/ Bewitched], a one-man research group.
+
==See Also==
+
* [http://www-static.cc.gatech.edu/gvu/ii/resources/infovis.html GVU's Information Visualization Resources link farm]
+
* [http://directory.google.com/Top/Science/Math/Combinatorics/Software/Graph_Drawing/ Google Directory of Graph Drawing Software]
+
* [http://directory.fsf.org/science/visual/ GNU Free Software directory of scientific visualization software]
+
* [http://www.manageability.org/blog/stuff/open-source-graph-network-visualization-in-java/view Open Source Graph Network Visualization in Java]
+
* [http://www.insna.org/INSNA/soft_inf.html INSNA's web page of Computer Programs for Social Network Analysis]
+
* http://www.dmoz.org/Science/Math/Combinatorics/Software/Graph_Drawing/
+
  
CAIDA has 15+ years of work visualizing Internet topologies. You may find their tools to be useful:
+
2008-12-23 14:25:42 DCIM/CANONMSC/M0100.CTG SHA1 changed 69b30c352ee802f49b1ea25325af9fa05c3ffca1 -> baa42c03a917b01b212fb7e538e5deb525995f31
* http://www.caida.org/tools/visualization/
+
2008-12-23 14:25:42 DCIM/CANONMSC/M0100.CTG crtime changed 1230070924 -> 1230071142
* http://www.caida.org/publications/visualizations/
+
2008-12-23 14:25:42 DCIM/CANONMSC/M0100.CTG mtime changed 1230070924 -> 1230071142
* http://www.caida.org/tools/visualization/walrus/gallery1/
+
2008-12-23 14:25:42 DCIM/CANONMSC/M0100.CTG resized 180 -> 188
* http://www.caida.org/research/topology/as_core_network/
+
2008-12-23 14:26:12 DCIM/100CANON/IMG_0041.JPG created
 +
$
 +
</pre>
 +
 
 +
Here are some more examples:
 +
* [[File:Idifference-demo1.txt]] --- idifference.py run on two disks from the 2009-M57 Patents scenario (Jo's November 23 vs. November 24th disk)
 +
 
 +
==See Also==
 +
*[http://dfrws.org/2012/proceedings/DFRWS2012-6.pdf A general strategy for differential forensic analysis]

Latest revision as of 21:21, 21 October 2013

Forensic Disk Differencing is the process of taking two or more disk images from the same computer and determining what changes in the first disk image might have resulted in the changes that are observed in the second. One common use of differencing is to determine what an attacker did during a break-in. To be used for this purpose, it is necessary to have a forensic disk image of the computer before the break-in and after the break-in.

Differencing Tools

idifference.py

idifference.py is part of the Digital Forensics XML Python Toolkit distributed with fiwalk. This tool will compare two different disk images and report changes in files between the first and the second. It also produces a timeline of changes.

For example, using the nps-2009-canon2 series of disk images:

$ python idifference.py /nps-2009-canon2-gen2.raw nps-2009-canon2-gen3.raw 
>>> Reading nps-2009-canon2-gen2.raw
>>> Reading nps-2009-canon2-gen3.raw

Disk image:/corp/drives/nps/nps-2009-canon2/nps-2009-canon2-gen3.raw 

New Files: 

2008-12-23 14:26:12	1315993	DCIM/100CANON/IMG_0041.JPG

Deleted Files: 

2008-12-23 14:12:38	855935	DCIM/100CANON/IMG_0001.JPG
2008-12-23 14:22:38	1347778	DCIM/100CANON/IMG_0037.JPG

Files with modified content (but size unchanged): 

Files with changed file properties: 

DCIM/CANONMSC/M0100.CTG	SHA1 changed	69b30c352ee802f49b1ea25325af9fa05c3ffca1	->	baa42c03a917b01b212fb7e538e5deb525995f31
DCIM/CANONMSC/M0100.CTG	crtime changed to	1230070924	->	1230071142
DCIM/CANONMSC/M0100.CTG	mtime changed to	1230070924	->	1230071142
DCIM/CANONMSC/M0100.CTG	resized	180	->	188

Timeline 

2008-12-23 14:25:42	DCIM/CANONMSC/M0100.CTG	SHA1 changed	69b30c352ee802f49b1ea25325af9fa05c3ffca1	->	baa42c03a917b01b212fb7e538e5deb525995f31
2008-12-23 14:25:42	DCIM/CANONMSC/M0100.CTG	crtime changed	1230070924	->	1230071142
2008-12-23 14:25:42	DCIM/CANONMSC/M0100.CTG	mtime changed	1230070924	->	1230071142
2008-12-23 14:25:42	DCIM/CANONMSC/M0100.CTG	resized	180	->	188
2008-12-23 14:26:12	DCIM/100CANON/IMG_0041.JPG	created
$

Here are some more examples:

  • File:Idifference-demo1.txt --- idifference.py run on two disks from the 2009-M57 Patents scenario (Jo's November 23 vs. November 24th disk)

See Also