Difference between pages "Tools:Visualization" and "Windows SuperFetch Format"

From ForensicsWiki
(Difference between pages)
Jump to: navigation, search
m (Commercial Tools)
 
 
Line 1: Line 1:
Although not strictly for forensic purposes, '''visualization tools''' such as the ones discussed here can be very useful for visualizing large data sets. As forensic practitioners need to process more and more data, it is likely that some of the techniques implemented by these tools will need to be adopted.
+
{{expand}}
  
=Open Source Visualization Toolkits=
+
SuperFetch, is a memory management scheme that enhances the least-recently accessed approach with historical information and proactive memory management. [http://technet.microsoft.com/en-us/magazine/2007.03.vistakernel.aspx]
  
; [http://public.kitware.com/VTK/ The Visualization Toolkit]
+
<b>Note that the following format specification are incomplete.</b>
: C++ multi-platform with interfaces available for Tcl/Tk, Java and Python. Professional support provided by [http://www.kitware.com/ Kitware].
+
  
; [http://www.graphviz.org/ Graphviz]
+
== SuperFetch DB files ==
: Originally developed by the [http://public.research.att.com/areas/visualization/ AT&T Information Visualization Gorup], designed for drawing connected graphs of nodes and edges. Neato is a similar system but does layout based on a spring model. Can produce output as [[PostScript]], [[PNG]], [[GIF]], or as an annotated graph file with the locations of all of the objects---ideal for drawing in a GUI. Runs from the command line on [[Unix]], [[Windows]] and [[Mac]], although there is also a [http://www.pixelglow.com/graphviz/ MacOS GUI version].
+
The <tt>Ag*.db</tt> files are of the SuperFetch file format. E.g.
 +
<pre>
 +
AgAppLaunch.db
 +
AgCx_SC*.db
 +
AgGlFaultHistory.db
 +
AgGlFgAppHistory.db
 +
AgGlGlobalHistory.db
 +
AgGlUAD_%SID%.db
 +
AgGlUAD_P_%SID%.db
 +
AgRobust.db
 +
</pre>
  
; [http://www.opendx.org/ OpenDX]
+
The SuperFetch DB files can be stored in uncompressed or compressed form, where different version of Windows use different compressed forms:
: Based on IBM's Visualization Data Explorer, runs on [[Unix]]/X11/Motif.
+
* Compressed SuperFetch DB - MEMO file format; Windows Vista
 +
* Compressed SuperFetch DB - MEM0 file format; Windows  7
 +
* Compressed SuperFetch DB - MAM file format; Windows 8
  
; [http://www.ssec.wisc.edu/~billh/visad.html#intro VisAD]
+
=== Compressed SuperFetch DB - MEMO file format ===
: A Java component library for interactive and collaborative visualization.
+
The MEM file consists of:
 +
* file header
 +
* compressed blocks
  
==Graph Drawing Tools==
+
This format uses the LZNT1 compression method
  
; [http://www.informatik.uni-bremen.de/uDrawGraph/en/uDrawGraph/uDrawGraph.html uDrawGraph]
+
==== File header ====
 +
The file header is 84 bytes of size and consists of:
 +
{| class="wikitable"
 +
|-
 +
! Offset
 +
! Size
 +
! Value
 +
! Description
 +
|-
 +
| 0
 +
| 4
 +
| "MEMO" (0x4d, 0x45, 0x4d, 0x4f)
 +
| Signature
 +
|-
 +
| 4
 +
| 4
 +
|
 +
| Uncompressed (total) data size
 +
|-
 +
|}
  
=Commercial Tools=
+
==== Compressed blocks ====
 +
The compressed block size is the chunk data size, which is part of the LZNT1 compressed data, + 2 bytes for the size of the chunk header itself.
  
; [http://www.aisee.com/ aiSee Graph Layout Software]
+
The uncompressed block size is 4096 (0x1000) or the remaining uncompressed data size for the last block.
: Supports 15 layout algorithms, recursive graph nesting, and easy printing. Runs on [[Windows]], [[Linux]], [[Solaris]], [[NetBSD]], and [[MacOS]]. 30-day trial and free registered versions available. Academic pricing available.
+
  
; [http://www.tomsawyer.com/ Tom Sawyer Software] Analysis, Visualizaiton, and Layout programs.
+
=== Compressed SuperFetch DB - MEM0 file format ===
: Heavy support for drawing graphs. Beautiful gallery. ActiveX, Java, C++ and .NET editions.
+
The MEM file consists of:
 +
* file header
 +
* compressed blocks
  
; [http://www.geomantics.com/ Geomantics]
+
This format uses the LZXPRESS Huffman compression method
: Geographical, Visualization and Graphics software. Runs on [[Windows]].
+
  
; [http://www.kylebank.com/ Graphis 2D and 3D graphing software]
+
==== File header ====
: Runs on [[Windows]]. Free 30-day evaluation copy available.
+
The file header is 84 bytes of size and consists of:
 +
{| class="wikitable"
 +
|-
 +
! Offset
 +
! Size
 +
! Value
 +
! Description
 +
|-
 +
| 0
 +
| 4
 +
| "MEM0" (0x4d, 0x45, 0x4d, 0x30)
 +
| Signature
 +
|-
 +
| 4
 +
| 4
 +
|
 +
| Uncompressed (total) data size
 +
|-
 +
|}
  
;[http://www.openviz.com/ OpenViz] and  [http://www.powerviz.com/ PowerViz]
+
==== Compressed blocks ====
: Both from Advanced Visual Systems, super high-end visualization toolkits. $$$$
+
The file header is followed by compressed blocks:
 +
{| class="wikitable"
 +
|-
 +
! Offset
 +
! Size
 +
! Value
 +
! Description
 +
|-
 +
| 0
 +
| 4
 +
|
 +
| Compressed data size
 +
|-
 +
| 4
 +
| ...
 +
|
 +
| Compressed data
 +
|-
 +
|}
  
=Other Resources=
+
The uncompressed block size is 65536 (0x10000) or the remaining uncompressed data size for the last block.
  
; [http://www.palgrave-journals.com/ivs/index.html Information Visualization Journal]
+
=== Compressed SuperFetch DB - MAM file format ===
 +
The MAM file consists of:
 +
* file header
 +
* compressed blocks
  
; [http://www.msi.umn.edu/user_support/scivis/scivis-list.html Scientific Visualization at the Supercomputing Institute]
+
This format uses the <b>TODO</b> compression method
  
; [http://directory.google.com/Top/Science/Math/Combinatorics/Software/Graph_Drawing/ Google Directory of Graph Drawing Software]
+
==== File header ====
 +
<b>TODO</b>
  
; [http://rw4.cs.uni-sb.de/~diehl/softvis/seminar/index.php?goto=seminar ACM Symposium on Software Visualization]
+
{| class="wikitable"
: May give you some ideas.
+
|-
 +
! Offset
 +
! Size
 +
! Value
 +
! Description
 +
|-
 +
| 0
 +
| 4
 +
| "MAM\x84" (0x4d, 0x41, 0x4d, 0x84)
 +
| Signature
 +
|-
 +
|}
  
; [http://directory.fsf.org/science/visual/ GNU Free Software directory of scientific visualization software]
+
==== Compressed blocks ====
 +
<b>TODO</b>
 +
 
 +
=== Uncompressed SuperFetch DB format ===
 +
<b>TODO</b>
 +
 
 +
==== File header ====
 +
<b>TODO</b>
 +
 
 +
{| class="wikitable"
 +
|-
 +
! Offset
 +
! Size
 +
! Value
 +
! Description
 +
|-
 +
| 0
 +
| 4
 +
| 0x0000000e
 +
| Unknown (Database type or signature?)
 +
|-
 +
| 4
 +
| 4
 +
|
 +
| Uncompressed (total) data size
 +
|-
 +
|}
 +
== TRX files ==
 +
The <tt>Ag*.db.trx</tt> files are of the TRX file format. E.g.
 +
<pre>
 +
AgCx_SC*.db.trx
 +
</pre>
 +
 
 +
<b>Note that the following format specification is incomplete.</b>
 +
 
 +
=== File header ===
 +
The file header is variable of size and consists of:
 +
{| class="wikitable"
 +
|-
 +
! Offset
 +
! Size
 +
! Value
 +
! Description
 +
|-
 +
| 0
 +
| 4
 +
| 1
 +
| Unknown (Version?)
 +
|-
 +
| 4
 +
| 4
 +
|
 +
| Unknown
 +
|-
 +
| 8
 +
| 4
 +
|
 +
| File size
 +
|-
 +
| 12
 +
| 4
 +
|
 +
| Maximum number of records (of the record offsets array)
 +
|-
 +
| 16
 +
| 4
 +
|
 +
| Number of records
 +
|-
 +
| 20
 +
| ...
 +
|
 +
| Record offsets array, where the record offset is a 32-bit integer. Unused record offset are set to 0.
 +
|-
 +
|}
 +
 
 +
=== Record ===
 +
<b>TODO describe</b>
 +
 
 +
== See Also ==
 +
* [[SuperFetch]]
 +
 
 +
== External Links ==
 +
* [http://technet.microsoft.com/en-us/magazine/2007.03.vistakernel.aspx Inside the Windows Vista Kernel: Part 2], by [[Mark Russinovich]], March 2007
 +
* [http://blog.rewolf.pl/blog/?p=214 Windows SuperFetch file format – partial specification], by ReWolf, October 5, 2011
 +
 
 +
[[Category:File Formats]]

Revision as of 00:41, 23 April 2014

Information icon.png

Please help to improve this article by expanding it.
Further information might be found on the discussion page.

SuperFetch, is a memory management scheme that enhances the least-recently accessed approach with historical information and proactive memory management. [1]

Note that the following format specification are incomplete.

SuperFetch DB files

The Ag*.db files are of the SuperFetch file format. E.g.

AgAppLaunch.db
AgCx_SC*.db
AgGlFaultHistory.db
AgGlFgAppHistory.db
AgGlGlobalHistory.db
AgGlUAD_%SID%.db
AgGlUAD_P_%SID%.db
AgRobust.db

The SuperFetch DB files can be stored in uncompressed or compressed form, where different version of Windows use different compressed forms:

  • Compressed SuperFetch DB - MEMO file format; Windows Vista
  • Compressed SuperFetch DB - MEM0 file format; Windows 7
  • Compressed SuperFetch DB - MAM file format; Windows 8

Compressed SuperFetch DB - MEMO file format

The MEM file consists of:

  • file header
  • compressed blocks

This format uses the LZNT1 compression method

File header

The file header is 84 bytes of size and consists of:

Offset Size Value Description
0 4 "MEMO" (0x4d, 0x45, 0x4d, 0x4f) Signature
4 4 Uncompressed (total) data size

Compressed blocks

The compressed block size is the chunk data size, which is part of the LZNT1 compressed data, + 2 bytes for the size of the chunk header itself.

The uncompressed block size is 4096 (0x1000) or the remaining uncompressed data size for the last block.

Compressed SuperFetch DB - MEM0 file format

The MEM file consists of:

  • file header
  • compressed blocks

This format uses the LZXPRESS Huffman compression method

File header

The file header is 84 bytes of size and consists of:

Offset Size Value Description
0 4 "MEM0" (0x4d, 0x45, 0x4d, 0x30) Signature
4 4 Uncompressed (total) data size

Compressed blocks

The file header is followed by compressed blocks:

Offset Size Value Description
0 4 Compressed data size
4 ... Compressed data

The uncompressed block size is 65536 (0x10000) or the remaining uncompressed data size for the last block.

Compressed SuperFetch DB - MAM file format

The MAM file consists of:

  • file header
  • compressed blocks

This format uses the TODO compression method

File header

TODO

Offset Size Value Description
0 4 "MAM\x84" (0x4d, 0x41, 0x4d, 0x84) Signature

Compressed blocks

TODO

Uncompressed SuperFetch DB format

TODO

File header

TODO

Offset Size Value Description
0 4 0x0000000e Unknown (Database type or signature?)
4 4 Uncompressed (total) data size

TRX files

The Ag*.db.trx files are of the TRX file format. E.g.

AgCx_SC*.db.trx

Note that the following format specification is incomplete.

File header

The file header is variable of size and consists of:

Offset Size Value Description
0 4 1 Unknown (Version?)
4 4 Unknown
8 4 File size
12 4 Maximum number of records (of the record offsets array)
16 4 Number of records
20 ... Record offsets array, where the record offset is a 32-bit integer. Unused record offset are set to 0.

Record

TODO describe

See Also

External Links