Difference between pages "Memory analysis" and "Libewf"

From ForensicsWiki
(Difference between pages)
Jump to: navigation, search
(External Links)
 
 
Line 1: Line 1:
'''Memory Analysis''' is the science of using a [[Memory Imaging|memory image]] to determine information about running programs, the [[operating system]], and the overall state of a computer. Because the analysis is highly dependent on the operating system, it has been divded into the following pages:
+
{{Infobox_Software |
 +
  name = libewf |
 +
  maintainer = [[Joachim Metz]] |
 +
  os = [[Linux]], [[FreeBSD]], [[NetBSD]], [[OpenBSD]], [[Mac OS X]], [[Windows]] |
 +
  genre = {{Disk imaging}} |
 +
  license = {{LGPL}} |
 +
  website = [http://code.google.com/p/libewf/ code.google.com/p/libewf/] |
 +
}}
  
* [[Windows Memory Analysis]]
+
'''Libewf''' is a library to access the [[Encase image file format|Expert Witness Compression Format (EWF)]].
* [[Linux Memory Analysis]]
+
  
== OS-Independent Analysis ==
+
== Features ==  
 +
Read or write supported EWF formats:
 +
* [[SMART]] .s01 (EWF-S01)
 +
* [[EnCase]] .E01 (EWF-E01) and .Ex01 (EWF2-Ex01)
  
At the IEEE Security and Privacy conference in May 2011, Brendan Dolan-Gavitt presented a novel system, [http://www.cc.gatech.edu/~brendan/Virtuoso_Oakland.pdf Virtuoso], that was able to perform operating-system independent memory analysis. Using virtual machine introspection accompanied by a number of formal program analysis techniques, his system was able to monitor the machine-level instructions and behavior of application actions (listing processes, network connections, etc) and then automatically generate Volatility plugins that replicated this analysis.
+
Read-only supported EWF formats:
 +
* Logical Evidence File (LEF) .L01 (EWF-L01) and .Lx01 (EWF2-Lx01)
  
== Encryption Keys ==
+
Other features:
 +
* empty-block compression
 +
* read/write access using delta (or shadow) files
 +
* write resume
  
Various types of encryption keys can be extracted during memory analysis.
+
== Tools ==
* [[AESKeyFinder]] extracts 128-bit and 256-bit [[AES]] keys and [[RSAKeyFinder]] and private and public [[RSA]] keys from a memory dump [http://citp.princeton.edu/memory/code/].
+
The '''libewf''' package contains the following tools:
* [http://jessekornblum.com/tools/volatility/cryptoscan.py cryptoscan.py], which is a [[List of Volatility Plugins|plugin for the Volatility framework]], scans a memory image for [[TrueCrypt]] passphrases
+
* '''ewfacquire''', which writes storage media data from devices and files to EWF files.
 +
* '''ewfacquirestream''', which writes data from stdin to EWF files.
 +
* '''ewfdebug'''; experimental tool does nothing at the moment.
 +
* '''ewfexport''', which exports storage media data in EWF files to (split) RAW format or a specific version of EWF files.
 +
* '''ewfinfo''', which shows the metadata in EWF files.
 +
* '''ewfmount''', which FUSE mounts EWF files.
 +
* '''ewfrecover'''; special variant of ewfexport to create a new set of EWF files from a corrupt set.
 +
* '''ewfverify''', which verifies the storage media data in EWF files.
  
== See Also ==
+
The '''libewf''' package also contains the following bindings:
 +
* '''ewf.net''', bindings for .Net
 +
* '''pyewf''', bindings for Python contributed by [[David Collett]] in 2008
  
* [[Memory Imaging]]
+
=== Contributions ===
* [[:Tools:Memory Imaging|Memory Imaging Tools]]
+
Tools that have been contributed to the project are provided as separate tools on the sourceforge libewf project site. These are:
* [[:Tools:Memory Analysis|Memory Analysis Tools]]
+
* '''mount_ewf.py''', which allows the storage media data in a EWF files to be mounted, contributed by [[David Loveall]] in 2007.
 +
* '''libewf-java''', Java (JNA) bindings were contributed by [[Bradley Schatz]] in 2009.
 +
* '''delphi imdisk proxy''', Borland Delphi imdisk proxy, as an alternative to mount_ewf.py for Windows, contributed by [[Brendan Berney]] in 2010.
 +
* '''jlibewf''', native Java EWF reader contributed by [[Bruce Allen]] in 2010.
 +
* '''libewfcs''', native C# EWF reader contributed by [[Bruce Allen]] in 2011.
 +
* '''ewfaquirestream-mt''', C++ 11 multi-threaded version of ewfacquirestream contributed by Bernhard Zach in 2013.
  
== External Links ==
+
A menu based interface for ewfacquirestream called pyEWF, contributed by [[Dennis Schreiber]], was originally also available on the uitwisselplatform project site. However this is currently no longer maintained and was not moved to the sourceforge project size. The uitwisselplatform no longer exists. The name pyewf was reused for the libewf Python bindings created by [[David Collett]] which is now included in the libewf package.
* [http://wiki.yobi.be/wiki/RAM_analysis YobiWiki: RAM analysis]
+
* [http://cryptome.org/0003/RAMisKey.pdf RAM is Key - Extracting Disk Encryption Keys From Volatile Memory], by [[Brian Kaplan]], May 2007
+
* [https://docs.google.com/presentation/d/1KsZGF6cQ-N8ngABFGCZf8pTQQ5CZ19VoAHq5cO5ZPdE/edit Memory Forensics With Volatility (Technology Preview)], by [[Michael Cohen]], October 2012
+
* [http://scudette.blogspot.com/2012/11/finding-kernel-debugger-block.html Finding the Kernel Debugger Block], by [[Michael Cohen]], November 18, 2012
+
* [http://belkasoft.com/download/info/Live_RAM_Analysis_in_Digital_Forensics.pdf Discovering ephemeral evidence with Live RAM analysis] by Oleg Afonin and Yuri Gubanov, 2013
+
* [http://www.dfrws.org/2013/proceedings/DFRWS2013-11.pdf An Evaluation Platform for Forensic Memory Acquisition Software] by Stefan Voemel and Johannes Stuettgen, DFRWS 2013
+
  
=== Anti-forensics ===
+
=== Examples ===  
* [http://blog.handlerdiaries.com/?p=363 Forensic Analysis of Anti-Forensic Activities], by [[Jack Crook]], January 29, 2014
+
* [http://volatility-labs.blogspot.com/2014/02/add-next-big-threat-to-memory.html ADD: The Next Big Threat To Memory Forensics....Or Not], by [[Michael Hale Ligh]], February 3, 2014
+
* [http://scudette.blogspot.com/2014/02/anti-forensics-and-memory-analysis.html Anti-forensics and memory analysis], by [[Michael Cohen]], February 7, 2014
+
  
=== Computer architecture ===
+
Imaging a device on a Unix-based system:
* [http://en.wikipedia.org/wiki/64-bit_computing Wikipedia: 64-bit computing]
+
<pre>
* [http://www.unix.org/version2/whatsnew/lp64_wp.html 64-Bit Programming Models: Why LP64?], The Open Group, 1997
+
ewfacquire /dev/sda
 +
</pre>
  
=== [http://volatility-labs.blogspot.com/ Volatility Labs] ===
+
Imaging a device on a Windows system:
* [http://volatility-labs.blogspot.com/2012/09/movp-11-logon-sessions-processes-and.html MoVP 1.1 Logon Sessions, Processes, and Images]
+
<pre>
* [http://volatility-labs.blogspot.com/2012/09/movp-12-window-stations-and-clipboard.html MoVP 1.2 Window Stations and Clipboard Malware]
+
ewfacquire \\.\PhysicalDrive0
* [http://volatility-labs.blogspot.com/2012/09/movp-13-desktops-heaps-and-ransomware.html MoVP 1.3 Desktops, Heaps, and Ransomware]
+
</pre>
* [http://volatility-labs.blogspot.com/2012/09/movp-14-average-coder-rootkit-bash.html MoVP 1.4 Average Coder Rootkit, Bash History, and Elevated Processes]
+
* [http://volatility-labs.blogspot.com/2012/09/movp-15-kbeast-rootkit-detecting-hidden.html MoVP 1.5 KBeast Rootkit, Detecting Hidden Modules, and sysfs]
+
* [http://volatility-labs.blogspot.com/2012/09/movp-21-atoms-new-mutex-classes-and-dll.html MoVP 2.1 Atoms (The New Mutex), Classes and DLL Injection]
+
* [http://volatility-labs.blogspot.com/2012/09/movp-22-malware-in-your-windows.html MoVP 2.2 Malware In Your Windows]
+
* [http://volatility-labs.blogspot.com/2012/09/movp-23-event-logs-and-service-sids.html MoVP 2.3 Event Logs and Service SIDs]
+
* [http://volatility-labs.blogspot.com/2012/09/movp-24-analyzing-jynx-rootkit-and.html MoVP 2.4 Analyzing the Jynx rootkit and LD_PRELOAD]
+
* [http://volatility-labs.blogspot.com/2012/09/movp-25-investigating-in-memory-network.html MoVP 2.5: Investigating In-Memory Network Data with Volatility]
+
* [http://volatility-labs.blogspot.com/2012/09/movp-31-detecting-malware-hooks-in.html MoVP 3.1 Detecting Malware Hooks in the Windows GUI Subsystem]
+
* [http://volatility-labs.blogspot.com/2012/09/howto-scan-for-internet-cachehistory.html HowTo: Scan for Internet Cache/History and URLs]
+
* [http://volatility-labs.blogspot.com/2012/09/movp-32-shellbags-in-memory-setregtime.html MoVP 3.2 Shellbags in Memory, SetRegTime, and TrueCrypt Volumes]
+
* [http://volatility-labs.blogspot.com/2012/09/movp-33-analyzing-user-handles-and.html MoVP 3.3 Analyzing USER Handles and the Win32k.sys Gahti]
+
* [http://volatility-labs.blogspot.com/2012/09/movp-34-recovering-tagclipdata-whats-in.html MoVP 3.4: Recovering tagCLIPDATA: What's In Your Clipboard?]
+
* [http://volatility-labs.blogspot.com/2012/09/movp-35-analyzing-2008-dfrws-challenge.html MoVP 3.5: Analyzing the 2008 DFRWS Challenge with Volatility]
+
* [http://volatility-labs.blogspot.com/2012/10/movp-41-detecting-malware-with-gdi.html MoVP 4.1 Detecting Malware with GDI Timers and Callbacks]
+
* [http://volatility-labs.blogspot.com/2012/10/movp-43-taking-screenshots-from-memory.html MoVP 4.2 Taking Screenshots from Memory Dumps]
+
* [http://volatility-labs.blogspot.com/2012/10/movp-43-recovering-master-boot-records.html MoVP 4.3 Recovering Master Boot Records (MBRs) from Memory]
+
* [http://volatility-labs.blogspot.com/2012/10/movp-44-cache-rules-everything-around.html MoVP 4.4 Cache Rules Everything Around Me(mory)]
+
* [http://volatility-labs.blogspot.com/2012/10/omfw-2012-malware-in-windows-gui.html OMFW 2012: Malware In the Windows GUI Subsystem]
+
* [http://volatility-labs.blogspot.com/2012/10/omfw-2012-reconstructing-mbr-and-mft.html OMFW 2012: Reconstructing the MBR and MFT from Memory]
+
* [http://volatility-labs.blogspot.com/2012/10/phalanx-2-revealed-using-volatility-to.html Phalanx 2 Revealed: Using Volatility to Analyze an Advanced Linux Rootkit]
+
* [http://volatility-labs.blogspot.com/2012/10/solving-grrcon-network-forensics.html Solving the GrrCon Network Forensics Challenge with Volatility]
+
* [http://volatility-labs.blogspot.com/2012/10/omfw-2012-analyzing-linux-kernel.html OMFW 2012: Analyzing Linux Kernel Rootkits with Volatility]
+
* [http://volatility-labs.blogspot.com/2012/10/omfw-2012-datalore-android-memory.html OMFW 2012: Datalore: Android Memory Analysis]
+
* [http://volatility-labs.blogspot.com/2012/10/movp-for-volatility-22-and-omfw-2012.html MoVP for Volatility 2.2 and OMFW 2012 Wrap-Up]
+
* [http://volatility-labs.blogspot.com/2012/10/reverse-engineering-poison-ivys.html Reverse Engineering Poison Ivy's Injected Code Fragments]
+
* [http://volatility-labs.blogspot.com/2012/10/omfw-2012-analysis-of-process-token.html OMFW 2012: The Analysis of Process Token Privileges]
+
* [http://volatility-labs.blogspot.com/2012/10/omfw-2012-mining-pfn-database-for.html OMFW 2012: Mining the PFN Database for Malware Artifacts]
+
* [http://volatility-labs.blogspot.com/2014/01/truecrypt-master-key-extraction-and.html TrueCrypt Master Key Extraction And Volume Identification], by [[Michael Hale Ligh]], January 14, 2014
+
  
=== Volatility Videos ===
+
Converting a RAW into an EWF image
* [http://sketchymoose.blogspot.com/2011/10/set-up-to-more-memory-forensics.html Set Up to More Memory Forensics!], October 2011
+
<pre>
* [http://www.youtube.com/watch?v=8HsZLge0wWc Using Volatility: Suspicious Process (1/2)]
+
ewfacquire myfile.raw
* [http://www.youtube.com/watch?v=XTZPNk-Esok Using Volatility: Suspicious Process (2/2)]
+
</pre>
  
=== WinDBG ===
+
or:
* [http://blog.opensecurityresearch.com/2013/12/getting-started-with-windbg-part-1.html Getting Started with WinDBG - Part 1], by [[Brad Antoniewicz]], December 17, 2013
+
<pre>
* [http://blog.opensecurityresearch.com/2013/12/getting-started-with-windbg-part-2.html Getting Started with WinDBG - Part 2], by [[Brad Antoniewicz]], December 24, 2013
+
ewfacquire -c best -m fixed -t myfile -S 1T -u [-q] myfile.raw
* [http://blog.opensecurityresearch.com/2013/12/getting-started-with-windbg-part-3.html Getting Started with WinDBG - Part 3], by [[Brad Antoniewicz]], December 31, 2013
+
</pre>
* [http://www.msuiche.net/2014/01/12/extengcpp-part-1/ Developing WinDbg ExtEngCpp Extension in C++ – Introduction – Part 1], by [[Matt Suiche]], January 12, 2014
+
 
* [http://www.msuiche.net/2014/01/15/developing-windbg-extengcpp-extension-in-c-com-interface/ Developing WinDbg ExtEngCpp Extension in C++ – COM Interface – Part 2], by [[Matt Suiche]], January 15, 2014
+
or
* [http://www.msuiche.net/2014/01/20/developing-windbg-extengcpp-extension-in-c-memory-debugger-markup-language-dml-part-3/ Developing WinDbg ExtEngCpp Extension in C++ – Memory & Debugger Markup Language (DML) – Part 3], by [[Matt Suiche]], January 20, 2014
+
 
 +
<pre>
 +
cat split.raw.??? | ewfacquirestream
 +
cat myfile.??? | ewfacquirestream  -c best -m fixed -t myfile -S 1T
 +
 
 +
</pre>
 +
 
 +
Converting an optical disc (split) RAW into an EWF image (libewf 20110109 or later)
 +
<pre>
 +
ewfacquire -T optical.cue optical.iso
 +
</pre>
 +
 
 +
Converting an EWF into another EWF format or a (split) RAW image
 +
<pre>
 +
ewfexport image.E01
 +
</pre>
 +
 
 +
Exporting files from a logical image (L01)
 +
<pre>
 +
ewfexport image.L01
 +
</pre>
 +
 
 +
FUSE mounting an EWF image (libewf 20110828 or later)
 +
<pre>
 +
ewfmount image.E01 mount_point
 +
</pre>
 +
 
 +
FUSE mounting a logical image (L01) (libewf 20111016 or later)
 +
<pre>
 +
ewfmount -f files image.L01 mount_point
 +
</pre>
 +
 
 +
Verify an single image with results to the screen
 +
<pre>
 +
ewfverify image.E01
 +
</pre>
 +
 
 +
From a linux shell, verify a group of images in subdirectories of the current directory creating a simple log file per image.
 +
<pre>
 +
find . -name \*.E01 -printf '%f %p\n' | xargs printf "ewfverify -l \$(basename -s .E01 %s).ewfverify.out  %s\n" | sh
 +
</pre>
 +
 
 +
or
 +
 
 +
<pre>
 +
find . -name '*.E01' | while read F
 +
do
 +
  echo ewfverify -l "$(basename -s .E01 $F).ewfverify.out" "$F"
 +
done
 +
</pre>
 +
 
 +
== History ==
 +
 
 +
Libewf was created by [[Joachim Metz]] in 2006, while working for [http://en.hoffmannbv.nl/ Hoffmann Investigations].
 +
 
 +
Libewf is a rewrite of earlier work on the EnCase 4 file format by [[Michael Cohen]] part of [[PyFlag]] and the [[:File:ASR Data's Expert Witness Compression Format.pdf|Expert Witness Compression Format]] Specification by [[Andrew Rosen]]. It has been updated to read and write EnCase version 1 to 7 .E01 files, EnCase 5 to 7 .L01 files, EnCase 7 .Ex01 and .Lx01 files and SMART .s01 files. Libewf has initiated an Extended EWF (EWF-X) specifications to bypass limitations on the format imposed by the EnCase .E01 format.
 +
 
 +
In 2007 [[David Loveall]] contributed mount_ewf.py to the libewf project. This application allows a [[fuse]] based mount of the storage media data in the EWF files to be mounted. Due to repeated issues with Python and the fuse Python-bindings on [[Mac OS X]] part of the functionality of these scripts has been rewritten into '''ewfmount'''.
 +
 
 +
As of version 20120715 support for EWF version 2 (.Ex01 and .Lx01) was added.
 +
 
 +
== External Links ==
  
[[Category:Memory Analysis]]
+
* [https://code.google.com/p/libewf/ Project site]
 +
* [https://code.google.com/p/libewf/wiki/Building Building libewf and tools from source]
 +
* [https://code.google.com/p/libewf/wiki/Mounting Mounting a set of EWF file(s)]
 +
* [http://libewf.sourceforge.net Old project site]

Revision as of 14:36, 16 February 2014

libewf
Maintainer: Joachim Metz
OS: Linux, FreeBSD, NetBSD, OpenBSD, Mac OS X, Windows
Genre: Disk imaging
License: LGPL
Website: code.google.com/p/libewf/

Libewf is a library to access the Expert Witness Compression Format (EWF).

Features

Read or write supported EWF formats:

  • SMART .s01 (EWF-S01)
  • EnCase .E01 (EWF-E01) and .Ex01 (EWF2-Ex01)

Read-only supported EWF formats:

  • Logical Evidence File (LEF) .L01 (EWF-L01) and .Lx01 (EWF2-Lx01)

Other features:

  • empty-block compression
  • read/write access using delta (or shadow) files
  • write resume

Tools

The libewf package contains the following tools:

  • ewfacquire, which writes storage media data from devices and files to EWF files.
  • ewfacquirestream, which writes data from stdin to EWF files.
  • ewfdebug; experimental tool does nothing at the moment.
  • ewfexport, which exports storage media data in EWF files to (split) RAW format or a specific version of EWF files.
  • ewfinfo, which shows the metadata in EWF files.
  • ewfmount, which FUSE mounts EWF files.
  • ewfrecover; special variant of ewfexport to create a new set of EWF files from a corrupt set.
  • ewfverify, which verifies the storage media data in EWF files.

The libewf package also contains the following bindings:

  • ewf.net, bindings for .Net
  • pyewf, bindings for Python contributed by David Collett in 2008

Contributions

Tools that have been contributed to the project are provided as separate tools on the sourceforge libewf project site. These are:

  • mount_ewf.py, which allows the storage media data in a EWF files to be mounted, contributed by David Loveall in 2007.
  • libewf-java, Java (JNA) bindings were contributed by Bradley Schatz in 2009.
  • delphi imdisk proxy, Borland Delphi imdisk proxy, as an alternative to mount_ewf.py for Windows, contributed by Brendan Berney in 2010.
  • jlibewf, native Java EWF reader contributed by Bruce Allen in 2010.
  • libewfcs, native C# EWF reader contributed by Bruce Allen in 2011.
  • ewfaquirestream-mt, C++ 11 multi-threaded version of ewfacquirestream contributed by Bernhard Zach in 2013.

A menu based interface for ewfacquirestream called pyEWF, contributed by Dennis Schreiber, was originally also available on the uitwisselplatform project site. However this is currently no longer maintained and was not moved to the sourceforge project size. The uitwisselplatform no longer exists. The name pyewf was reused for the libewf Python bindings created by David Collett which is now included in the libewf package.

Examples

Imaging a device on a Unix-based system:

ewfacquire /dev/sda

Imaging a device on a Windows system:

ewfacquire \\.\PhysicalDrive0

Converting a RAW into an EWF image

ewfacquire myfile.raw

or:

ewfacquire -c best -m fixed -t myfile -S 1T -u [-q] myfile.raw

or

cat split.raw.??? | ewfacquirestream
cat myfile.??? | ewfacquirestream  -c best -m fixed -t myfile -S 1T 

Converting an optical disc (split) RAW into an EWF image (libewf 20110109 or later)

ewfacquire -T optical.cue optical.iso

Converting an EWF into another EWF format or a (split) RAW image

ewfexport image.E01

Exporting files from a logical image (L01)

ewfexport image.L01

FUSE mounting an EWF image (libewf 20110828 or later)

ewfmount image.E01 mount_point

FUSE mounting a logical image (L01) (libewf 20111016 or later)

ewfmount -f files image.L01 mount_point

Verify an single image with results to the screen

ewfverify image.E01

From a linux shell, verify a group of images in subdirectories of the current directory creating a simple log file per image.

find . -name \*.E01 -printf '%f %p\n' | xargs printf "ewfverify -l \$(basename -s .E01 %s).ewfverify.out  %s\n" | sh

or

find . -name '*.E01' | while read F
do
  echo ewfverify -l "$(basename -s .E01 $F).ewfverify.out" "$F"
done

History

Libewf was created by Joachim Metz in 2006, while working for Hoffmann Investigations.

Libewf is a rewrite of earlier work on the EnCase 4 file format by Michael Cohen part of PyFlag and the Expert Witness Compression Format Specification by Andrew Rosen. It has been updated to read and write EnCase version 1 to 7 .E01 files, EnCase 5 to 7 .L01 files, EnCase 7 .Ex01 and .Lx01 files and SMART .s01 files. Libewf has initiated an Extended EWF (EWF-X) specifications to bypass limitations on the format imposed by the EnCase .E01 format.

In 2007 David Loveall contributed mount_ewf.py to the libewf project. This application allows a fuse based mount of the storage media data in the EWF files to be mounted. Due to repeated issues with Python and the fuse Python-bindings on Mac OS X part of the functionality of these scripts has been rewritten into ewfmount.

As of version 20120715 support for EWF version 2 (.Ex01 and .Lx01) was added.

External Links