Difference between revisions of "Memory analysis"

From ForensicsWiki
Jump to: navigation, search
(Volatility Labs)
(External Links)
 
(27 intermediate revisions by the same user not shown)
Line 1: Line 1:
'''Memory Analysis''' is the science of using a [[Memory Imaging|memory image]] to determine information about running programs, the [[operating system]], and the overall state of a computer. Because the analysis is highly dependent on the operating system, it has been divded into the following pages:
+
'''Memory Analysis''' is the science of using a [[Memory Imaging|memory image]] to determine information about running programs, the [[operating system]], and the overall state of a computer. Because the analysis is highly dependent on the operating system, it has been divided into the following pages:
  
* [[Windows Memory Analysis]]
 
 
* [[Linux Memory Analysis]]
 
* [[Linux Memory Analysis]]
 +
* [[Mac OS X Memory Analysis]]
 +
* [[Windows Memory Analysis]]
 +
 +
== Data types ==
 +
 +
Different programming languages provide different primitive data types, e.g. in C it is common to have "char", "int", "long", "float", "double".
 +
The size of these primitive data types is dependent on multiple factors like the CPU, the programming language, the compiler, etc. [http://en.wikipedia.org/wiki/Integer_(computer_science)] e.g. on 64-bit system the size of an "int" can vary based on the data-model [http://en.wikipedia.org/wiki/64-bit_computing].
 +
 +
<b>Note that some memory analysis resources and tools are known to completely disregard this aspect of 64-bit computing and claim incorrectly an integer is always 32-bit on 64-bit systems or claim the size of the integer is operating system dependent.</b>
 +
 +
When reading primitive data types from a byte stream the endianness specifies how the least and most significant parts of the data are stored [http://en.wikipedia.org/wiki/Endianness].
 +
 +
Composite data types (also referred to as data structures) consists of primitive and/or composite data types.
 +
 +
When reading data structures from memory note that data structure alignment [http://en.wikipedia.org/wiki/Data_structure_alignment] can be applied.
 +
 +
=== Character and string data types ===
 +
Note that this subject could fill books by itself and the following information is a very brief overview of some of the characteristics of character and string data types.
 +
 +
A programming language can distinguish between Unicode, non-Unicode and binary strings [http://en.wikipedia.org/wiki/Primitive_data_type#Characters_and_strings].
 +
* Unicode strings typically are stored in a Tranform format [http://en.wikipedia.org/wiki/UTF]
 +
** There are multiple versions of Unicode
 +
** A predecessor to Unicode is Universal Character Set (UCS) [http://en.wikipedia.org/wiki/Universal_Character_Set]
 +
* non-Unicode strings can be stored in various encodings e.g. ASCII [http://en.wikipedia.org/wiki/Character_encoding]
 +
** typical the encoding is identified by a codepage e.g. for CP1252 for Windows [http://en.wikipedia.org/wiki/Windows_code_page]
 +
** In Windows the supported set of non-Unicode strings is referred to as ANSI-strings, which is technically incorrect but a too widely used term to be ignored.
 +
** In other texts or contexts non-Unicode strings can be referred to as extended ASCII or ASCII with codepage strings, which, dependent on the context, can also be technically incorrect since non-Unicode strings can be stored in non-ASCII methods as well, e.g. EBIDIC or the codepage in which the string is stored has no historical relation to ASCII.
 +
** There are multiple variants of non-Unicode strings sometimes divided in single byte character (SBC) and multi byte character (MBC) strings. <b>Note that character here refers to the primitive data type used, not a textual character of the string.</b>
 +
* Binary strings can contain bit- or byte streams. Since bitstreams are often stored in bytes the endianess of the bits within the byte is relevant when reading or writing the string.
 +
 +
Another distinction made in some programming languages is to have a default (or narrow) character type, e.g. in C "char", and a separate wide character type, e.g. in C "wchar_t".
 +
The "wchar_t" is typically associated with Unicode strings. <b>Note that this should be taken as a very loose association, since the "char" can be used to store Unicode strings in UTF-8 and wchar_t can be used to store non-Unicode strings.</b> The typical size of char is 8-bits where the MSB is the sign bit. The typical size of wchar_t varies per "platform", e.g. it is common to see wchar_t to be 32-bit of size on Linux with gcc and 16-bit of size on Windows with MSC.
  
 
== OS-Independent Analysis ==
 
== OS-Independent Analysis ==
Line 19: Line 50:
 
* [[:Tools:Memory Imaging|Memory Imaging Tools]]
 
* [[:Tools:Memory Imaging|Memory Imaging Tools]]
 
* [[:Tools:Memory Analysis|Memory Analysis Tools]]
 
* [[:Tools:Memory Analysis|Memory Analysis Tools]]
 +
* [[Linux Memory Analysis]]
 +
* [[Mac OS X Memory Analysis]]
 +
* [[Virtualization Memory Analysis]]
 +
* [[Windows Memory Analysis]]
  
 
== External Links ==
 
== External Links ==
* [http://belkasoft.com/download/info/Live_RAM_Analysis_in_Digital_Forensics.pdf Discovering ephemeral evidence with Live RAM analysis] by Oleg Afonin and Yuri Gubanov, 2013
+
* [http://wiki.yobi.be/wiki/RAM_analysis YobiWiki: RAM analysis]
 
* [http://cryptome.org/0003/RAMisKey.pdf RAM is Key - Extracting Disk Encryption Keys From Volatile Memory], by [[Brian Kaplan]], May 2007
 
* [http://cryptome.org/0003/RAMisKey.pdf RAM is Key - Extracting Disk Encryption Keys From Volatile Memory], by [[Brian Kaplan]], May 2007
 +
* [http://www.cse.unt.edu/~song/csce5933-003/readings/HayOSR08.pdf Forensics Examination of Volatile System Data Using Virtual Introspection], by Brian Hay and Kara Nance, 2008
 
* [https://docs.google.com/presentation/d/1KsZGF6cQ-N8ngABFGCZf8pTQQ5CZ19VoAHq5cO5ZPdE/edit Memory Forensics With Volatility (Technology Preview)], by [[Michael Cohen]], October 2012
 
* [https://docs.google.com/presentation/d/1KsZGF6cQ-N8ngABFGCZf8pTQQ5CZ19VoAHq5cO5ZPdE/edit Memory Forensics With Volatility (Technology Preview)], by [[Michael Cohen]], October 2012
 +
* [http://scudette.blogspot.com/2012/11/finding-kernel-debugger-block.html Finding the Kernel Debugger Block], by [[Michael Cohen]], November 18, 2012
 +
* [http://belkasoft.com/download/info/Live_RAM_Analysis_in_Digital_Forensics.pdf Discovering ephemeral evidence with Live RAM analysis] by Oleg Afonin and Yuri Gubanov, 2013
 
* [http://www.dfrws.org/2013/proceedings/DFRWS2013-11.pdf An Evaluation Platform for Forensic Memory Acquisition Software] by Stefan Voemel and Johannes Stuettgen, DFRWS 2013
 
* [http://www.dfrws.org/2013/proceedings/DFRWS2013-11.pdf An Evaluation Platform for Forensic Memory Acquisition Software] by Stefan Voemel and Johannes Stuettgen, DFRWS 2013
 +
* [http://rekall-forensic.blogspot.com/2014/02/do-we-need-kernel-debugging-block.html Do we need the Kernel Debugging Block?], by [[Michael Cohen]], February 21, 2014
 +
* [http://rekall-forensic.blogspot.com/2014/10/windows-virtual-address-translation-and.html Windows Virtual Address Translation and the Pagefile], by [[Michael Cohen]], October 31, 2014
 +
 +
=== Anti-forensics ===
 +
* [https://www.blackhat.com/presentations/bh-dc-07/Walters/Paper/bh-dc-07-Walters-WP.pdf Integrating Volatile Memory Forensics into the Digital Investigation Process], by [[AAron Walter]], [[Nick Petroni]], Blackhat 2007
 +
* [http://blog.handlerdiaries.com/?p=363 Forensic Analysis of Anti-Forensic Activities], by [[Jack Crook]], January 29, 2014
 +
* [http://volatility-labs.blogspot.com/2014/02/add-next-big-threat-to-memory.html ADD: The Next Big Threat To Memory Forensics....Or Not], by [[Michael Hale Ligh]], February 3, 2014
 +
* [http://scudette.blogspot.com/2014/02/anti-forensics-and-memory-analysis.html Anti-forensics and memory analysis], by [[Michael Cohen]], February 7, 2014
 +
* [http://takahiroharuyama.github.io/blog/2014/04/21/memory-forensics-still-aborted/ Memory Forensics: Still Aborted], by Takahiro Haruyama, April 21, 2014
  
 
=== Computer architecture ===
 
=== Computer architecture ===
 
* [http://en.wikipedia.org/wiki/64-bit_computing Wikipedia: 64-bit computing]
 
* [http://en.wikipedia.org/wiki/64-bit_computing Wikipedia: 64-bit computing]
 +
* [http://download.intel.com/design/processor/manuals/253665.pdf Intel® 64 and IA-32 Architectures Software Developer’s Manual Volume 1: Basic Architecture], by Intel, May 2011
 
* [http://www.unix.org/version2/whatsnew/lp64_wp.html 64-Bit Programming Models: Why LP64?], The Open Group, 1997
 
* [http://www.unix.org/version2/whatsnew/lp64_wp.html 64-Bit Programming Models: Why LP64?], The Open Group, 1997
 +
 +
=== Data types ===
 +
* [http://en.wikipedia.org/wiki/Primitive_data_type Wikipedia: Primitive data type]
 +
* [http://en.wikipedia.org/wiki/Integer_(computer_science) Wikipedia: Integer]
 +
* [http://en.wikipedia.org/wiki/64-bit_computing Wikipedia: 64-bit computing]
 +
* [http://en.wikipedia.org/wiki/Endianness Wikipedia: Endianness]
 +
* [https://cygwin.com/cygwin-ug-net/programming.html Chapter 4. Programming with Cygwin]
 +
* [http://en.wikipedia.org/wiki/Data_structure_alignment Wikipedia: Data structure alignment]
 +
* [http://lucacardelli.name/Papers/TypeSystems.pdf Type Systems], by Luca Cardelli
  
 
=== [http://volatility-labs.blogspot.com/ Volatility Labs] ===
 
=== [http://volatility-labs.blogspot.com/ Volatility Labs] ===
Line 69: Line 126:
  
 
=== WinDBG ===
 
=== WinDBG ===
* [http://blog.opensecurityresearch.com/2013/12/getting-started-with-windbg-part-1.html Getting Started with WinDBG - Part 1], by Brad Antoniewicz, December 17, 2013
+
* [http://blog.opensecurityresearch.com/2013/12/getting-started-with-windbg-part-1.html Getting Started with WinDBG - Part 1], by [[Brad Antoniewicz]], December 17, 2013
* [http://blog.opensecurityresearch.com/2013/12/getting-started-with-windbg-part-2.html Getting Started with WinDBG - Part 2], by Brad Antoniewicz, December 24, 2013
+
* [http://blog.opensecurityresearch.com/2013/12/getting-started-with-windbg-part-2.html Getting Started with WinDBG - Part 2], by [[Brad Antoniewicz]], December 24, 2013
* [http://blog.opensecurityresearch.com/2013/12/getting-started-with-windbg-part-3.html Getting Started with WinDBG - Part 3], by Brad Antoniewicz, December 31, 2013
+
* [http://blog.opensecurityresearch.com/2013/12/getting-started-with-windbg-part-3.html Getting Started with WinDBG - Part 3], by [[Brad Antoniewicz]], December 31, 2013
 +
* [http://www.msuiche.net/2014/01/12/extengcpp-part-1/ Developing WinDbg ExtEngCpp Extension in C++ – Introduction – Part 1], by [[Matt Suiche]], January 12, 2014
 +
* [http://www.msuiche.net/2014/01/15/developing-windbg-extengcpp-extension-in-c-com-interface/ Developing WinDbg ExtEngCpp Extension in C++ – COM Interface – Part 2], by [[Matt Suiche]], January 15, 2014
 +
* [http://www.msuiche.net/2014/01/20/developing-windbg-extengcpp-extension-in-c-memory-debugger-markup-language-dml-part-3/ Developing WinDbg ExtEngCpp Extension in C++ – Memory & Debugger Markup Language (DML) – Part 3], by [[Matt Suiche]], January 20, 2014
  
 
[[Category:Memory Analysis]]
 
[[Category:Memory Analysis]]
 +
[[Category:Analysis Techniques]]

Latest revision as of 06:37, 1 November 2014

Memory Analysis is the science of using a memory image to determine information about running programs, the operating system, and the overall state of a computer. Because the analysis is highly dependent on the operating system, it has been divided into the following pages:

Data types

Different programming languages provide different primitive data types, e.g. in C it is common to have "char", "int", "long", "float", "double". The size of these primitive data types is dependent on multiple factors like the CPU, the programming language, the compiler, etc. [1] e.g. on 64-bit system the size of an "int" can vary based on the data-model [2].

Note that some memory analysis resources and tools are known to completely disregard this aspect of 64-bit computing and claim incorrectly an integer is always 32-bit on 64-bit systems or claim the size of the integer is operating system dependent.

When reading primitive data types from a byte stream the endianness specifies how the least and most significant parts of the data are stored [3].

Composite data types (also referred to as data structures) consists of primitive and/or composite data types.

When reading data structures from memory note that data structure alignment [4] can be applied.

Character and string data types

Note that this subject could fill books by itself and the following information is a very brief overview of some of the characteristics of character and string data types.

A programming language can distinguish between Unicode, non-Unicode and binary strings [5].

  • Unicode strings typically are stored in a Tranform format [6]
    • There are multiple versions of Unicode
    • A predecessor to Unicode is Universal Character Set (UCS) [7]
  • non-Unicode strings can be stored in various encodings e.g. ASCII [8]
    • typical the encoding is identified by a codepage e.g. for CP1252 for Windows [9]
    • In Windows the supported set of non-Unicode strings is referred to as ANSI-strings, which is technically incorrect but a too widely used term to be ignored.
    • In other texts or contexts non-Unicode strings can be referred to as extended ASCII or ASCII with codepage strings, which, dependent on the context, can also be technically incorrect since non-Unicode strings can be stored in non-ASCII methods as well, e.g. EBIDIC or the codepage in which the string is stored has no historical relation to ASCII.
    • There are multiple variants of non-Unicode strings sometimes divided in single byte character (SBC) and multi byte character (MBC) strings. Note that character here refers to the primitive data type used, not a textual character of the string.
  • Binary strings can contain bit- or byte streams. Since bitstreams are often stored in bytes the endianess of the bits within the byte is relevant when reading or writing the string.

Another distinction made in some programming languages is to have a default (or narrow) character type, e.g. in C "char", and a separate wide character type, e.g. in C "wchar_t". The "wchar_t" is typically associated with Unicode strings. Note that this should be taken as a very loose association, since the "char" can be used to store Unicode strings in UTF-8 and wchar_t can be used to store non-Unicode strings. The typical size of char is 8-bits where the MSB is the sign bit. The typical size of wchar_t varies per "platform", e.g. it is common to see wchar_t to be 32-bit of size on Linux with gcc and 16-bit of size on Windows with MSC.

OS-Independent Analysis

At the IEEE Security and Privacy conference in May 2011, Brendan Dolan-Gavitt presented a novel system, Virtuoso, that was able to perform operating-system independent memory analysis. Using virtual machine introspection accompanied by a number of formal program analysis techniques, his system was able to monitor the machine-level instructions and behavior of application actions (listing processes, network connections, etc) and then automatically generate Volatility plugins that replicated this analysis.

Encryption Keys

Various types of encryption keys can be extracted during memory analysis.

See Also

External Links

Anti-forensics

Computer architecture

Data types

Volatility Labs

Volatility Videos

WinDBG