Difference between revisions of "Memory analysis"

From ForensicsWiki
Jump to: navigation, search
(Computer architecture)
 
(40 intermediate revisions by 2 users not shown)
Line 1: Line 1:
'''Memory Analysis''' is the science of using a [[Memory Imaging|memory image]] to determine information about running programs, the [[operating system]], and the overall state of a computer. Because the analysis is highly dependent on the operating system, it has been divded into the following pages:
+
'''Memory Analysis''' is the science of using a [[Memory Imaging|memory image]] to determine information about running programs, the [[operating system]], and the overall state of a computer. Because the analysis is highly dependent on the operating system, it has been divided into the following pages:
  
* [[Windows Memory Analysis]]
 
 
* [[Linux Memory Analysis]]
 
* [[Linux Memory Analysis]]
 +
* [[Mac OS X Memory Analysis]]
 +
* [[Windows Memory Analysis]]
 +
 +
== Data types ==
 +
 +
Different programming languages provide different primitive data types, e.g. in C it is common to have "char", "int", "long", "float", "double".
 +
The size of these primitive data types is dependent on multiple factors like the CPU, the programming language, the compiler, etc. [http://en.wikipedia.org/wiki/Integer_(computer_science)] e.g. on 64-bit system the size of an "int" can vary based on the data-model [http://en.wikipedia.org/wiki/64-bit_computing].
 +
 +
<b>Note that some memory analysis resources and tools are known to completely disregard this aspect of 64-bit computing and claim incorrectly an integer is always 32-bit on 64-bit systems or claim the size of the integer is operating system dependent.</b>
 +
 +
When reading primitive data types from a byte stream the endianness specifies how the least and most significant parts of the data are stored [http://en.wikipedia.org/wiki/Endianness].
 +
 +
Composite data types (also referred to as data structures) consists of primitive and/or composite data types.
 +
 +
When reading data structures from memory note that data structure alignment [http://en.wikipedia.org/wiki/Data_structure_alignment] can be applied.
 +
 +
=== Character and string data types ===
 +
Note that this subject could fill books by itself and the following information is a very brief overview of some of the characteristics of character and string data types.
 +
 +
A programming language can distinguish between Unicode, non-Unicode and binary strings [http://en.wikipedia.org/wiki/Primitive_data_type#Characters_and_strings].
 +
* Unicode strings typically are stored in a Tranform format [http://en.wikipedia.org/wiki/UTF]
 +
** There are multiple versions of Unicode
 +
** A predecessor to Unicode is Universal Character Set (UCS) [http://en.wikipedia.org/wiki/Universal_Character_Set]
 +
* non-Unicode strings can be stored in various encodings e.g. ASCII [http://en.wikipedia.org/wiki/Character_encoding]
 +
** typical the encoding is identified by a codepage e.g. for CP1252 for Windows [http://en.wikipedia.org/wiki/Windows_code_page]
 +
** In Windows the supported set of non-Unicode strings is referred to as ANSI-strings, which is technically incorrect but a too widely used term to be ignored.
 +
** In other texts or contexts non-Unicode strings can be referred to as extended ASCII or ASCII with codepage strings, which, dependent on the context, can also be technically incorrect since non-Unicode strings can be stored in non-ASCII methods as well, e.g. EBIDIC or the codepage in which the string is stored has no historical relation to ASCII.
 +
** There are multiple variants of non-Unicode strings sometimes divided in single byte character (SBC) and multi byte character (MBC) strings. <b>Note that character here refers to the primitive data type used, not a textual character of the string.</b>
 +
* Binary strings can contain bit- or byte streams. Since bitstreams are often stored in bytes the endianess of the bits within the byte is relevant when reading or writing the string.
 +
 +
Another distinction made in some programming languages is to have a default (or narrow) character type, e.g. in C "char", and a separate wide character type, e.g. in C "wchar_t".
 +
The "wchar_t" is typically associated with Unicode strings. <b>Note that this should be taken as a very loose association, since the "char" can be used to store Unicode strings in UTF-8 and wchar_t can be used to store non-Unicode strings.</b> The typical size of char is 8-bits where the MSB is the sign bit. The typical size of wchar_t varies per "platform", e.g. it is common to see wchar_t to be 32-bit of size on Linux with gcc and 16-bit of size on Windows with MSC.
  
 
== OS-Independent Analysis ==
 
== OS-Independent Analysis ==
Line 19: Line 50:
 
* [[:Tools:Memory Imaging|Memory Imaging Tools]]
 
* [[:Tools:Memory Imaging|Memory Imaging Tools]]
 
* [[:Tools:Memory Analysis|Memory Analysis Tools]]
 
* [[:Tools:Memory Analysis|Memory Analysis Tools]]
 +
* [[Linux Memory Analysis]]
 +
* [[Mac OS X Memory Analysis]]
 +
* [[Virtualization Memory Analysis]]
 +
* [[Windows Memory Analysis]]
  
 
== External Links ==
 
== External Links ==
=== Volatility Labs ===
+
* [http://wiki.yobi.be/wiki/RAM_analysis YobiWiki: RAM analysis]
* [http://volatility-labs.blogspot.ch/2012/09/movp-11-logon-sessions-processes-and.html MoVP 1.1 Logon Sessions, Processes, and Images]
+
* [http://cryptome.org/0003/RAMisKey.pdf RAM is Key - Extracting Disk Encryption Keys From Volatile Memory], by [[Brian Kaplan]], May 2007
* [http://volatility-labs.blogspot.ch/2012/09/movp-12-window-stations-and-clipboard.html MoVP 1.2 Window Stations and Clipboard Malware]
+
* [http://www.cse.unt.edu/~song/csce5933-003/readings/HayOSR08.pdf Forensics Examination of Volatile System Data Using Virtual Introspection], by Brian Hay and Kara Nance, 2008
* [http://volatility-labs.blogspot.ch/2012/09/movp-13-desktops-heaps-and-ransomware.html MoVP 1.3 Desktops, Heaps, and Ransomware]
+
* [https://docs.google.com/presentation/d/1KsZGF6cQ-N8ngABFGCZf8pTQQ5CZ19VoAHq5cO5ZPdE/edit Memory Forensics With Volatility (Technology Preview)], by [[Michael Cohen]], October 2012
* [http://volatility-labs.blogspot.ch/2012/09/movp-14-average-coder-rootkit-bash.html MoVP 1.4 Average Coder Rootkit, Bash History, and Elevated Processes]
+
* [http://scudette.blogspot.com/2012/11/finding-kernel-debugger-block.html Finding the Kernel Debugger Block], by [[Michael Cohen]], November 18, 2012
* [http://volatility-labs.blogspot.ch/2012/09/movp-15-kbeast-rootkit-detecting-hidden.html MoVP 1.5 KBeast Rootkit, Detecting Hidden Modules, and sysfs]
+
* [http://belkasoft.com/download/info/Live_RAM_Analysis_in_Digital_Forensics.pdf Discovering ephemeral evidence with Live RAM analysis] by Oleg Afonin and Yuri Gubanov, 2013
* [http://volatility-labs.blogspot.ch/2012/09/movp-21-atoms-new-mutex-classes-and-dll.html MoVP 2.1 Atoms (The New Mutex), Classes and DLL Injection]
+
* [http://www.dfrws.org/2013/proceedings/DFRWS2013-11.pdf An Evaluation Platform for Forensic Memory Acquisition Software] by Stefan Voemel and Johannes Stuettgen, DFRWS 2013
* [http://volatility-labs.blogspot.ch/2012/09/movp-22-malware-in-your-windows.html MoVP 2.2 Malware In Your Windows]
+
* [http://rekall-forensic.blogspot.com/2014/02/do-we-need-kernel-debugging-block.html Do we need the Kernel Debugging Block?], by [[Michael Cohen]], February 21, 2014
* [http://volatility-labs.blogspot.ch/2012/09/movp-23-event-logs-and-service-sids.html MoVP 2.3 Event Logs and Service SIDs]
+
 
* [http://volatility-labs.blogspot.ch/2012/09/movp-24-analyzing-jynx-rootkit-and.html MoVP 2.4 Analyzing the Jynx rootkit and LD_PRELOAD]
+
=== Anti-forensics ===
 +
* [https://www.blackhat.com/presentations/bh-dc-07/Walters/Paper/bh-dc-07-Walters-WP.pdf Integrating Volatile Memory Forensics into the Digital Investigation Process], by [[AAron Walter]], [[Nick Petroni]], Blackhat 2007
 +
* [http://blog.handlerdiaries.com/?p=363 Forensic Analysis of Anti-Forensic Activities], by [[Jack Crook]], January 29, 2014
 +
* [http://volatility-labs.blogspot.com/2014/02/add-next-big-threat-to-memory.html ADD: The Next Big Threat To Memory Forensics....Or Not], by [[Michael Hale Ligh]], February 3, 2014
 +
* [http://scudette.blogspot.com/2014/02/anti-forensics-and-memory-analysis.html Anti-forensics and memory analysis], by [[Michael Cohen]], February 7, 2014
 +
* [http://takahiroharuyama.github.io/blog/2014/04/21/memory-forensics-still-aborted/ Memory Forensics: Still Aborted], by Takahiro Haruyama, April 21, 2014
 +
 
 +
=== Computer architecture ===
 +
* [http://en.wikipedia.org/wiki/64-bit_computing Wikipedia: 64-bit computing]
 +
* [http://download.intel.com/design/processor/manuals/253665.pdf Intel® 64 and IA-32 Architectures Software Developer’s Manual Volume 1: Basic Architecture], by Intel, May 2011
 +
* [http://www.unix.org/version2/whatsnew/lp64_wp.html 64-Bit Programming Models: Why LP64?], The Open Group, 1997
 +
 
 +
=== Data types ===
 +
* [http://en.wikipedia.org/wiki/Primitive_data_type Wikipedia: Primitive data type]
 +
* [http://en.wikipedia.org/wiki/Integer_(computer_science) Wikipedia: Integer]
 +
* [http://en.wikipedia.org/wiki/64-bit_computing Wikipedia: 64-bit computing]
 +
* [http://en.wikipedia.org/wiki/Endianness Wikipedia: Endianness]
 +
* [https://cygwin.com/cygwin-ug-net/programming.html Chapter 4. Programming with Cygwin]
 +
* [http://en.wikipedia.org/wiki/Data_structure_alignment Wikipedia: Data structure alignment]
 +
* [http://lucacardelli.name/Papers/TypeSystems.pdf Type Systems], by Luca Cardelli
 +
 
 +
=== [http://volatility-labs.blogspot.com/ Volatility Labs] ===
 +
* [http://volatility-labs.blogspot.com/2012/09/movp-11-logon-sessions-processes-and.html MoVP 1.1 Logon Sessions, Processes, and Images]
 +
* [http://volatility-labs.blogspot.com/2012/09/movp-12-window-stations-and-clipboard.html MoVP 1.2 Window Stations and Clipboard Malware]
 +
* [http://volatility-labs.blogspot.com/2012/09/movp-13-desktops-heaps-and-ransomware.html MoVP 1.3 Desktops, Heaps, and Ransomware]
 +
* [http://volatility-labs.blogspot.com/2012/09/movp-14-average-coder-rootkit-bash.html MoVP 1.4 Average Coder Rootkit, Bash History, and Elevated Processes]
 +
* [http://volatility-labs.blogspot.com/2012/09/movp-15-kbeast-rootkit-detecting-hidden.html MoVP 1.5 KBeast Rootkit, Detecting Hidden Modules, and sysfs]
 +
* [http://volatility-labs.blogspot.com/2012/09/movp-21-atoms-new-mutex-classes-and-dll.html MoVP 2.1 Atoms (The New Mutex), Classes and DLL Injection]
 +
* [http://volatility-labs.blogspot.com/2012/09/movp-22-malware-in-your-windows.html MoVP 2.2 Malware In Your Windows]
 +
* [http://volatility-labs.blogspot.com/2012/09/movp-23-event-logs-and-service-sids.html MoVP 2.3 Event Logs and Service SIDs]
 +
* [http://volatility-labs.blogspot.com/2012/09/movp-24-analyzing-jynx-rootkit-and.html MoVP 2.4 Analyzing the Jynx rootkit and LD_PRELOAD]
 +
* [http://volatility-labs.blogspot.com/2012/09/movp-25-investigating-in-memory-network.html MoVP 2.5: Investigating In-Memory Network Data with Volatility]
 +
* [http://volatility-labs.blogspot.com/2012/09/movp-31-detecting-malware-hooks-in.html MoVP 3.1 Detecting Malware Hooks in the Windows GUI Subsystem]
 +
* [http://volatility-labs.blogspot.com/2012/09/howto-scan-for-internet-cachehistory.html HowTo: Scan for Internet Cache/History and URLs]
 +
* [http://volatility-labs.blogspot.com/2012/09/movp-32-shellbags-in-memory-setregtime.html MoVP 3.2 Shellbags in Memory, SetRegTime, and TrueCrypt Volumes]
 +
* [http://volatility-labs.blogspot.com/2012/09/movp-33-analyzing-user-handles-and.html MoVP 3.3 Analyzing USER Handles and the Win32k.sys Gahti]
 +
* [http://volatility-labs.blogspot.com/2012/09/movp-34-recovering-tagclipdata-whats-in.html MoVP 3.4: Recovering tagCLIPDATA: What's In Your Clipboard?]
 +
* [http://volatility-labs.blogspot.com/2012/09/movp-35-analyzing-2008-dfrws-challenge.html MoVP 3.5: Analyzing the 2008 DFRWS Challenge with Volatility]
 +
* [http://volatility-labs.blogspot.com/2012/10/movp-41-detecting-malware-with-gdi.html MoVP 4.1 Detecting Malware with GDI Timers and Callbacks]
 +
* [http://volatility-labs.blogspot.com/2012/10/movp-43-taking-screenshots-from-memory.html MoVP 4.2 Taking Screenshots from Memory Dumps]
 +
* [http://volatility-labs.blogspot.com/2012/10/movp-43-recovering-master-boot-records.html MoVP 4.3 Recovering Master Boot Records (MBRs) from Memory]
 +
* [http://volatility-labs.blogspot.com/2012/10/movp-44-cache-rules-everything-around.html MoVP 4.4 Cache Rules Everything Around Me(mory)]
 +
* [http://volatility-labs.blogspot.com/2012/10/omfw-2012-malware-in-windows-gui.html OMFW 2012: Malware In the Windows GUI Subsystem]
 +
* [http://volatility-labs.blogspot.com/2012/10/omfw-2012-reconstructing-mbr-and-mft.html OMFW 2012: Reconstructing the MBR and MFT from Memory]
 +
* [http://volatility-labs.blogspot.com/2012/10/phalanx-2-revealed-using-volatility-to.html Phalanx 2 Revealed: Using Volatility to Analyze an Advanced Linux Rootkit]
 +
* [http://volatility-labs.blogspot.com/2012/10/solving-grrcon-network-forensics.html Solving the GrrCon Network Forensics Challenge with Volatility]
 +
* [http://volatility-labs.blogspot.com/2012/10/omfw-2012-analyzing-linux-kernel.html OMFW 2012: Analyzing Linux Kernel Rootkits with Volatility]
 +
* [http://volatility-labs.blogspot.com/2012/10/omfw-2012-datalore-android-memory.html OMFW 2012: Datalore: Android Memory Analysis]
 +
* [http://volatility-labs.blogspot.com/2012/10/movp-for-volatility-22-and-omfw-2012.html MoVP for Volatility 2.2 and OMFW 2012 Wrap-Up]
 +
* [http://volatility-labs.blogspot.com/2012/10/reverse-engineering-poison-ivys.html Reverse Engineering Poison Ivy's Injected Code Fragments]
 +
* [http://volatility-labs.blogspot.com/2012/10/omfw-2012-analysis-of-process-token.html OMFW 2012: The Analysis of Process Token Privileges]
 +
* [http://volatility-labs.blogspot.com/2012/10/omfw-2012-mining-pfn-database-for.html OMFW 2012: Mining the PFN Database for Malware Artifacts]
 +
* [http://volatility-labs.blogspot.com/2014/01/truecrypt-master-key-extraction-and.html TrueCrypt Master Key Extraction And Volume Identification], by [[Michael Hale Ligh]], January 14, 2014
 +
 
 +
=== Volatility Videos ===
 +
* [http://sketchymoose.blogspot.com/2011/10/set-up-to-more-memory-forensics.html Set Up to More Memory Forensics!], October 2011
 +
* [http://www.youtube.com/watch?v=8HsZLge0wWc Using Volatility: Suspicious Process (1/2)]
 +
* [http://www.youtube.com/watch?v=XTZPNk-Esok Using Volatility: Suspicious Process (2/2)]
 +
 
 +
=== WinDBG ===
 +
* [http://blog.opensecurityresearch.com/2013/12/getting-started-with-windbg-part-1.html Getting Started with WinDBG - Part 1], by [[Brad Antoniewicz]], December 17, 2013
 +
* [http://blog.opensecurityresearch.com/2013/12/getting-started-with-windbg-part-2.html Getting Started with WinDBG - Part 2], by [[Brad Antoniewicz]], December 24, 2013
 +
* [http://blog.opensecurityresearch.com/2013/12/getting-started-with-windbg-part-3.html Getting Started with WinDBG - Part 3], by [[Brad Antoniewicz]], December 31, 2013
 +
* [http://www.msuiche.net/2014/01/12/extengcpp-part-1/ Developing WinDbg ExtEngCpp Extension in C++ – Introduction – Part 1], by [[Matt Suiche]], January 12, 2014
 +
* [http://www.msuiche.net/2014/01/15/developing-windbg-extengcpp-extension-in-c-com-interface/ Developing WinDbg ExtEngCpp Extension in C++ – COM Interface – Part 2], by [[Matt Suiche]], January 15, 2014
 +
* [http://www.msuiche.net/2014/01/20/developing-windbg-extengcpp-extension-in-c-memory-debugger-markup-language-dml-part-3/ Developing WinDbg ExtEngCpp Extension in C++ – Memory & Debugger Markup Language (DML) – Part 3], by [[Matt Suiche]], January 20, 2014
  
 
[[Category:Memory Analysis]]
 
[[Category:Memory Analysis]]
 +
[[Category:Analysis Techniques]]

Latest revision as of 01:41, 25 August 2014

Memory Analysis is the science of using a memory image to determine information about running programs, the operating system, and the overall state of a computer. Because the analysis is highly dependent on the operating system, it has been divided into the following pages:

Data types

Different programming languages provide different primitive data types, e.g. in C it is common to have "char", "int", "long", "float", "double". The size of these primitive data types is dependent on multiple factors like the CPU, the programming language, the compiler, etc. [1] e.g. on 64-bit system the size of an "int" can vary based on the data-model [2].

Note that some memory analysis resources and tools are known to completely disregard this aspect of 64-bit computing and claim incorrectly an integer is always 32-bit on 64-bit systems or claim the size of the integer is operating system dependent.

When reading primitive data types from a byte stream the endianness specifies how the least and most significant parts of the data are stored [3].

Composite data types (also referred to as data structures) consists of primitive and/or composite data types.

When reading data structures from memory note that data structure alignment [4] can be applied.

Character and string data types

Note that this subject could fill books by itself and the following information is a very brief overview of some of the characteristics of character and string data types.

A programming language can distinguish between Unicode, non-Unicode and binary strings [5].

  • Unicode strings typically are stored in a Tranform format [6]
    • There are multiple versions of Unicode
    • A predecessor to Unicode is Universal Character Set (UCS) [7]
  • non-Unicode strings can be stored in various encodings e.g. ASCII [8]
    • typical the encoding is identified by a codepage e.g. for CP1252 for Windows [9]
    • In Windows the supported set of non-Unicode strings is referred to as ANSI-strings, which is technically incorrect but a too widely used term to be ignored.
    • In other texts or contexts non-Unicode strings can be referred to as extended ASCII or ASCII with codepage strings, which, dependent on the context, can also be technically incorrect since non-Unicode strings can be stored in non-ASCII methods as well, e.g. EBIDIC or the codepage in which the string is stored has no historical relation to ASCII.
    • There are multiple variants of non-Unicode strings sometimes divided in single byte character (SBC) and multi byte character (MBC) strings. Note that character here refers to the primitive data type used, not a textual character of the string.
  • Binary strings can contain bit- or byte streams. Since bitstreams are often stored in bytes the endianess of the bits within the byte is relevant when reading or writing the string.

Another distinction made in some programming languages is to have a default (or narrow) character type, e.g. in C "char", and a separate wide character type, e.g. in C "wchar_t". The "wchar_t" is typically associated with Unicode strings. Note that this should be taken as a very loose association, since the "char" can be used to store Unicode strings in UTF-8 and wchar_t can be used to store non-Unicode strings. The typical size of char is 8-bits where the MSB is the sign bit. The typical size of wchar_t varies per "platform", e.g. it is common to see wchar_t to be 32-bit of size on Linux with gcc and 16-bit of size on Windows with MSC.

OS-Independent Analysis

At the IEEE Security and Privacy conference in May 2011, Brendan Dolan-Gavitt presented a novel system, Virtuoso, that was able to perform operating-system independent memory analysis. Using virtual machine introspection accompanied by a number of formal program analysis techniques, his system was able to monitor the machine-level instructions and behavior of application actions (listing processes, network connections, etc) and then automatically generate Volatility plugins that replicated this analysis.

Encryption Keys

Various types of encryption keys can be extracted during memory analysis.

See Also

External Links

Anti-forensics

Computer architecture

Data types

Volatility Labs

Volatility Videos

WinDBG