Difference between pages "Upcoming events" and "Carver 2.0 Planning Page"

From ForensicsWiki
(Difference between pages)
Jump to: navigation, search
(Conferences)
 
m (Implementation Timeline)
 
Line 1: Line 1:
<b>PLEASE READ BEFORE YOU EDIT THE LISTS BELOW</b><br>
+
This page is for planning Carver 2.0.
Events should be posted in the correct section, and in date order.  An event should NEVER be listed in more than one section (i.e. Ongoing/Continuous events should not be listed in Scheduled Training).  When events begin the same day, events of a longer length should be listed first.  New postings of events with the same date(s) as other events should be added after events already in the list. If a provider offers the same event at several locations simultaneously, the listing should have a single (ONE) entry in the list with the date(s) and ALL locations for the event. Please use three-letter month abbreviations (i.e. Sep, NOT Sept. or September), use two digit dates (i.e. Jan 01 NOT Jan 1), and use date ranges rather than listing every date during an event(i.e. Jan 02-05, NOT Jan 02, 03, 04, 05).<br>
+
<i>Some events may be <u>limited</u> to <b>Law Enforcement Only</b> or to a specific audience. Such restrictions should be noted when known.</i>
+
  
This is a BY DATE listing of upcoming events relevant to [[digital forensics]].  It is not an all inclusive list, but includes most well-known activities. Some events may duplicate events on the generic [[conferences]] page, but entries in this list have specific dates and locations for the upcoming event.
+
Please, do not delete text (ideas) here. Use something like this:
  
This listing is divided into four sections (described as follows):<br>
+
<pre>
<ol><li><b><u>Calls For Papers</u></b> - Calls for papers for either Journals or for Conferences, relevant to Digital Forensics (Name, Closing Date, URL)</li><br>
+
<s>bad idea</s>
<li><b><u>Conferences</u></b> - Conferences relevant for Digital Forensics (Name, Date, Location, URL)</li><br>
+
:: good idea
<li><b><u>On-Going / Continuous Training</u></b> - Training opportunities that are either always available online/distance learning format (start anytime) or that are offered the same time every month (Name, date-if applicable, URL)</li><br>
+
</pre>
<li><b><u>[[Scheduled Training Courses]]</u></b> - Training Classes/Courses that are scheduled for specific dates/locations.  This would include online (or distance learning format) courses which begin on specific dates, instead of the "start anytime" courses listed in the previous section. (Provider, URL) (''note: this has been moved to its own page.'')<br></li></ol>
+
  
The Conference and Training List is provided by the American Academy of Forensic Sciences (AAFS) Digital and Multimedia Sciences Section Listserv. 
+
This will look like:
<i> (Subscribe by sending an email to listserv@lists.mitre.org with message body containing SUBSCRIBE AAFS-DIGITAL-MULTIMEDIA-LIST)</i>
+
Requests for additions, deletions or corrections to this list may be sent by email to David Baker <i>(bakerd AT mitre.org)</i>.
+
  
== Calls For Papers ==
+
<s>bad idea</s>
{| border="0" cellpadding="2" cellspacing="2" align="top"
+
:: good idea
|- style="background:#bfbfbf; font-weight: bold"
+
! width="30%|Title
+
! width="15%"|Due Date
+
! width="15%"|Notification Date
+
! width="40%"|Website
+
|-
+
|ShmooCon 2009
+
|Dec 01, 2008
+
|Jan 01, 2009
+
|http://www.shmoocon.org/cfp.html
+
|-
+
|Security Opus
+
|Dec 01, 2008
+
|Jan 31, 2009
+
|http://www.securityopus.com/SORpapers.php
+
|-
+
|AusCERT Conference 2009
+
|Dec 05, 2008
+
|Jan 30, 2009
+
|http://conference.auscert.org.au/conf2009/cfp2009.html
+
|-
+
|Blackhat Briefings - Washington DC
+
|Jan 01, 2009
+
|Jan 16, 1009
+
|https://www.blackhat.com/html/bh-dc-09/bh-dc-09-cfp.html
+
|-
+
|3rd Edition of Small Scale Digital Device Forensics Journal
+
|Jan 31, 2009
+
|
+
|http://www.ssddfj.org/Call.asp
+
|-
+
|4rd International Workshop on Systematic Approaches to Digital Forensic Engineering (SADFE-2009)
+
|Feb 01, 2009
+
|
+
|http://conf.ncku.edu.tw/sadfe/sadfe09/
+
|-
+
|Blackhat Briefings - Europe
+
|Feb 01, 2009
+
|Feb 15, 2009
+
|https://www.blackhat.com/html/bh-europe-09/bh-eu-09-cfp.html.
+
|-
+
|Usenix Security 2009
+
|Feb 04, 2009
+
|Apr 13, 2009
+
|http://www.usenix.org/events/sec09/cfp
+
|-
+
|2009 ADFSL Conference on Digital Forensics, Security and Law
+
|Feb 20, 2009
+
|
+
|http://www.digitalforensics-conference.org/callforpapers.htm
+
|-
+
|KDDD 2009
+
|Feb 02, 2009
+
|Apr 10, 2009
+
|http://www.sigkdd.org/kdd2009/
+
|-
+
|DFRWS 2009
+
|Mar 16, 2009
+
|Apr 28, 2009
+
|http://www.dfrws.org/2009/cfp.shtml
+
|-
+
|ACM CCS 2009
+
|Apr 2009
+
|
+
|http://www.sigsac.org/ccs
+
|-
+
|New Security Paradigms Conference 2009
+
|Apr 2009
+
|
+
|http://www.nspw.org/current/
+
|-
+
|IEEE Symposium on Security and Privacy 2010
+
|Nov 2009
+
|
+
|
+
|-
+
|}
+
  
== Conferences ==
+
= License =
{| border="0" cellpadding="2" cellspacing="2" align="top"
+
|- style="background:#bfbfbf; font-weight: bold"
+
! width="40%"|Title
+
! width="20%"|Date/Location
+
! width="40%"|Website
+
|-
+
|Computer Security Institute Annual Conference 2008
+
|Nov 15-21<br>Washington, DC
+
|http://www.csiannual.com/conference/
+
|-
+
|6th Australian Digital Forensics Conference
+
|Dec 01-03<br>Mount Lawley, WA, Australia
+
|http://scissec.scis.ecu.edu.au/conferences2008/index.php?cf=2
+
|-
+
|Pacific Information Security Forum
+
|Dec 02-03, San Francisco, CA
+
|http://www.ianetsec.com/forums/event_summary.html?label=45
+
|-
+
|IEEE International Workshop on Information and Data Assurance
+
|Dec 07<br>Austin, TX
+
|http://ipccc.org/ipccc2008/main.php?page=6#workshop3
+
|-
+
|Digital Forensics Forum Arabia 2008
+
|Dec 15-17<br>Manama, Bahrain
+
|http://dff-worldwide.com/index.php?page=dff-arabia-2008-conference&hl=en_US
+
|-
+
|e-Forensics 2009
+
|Jan 19-21<br>Adelaide, Australia
+
|http://www.e-forensics.eu/
+
|-
+
|2009 DoD Cyber Crime Conference
+
|Jan 24-30<br>St. Louis, MO
+
|http://www.dodcybercrime.com/
+
|-
+
|5th Annual IFIP WG 11.9 International Conference on Digital Forensics
+
|Jan 25-28<br>Orlando, FL
+
|http://www.ifip119.org/Conferences/
+
|-
+
|ShmooCon 2009
+
|Feb 06-08<br>Washington, DC
+
|http://www.shmoocon.org/
+
|-
+
|American Academy of Forensic Sciences Annual Meeting
+
|Feb 16-21<br>Denver, CO
+
|http://www.aafs.org/default.asp?section_id=meetings&page_id=aafs_annual_meeting
+
|-
+
|Blackhat DC
+
|Feb 16-19<br>Washington, DC
+
|https://www.blackhat.com/html/bh-dc-09/bh-dc-09-main.html
+
|-
+
|24th Annual ACM Symposium on Applied Computing - Computer Forensics Track
+
|Mar 08-12<br>Honolulu, HI
+
|http://www.acm.org/conferences/sac/sac2009
+
|-
+
|ARES 2009 Conference
+
|Mar 16-19<br>Fukuoka, Japan
+
|http://www.ares-conference.eu/conf/
+
|-
+
|Security Opus
+
|Mar 17-18<br>San Francisco, CA
+
|http://www.securityopus.com
+
|-
+
|e-Crime Congress 2009
+
|Mar 24-25, London, United Kingdom
+
|http://www.e-crimecongress.org/ecrime2009/
+
|-
+
|Blackhat Europe
+
|Apr 14-17<br>Amsterdam, The Netherlands
+
|https://www.blackhat.com/html/bh-europe-09/bh-eu-09-main.html
+
|-
+
|AusCERT2009
+
|May 17-22<br>Gold Coast, Australia
+
|http://conference.auscert.org.au/conf2009/
+
|-
+
|Computer Security Institute: Security Exchange
+
|May 17-22<br>Las Vegas, NV
+
|http://www.csisx.com/
+
|-
+
|ADFSL 2009 Conference on Digital Forensics, Security and Law
+
|May 20-22<br>Burlington, VT
+
|http://www.digitalforensics-conference.org
+
|-
+
|Fourth International Workshop on Systematic Approaches to Digital Forensic Engineering
+
|May 22<br>Oakland, CA
+
|http://conf.ncku.edu.tw/sadfe/sadfe09/
+
|-
+
|2009 Techno Security Conference
+
|May 31-Jun 03<br>Myrtle Beach, SC
+
|http://www.techsec.com/index.html
+
|-
+
|Mobile Forensics World 2009
+
|Jun 03-06<br>Chicago, IL
+
|http://www.mobileforensicsworld.com
+
|-
+
|IEEE ICC Communication and Information Systems Security (CISS) Symposium
+
|Jun 14-18<br>Dresden, Germany
+
|http://www.ieee-icc.org/2009/
+
|-
+
|Blackhat USA 2009
+
|Jul 25-30<br>Las Vegas, NV
+
|https://www.blackhat.com/
+
|-
+
|DefCon 17
+
|Jul 31-Aug 02<br>Las Vegas, NV
+
|http://www.defcon.org/
+
|-
+
|Usenix Security Sypmosium
+
|Aug 10-14<br>Montreal, Quebec, Canada
+
|http://www.usenix.org/events/sec09/
+
|-
+
|Digital Forensic Research Workshop
+
|Aug 17-19<br>Montreal, Quebec, Canada
+
|http://www.dfrws.org
+
|-
+
|Triennial Meeting of the European Academy of Forensic Science
+
|Sep 08-11<br>Glasgow, Scotland, UK
+
|http://www.eafs2009.com/
+
|-
+
|}
+
  
== On-going / Continuous Training ==
+
BSD-3.
{| border="0" cellpadding="2" cellspacing="2" align="top"
+
 
|- style="background:#bfbfbf; font-weight: bold"
+
= OS =
! width="40%"|Title
+
 
! width="20%"|Date/Location
+
Linux/FreeBSD/MacOS
! width="40%"|Website
+
: (shouldn't this just match what the underlying afflib & sleuthkit cover? [[User:RB|RB]])
|-
+
:: Yes, but you need to test and validate on each. Question: Do we want to support windows? [[User:Simsong|Simsong]] 21:09, 30 October 2008 (UTC)
| ----DISTANCE LEARNING----
+
 
|-
+
= Requirements =
|Basic Computer Examiner Course - Computer Forensic Training Online
+
* AFF and EWF file images supported from scratch.
|Distance Learning Format
+
* File system aware layer.
|http://www.cftco.com
+
** By default, files are not carved. (clarify: only identified? [[User:RB|RB]]; I guess that it operates like [[Selective file dumper]] [[User:.FUF|.FUF]] 07:00, 29 October 2008 (UTC))
|-
+
* Plug-in architecture for identification/validation.
|Linux Data Forensics Training
+
** Can handle config files,like Revit07, to enter different file formats used by the carver.
|Distance Learning Format
+
* Ship with validators for:
|http://www.crazytrain.com/training.html
+
** JPEG
|-
+
** PNG
|SANS On-Demand Training
+
** GIF
|Distance Learning Format
+
** MSOLE
|http://www.sans.org/ondemand/?portal=69456f95660ade45be29c00b0c14aea1
+
** ZIP
|-
+
** TAR (gz/bz2)
| ----RECURRING TRAINING----
+
* Simple fragment recovery carving using gap carving.  
|-
+
* Recovering of individual ZIP sections and JPEG icons that are not sector aligned.
|MaresWare Suite Training
+
* Autonomous operation (some mode of operation should be completely non-interactive, requiring no human intervention to complete [[User:RB|RB]])
|First full week every month<br>Atlanta, GA
+
* Tested on 500GB-sized images. Should be able to carve a 500GB image in roughly 50% longer than it takes to read the image.
|http://www.maresware.com/maresware/training/maresware.htm
+
** Perhaps allocate a percentage budget per-validator (i.e. each validator adds N% to the carving time)
|-
+
* Parallelizable.
|Evidence Recovery for Windows Vista&trade;
+
* Configuration:
|First full week every month<br>Brunswick, GA
+
** Capability to parse some existing carvers' configuration files, either on-the-fly or as a one-way converter.
|http://www.internetcrimes.net
+
** Disengage internal configuration structure from configuration files, create parsers that present the expected structure
|-
+
**  Either extend Scalpel/Foremost syntaxes for extended features or use a tertiary syntax
|Evidence Recovery for Windows Server&reg; 2003 R2
+
* Can output audit.txt file.
|Second full week every month<br>Brunswick, GA
+
* Easy integration into ascription software.
|http://www.internetcrimes.net
+
 
|-
+
= Ideas =
|Evidence Recovery for the Windows XP&trade; operating system
+
* Use as much TSK if possible. Don't carry your own FS implementation the way photorec does.
|Third full week every month<br>Brunswick, GA
+
* Extracting/carving data from [[Thumbs.db]]? I've used [[foremost]] for it with some success. [[Vinetto]] has some critical bugs :( [[User:.FUF|.FUF]] 19:18, 28 October 2008 (UTC)
|http://www.internetcrimes.net
+
* Carving data structures. For example, extract all TCP headers from image by defining TCP header structure and some fields (e.g. source port > 1024, dest port = 80). This will extract all data matching the pattern and write a file with other fields. Another example is carving INFO2 structures and URL activity records from index.dat [[User:.FUF|.FUF]] 20:51, 28 October 2008 (UTC)
|-
+
** This has the opportunity to be extended to the concept of "point at blob FOO and interpret it as BAR"
|Computer Forensics Training and CCE&trade; Testing for Litigation Support Professionals
+
 
|Third weekend of every month(Fri-Mon)<br>Dallas, TX
+
.FUF added:
|http://www.md5group.com
+
The main idea is to allow users to define structures, for example (in pascal-like form):
|-
+
 
|}
+
<pre>
==See Also==
+
Field1: Byte = 123;
* [[Scheduled Training Courses]]
+
SomeTextLength: DWORD;
==References==
+
SomeText: string[SomeTextLength];
* [http://faculty.cs.tamu.edu/guofei/sec_conf_stat.htm Computer Security Conference Ranking and Statistic]
+
Field4: Char = 'r';
* [http://www.kdnuggets.com/meetings/ Meetings and Conferences in Data Mining and Discovery]
+
...
* http://www.conferencealerts.com/data.htm Data Mining Conferences World-Wide]
+
</pre>
 +
 
 +
This will produce something like this:
 +
<pre>
 +
Field1 = 123
 +
SomeTextLength = 5
 +
SomeText = 'abcd1'
 +
Field4 = 'r'
 +
</pre>
 +
 
 +
(In text or raw forms.)
 +
 
 +
Opinions?
 +
 
 +
Opinion: Simple pattern identification like that may not suffice, I think Simson's original intent was not only to identify but to allow for validation routines (plugins, as the original wording was).  As such, the format syntax would need to implement a large chunk of some programming language in order to be sufficiently flexible. [[User:RB|RB]]
 +
 
 +
=File System Awareness =
 +
==Background: Why be File System Aware?==
 +
Advantages of being FS aware:
 +
* You can pick up sector allocation sizes
 +
* Some file systems may store things off sector boundaries. (ReiserFS with tail packing)
 +
* Increasingly file systems have compression (NTFS compression)
 +
* Carve just the sectors that are not in allocated files.
 +
==Tasks that would be required==
 +
 
 +
==Discussion==
 +
:: As noted above, TSK should be utilized as much as possible, particularly the filesystem-aware portion.  If we want to identify filesystems outside of its supported set, it would be more worth our time to work on implementing them there than in the carver itself.  [[User:RB|RB]]
 +
 
 +
:::: I guess this tool operates like [[Selective file dumper]] and can recover files in both ways (or not?). Recovering files by using carving can recover files in situations where sleuthkit does nothing (e.g. file on NTFS was deleted using ntfs-3g, or filesystem was destroyed or just unknown). And we should build the list of filesystems supported by carver, not by TSK. [[User:.FUF|.FUF]] 07:08, 29 October 2008 (UTC)
 +
 
 +
:: This tool is still in the early planning stages (requirements discovery), hence few operational details (like precise modes of operation) have been fleshed out - those will and should come later.  The justification for strictly using TSK for the filesystem-sensitive approach is simple: TSK has good filesystem APIs, and it would be foolish to create yet another standalone, incompatible implementation of filesystem(foo) when time would be better spent improving those in TSK, aiding other methods of analysis as well.  This is the same reason individuals that have implemented several other carvers are participating: de-duplication of effort.  [[User:RB|RB]]
 +
 
 +
=Validator Construction=
 +
Options:
 +
* Write validators in C/C++
 +
* Have a scripting language for writing them (python? Perl?) our own?
 +
* Use existing programs (libjpeg?) as plug-in validators?
 +
=Existing Code that we have=
 +
=Implementation Timeline=
 +
# gather the available resources/ideas/wishes/needs etc. (I guess we're in this phase)
 +
# start discussing a high level design (in terms of algorithm, facilities, information needed)
 +
## input formats facility
 +
## partition/volume facility
 +
## file system facility
 +
## file format facility
 +
## content facility
 +
## how to deal with fragment detection (do the validators allow for fragment detection?)
 +
## how to deal with recombination of fragments
 +
## do we want multiple carving phases in light of speed/precision tradeoffs
 +
# start detailing parts of the design
 +
## Discuss options for a grammar driven validator?
 +
## Hard-coded plug-ins?
 +
## Which exsisting code can we use?
 +
# start building/assembling parts of the tooling for a prototype
 +
## Implement simple file carving with validation.
 +
## Implement gap carving
 +
# Initial Release
 +
# Implement the ''threaded carving'' that [[User:.FUF|.FUF]] is describing above.

Revision as of 17:13, 30 October 2008

This page is for planning Carver 2.0.

Please, do not delete text (ideas) here. Use something like this:

<s>bad idea</s>
:: good idea

This will look like:

bad idea

good idea

License

BSD-3.

OS

Linux/FreeBSD/MacOS

(shouldn't this just match what the underlying afflib & sleuthkit cover? RB)
Yes, but you need to test and validate on each. Question: Do we want to support windows? Simsong 21:09, 30 October 2008 (UTC)

Requirements

  • AFF and EWF file images supported from scratch.
  • File system aware layer.
    • By default, files are not carved. (clarify: only identified? RB; I guess that it operates like Selective file dumper .FUF 07:00, 29 October 2008 (UTC))
  • Plug-in architecture for identification/validation.
    • Can handle config files,like Revit07, to enter different file formats used by the carver.
  • Ship with validators for:
    • JPEG
    • PNG
    • GIF
    • MSOLE
    • ZIP
    • TAR (gz/bz2)
  • Simple fragment recovery carving using gap carving.
  • Recovering of individual ZIP sections and JPEG icons that are not sector aligned.
  • Autonomous operation (some mode of operation should be completely non-interactive, requiring no human intervention to complete RB)
  • Tested on 500GB-sized images. Should be able to carve a 500GB image in roughly 50% longer than it takes to read the image.
    • Perhaps allocate a percentage budget per-validator (i.e. each validator adds N% to the carving time)
  • Parallelizable.
  • Configuration:
    • Capability to parse some existing carvers' configuration files, either on-the-fly or as a one-way converter.
    • Disengage internal configuration structure from configuration files, create parsers that present the expected structure
    • Either extend Scalpel/Foremost syntaxes for extended features or use a tertiary syntax
  • Can output audit.txt file.
  • Easy integration into ascription software.

Ideas

  • Use as much TSK if possible. Don't carry your own FS implementation the way photorec does.
  • Extracting/carving data from Thumbs.db? I've used foremost for it with some success. Vinetto has some critical bugs :( .FUF 19:18, 28 October 2008 (UTC)
  • Carving data structures. For example, extract all TCP headers from image by defining TCP header structure and some fields (e.g. source port > 1024, dest port = 80). This will extract all data matching the pattern and write a file with other fields. Another example is carving INFO2 structures and URL activity records from index.dat .FUF 20:51, 28 October 2008 (UTC)
    • This has the opportunity to be extended to the concept of "point at blob FOO and interpret it as BAR"

.FUF added: The main idea is to allow users to define structures, for example (in pascal-like form):

Field1: Byte = 123;
SomeTextLength: DWORD;
SomeText: string[SomeTextLength];
Field4: Char = 'r';
...

This will produce something like this:

Field1 = 123
SomeTextLength = 5
SomeText = 'abcd1'
Field4 = 'r'

(In text or raw forms.)

Opinions?

Opinion: Simple pattern identification like that may not suffice, I think Simson's original intent was not only to identify but to allow for validation routines (plugins, as the original wording was). As such, the format syntax would need to implement a large chunk of some programming language in order to be sufficiently flexible. RB

File System Awareness

Background: Why be File System Aware?

Advantages of being FS aware:

  • You can pick up sector allocation sizes
  • Some file systems may store things off sector boundaries. (ReiserFS with tail packing)
  • Increasingly file systems have compression (NTFS compression)
  • Carve just the sectors that are not in allocated files.

Tasks that would be required

Discussion

As noted above, TSK should be utilized as much as possible, particularly the filesystem-aware portion. If we want to identify filesystems outside of its supported set, it would be more worth our time to work on implementing them there than in the carver itself. RB
I guess this tool operates like Selective file dumper and can recover files in both ways (or not?). Recovering files by using carving can recover files in situations where sleuthkit does nothing (e.g. file on NTFS was deleted using ntfs-3g, or filesystem was destroyed or just unknown). And we should build the list of filesystems supported by carver, not by TSK. .FUF 07:08, 29 October 2008 (UTC)
This tool is still in the early planning stages (requirements discovery), hence few operational details (like precise modes of operation) have been fleshed out - those will and should come later. The justification for strictly using TSK for the filesystem-sensitive approach is simple: TSK has good filesystem APIs, and it would be foolish to create yet another standalone, incompatible implementation of filesystem(foo) when time would be better spent improving those in TSK, aiding other methods of analysis as well. This is the same reason individuals that have implemented several other carvers are participating: de-duplication of effort. RB

Validator Construction

Options:

  • Write validators in C/C++
  • Have a scripting language for writing them (python? Perl?) our own?
  • Use existing programs (libjpeg?) as plug-in validators?

Existing Code that we have

Implementation Timeline

  1. gather the available resources/ideas/wishes/needs etc. (I guess we're in this phase)
  2. start discussing a high level design (in terms of algorithm, facilities, information needed)
    1. input formats facility
    2. partition/volume facility
    3. file system facility
    4. file format facility
    5. content facility
    6. how to deal with fragment detection (do the validators allow for fragment detection?)
    7. how to deal with recombination of fragments
    8. do we want multiple carving phases in light of speed/precision tradeoffs
  3. start detailing parts of the design
    1. Discuss options for a grammar driven validator?
    2. Hard-coded plug-ins?
    3. Which exsisting code can we use?
  4. start building/assembling parts of the tooling for a prototype
    1. Implement simple file carving with validation.
    2. Implement gap carving
  5. Initial Release
  6. Implement the threaded carving that .FUF is describing above.