Difference between pages "National Software Reference Library" and "New Technology File System (NTFS)"

From ForensicsWiki
(Difference between pages)
Jump to: navigation, search
m
 
(Changes in Windows Vista)
 
Line 1: Line 1:
The '''National Software Reference Library''' (NSRL) is the National Institute of Standards and Technology's National Software Reference Library. The NSRL is a physical resource located in Gaithersburg Maryland. The NSRL consists of more than 21,000 individual software packages. NIST has the original packaging and distribution media for 15,000 packages, and archived digital distributions for the remainder.
+
The '''New Technology File System''' ('''NTFS''') is a [[file system]] developed and introduced by [[Microsoft]] in 1995 with [[Windows]] NT. As a replacement for the [[FAT]] file system, it quickly became the standard for [[Windows 2000]], [[Windows XP]] and [[Windows Server 2003]].
  
RDS is the Reference Data Set. The RDS consists of the metadata describing software package titles, manufacturers, operating systems, file metadata and hash codes of the files in the NSRL. Originally (ca. 2000-2002) it was created by installing the software on systems and then generating a list of the hash codes. During 2003-2012 it was created by processing only the distributed media and extracting the installation files from the Microsoft .CAB, .MSI and .ZIP files, Unix/Linux .RPM and .DEB files, Apple .DMG files, and generic archive files. As of 2013, operating systems and packages are installed in virtual machines to facilitate collection of metadata and hashes from those installations.
+
The features of NTFS include:
  
The RDS is typically used for [[Data Reduction|data reduction]]. That is, the set of hash codes is used as a filter to eliminate or highlight files from examination.  Most frequently the RDS used as a list of ''known goods'' that can be safely suppressed.  ''This is an incorrect usage of the RDS and should be discouraged,'' because the RDS does not indicate if a file is known good or bad, only that it is known. Indeed, the RDS has many files that were once throught to be good, but are now thought to be bad---for example, versions of Adobe Flash with known security vulnerabilities. Categories such as "Steganography" or "Keylogger" are assigned to allow filtering by need.
+
* [[Hard-links]]
 +
* Improved performance, reliability and disk space utilization
 +
* Security [[access control lists]]
 +
* File system journaling
  
The NSRL is distributed online can be downloaded from the [http://www.nsrl.nist.gov/Downloads.htm NSRL website]. The most recent release was version 2.40 in March 2013.
+
== Time Stamps ==
  
== RDS File Format ==
+
NTFS keeps track of lots of time stamps. Each file has a time stamp for 'Create', 'Modify', 'Access', and 'Entry Modified'. The latter refers to the time when the MFT entry itself was modified. These four values are commonly abbreviated as the 'MACE' values. Note that other attributes in each MFT record may also contain timestamps that are of forensic value.
  
Each RDS consists of several files, but the hashes are stored in <tt>NSRLFile.txt</tt>. These files have a header followed by many hash records. The header denotes the columns in each file. (See the External Links for the complete specification). RDS files can be used directly with programs like [[md5deep]], [[Forensic Toolkit|FTK]], and [[EnCase]].
+
Additional information on how NTFS timestamps work when files are moved or copied is available here: [http://support.microsoft.com/kb/299648 Microsoft KB 299648]
  
The file format has changed slightly over time. Releases occur four times per year. The latest version was dated 1 Mar 2013:
+
=== Changes in Windows Vista  ===
 +
In Windows Vista, NTFS no longer tracks the Last Access time of a file by default. This feature can be enabled by setting the NtfsDisableLastAccessUpdate value to '0' in the Registry key:
 +
<pre>HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\FileSystem</pre>
  
=== Version 2.0 ===
+
Note that this feature has been around since as early as Windows 2000 [http://technet.microsoft.com/en-us/library/cc959914.aspx].
  
Starting in version 2.0, the NSRL moved the hashes to the start of each line and dropped the [[MD4]] hash. The file header:
+
== Alternate Data Streams ==
 +
The '''NTFS''' file system includes a feature referred to as Alternate Data Streams (ADSs). This feature has also been referred to as "multiple data streams", "alternative data streams", etc.  ADSs were included in '''NTFS''' in order to support the resource forks employed by the Hierarchal File System ([[HFS]]) employed by Macintosh systems.
  
<pre>"SHA-1","MD5","CRC32","FileName","FileSize","ProductCode","OpSystemCode","SpecialCode"</pre>
+
As of [[Windows XP]] SP2, files downloaded via Internet Explorer, Outlook, and Windows Messenger were automatically given specific "zoneid" ADSs.  The [[Windows]] Explorer shell would then display a warning when the user attempted to execute these files (by double-clicking them).
  
=== Version 1.5 ===
+
Sysadmins should be aware that prior to Vista, there are no tools native to the [[Windows]] platform that would allow you to view the existence of arbitrary ADSs.  While ADSs can be created and their contents executed or viewed, it wasn't until the "/r" switch was introduced with the "dir" command on Vista that arbitrary ADSs would be visible.  Prior to this, tools such as [http://www.heysoft.de/Frames/f_sw_la_en.htm LADS] could be used to view the existence of these files.
  
Information on the older header version is kept here so that programs can read older files. The file header:
+
Microsoft FSRM (File System Resource Manager) also uses ADS as part of 'file classification'.
  
<pre>"SHA-1","FileName","FileSize","ProductCode","OpSystemCode","MD4","MD5","CRC32","SpecialCode"</pre>
+
Examiners should be aware that most forensic analysis applications, including [[EnCase]] and ProDiscover, will display ADSs found in acquired images in red.
  
<tt>OpSystemCode</tt> refers to the operating system code. The <tt>SpecialCode</tt> is a single character that can be used to mark records. A normal file has a blank value here. An <tt>M</tt> in this field denotes a malicious file.
+
== Advanced Format (4KB Sector) Hard Drives ==
 +
NTFS does not natively handle drives that use the new standard of 4KB sectors. For information on this, see [[Advanced Format]].
  
== External Links ==
+
== Transactional NTFS (TxF) ==
  
* [http://www.nsrl.nist.gov/ NSRL website]
+
According to MSDN Transactional NTFS (TxF) allows file operations on an NTFS file system volume to be performed in a transaction.  
* [http://www.nsrl.nist.gov/documents/Data-Formats-of-the-NSRL-Reference-Data-Set-14.pdf NSRL RDS Data File Format] - Describes the format of the hash files
+
  
[[Category:Hashing]]
+
Several TxF related file-system-metadata files can be found in the file-system-metadata directory: \$Extend\$RmMetadata\. TxF also uses the MFT attribute $LOGGING_UTILITY_STREAM with the name $TXF_DATA.
 +
 
 +
TxF uses the [[Common Log File System (CLFS)]]
 +
 
 +
== External links ==
 +
* [http://en.wikipedia.org/wiki/NTFS Wikipedia: NTFS]
 +
* [http://msdn.microsoft.com/en-us/library/bb968806%28v=VS.85%29.aspx MSDN on Transactional NTFS]
 +
* [http://en.wikipedia.org/wiki/Transactional_NTFS Wikipedia on Transactional NTFS]
 +
* [http://www.tzworks.net/prototype_page.php?proto_id=12  Windows NTFS Metadata Extractor Utility] Free tool that can be run on Windows, Linux or Mac OS-X
 +
* [http://www.tzworks.net/prototype_page.php?proto_id=28 Graphic Engine for NTFS Analysis (gena)] (GUI to view NTFS internals/extract data on live systems)
 +
* [http://sourceforge.net/projects/linux-ntfs/files/NTFS%20Documentation/ Linux-ntfs Documentation] Detailed documentation of the NTFS format by the Linux-NTFS driver creators.
 +
* [http://support.microsoft.com/kb/140365 Default cluster size for NTFS, FAT, and exFAT]
 +
* [http://code.google.com/p/libfslibs/downloads/detail?name=New%20Technologies%20File%20System%20%28NTFS%29.pdf New Technologies File System (NTFS)], by the [[libfslibs|libfslibs project]], August 2009
 +
 
 +
[[Category:File Systems]]

Revision as of 09:33, 14 September 2013

The New Technology File System (NTFS) is a file system developed and introduced by Microsoft in 1995 with Windows NT. As a replacement for the FAT file system, it quickly became the standard for Windows 2000, Windows XP and Windows Server 2003.

The features of NTFS include:

Time Stamps

NTFS keeps track of lots of time stamps. Each file has a time stamp for 'Create', 'Modify', 'Access', and 'Entry Modified'. The latter refers to the time when the MFT entry itself was modified. These four values are commonly abbreviated as the 'MACE' values. Note that other attributes in each MFT record may also contain timestamps that are of forensic value.

Additional information on how NTFS timestamps work when files are moved or copied is available here: Microsoft KB 299648

Changes in Windows Vista

In Windows Vista, NTFS no longer tracks the Last Access time of a file by default. This feature can be enabled by setting the NtfsDisableLastAccessUpdate value to '0' in the Registry key:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\FileSystem

Note that this feature has been around since as early as Windows 2000 [1].

Alternate Data Streams

The NTFS file system includes a feature referred to as Alternate Data Streams (ADSs). This feature has also been referred to as "multiple data streams", "alternative data streams", etc. ADSs were included in NTFS in order to support the resource forks employed by the Hierarchal File System (HFS) employed by Macintosh systems.

As of Windows XP SP2, files downloaded via Internet Explorer, Outlook, and Windows Messenger were automatically given specific "zoneid" ADSs. The Windows Explorer shell would then display a warning when the user attempted to execute these files (by double-clicking them).

Sysadmins should be aware that prior to Vista, there are no tools native to the Windows platform that would allow you to view the existence of arbitrary ADSs. While ADSs can be created and their contents executed or viewed, it wasn't until the "/r" switch was introduced with the "dir" command on Vista that arbitrary ADSs would be visible. Prior to this, tools such as LADS could be used to view the existence of these files.

Microsoft FSRM (File System Resource Manager) also uses ADS as part of 'file classification'.

Examiners should be aware that most forensic analysis applications, including EnCase and ProDiscover, will display ADSs found in acquired images in red.

Advanced Format (4KB Sector) Hard Drives

NTFS does not natively handle drives that use the new standard of 4KB sectors. For information on this, see Advanced Format.

Transactional NTFS (TxF)

According to MSDN Transactional NTFS (TxF) allows file operations on an NTFS file system volume to be performed in a transaction.

Several TxF related file-system-metadata files can be found in the file-system-metadata directory: \$Extend\$RmMetadata\. TxF also uses the MFT attribute $LOGGING_UTILITY_STREAM with the name $TXF_DATA.

TxF uses the Common Log File System (CLFS)

External links