OSDIHPFT - HPFS AND FAT FILE SYSTEMS 01/29/96 ======================================================================= HPFS and FAT File Systems--All OS/2 Releases ======================================================================= Please Read Entire Document for Full Explanation of Procedures -------------------------------------------------------------- DESCRIPTION This document explains the differences between a high performance file system (HPFS) and a file allocation table (FAT) file system and how they interface with each other under OS/2. RESOLUTION The following is an excerpt from A Technical Guide to OS/2 2.0, by Martin McElroy, IBM United Kingdom. OS/2 offers superior file system support, leading to increased performance. The OS/2 architecture allows other file systems to be installed in a modular fashion (Installable File System or IFS). It is more flexible in design to accommodate future enhancements; for example, now OS/2 includes CD-ROM support via an IFS. OS/2 also provides support for very large disks, a consideration particularly relevant to server environments. OS/2 supports hard disks up to 2 GB in size. In Windows 3.1 the limit is only 1 GB. In addition, OS/2 provides file I/O services not only to OS/2 applications but to DOS and Windows applications running in VDMs. Therefore, DOS and Windows applications can take advantage of advanced functions, without having to be modified, since the DOS emulation of MVDM provides a compatible interface to the file system for DOS applications. Both major OS/2 file systems have been improved under OS/2, allowing better performance not only for applications, but also in paging performed by systems when using virtual memory. High Performance File System (HPFS) ----------------------------------- The following is an excerpt from A Technical Guide to OS/2 2.0, by Martin McElroy, IBM United Kingdom. HPFS was first introduced to OS/2 Version 1.2, and is an example of the kind of advanced function that has not yet been implemented in less sophisticated systems like DOS. It was introduced as an alternative to the File Allocation Table (FAT) system which came from DOS. HPFS is particularly good for managing large disks, partitions, and large files. It provides fast and consistent performance, outperforming DOS-based FAT systems in nearly all cases. OS/2 implements an enhanced FAT system that uses similar caching features as HPFS, which also give high performance in many circumstances--see below. HPFS is particularly good in disk utilization compared to FAT. It uses a highly contiguous file allocation system, which results in good performance (relative to FAT) in accessing files or data in a cluttered or full partition. It implements a B-Tree directory structure and search algorithm, as opposed to sequential under FAT. HPFS also allows for multi-threaded I/O, caching of directory pointers in memory for quicker access of last directories used, and read-ahead and lazy write (lazy write buffers up write requests from applications and commits them to disk after a given time or during disk inactivity.) These advanced features allow for substantial performance increases and greater tuning. HPFS can provide write error recovery on the fly with 'hotfix' facilities. Since it is FAT-compatible at the API level, applications running under OS/2 can use either system, and do not have to be written specifically for one or the other. It presents a consistent interface to other components of OS/2 like MVDM, to allow DOS and Windows to use HPFS volumes as if they were FAT. HPFS supports the use of long file names, for greater usability, instead of using LJS1290.TXT you can have a file name, LETTER TO JOHN SMITH SEPTEMBER 90. Accordingly, applications need to be coded with this in mind. DOS and Windows applications use the 8.3 naming system on HPFS without any difficulty. HPFS has been enhanced in OS/2 to add performance-related features such as command chaining (providing a list of contiguous sector requests required to fulfill an I/O request) and scatter/gather facilities that are supported in Small Computer Systems Interface (SCSI) adapters to gather physically discontiguous pages in a data buffer, and perform I/O in a single operation. High Performance File System Changes ------------------------------------ The following is an excerpt from Chapter 6 of the IBM OS/2 Version 2.0 Technical Redbooks, Volume 1: Control Program (Document Number GG24-3730-00), published by the IBM Corporation. The following changes have been made to the High Performance File System (HPFS) driver under OS/2. At initialization time, the level of support provided by the device driver is determined using the new device command 1DH-GetDeviceSupport. The HPFS driver passes physical addresses for data pointers, in the appropriate request format, to the device driver. The HPFS driver now supports command chaining, calling the volume manager with a list of all contiguous sector requests required to fulfill an I/O request. This function is supported for all DASD types. The HPFS driver supports scatter/gather by passing physical pointers to each page in the data buffer (physically discontiguous) as part of the I/O request. This allows I/O controllers such as the IBM SCSI adapters which support the scatter/gather capability to perform the I/O in a single operation. * The HPFS driver supports disk caching in the IFS driver, rather than in the device driver. * The HPFS driver is able to recognize devices which have outboard caches (non-system memory), and incorporates them into the total caching scheme. * The HPFS file system under OS/2 supports a maximum file size of 2 GB. The maximum size for an HPFS volume is 512 GB. Enhanced FAT (Super FAT) File System ------------------------------------ The following is an excerpt from A Technical Guide to OS/2 2.0,by Martin McElroy, IBM United Kingdom. OS/2 includes an enhanced version of the FAT file system which is completely compatible with the FAT system under DOS. This gives greater performance but full compatibility with existing FAT systems. It adds features such as lazy write and improves caching to FAT. This means that DOS applications running with a FAT file system under OS/2 will be substantially faster for disk-based operations than under DOS. All the benefits can be obtained without having to reformat your hard disk. The OS/2 enhanced FAT driver works with existing DOS FAT volumes. FAT File System Changes ----------------------- The following is an excerpt from Chapter 6 of the IBM OS/2 Version 2.0 Technical Redbooks, Volume 1: Control Program (Document Number GG24-3730-00), published by IBM Corporation. Changes have been made to the FAT file system driver under OS/2, to provide improved performance and enhanced support for disk hardware devices. * The FAT driver supports command chaining. * The FAT driver attempts to call the volume manager with a list of all contiguous sector requests required to fulfill an I/O request, thus allowing multiple page-in and page-out requests in a single logical operation. * The FAT driver provides faster allocation of free space on the logical drive, using a bit map to track free clusters on the disk. Disk caching is now supported within the FAT driver, and has been removed from the device driver. A cache buffer is provided to support disk caching with the following features: * Lazy writing. * Lazy reading on writes. Meaning, the ability to write to the cache and flush the cache to disk, then, to read the updated information from the cache rather than requiring a physical disk read operation. * Asynchronous read-ahead through a multi-purpose asynchronous read thread. * Large cache size (theoretical maximum of 64 MB, although practical limitations will necessitate a smaller cache). * The ability to dynamically enable and disable the cache in response to a user command. * Bad sectors are automatically bypassed on reads. There are several advantages in performing caching in the FAT driver rather than the device driver; more operating-system kernel services are available at this level, and intelligent read-ahead operations are more easily performed. Lazy writing is easily implemented at the file system level than at the device driver level. The FAT file system under OS/2 supports a maximum file size of 2 GB. The maximum supported size for a FAT volume is also 2 GB. ______________________________________________________________________ IBM disclaims all warranties, whether express or implied, including without limitation, warranties of fitness and merchantability with respect to the information in this document. By furnishing this document, IBM grants no licenses to any related patents or copyrights. Copyright (c) 1994, 1996 IBM Corporation. Any trademarks and product or brand names referenced in this document are the property of their respective owners. Consult your product manuals for complete trademark information.