FAT times at Ridgemont High
Given that it was originally a quick and dirty clone of a file system designed for 8-bit microcomputers in the 1970s that was itself a quick-and-dirty hack that mimicked the minicomputers of a decade earlier, FAT was not really up for very much. It retained CP/M's "8 and 3" file name limit, and the way it stored files was designed around the physical structure of the floppy disk drive, the primary storage device of the day.
The File Allocation Table described which areas of the disk were allocated to files, which were free space, and which were damaged and unusable (called "bad sectors"). Because each floppy disk had very little space (the first single-sided disks in the IBM PC could store only 160 kilobytes) the table itself needed to be very small. To keep it small, the disk was divided into clusters—groups of sectors stored next to each other on the disk. The first version of FAT was called FAT-12, because it used a 12-bit number to count the clusters (2 to the power of 12 is 4096, and each cluster was 8KB, so the maximum volume size was 32MB).
The original IBM PC 5.25-inch floppy disk, with superimposed tracks/sectors
FAT had no clue about directories or optimizing file storage, and just threw all the bits of files on the first part of the disk where it found any space. For a floppy disk, this didn't matter much, because you typically only stored a few files on it anyway. However, IBM was getting ready to release their PC-XT with an optional 20MB hard disk. This (for the time) enormous space meant that FAT would need a way to store files in a proper hierarchy. For MS-DOS 2.0, Microsoft added nested directories, labeled with the backslash () as the separator, so a file might be stored in, say, C:MYFILESNOTES. Why a backslash and not a forward slash, as God (and Unix) intended? Well, the forward slash was already used by some MS-DOS 1.0 programs as a modifier argument (say, FORMAT A: /S to add system files after formatting) and it was just too much work to change it now. Why C:? Well, A: and B: were reserved for the first and second floppy disk drives that everyone had to have before hard disks became standard.
The introduction of hard disks soon made FAT-12 obsolete, so Microsoft came up with a 16-bit version in DOS 3.31, released in 1987. It had 32KB clusters and could access 216 of them, for an astounding 2GB maximum disk size.
And what about the problem of storage optimization? With a hard drive, a user was always copying and deleting files, and this left holes that new files were crammed into (since FAT just looked for the first space it could store stuff) which led to real problems with fragmentation. This made hard drives work harder than ever, jumping around willy-nilly trying to find all the bits of a file that were strewn about the drive. Did Microsoft fix this problem with FAT? Not at all. They let other companies make programs like Norton Utilities and PC-Tools that would defragment the whole disk in one go, giving users happy evenings to remember forever, sitting watching the screen move little rectangles around.
By 1995, hard disks were getting larger than the 2GB limit, and the 8.3 file name limit seemed even more archaic than it had been 20 years ago. To solve both these problems, Microsoft introduced FAT-32, with an 8TB (terabyte) limit and a special magic ability called VFAT that gave FAT the ability to have long file names without really having them.
VFAT took a long file name (up to 255 characters) and created a short version that fit into the 8.3 straitjacket. For example, "Super-long file name.txt" would be stored as "SUPER-~1.TXT". The remaining letters were thrown into a very strange kind of nonexistence, like living on an astral plane. They were sliced up into 13-letter chunks and stored as phantom directories that were marked with the metadata attributes of Volume Label, System, Hidden, and Read-Only, a combination that confused older versions of MS-DOS so much that they flat-out ignored them. However, if a long file name was deleted from DOS, it considered the associated phantom directory full of Volume Label entries to be empty and deleted it as well. Even stranger tricks were added to check whether the long file name matched the 8.3 one.
Some of these tricks still haunt us today, thanks to shaky third-party implementations of VFAT. For example, while writing this article I had the fun experience of watching a directory on my USB flash drive spontaneously change from "Williams files" to "WILLIA~1" when transferring a file to my iBook.
So did Microsoft address the horrible fragmentation problems of FAT in this version either? The answer is: not so much. But it did at least include a home-grown defragmenting program with Windows 95. Thanks to the power of multitasking, you could even do other things while it was defragging! Except you couldn't, because it would complain that "files had changed" and keep starting over.
This strange VFAT solution was a ghastly hack, but it did allow seamless upgrades of DOS and Windows 3.1 systems to Windows 95 and long file names, while still letting poor DOS users access these files. It also caused Macintosh users to laugh and make "MICROS~1" jokes that nobody ever understood. They probably shouldn't have been laughing, however, because the Macintosh file system had even stranger limitations.