文件系统介绍

refer:  http://en.wikipedia.org/wiki/List_of_flash_file_systems#File_systems_optimized_for_flash_memory.2C_solid_state_media

Disk file systems

Disk file systems are usually block-oriented. Files in a block-oriented file system are sequences of blocks, often featuring fully random-access read, write, and modify operations.

File systems with built-in fault-tolerance

These file systems have built-in checksumming and either mirroring or parity for extra redundancy on one or several block devices.

File systems optimized for flash memory, solid state media

Solid state media, like flash memory, are similar to disks in their interfaces, but have different problems. On low level, they require special handling such as wear leveling and different error detection and correction algorithms. Typically a device such as solid-state disk handles such operations internally and therefore a regular file system can be used. However, for certain specialized installations (embedded systems, industrial applications) a file system optimized for plain flash memory is needed.

  • CASL is a filesystem designed by Nimble Storage that uses Solid State Devices to cache traditional hard drives.
  • ETFS - Embedded Transactional File System. Designed primarily for NAND devices by QNX Software Systems.
  • exFAT - Microsoft proprietary system intended for flash cards[5]
  • ExtremeFFS - Internal file system for SSDs.
  • F2FS – Flash-Friendly File System. An open source Linux file system introduced by Samsung in 2012.[6]
  • FFS2 (presumably preceded by FFS1), one of the earliest flash file systems. Developed and patented by Microsoft in the early 1990s.[7]
  • JFFS – Original log structured Linux file system for NOR flash media
  • JFFS2 – Successor of JFFS, for NAND and NOR flash
  • LogFS – Intended to replace JFFS2, better scalability. In early development.
  • Non-Volatile File System—the "non-volatile file system" for flash memory introduced by Palm, Inc..
  • OneFS - OneFS is a file system utilized by Isilon. It supports selective placement of meta-data directly onto flash SSD.
  • Segger Microcontroller Systems emFile - File system for deeply embedded applications which supports both NAND and NOR flashes. Wear leveling, fast read and write, and very low RAM usage.
  • TFAT – A transactional version of the FAT filesystem.
  • TrueFFS - Internal file system for SSDs, implementing error correction, bad block re-mapping and wear levelling.
  • UBIFS – Successor of JFFS2 optimized to utilize non-volatile DRAM
  • UFFS - Ultra low cost flash file system for embedded system [8]
  • Unison RTOS - Fsys-Nand/Nor small footprint low cost flash file system for embedded systems [9]
  • Write Anywhere File Layout - WAFL is an internal file system utilized by NetApp within their DataONTAP OS, originally optimized to use non-volatile DRAM
  • XCFiles – an exFAT implementation from Datalight for Wind River VxWorks and other embedded operating systems
  • YAFFS – A Log structured file system designed for NAND flash, but also used with NOR flash.
  • ZFS - Allows placing write-ahead log (ZIL) on flash, and using flash as a second-level read cache (L2ARC)

Record-oriented file systems

In record-oriented file systems files are stored as a collection of records. They are typically associated with mainframe and minicomputer operating systems. Programs read and write whole records, rather than bytes or arbitrary byte ranges, and can seek to a record boundary but not within records. The more sophisticated record-oriented file systems have more in common with simple databases than with other file systems.

Shared disk file systems

Shared disk file systems (also called shared storage file systems, SAN file system, Clustered file system or even cluster file systems) are primarily used in a storage area network where all nodes directly access the block storage where the file system is located. This makes it possible for nodes to fail without affecting access to the file system from the other nodes. Shared disk file systems are normally used in a high-availability cluster together with storage on hardware RAID. Shared disk file systems normally do not scale over 64 or 128 nodes.

Shared disk file systems may be symmetric where metadata is distributed among the nodes or asymmetric with centralized metadata servers.

Distributed file systems

Distributed file systems are also called network file systems. Many implementations have been made, they are location dependent and they have access control lists (ACLs), unless otherwise stated below.

Distributed fault-tolerant file systems

Distributed fault-tolerant replication of data between nodes (between servers or servers/clients) for high availability and offline (disconnected) operation.

Distributed parallel file systems

Distributed parallel file systems stripe data over multiple servers for high performance. They are normally used in high-performance computing (HPC).

Some of the distributed parallel file systems use object storage device (OSD) (In Lustre called OST) for chunks of data together with centralized metadata servers.

  • Fraunhofer Parallel File System (FhGFS) from the Fraunhofer Society Competence Center for High Performance Computing. Available free of charge for Linux under a proprietary license. (High availability features are on the roadmap.)
  • Parallel Virtual File System (PVFS, PVFS2). Developed to store virtual system images, with a focus on non shared writing optimizations. Available for Linux under GPL.
  • Starfish is a POSIX-compatible, N-way redundant file system created by Digital Bazaar Inc. and published under a pseudo open source license. Available for Linux and Mac OS. Windows support is available via Samba.

Distributed parallel fault-tolerant file systems

Distributed file systems, which also are parallel and fault tolerant, stripe and replicate data over multiple servers for high performance and to maintain data integrity. Even if a server fails no data is lost. The file systems are used in both high-performance computing (HPC) and high-availability clusters.

All file systems listed here focus on high availability, scalability and high performance unless otherwise stated below.

NameByLicenseOSDescription
CephInktankLGPLLinuxA massively scalable object store. Ceph was merged into the linux kernel in 2010. Ceph’s foundation is the Reliable Autonomic Distributed Object Store (RADOS), which provides object storage via programmatic interface and S3 or Swift REST APIs, block storage to QEMU/KVM/Linux hosts, and POSIX filesystem storage which can be mounted by linux kernel and FUSE clients.
CloudStoreKosmixApache License 2.0 Google File System workalike
CosmosMicrosoft internalinternal software Focuses on fault tolerance, high throughput and scalability. Designed for terabyte and petabyte sized data sets and processing with Dryad.
dCacheDESY and others  A write once filesystem, accessible via various protocols
ExaFSExanetproprietary software Distributed file system, runs as part of ExaStore, a Linux based NAS solution that runs on commodity Intel based hardware, serving NFS v2/v3, SMB/CIFS and AFP to Windows, Mac OS, Linux and other UNIX clients.
FS-ManagerCDNetworksproprietary softwareLinuxFocused on Content Delivery Network
Gfarm file systemAsia Pacific GridX11 LicenseLinux, Mac OS X, FreeBSD, NetBSD and SolarisUses OpenLDAP or PostgreSQL for metadata and FUSE or LUFS for mounting
General Parallel File System (GPFS)IBMproprietaryAIX, Linux and WindowsSupport replication between attached block storage. Symmetric or asymmetric (configurable)
GlusterFSGluster, a company acquired by Red HatGNU General Public License v3Linux, Mac OS X, NetBSD, FreeBSD, OpenSolarisA general purpose distributed file system for scalable storage. It aggregates various storage bricks over Infiniband RDMA or TCP/IP interconnect into one large parallel network file system. GlusterFS is the main component in Red Hat Storage Server.
Google File System (GFS)Googleinternal softwareGoogle App EngineFocus on fault tolerance, high throughput and scalability
IBRIX FusionIBRIXproprietary softwareLinux 
Lustreoriginally developed by Cluster File Systems and currently supported by Intel (formerly Whamcloud)GPLLinuxA POSIX-compliant, high-performance filesystem. Lustre has high availability via storage failover
MogileFSDanga InteractiveGPLLinux (but may be ported)Is not POSIX compliant, uses a flat namespace, application level, uses MySQL or Postgres for metadata and HTTP for transport.
OneFS distributed file systemIsilon FreeBSDBSD based OS on dedicated Intel based hardware, serving NFS v3 and SMB/CIFS to Windows, Mac OS, Linux and other UNIX clients under a proprietary software
Panasas ActiveScale File System (PanFS)Panasasproprietary softwareLinuxUses object storage devices
PeerFSRadiant Data Corporationproprietary softwareLinuxFocus on high availability and high performance and uses peer-to-peer replication with multiple sources and targets
Tahoe-LAFSTahoe-LAFS Software FoundationGNU GPL 2+ and other[16]Windows, Linux, OS Xsecure, decentralized, fault-tolerant, peer-to-peer distributed data store and distributed file system
TerraGrid Cluster File SystemTerrascale Technologies Incproprietary softwareLinuxImplements on demand cache coherency and uses industrial standard iSCSI and a modified version of the XFS file system
XtreemFS [3]Contrail E.U. project, the German MoSGrid project and the German project "First We Take Berlin"open-source (GPL)Linux, Solariscross-platform file system for wide area networks. It replicates the data for fault tolerance and caches metadata and data to improve performance over high-latency links. SSL and X.509 certificates support makes XtreemFS usable over public networks. It also supports Striping for usage in a cluster.
Chiron FS   [17] is a fuse-based, transparent replication file system, layering on an existing file system and implementing at the file system level what RAID 1 does at the device level. A notably convenient consequence is the possibility of picking single target directories, without the need of replicating entire partitions. (The project has no visible activity after 2008, a status request in Oct. 2009 in the chironfs-forum is unanswered)

In development:

  • PlasmaFS [18] is a free and open-source (GPL) userspace filesystem focusing on data safety and security. PlasmaFS provides a transactional API which is accessible over a SunRPC-based protocol. PlasmaFS can also be mounted as NFS volume, and is POSIX-compliant. Both data and metadata are replicated.
  • WebDFS An Open Source scalable, decentralized file store similar to MogileFS in function and purpose. Uses HTTP as the transport. Data is automatically and optimally re-arranged to accommodate the addition of new resources. The lack of central meta data management greatly simplifies deployment and use.
  • zFS from IBM (not to be confused with ZFS from Sun Microsystems or the zFS file system provided with IBM's z/OS operating system) focus on cooperative cache and distributed transactions and uses object storage devices. Under development and not freely available.
  • Hadoop Distributed File System - free GoogleFS clone produced by Apache. http://hadoop.apache.org/
  • HAMMER/ANVIL by Matt Dillon
  • OASIS from ETRI. Very similar to the Lustre or Panasas. Available for Linux via. special technology transfer program provided by ETRI.
  • GLORY-FS also from ETRI. Very similar to the Google File System or Hadoop, but it is fully POSIX compliant. It is specially optimized for large-scale web 2.0 content services. Version 2.5 is available for Linux via. special technology transfer program provided by ETRI. Windows version is under development.
  • parallax [19]
  • PNFS (Parallel NFS) - Clients available for Linux and OpenSolaris and back-ends from NetApp, Panasas, EMC Highroad and IBM GPFS [20]
  • Coherent Remote File System (CRFS) - requires Btrfs
  • Parallel Optimized Host Message Exchange Layered File System (POHMELFS) and Distributed STorage (DST). POSIX compliant, added to Linux kernel 2.6.30
  • Sector [4] from National Center for Data Mining. Sector is a high performance, scalable, and secure distributed file system. Available under Apache License 2.0
  • StarFS from CDNetworks. The StarFS is a global storage platform which supports virtualization of distributed file system and event-driven file synchronization with remote StarFS clusters.
  • Unilium [5] provides a decentralized, versioning file system stored in content addressable storage, whose data may be hosted across heterogeneous data storage nodes.

Peer-to-peer file systems

Some of these may be called cooperative storage cloud.

  • CFS is a read-only file system based on the Chord DHT
  • Cleversafe uses Cauchy Reed-Solomon Information Dispersal Algorithms (IDAs) to separate data into unrecognizable slices and distribute them, via secure Internet connections, to multiple storage locations.
  • Infinit is a peer-to-peer file system, coded in C++, allowing users to store, access and share their files in a safe and collaborative way from any Internet-connected device.
  • Ivy [21] is a multi-user read/write peer-to-peer file system. Ivy has no centralized or dedicated components, and it provides useful integrity properties without requiring users to fully trust either the underlying peer-to-peer storage system or the other users of the file system.
  • Pastis file system is a French peer-to-peer file system developed in Java
  • ColonyFS emphasises anonymity, security and dependability, is written in Java and C#, and is released under the GPL

Special purpose file systems

  • archfs (archive)
  • aufs an enhanced version of UnionFS stackable unification file system
  • AXFS (small footprint compressed read-only, with XIP)
  • Barracuda WebDAV plug-in. Secure Network File Server for embedded devices.
  • Boot File System was used on UnixWare to store files necessary to its boot process.
  • Cascade File System – provides file system access to Subversion and Perforce repositories and caches their contents locally
  • cdfs (reading and writing of CDs)
  • Compact Disc File System (reading and writing of CDs; experimental)
  • cfs (caching)
  • cvsfs (presents the CVS contents as mountable file system).
  • Dokan LGPL FUSE for Windows analog
  • compFUSEd (overlay transparent read-write compression, FUSE based)
  • FuseCompress (overlay transparent read-write compression, FUSE based)
  • Cramfs (small footprint compressed read-only)
  • Cromfs is a user-space (FUSE based) read-only filesystem using an efficient LZMA compression algorithm.
  • Davfs2 (WebDAV)-
  • Freenet – Decentralized, censorship-resistant
  • FTPFS/CurlFtpFS (ftp access)
  • GmailFS (Google Mail File System)
  • lnfs (long names)
  • mhddfs - Join several filesystems together to form a single larger one
  • mini fo (The mini fanout overlay file system) – Redirects modifying operations to a writeable location called "storage directory", and leaving the original data in the "base directory" untouched. When reading, the file system merges the modified and original data so that only the newest versions will appear. Most prominently used in OpenWrt[22]
  • MVFS – MultiVersion File System, proprietary, used by Rational ClearCase.
  • MTFS – stackable file system, reliability improvement mechanism for Lustre and other file systems based on replication.
  • nntpfs (netnews)
  • ParFiSys (Experimental parallel file system for massively parallel processing)
  • pramfs - Protected and Persistent RAM Filesystem
  • RAIF Redundant Array of Independent Filesystems - stackable RAID-like file system
  • romfs
  • SODA: a Lease-based Consistent Distributed File System - (early 1990s)
  • SquashFS (compressed read-only)
  • SysmanFS (based on FUSE, a virtual file system for cluster system management)
  • tmpfs in-memory temporary file system (on Linux platforms).
  • UMSDOS - FAT file system extended to store permissions and metadata, used for Linux
  • UnionFS - stackable unification file system, which can appear to merge the contents of several directories (branches), while keeping their physical content separate
  • Venti - Plan 9 de-duplicated storage used by Fossil.
  • wikifs (Plan 9) (wiki wiki)
  • WDK.VFS - SiteAdmin CMS Virtual File System introduced by Evgenios Skitsanos
  • Datalight Reliance - transactional file system for 32-bit embedded systems from Datalight, Inc.
  • ERTFS ProPlus64 - it comes with integrated Failsafe operation, it contains a default journaling mode.
  • WBFS - Wii Backup FileSystem
  • whefs - WanderingHorse.net Embedded Filesystem is an open source C library implementing an embedded/embeddable filesystem.

Pseudo- and virtual file systems

  • devfs – Virtual file system in Unix-like operating systems for managing devices on-the-fly
  • procfs – Pseudo-file system, used to access kernel information about processes
  • specfs – Special File System for device files
  • sysfs – Virtual file system in Unix-like operating systems holding information about buses, devices, firmware, filesystems, etc.
  • WinFS – Windows Future Storage, was planned as the successor to NTFS for Windows Vista.

Encrypted file systems

Files system interfaces

These are not really file systems; they allow access to file systems from an operating system standpoint.

  • FUSE (file system in userspace, like LUFS but better maintained)
  • LUFS (Linux userland file system - seems to be abandoned in favour of FUSE)
  • VFS Virtual Filesystem

See also

References

  1. ^ Zhu, Ben; Li, Kai; Patterson, Hugo (2008). "Avoiding the Disk Bottleneck in the Data Domain Deduplication File System". Proceedings of the 6th USENIX Conference on File and Storage Technologies (FAST ’08). San Jose, CA. pp. 269-282.
  2. ^ Corbet, Jonathan. "The Next3 filesystem". LWN.
  3. ^ a b "Help - IBM AIX Compilers". Publib.boulder.ibm.com. Retrieved 2012-06-15.
  4. ^ "ZFS on Linux". ZFS on Linux. Retrieved 2012-06-15.
  5. ^ "Description of the exFAT file system driver update package". Support.microsoft.com. 2011-10-08. Retrieved 2012-06-15.
  6. ^ Michael Larabel (2011-10-05). "Samsung Introduces New Linux File-System: F2FS". phoronix.com. Retrieved 2012-12-07.
  7. ^ "United States Patent: 5392427". Patft.uspto.gov. Retrieved 2012-06-15.
  8. ^ "gouffs". Sites.google.com. Retrieved 2012-06-15.
  9. ^ "Embedded File System". RoweBots. Retrieved 2012-06-15.
  10. ^ Pirkola, G. C. (June 1975). "A file system for a general-purpose time-sharing environment". Proceedings of the IEEE 63 (6): 918–924. doi:10.1109/PROC.1975.9856. ISSN 0018-9219.
  11. ^ "Encina". Zois.co.uk. Retrieved 2012-06-15.
  12. ^ "IBM - TXSeries for Multiplatforms, V7.1 - TXSeries for Multiplatforms - Software". 306.ibm.com. Retrieved 2012-06-15.
  13. ^ "Melio FS". Sanbolic. Retrieved 2012-09-21.
  14. ^ http://www.polyserve.com/matrix_server_windows.php
  15. ^ "tahoe-lafs". Allmydata.org. Retrieved 2012-06-15.
  16. ^ https://tahoe-lafs.org/trac/tahoe-lafs/browser/git/docs/about.rst
  17. ^ "ChironFS - A filesystem must go on, even if it is hurt". Furquim.org. Retrieved 2012-06-15.
  18. ^ "The Plasma Project". Plasma.camlcity.org. 2011-10-02. Retrieved 2012-06-15.
  19. ^ "Parallax: Managing Storage for a Million Machines". University of Cambridge Computer Laboratory. Retrieved 2 December 2008.
  20. ^ "pNFS".
  21. ^ http://pdos.csail.mit.edu/ivy/osdi02.pdf
  22. ^ "Flash Layout - OpenWrt Wiki". Wiki.openwrt.org. 2011-12-20. Retrieved 2012-06-15.

评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值