zfsprops

title: zfsprops(7)

text:
# zfsprops(7)

## NAME
zfsprops — native and user-defined properties of ZFS datasets

## DESCRIPTION
Properties are divided into two types, native properties and user-defined (or "user") properties. Native properties either export internal statistics or control ZFS behavior. In addition, native properties are either editable or read-only. User properties have no effect on ZFS behavior, but you can use them to annotate datasets in a way that is meaningful in your environment. For more information about user properties, see the User Properties section, below.

### Native Properties
Every dataset has a set of properties that export statistics about the dataset as well as control various behaviors. Properties are inherited from the parent unless overridden by the child. Some properties apply only to certain types of datasets (file systems, volumes, or snapshots).

The values of numeric properties can be specified using human-readable suffixes (for example, k, KB, M, Gb, and so forth, up to Z for zettabyte). The following are all valid (and equal) specifications: 1536M, 1.5g, 1.50GB.

The values of non-numeric properties are case sensitive and must be lowercase, except for mountpoint, sharenfs, and sharesmb.

The following native properties consist of read-only statistics about the dataset. These properties can be neither set, nor inherited. Native properties apply to all dataset types unless otherwise noted.

available

The amount of space available to the dataset and all its children, assuming that there is no other activity in the pool. Because space is shared within a pool, availability can be limited by any number of factors, including physical pool size, quotas, reservations, or other datasets within the pool. This property can also be referred to by its shortened column name, avail.

compressratio

For non-snapshots, the compression ratio achieved for the used space of this dataset, expressed as a multiplier. The used property includes descendant datasets, and, for clones, does not include the space shared with the origin snapshot. For snapshots, the compressratio is the same as the refcompressratio property. Compression can be turned on by running: zfs set compression=on dataset. The default value is off.

createtxg

The transaction group (txg) in which the dataset was created. Bookmarks have the same createtxg as the snapshot they are initially tied to. This property is suitable for ordering a list of snapshots, e.g. for incremental send and receive.

creation

The time this dataset was created.

clones

For snapshots, this property is a comma-separated list of filesystems or volumes which are clones of this snapshot. The clones' origin property is this snapshot. If the clones property is not empty, then this snapshot can not be destroyed (even with the -r or -f options). The roles of origin and clone can be swapped by promoting the clone with the zfs promote command.

defer_destroy

This property is on if the snapshot has been marked for deferred destroy by using the zfs destroy -d command. Otherwise, the property is off.

encryptionroot

For encrypted datasets, indicates where the dataset is currently inheriting its encryption key from. Loading or unloading a key for the encryptionroot will implicitly load / unload the key for any inheriting datasets (see zfs load-key and zfs unload-key for details). Clones will always share an encryption key with their origin. See the Encryption section of zfs-load-key(8) for details.

filesystem_count

The total number of filesystems and volumes that exist under this location in the dataset tree. This value is only available when a filesystem_limit has been set somewhere in the tree under which the dataset resides.

keystatus

Indicates if an encryption key is currently loaded into ZFS. The possible values are none, available, and unavailable. See zfs load-key and zfs unload-key.

guid

The 64 bit GUID of this dataset or bookmark which does not change over its entire lifetime. When a snapshot is sent to another pool, the received snapshot has the same GUID. Thus, the guid is suitable to identify a snapshot across pools.

logicalreferenced

The amount of space that is "logically" accessible by this dataset. See the referenced property. The logical space ignores the effect of the compression and copies properties, giving a quantity closer to the amount of data that applications see. However, it does include space consumed by metadata. This property can also be referred to by its shortened column name, lrefer.

logicalused

The amount of space that is "logically" consumed by this dataset and all its descendents. See the used property. The logical space ignores the effect of the compression and copies properties, giving a quantity closer to the amount of data that applications see. However, it does include space consumed by metadata. This property can also be referred to by its shortened column name, lused.

mounted

For file systems, indicates whether the file system is currently mounted. This property can be either yes or no.

objsetid

A unique identifier for this dataset within the pool. Unlike the dataset's guid, the objsetid of a dataset is not transferred to other pools when the snapshot is copied with a send/receive operation. The objsetid can be reused (for a new dataset) after the dataset is deleted.

origin

For cloned file systems or volumes, the snapshot from which the clone was created. See also the clones property.

receive_resume_token

For filesystems or volumes which have saved partially-completed state from zfs receive -s, this opaque token can be provided to zfs send -t to resume and complete the zfs receive.

redact_snaps

For bookmarks, this is the list of snapshot guids the bookmark contains a redaction list for. For snapshots, this is the list of snapshot guids the snapshot is redacted with respect to.

referenced

The amount of data that is accessible by this dataset, which may or may not be shared with other datasets in the pool. When a snapshot or clone is created, it initially references the same amount of space as the file system or snapshot it was created from, since its contents are identical. This property can also be referred to by its shortened column name, refer.

refcompressratio

The compression ratio achieved for the referenced space of this dataset, expressed as a multiplier. See also the compressratio property.

snapshot_count

The total number of snapshots that exist under this location in the dataset tree. This value is only available when a snapshot_limit has been set somewhere in the tree under which the dataset resides.

type

The type of dataset: filesystem, volume, snapshot, or bookmark.

used

The amount of space consumed by this dataset and all its descendents. This is the value that is checked against this dataset's quota and reservation. The space used does not include this dataset's reservation, but does take into account the reservations of any descendent datasets. The amount of space that a dataset consumes from its parent, as well as the amount of space that is freed if this dataset is recursively destroyed, is the greater of its space used and its reservation. The used space of a snapshot (see the Snapshots section of zfsconcepts(7)) is space that is referenced exclusively by this snapshot. If this snapshot is destroyed, the amount of used space will be freed. Space that is shared by multiple snapshots isn't accounted for in this metric. When a snapshot is destroyed, space that was previously shared with this snapshot can become unique to snapshots adjacent to it, thus changing the used space of those snapshots. The used space of the latest snapshot can also be affected by changes in the file system. Note that the used space of a snapshot is a subset of the written space of the snapshot.

The amount of space used, available, or referenced does not take into account pending changes. Pending changes are generally accounted for within a few seconds. Committing a change to a disk using fsync(2) or O_SYNC does not necessarily guarantee that the space usage information is updated immediately.

usedby*

The usedby* properties decompose the used properties into the various reasons that space is used. Specifically, used = usedbychildren + usedbydataset + usedbyrefreservation + usedbysnapshots. These properties are only available for datasets created on zpool "version 13" pools.

usedbychildren

The amount of space used by children of this dataset, which would be freed if all the dataset's children were destroyed.

usedbydataset

The amount of space used by this dataset itself, which would be freed if the dataset were destroyed (after first removing any refreservation and destroying any necessary snapshots or descendents).

usedbyrefreservation

The amount of space used by a refreservation set on this dataset, which would be freed if the refreservation was removed.

usedbysnapshots

The amount of space consumed by snapshots of this dataset. In particular, it is the amount of space that would be freed if all of this dataset's snapshots were destroyed. Note that this is not simply the sum of the snapshots' used properties because space can be shared by multiple snapshots.

userused@user

The amount of space consumed by the specified user in this dataset. Space is charged to the owner of each file, as displayed by ls -l. The amount of space charged is displayed by du and ls -s. See the zfs userspace command for more information. Unprivileged users can access only their own space usage. The root user, or a user who has been granted the userused privilege with zfs allow, can access everyone's usage.

The userused@... properties are not displayed by zfs get all. The user's name must be appended after the @ symbol, using one of the following forms:

- POSIX name ("joe")

- POSIX numeric ID ("789")

- SID name ("joe.smith@mydomain")

- SID numeric ID ("S-1-123-456-789")

Files created on Linux always have POSIX owners.

userobjused@user

The userobjused property is similar to userused but instead it counts the number of objects consumed by a user. This property counts all objects allocated on behalf of the user, it may differ from the results of system tools such as df -i. When the property xattr=on is set on a file system additional objects will be created per-file to store extended attributes. These additional objects are reflected in the userobjused value and are counted against the user's userobjquota. When a file system is configured to use xattr=sa no additional internal objects are normally required.

userrefs

This property is set to the number of user holds on this snapshot. User holds are set by using the zfs hold command.

groupused@group

The amount of space consumed by the specified group in this dataset. Space is charged to the group of each file, as displayed by ls -l. See the userused@user property for more information. Unprivileged users can only access their own groups' space usage. The root user, or a user who has been granted the groupused privilege with zfs allow, can access all groups' usage.

groupobjused@group

The number of objects consumed by the specified group in this dataset. Multiple objects may be charged to the group for each file when extended attributes are in use. See the userobjused@user property for more information. Unprivileged users can only access their own groups' space usage. The root user, or a user who has been granted the groupobjused privilege with zfs allow, can access all groups' usage.

projectused@project

The amount of space consumed by the specified project in this dataset. Project is identified via the project identifier (ID) that is object-based numeral attribute. An object can inherit the project ID from its parent object (if the parent has the flag of inherit project ID that can be set and changed via chattr -/+P or zfs project -s) when being created. The privileged user can set and change object's project ID via chattr -p or zfs project -s anytime. Space is charged to the project of each file, as displayed by lsattr -p or zfs project. See the userused@user property for more information. The root user, or a user who has been granted the projectused privilege with zfs allow, can access all projects' usage.

projectobjused@project

The projectobjused is similar to projectused but instead it counts the number of objects consumed by project. When the property xattr=on is set on a fileset, ZFS will create additional objects per-file to store extended attributes. These additional objects are reflected in the projectobjused value and are counted against the project's projectobjquota. When a filesystem is configured to use xattr=sa no additional internal objects are required. See the userobjused@user property for more information. The root user, or a user who has been granted the projectobjused privilege with zfs allow, can access all projects' objects usage.

volblocksize

For volumes, specifies the block size of the volume. The blocksize cannot be changed once the volume has been written, so it should be set at volume creation time. The default blocksize for volumes is 8 Kbytes. Any power of 2 from 512 bytes to 128 Kbytes is valid. This property can also be referred to by its shortened column name, volblock.

written

The amount of space referenced by this dataset, that was written since the previous snapshot (i.e. that is not referenced by the previous snapshot).

written@snapshot

The amount of referenced space written to this dataset since the specified snapshot. This is the space that is referenced by this dataset but was not referenced by the specified snapshot. The snapshot may be specified as a short snapshot name (just the part after the @), in which case it will be interpreted as a snapshot in the same filesystem as this dataset. The snapshot may be a full snapshot name (filesystem@snapshot), which for clones may be a snapshot in the origin's filesystem (or the origin of the origin's filesystem, etc.)

The following native properties can be used to change the behavior of a ZFS dataset.

aclinherit=discard|noallow|restricted|passthrough|passthrough-x

Controls how ACEs are inherited when files and directories are created. discard

does not inherit any ACEs.

noallow

only inherits inheritable ACEs that specify "deny" permissions.

restricted

default, removes the write_acl and write_owner permissions when the ACE is inherited.

passthrough

inherits all inheritable ACEs without any modifications.

passthrough-x

same meaning as passthrough, except that the owner@, group@, and everyone@ ACEs inherit the execute permission only if the file creation mode also requests the execute bit.

When the property value is set to passthrough, files are created with a mode determined by the inheritable ACEs. If no inheritable ACEs exist that affect the mode, then the mode is set in accordance to the requested mode from the application.

The aclinherit property does not apply to POSIX ACLs.

aclmode=discard|groupmask|passthrough|restricted

Controls how an ACL is modified during chmod(2) and how inherited ACEs are modified by the file creation mode: discard

default, deletes all ACEs except for those representing the mode of the file or directory requested by chmod(2).

groupmask

reduces permissions granted in all ALLOW entries found in the ACL such that they are no greater than the group permissions specified by chmod(2).

passthrough

indicates that no changes are made to the ACL other than creating or updating the necessary ACL entries to represent the new mode of the file or directory.

restricted

will cause the chmod(2) operation to return an error when used on any file or directory which has a non-trivial ACL whose entries can not be represented by a mode. chmod(2) is required to change the set user ID, set group ID, or sticky bits on a file or directory, as they do not have equivalent ACL entries. In order to use chmod(2) on a file or directory with a non-trivial ACL when aclmode is set to restricted, you must first remove all ACL entries which do not represent the current mode.

acltype=off|nfsv4|posix

Controls whether ACLs are enabled and if so what type of ACL to use. When this property is set to a type of ACL not supported by the current platform, the behavior is the same as if it were set to off. off

default on Linux, when a file system has the acltype property set to off then ACLs are disabled.

noacl

an alias for off

nfsv4

default on FreeBSD, indicates that NFSv4-style ZFS ACLs should be used. These ACLs can be managed with the getfacl(1) and setfacl(1). The nfsv4 ZFS ACL type is not yet supported on Linux.

posix

indicates POSIX ACLs should be used. POSIX ACLs are specific to Linux and are not functional on other platforms. POSIX ACLs are stored as an extended attribute and therefore will not overwrite any existing NFSv4 ACLs which may be set.

posixacl

an alias for posix

To obtain the best performance when setting posix users are strongly encouraged to set the xattr=sa property. This will result in the POSIX ACL being stored more efficiently on disk. But as a consequence, all new extended attributes will only be accessible from OpenZFS implementations which support the xattr=sa property. See the xattr property for more details.

atime=on|off

Controls whether the access time for files is updated when they are read. Turning this property off avoids producing write traffic when reading files and can result in significant performance gains, though it might confuse mailers and other similar utilities. The values on and off are equivalent to the atime and noatime mount options. The default value is on. See also relatime below.

canmount=on|off|noauto

If this property is set to off, the file system cannot be mounted, and is ignored by zfs mount -a. Setting this property to off is similar to setting the mountpoint property to none, except that the dataset still has a normal mountpoint property, which can be inherited. Setting this property to off allows datasets to be used solely as a mechanism to inherit properties. One example of setting canmount=off is to have two datasets with the same mountpoint, so that the children of both datasets appear in the same directory, but might have different inherited characteristics. When set to noauto, a dataset can only be mounted and unmounted explicitly. The dataset is not mounted automatically when the dataset is created or imported, nor is it mounted by the zfs mount -a command or unmounted by the zfs unmount -a command.

This property is not inherited.

checksum=on|off|fletcher2|fletcher4|sha256|noparity|sha512|skein|edonr

Controls the checksum used to verify data integrity. The default value is on, which automatically selects an appropriate algorithm (currently, fletcher4, but this may change in future releases). The value off disables integrity checking on user data. The value noparity not only disables integrity but also disables maintaining parity for user data. This setting is used internally by a dump device residing on a RAID-Z pool and should not be used by any other dataset. Disabling checksums is NOT a recommended practice. The sha512, skein, and edonr checksum algorithms require enabling the appropriate features on the pool. FreeBSD does not support the edonr algorithm.

Please see zpool-features(7) for more information on these algorithms.

Changing this property affects only newly-written data.

compression=on|off|gzip|gzip-N|lz4|lzjb|zle|zstd|zstd-N|zstd-fast|zstd-fast-N

Controls the compression algorithm used for this dataset. Setting compression to on indicates that the current default compression algorithm should be used. The default balances compression and decompression speed, with compression ratio and is expected to work well on a wide variety of workloads. Unlike all other settings for this property, on does not select a fixed compression type. As new compression algorithms are added to ZFS and enabled on a pool, the default compression algorithm may change. The current default compression algorithm is either lzjb or, if the lz4_compress feature is enabled, lz4.

The lz4 compression algorithm is a high-performance replacement for the lzjb algorithm. It features significantly faster compression and decompression, as well as a moderately higher compression ratio than lzjb, but can only be used on pools with the lz4_compress feature set to enabled. See zpool-features(7) for details on ZFS feature flags and the lz4_compress feature.

The lzjb compression algorithm is optimized for performance while providing decent data compression.

The gzip compression algorithm uses the same compression as the gzip(1) command. You can specify the gzip level by using the value gzip-N, where N is an integer from 1 (fastest) to 9 (best compression ratio). Currently, gzip is equivalent to gzip-6 (which is also the default for gzip(1)).

The zstd compression algorithm provides both high compression ratios and good performance. You can specify the zstd level by using the value zstd-N, where N is an integer from 1 (fastest) to 19 (best compression ratio). zstd is equivalent to zstd-3.

Faster speeds at the cost of the compression ratio can be requested by setting a negative zstd level. This is done using zstd-fast-N, where N is an integer in [1-9,10,20,30,...,100,500,1000] which maps to a negative zstd level. The lower the level the faster the compression - 1000 provides the fastest compression and lowest compression ratio. zstd-fast is equivalent to zstd-fast-1.

The zle compression algorithm compresses runs of zeros.

This property can also be referred to by its shortened column name compress. Changing this property affects only newly-written data.

When any setting except off is selected, compression will explicitly check for blocks consisting of only zeroes (the NUL byte). When a zero-filled block is detected, it is stored as a hole and not compressed using the indicated compression algorithm.

Any block being compressed must be no larger than 7/8 of its original size after compression, otherwise the compression will not be considered worthwhile and the block saved uncompressed. Note that when the logical block is less than 8 times the disk sector size this effectively reduces the necessary compression ratio; for example, 8kB blocks on disks with 4kB disk sectors must compress to 1/2 or less of their original size.

context=none|SELinux-User:SELinux-Role:SELinux-Type:Sensitivity-Level

This flag sets the SELinux context for all files in the file system under a mount point for that file system. See selinux(8) for more information.

fscontext=none|SELinux-User:SELinux-Role:SELinux-Type:Sensitivity-Level

This flag sets the SELinux context for the file system file system being mounted. See selinux(8) for more information.

defcontext=none|SELinux-User:SELinux-Role:SELinux-Type:Sensitivity-Level

This flag sets the SELinux default context for unlabeled files. See selinux(8) for more information.

rootcontext=none|SELinux-User:SELinux-Role:SELinux-Type:Sensitivity-Level

This flag sets the SELinux context for the root inode of the file system. See selinux(8) for more information.

copies=1|2|3

Controls the number of copies of data stored for this dataset. These copies are in addition to any redundancy provided by the pool, for example, mirroring or RAID-Z. The copies are stored on different disks, if possible. The space used by multiple copies is charged to the associated file and dataset, changing the used property and counting against quotas and reservations. Changing this property only affects newly-written data. Therefore, set this property at file system creation time by using the -o copies=N option.

Remember that ZFS will not import a pool with a missing top-level vdev. Do NOT create, for example a two-disk striped pool and set copies=2 on some datasets thinking you have setup redundancy for them. When a disk fails you will not be able to import the pool and will have lost all of your data.

Encrypted datasets may not have copies=3 since the implementation stores some encryption metadata where the third copy would normally be.

devices=on|off

Controls whether device nodes can be opened on this file system. The default value is on. The values on and off are equivalent to the dev and nodev mount options.

dedup=off|on|verify|sha256[,verify]|sha512[,verify]|skein[,verify]|edonr,verify

Configures deduplication for a dataset. The default value is off. The default deduplication checksum is sha256 (this may change in the future). When dedup is enabled, the checksum defined here overrides the checksum property. Setting the value to verify has the same effect as the setting sha256,verify. If set to verify, ZFS will do a byte-to-byte comparison in case of two blocks having the same signature to make sure the block contents are identical. Specifying verify is mandatory for the edonr algorithm.

Unless necessary, deduplication should not be enabled on a system. See the Deduplication section of zfsconcepts(7).

dnodesize=legacy|auto|1k|2k|4k|8k|16k

Specifies a compatibility mode or literal value for the size of dnodes in the file system. The default value is legacy. Setting this property to a value other than legacy requires the large_dnode pool feature to be enabled. Consider setting dnodesize to auto if the dataset uses the xattr=sa property setting and the workload makes heavy use of extended attributes. This may be applicable to SELinux-enabled systems, Lustre servers, and Samba servers, for example. Literal values are supported for cases where the optimal size is known in advance and for performance testing.

Leave dnodesize set to legacy if you need to receive a send stream of this dataset on a pool that doesn't enable the large_dnode feature, or if you need to import this pool on a system that doesn't support the large_dnode feature.

This property can also be referred to by its shortened column name, dnsize.

encryption=off|on|aes-128-ccm|aes-192-ccm|aes-256-ccm|aes-128-gcm|aes-192-gcm|aes-256-gcm

Controls the encryption cipher suite (block cipher, key length, and mode) used for this dataset. Requires the encryption feature to be enabled on the pool. Requires a keyformat to be set at dataset creation time. Selecting encryption=on when creating a dataset indicates that the default encryption suite will be selected, which is currently aes-256-gcm. In order to provide consistent data protection, encryption must be specified at dataset creation time and it cannot be changed afterwards.

For more details and caveats about encryption see the Encryption section of zfs-load-key(8).

keyformat=raw|hex|passphrase

Controls what format the user's encryption key will be provided as. This property is only set when the dataset is encrypted. Raw keys and hex keys must be 32 bytes long (regardless of the chosen encryption suite) and must be randomly generated. A raw key can be generated with the following command:

\# dd if=/dev/urandom bs=32 count=1 of=/path/to/output/key

Passphrases must be between 8 and 512 bytes long and will be processed through PBKDF2 before being used (see the pbkdf2iters property). Even though the encryption suite cannot be changed after dataset creation, the keyformat can be with zfs change-key.

keylocation=prompt|file://</absolute/file/path>|https://<address> |http://<address>

Controls where the user's encryption key will be loaded from by default for commands such as zfs load-key and zfs mount -l. This property is only set for encrypted datasets which are encryption roots. If unspecified, the default is prompt. Even though the encryption suite cannot be changed after dataset creation, the keylocation can be with either zfs set or zfs change-key. If prompt is selected ZFS will ask for the key at the command prompt when it is required to access the encrypted data (see zfs load-key for details). This setting will also allow the key to be passed in via the standard input stream, but users should be careful not to place keys which should be kept secret on the command line. If a file URI is selected, the key will be loaded from the specified absolute file path. If an HTTPS or HTTP URL is selected, it will be GETted using fetch(3), libcurl, or nothing, depending on compile-time configuration and run-time availability. The SSL_CA_CERT_FILE environment variable can be set to set the location of the concatenated certificate store. The SSL_CA_CERT_PATH environment variable can be set to override the location of the directory containing the certificate authority bundle. The SSL_CLIENT_CERT_FILE and SSL_CLIENT_KEY_FILE environment variables can be set to configure the path to the client certificate and its key.

pbkdf2iters=iterations

Controls the number of PBKDF2 iterations that a passphrase encryption key should be run through when processing it into an encryption key. This property is only defined when encryption is enabled and a keyformat of passphrase is selected. The goal of PBKDF2 is to significantly increase the computational difficulty needed to brute force a user's passphrase. This is accomplished by forcing the attacker to run each passphrase through a computationally expensive hashing function many times before they arrive at the resulting key. A user who actually knows the passphrase will only have to pay this cost once. As CPUs become better at processing, this number should be raised to ensure that a brute force attack is still not possible. The current default is 350000 and the minimum is 100000. This property may be changed with zfs change-key.

exec=on|off

Controls whether processes can be executed from within this file system. The default value is on. The values on and off are equivalent to the exec and noexec mount options.

filesystem_limit=count|none

Limits the number of filesystems and volumes that can exist under this point in the dataset tree. The limit is not enforced if the user is allowed to change the limit. Setting a filesystem_limit to on a descendent of a filesystem that already has a filesystem_limit does not override the ancestor's filesystem_limit, but rather imposes an additional limit. This feature must be enabled to be used (see zpool-features(7)).

special_small_blocks=size

This value represents the threshold block size for including small file blocks into the special allocation class. Blocks smaller than or equal to this value will be assigned to the special allocation class while greater blocks will be assigned to the regular class. Valid values are zero or a power of two from 512B up to 1M. The default size is 0 which means no small file blocks will be allocated in the special class. Before setting this property, a special class vdev must be added to the pool. See zpoolconcepts(7) for more details on the special allocation class.

mountpoint=path|none|legacy

Controls the mount point used for this file system. See the Mount Points section of zfsconcepts(7) for more information on how this property is used. When the mountpoint property is changed for a file system, the file system and any children that inherit the mount point are unmounted. If the new value is legacy, then they remain unmounted. Otherwise, they are automatically remounted in the new location if the property was previously legacy or none, or if they were mounted before the property was changed. In addition, any shared file systems are unshared and shared in the new location.

nbmand=on|off

Controls whether the file system should be mounted with nbmand (Non-blocking mandatory locks). This is used for SMB clients. Changes to this property only take effect when the file system is umounted and remounted. Support for these locks is scarce and not described by POSIX.

overlay=on|off

Allow mounting on a busy directory or a directory which already contains files or directories. This is the default mount behavior for Linux and FreeBSD file systems. On these platforms the property is on by default. Set to off to disable overlay mounts for consistency with OpenZFS on other platforms.

primarycache=all|none|metadata

Controls what is cached in the primary cache (ARC). If this property is set to all, then both user data and metadata is cached. If this property is set to none, then neither user data nor metadata is cached. If this property is set to metadata, then only metadata is cached. The default value is all.

quota=size|none

Limits the amount of space a dataset and its descendents can consume. This property enforces a hard limit on the amount of space used. This includes all space consumed by descendents, including file systems and snapshots. Setting a quota on a descendent of a dataset that already has a quota does not override the ancestor's quota, but rather imposes an additional limit. Quotas cannot be set on volumes, as the volsize property acts as an implicit quota.

snapshot_limit=count|none

Limits the number of snapshots that can be created on a dataset and its descendents. Setting a snapshot_limit on a descendent of a dataset that already has a snapshot_limit does not override the ancestor's snapshot_limit, but rather imposes an additional limit. The limit is not enforced if the user is allowed to change the limit. For example, this means that recursive snapshots taken from the global zone are counted against each delegated dataset within a zone. This feature must be enabled to be used (see zpool-features(7)).

userquota@user=size|none

Limits the amount of space consumed by the specified user. User space consumption is identified by the userspace@user property. Enforcement of user quotas may be delayed by several seconds. This delay means that a user might exceed their quota before the system notices that they are over quota and begins to refuse additional writes with the EDQUOT error message. See the zfs userspace command for more information.

Unprivileged users can only access their own groups' space usage. The root user, or a user who has been granted the userquota privilege with zfs allow, can get and set everyone's quota.

This property is not available on volumes, on file systems before version 4, or on pools before version 15. The userquota@... properties are not displayed by zfs get all. The user's name must be appended after the @ symbol, using one of the following forms:

- POSIX name ("joe")

- POSIX numeric ID ("789")

- SID name ("joe.smith@mydomain")

- SID numeric ID ("S-1-123-456-789")

Files created on Linux always have POSIX owners.

userobjquota@user=size|none

The userobjquota is similar to userquota but it limits the number of objects a user can create. Please refer to userobjused for more information about how objects are counted.

groupquota@group=size|none

Limits the amount of space consumed by the specified group. Group space consumption is identified by the groupused@group property. Unprivileged users can access only their own groups' space usage. The root user, or a user who has been granted the groupquota privilege with zfs allow, can get and set all groups' quotas.

groupobjquota@group=size|none

The groupobjquota is similar to groupquota but it limits number of objects a group can consume. Please refer to userobjused for more information about how objects are counted.

projectquota@project=size|none

Limits the amount of space consumed by the specified project. Project space consumption is identified by the projectused@project property. Please refer to projectused for more information about how project is identified and set/changed. The root user, or a user who has been granted the projectquota privilege with zfs allow, can access all projects' quota.

projectobjquota@project=size|none

The projectobjquota is similar to projectquota but it limits number of objects a project can consume. Please refer to userobjused for more information about how objects are counted.

readonly=on|off

Controls whether this dataset can be modified. The default value is off. The values on and off are equivalent to the ro and rw mount options. This property can also be referred to by its shortened column name, rdonly.

recordsize=size

Specifies a suggested block size for files in the file system. This property is designed solely for use with database workloads that access files in fixed-size records. ZFS automatically tunes block sizes according to internal algorithms optimized for typical access patterns. For databases that create very large files but access them in small random chunks, these algorithms may be suboptimal. Specifying a recordsize greater than or equal to the record size of the database can result in significant performance gains. Use of this property for general purpose file systems is strongly discouraged, and may adversely affect performance.

The size specified must be a power of two greater than or equal to 512B and less than or equal to 128kB. If the large_blocks feature is enabled on the pool, the size may be up to 1MB. See zpool-features(7) for details on ZFS feature flags.

Changing the file system's recordsize affects only files created afterward; existing files are unaffected.

This property can also be referred to by its shortened column name, recsize.

redundant_metadata=all|most|some|none

Controls what types of metadata are stored redundantly. ZFS stores an extra copy of metadata, so that if a single block is corrupted, the amount of user data lost is limited. This extra copy is in addition to any redundancy provided at the pool level (e.g. by mirroring or RAID-Z), and is in addition to an extra copy specified by the copies property (up to a total of 3 copies). For example if the pool is mirrored, copies=2, and redundant_metadata=most, then ZFS stores 6 copies of most metadata, and 4 copies of data and some metadata. When set to all, ZFS stores an extra copy of all metadata. If a single on-disk block is corrupt, at worst a single block of user data (which is recordsize bytes long) can be lost.

When set to most, ZFS stores an extra copy of most types of metadata. This can improve performance of random writes, because less metadata must be written. In practice, at worst about 1000 blocks (of recordsize bytes each) of user data can be lost if a single on-disk block is corrupt. The exact behavior of which metadata blocks are stored redundantly may change in future releases.

When set to some, ZFS stores an extra copy of only critical metadata. This can improve file create performance since less metadata needs to be written. If a single on-disk block is corrupt, at worst a single user file can be lost.

When set to none, ZFS does not store any copies of metadata redundantly. If a single on-disk block is corrupt, an entire dataset can be lost.

The default value is all.

refquota=size|none

Limits the amount of space a dataset can consume. This property enforces a hard limit on the amount of space used. This hard limit does not include space used by descendents, including file systems and snapshots.

refreservation=size|none|auto

The minimum amount of space guaranteed to a dataset, not including its descendents. When the amount of space used is below this value, the dataset is treated as if it were taking up the amount of space specified by refreservation. The refreservation reservation is accounted for in the parent datasets' space used, and counts against the parent datasets' quotas and reservations. If refreservation is set, a snapshot is only allowed if there is enough free pool space outside of this reservation to accommodate the current number of "referenced" bytes in the dataset.

If refreservation is set to auto, a volume is thick provisioned (or "not sparse"). refreservation=auto is only supported on volumes. See volsize in the Native Properties section for more information about sparse volumes.

This property can also be referred to by its shortened column name, refreserv.

relatime=on|off

Controls the manner in which the access time is updated when atime=on is set. Turning this property on causes the access time to be updated relative to the modify or change time. Access time is only updated if the previous access time was earlier than the current modify or change time or if the existing access time hasn't been updated within the past 24 hours. The default value is off. The values on and off are equivalent to the relatime and norelatime mount options.

reservation=size|none

The minimum amount of space guaranteed to a dataset and its descendants. When the amount of space used is below this value, the dataset is treated as if it were taking up the amount of space specified by its reservation. Reservations are accounted for in the parent datasets' space used, and count against the parent datasets' quotas and reservations. This property can also be referred to by its shortened column name, reserv.

secondarycache=all|none|metadata

Controls what is cached in the secondary cache (L2ARC). If this property is set to all, then both user data and metadata is cached. If this property is set to none, then neither user data nor metadata is cached. If this property is set to metadata, then only metadata is cached. The default value is all.

setuid=on|off

Controls whether the setuid bit is respected for the file system. The default value is on. The values on and off are equivalent to the suid and nosuid mount options.

sharesmb=on|off|opts

Controls whether the file system is shared by using Samba USERSHARES and what options are to be used. Otherwise, the file system is automatically shared and unshared with the zfs share and zfs unshare commands. If the property is set to on, the net(8) command is invoked to create a USERSHARE. Because SMB shares requires a resource name, a unique resource name is constructed from the dataset name. The constructed name is a copy of the dataset name except that the characters in the dataset name, which would be invalid in the resource name, are replaced with underscore (_) characters. Linux does not currently support additional options which might be available on Solaris.

If the sharesmb property is set to off, the file systems are unshared.

The share is created with the ACL (Access Control List) "Everyone:F" ("F" stands for "full permissions", i.e. read and write permissions) and no guest access (which means Samba must be able to authenticate a real user, system passwd/shadow, LDAP or smbpasswd based) by default. This means that any additional access control (disallow specific user specific access etc) must be done on the underlying file system.

sharenfs=on|off|opts

Controls whether the file system is shared via NFS, and what options are to be used. A file system with a sharenfs property of off is managed with the exportfs(8) command and entries in the /etc/exports file. Otherwise, the file system is automatically shared and unshared with the zfs share and zfs unshare commands. If the property is set to on, the dataset is shared using the default options: sec=sys,rw,crossmnt,no_subtree_check

Please note that the options are comma-separated, unlike those found in exports(5). This is done to negate the need for quoting, as well as to make parsing with scripts easier.

See exports(5) for the meaning of the default options. Otherwise, the exportfs(8) command is invoked with options equivalent to the contents of this property.

When the sharenfs property is changed for a dataset, the dataset and any children inheriting the property are re-shared with the new options, only if the property was previously off, or if they were shared before the property was changed. If the new property is off, the file systems are unshared.

logbias=latency|throughput

Provide a hint to ZFS about handling of synchronous requests in this dataset. If logbias is set to latency (the default), ZFS will use pool log devices (if configured) to handle the requests at low latency. If logbias is set to throughput, ZFS will not use configured pool log devices. ZFS will instead optimize synchronous operations for global pool throughput and efficient use of resources.

snapdev=hidden|visible

Controls whether the volume snapshot devices under /dev/zvol/⟨pool⟩ are hidden or visible. The default value is hidden.

snapdir=hidden|visible

Controls whether the .zfs directory is hidden or visible in the root of the file system as discussed in the Snapshots section of zfsconcepts(7). The default value is hidden.

sync=standard|always|disabled

Controls the behavior of synchronous requests (e.g. fsync, O_DSYNC). standard is the POSIX-specified behavior of ensuring all synchronous requests are written to stable storage and all devices are flushed to ensure data is not cached by device controllers (this is the default). always causes every file system transaction to be written and flushed before its system call returns. This has a large performance penalty. disabled disables synchronous requests. File system transactions are only committed to stable storage periodically. This option will give the highest performance. However, it is very dangerous as ZFS would be ignoring the synchronous transaction demands of applications such as databases or NFS. Administrators should only use this option when the risks are understood.

version=N|current

The on-disk version of this file system, which is independent of the pool version. This property can only be set to later supported versions. See the zfs upgrade command.

volsize=size

For volumes, specifies the logical size of the volume. By default, creating a volume establishes a reservation of equal size. For storage pools with a version number of 9 or higher, a refreservation is set instead. Any changes to volsize are reflected in an equivalent change to the reservation (or refreservation). The volsize can only be set to a multiple of volblocksize, and cannot be zero. The reservation is kept equal to the volume's logical size to prevent unexpected behavior for consumers. Without the reservation, the volume could run out of space, resulting in undefined behavior or data corruption, depending on how the volume is used. These effects can also occur when the volume size is changed while it is in use (particularly when shrinking the size). Extreme care should be used when adjusting the volume size.

Though not recommended, a "sparse volume" (also known as "thin provisioned") can be created by specifying the -s option to the zfs create -V command, or by changing the value of the refreservation property (or reservation property on pool version 8 or earlier) after the volume has been created. A "sparse volume" is a volume where the value of refreservation is less than the size of the volume plus the space required to store its metadata. Consequently, writes to a sparse volume can fail with ENOSPC when the pool is low on space. For a sparse volume, changes to volsize are not reflected in the refreservation. A volume that is not sparse is said to be "thick provisioned". A sparse volume can become thick provisioned by setting refreservation to auto.

volmode=default|full|geom|dev|none

This property specifies how volumes should be exposed to the OS. Setting it to full exposes volumes as fully fledged block devices, providing maximal functionality. The value geom is just an alias for full and is kept for compatibility. Setting it to dev hides its partitions. Volumes with property set to none are not exposed outside ZFS, but can be snapshotted, cloned, replicated, etc, that can be suitable for backup purposes. Value default means that volumes exposition is controlled by system-wide tunable zvol_volmode, where full, dev and none are encoded as 1, 2 and 3 respectively. The default value is full.

vscan=on|off

Controls whether regular files should be scanned for viruses when a file is opened and closed. In addition to enabling this property, the virus scan service must also be enabled for virus scanning to occur. The default value is off. This property is not used on Linux.

xattr=on|off|sa

Controls whether extended attributes are enabled for this file system. Two styles of extended attributes are supported: either directory based or system attribute based. The default value of on enables directory based extended attributes. This style of extended attribute imposes no practical limit on either the size or number of attributes which can be set on a file. Although under Linux the getxattr(2) and setxattr(2) system calls limit the maximum size to 64K. This is the most compatible style of extended attribute and is supported by all ZFS implementations.

System attribute based xattrs can be enabled by setting the value to sa. The key advantage of this type of xattr is improved performance. Storing extended attributes as system attributes significantly decreases the amount of disk IO required. Up to 64K of data may be stored per-file in the space reserved for system attributes. If there is not enough space available for an extended attribute then it will be automatically written as a directory based xattr. System attribute based extended attributes are not accessible on platforms which do not support the xattr=sa feature.

The use of system attribute based xattrs is strongly encouraged for users of SELinux or POSIX ACLs. Both of these features heavily rely on extended attributes and benefit significantly from the reduced access time.

The values on and off are equivalent to the xattr and noxattr mount options.

jailed=off|on

Controls whether the dataset is managed from a jail. See zfs-jail(8) for more information. Jails are a FreeBSD feature and are not relevant on other platforms. The default value is off.

zoned=on|off

Controls whether the dataset is managed from a non-global zone. Zones are a Solaris feature and are not relevant on other platforms. The default value is off.

The following three properties cannot be changed after the file system is created, and therefore, should be set when the file system is created. If the properties are not set with the zfs create or zpool create commands, these properties are inherited from the parent dataset. If the parent dataset lacks these properties due to having been created prior to these features being supported, the new file system will have the default values for these properties.

casesensitivity=sensitive|insensitive|mixed

Indicates whether the file name matching algorithm used by the file system should be case-sensitive, case-insensitive, or allow a combination of both styles of matching. The default value for the casesensitivity property is sensitive. Traditionally, UNIX and POSIX file systems have case-sensitive file names. The mixed value for the casesensitivity property indicates that the file system can support requests for both case-sensitive and case-insensitive matching behavior. Currently, case-insensitive matching behavior on a file system that supports mixed behavior is limited to the SMB server product. For more information about the mixed value behavior, see the "ZFS Administration Guide".

normalization=none|formC|formD|formKC|formKD

Indicates whether the file system should perform a unicode normalization of file names whenever two file names are compared, and which normalization algorithm should be used. File names are always stored unmodified, names are normalized as part of any comparison process. If this property is set to a legal value other than none, and the utf8only property was left unspecified, the utf8only property is automatically set to on. The default value of the normalization property is none. This property cannot be changed after the file system is created.

utf8only=on|off

Indicates whether the file system should reject file names that include characters that are not present in the UTF-8 character code set. If this property is explicitly set to off, the normalization property must either not be explicitly set or be set to none. The default value for the utf8only property is off. This property cannot be changed after the file system is created.

The casesensitivity, normalization, and utf8only properties are also new permissions that can be assigned to non-privileged users by using the ZFS delegated administration feature.

### Temporary Mount Point Properties
When a file system is mounted, either through mount(8) for legacy mounts or the zfs mount command for normal file systems, its mount options are set according to its properties. The correlation between properties and mount options is as follows:

atime

atime/noatime

canmount

auto/noauto

devices

dev/nodev

exec

exec/noexec

readonly

ro/rw

relatime

relatime/norelatime

setuid

suid/nosuid

xattr

xattr/noxattr

nbmand

mand/nomand

context=

context=

fscontext=

fscontext=

defcontext=

defcontext=

rootcontext=

rootcontext=

In addition, these options can be set on a per-mount basis using the -o option, without affecting the property that is stored on disk. The values specified on the command line override the values stored in the dataset. The nosuid option is an alias for nodevices,nosetuid. These properties are reported as "temporary" by the zfs get command. If the properties are changed while the dataset is mounted, the new setting overrides any temporary settings.

### User Properties
In addition to the standard native properties, ZFS supports arbitrary user properties. User properties have no effect on ZFS behavior, but applications or administrators can use them to annotate datasets (file systems, volumes, and snapshots).

User property names must contain a colon (":") character to distinguish them from native properties. They may contain lowercase letters, numbers, and the following punctuation characters: colon (":"), dash ("-"), period ("."), and underscore ("_"). The expected convention is that the property name is divided into two portions such as module:property, but this namespace is not enforced by ZFS. User property names can be at most 256 characters, and cannot begin with a dash ("-").

When making programmatic use of user properties, it is strongly suggested to use a reversed DNS domain name for the module component of property names to reduce the chance that two independently-developed packages use the same property name for different purposes.

The values of user properties are arbitrary strings, are always inherited, and are never validated. All of the commands that operate on properties (zfs list, zfs get, zfs set, and so forth) can be used to manipulate both native properties and user properties. Use the zfs inherit command to clear a user property. If the property is not defined in any parent dataset, it is removed entirely. Property values are limited to 8192 bytes.


zpool-features

package: zfs-utils

title: zpool-features(7)

summary: description of ZFS pool features

text:
# zpool-features(7)

## NAME
zpool-features — description of ZFS pool features

## DESCRIPTION
ZFS pool on-disk format versions are specified via "features" which replace the old on-disk format numbers (the last supported on-disk format number is 28). To enable a feature on a pool use the zpool upgrade, or set the feature@feature-name property to enabled. Please also see the Compatibility feature sets section for information on how sets of features may be enabled together.

The pool format does not affect file system version compatibility or the ability to send file systems between pools.

Since most features can be enabled independently of each other, the on-disk format of the pool is specified by the set of all features marked as active on the pool. If the pool was created by another software version this set may include unsupported features.

### Identifying features
Every feature has a GUID of the form com.example:feature-name. The reversed DNS name ensures that the feature's GUID is unique across all ZFS implementations. When unsupported features are encountered on a pool they will be identified by their GUIDs. Refer to the documentation for the ZFS implementation that created the pool for information about those features.

Each supported feature also has a short name. By convention a feature's short name is the portion of its GUID which follows the ‘:’ (i.e. com.example:feature-name would have the short name feature-name), however a feature's short name may differ across ZFS implementations if following the convention would result in name conflicts.

### Feature states
Features can be in one of three states:

active

This feature's on-disk format changes are in effect on the pool. Support for this feature is required to import the pool in read-write mode. If this feature is not read-only compatible, support is also required to import the pool in read-only mode (see Read-only compatibility).

enabled

An administrator has marked this feature as enabled on the pool, but the feature's on-disk format changes have not been made yet. The pool can still be imported by software that does not support this feature, but changes may be made to the on-disk format at any time which will move the feature to the active state. Some features may support returning to the enabled state after becoming active. See feature-specific documentation for details.

disabled

This feature's on-disk format changes have not been made and will not be made unless an administrator moves the feature to the enabled state. Features cannot be disabled once they have been enabled.

The state of supported features is exposed through pool properties of the form feature@short-name.

### Read-only compatibility
Some features may make on-disk format changes that do not interfere with other software's ability to read from the pool. These features are referred to as “read-only compatible”. If all unsupported features on a pool are read-only compatible, the pool can be imported in read-only mode by setting the readonly property during import (see zpool-import(8) for details on importing pools).

### Unsupported features
For each unsupported feature enabled on an imported pool, a pool property named unsupported@feature-name will indicate why the import was allowed despite the unsupported feature. Possible values for this property are:

inactive

The feature is in the enabled state and therefore the pool's on-disk format is still compatible with software that does not support this feature.

readonly

The feature is read-only compatible and the pool has been imported in read-only mode.

### Feature dependencies
Some features depend on other features being enabled in order to function. Enabling a feature will automatically enable any features it depends on.

### Compatibility feature sets
It is sometimes necessary for a pool to maintain compatibility with a specific on-disk format, by enabling and disabling particular features. The compatibility feature facilitates this by allowing feature sets to be read from text files. When set to off (the default), compatibility feature sets are disabled (i.e. all features are enabled); when set to legacy, no features are enabled. When set to a comma-separated list of filenames (each filename may either be an absolute path, or relative to /etc/zfs/compatibility.d or /usr/share/zfs/compatibility.d), the lists of requested features are read from those files, separated by whitespace and/or commas. Only features present in all files are enabled.

Simple sanity checks are applied to the files: they must be between 1B and 16kB in size, and must end with a newline character.

The requested features are applied when a pool is created using zpool create -o compatibility=… and controls which features are enabled when using zpool upgrade. zpool status will not show a warning about disabled features which are not part of the requested feature set.

The special value legacy prevents any features from being enabled, either via zpool upgrade or zpool set feature@feature-name=enabled. This setting also prevents pools from being upgraded to newer on-disk versions. This is a safety measure to prevent new features from being accidentally enabled, breaking compatibility.

By convention, compatibility files in /usr/share/zfs/compatibility.d are provided by the distribution, and include feature sets supported by important versions of popular distributions, and feature sets commonly supported at the start of each year. Compatibility files in /etc/zfs/compatibility.d, if present, will take precedence over files with the same name in /usr/share/zfs/compatibility.d.

If an unrecognized feature is found in these files, an error message will be shown. If the unrecognized feature is in a file in /etc/zfs/compatibility.d, this is treated as an error and processing will stop. If the unrecognized feature is under /usr/share/zfs/compatibility.d, this is treated as a warning and processing will continue. This difference is to allow distributions to include features which might not be recognized by the currently-installed binaries.

Compatibility files may include comments: any text from ‘#’ to the end of the line is ignored.

Example:

```
example# cat /usr/share/zfs/compatibility.d/grub2
# Features which are supported by GRUB2
async_destroy
bookmarks
embedded_data
empty_bpobj
enabled_txg
extensible_dataset
filesystem_limits
hole_birth
large_blocks
lz4_compress
spacemap_histogram

example# zpool create -o compatibility=grub2 bootpool vdev
```

See zpool-create(8) and zpool-upgrade(8) for more information on how these commands are affected by feature sets.

## FEATURES
The following features are supported on this system:

allocation_classes

GUID

org.zfsonlinux:allocation_classes

READ-ONLY COMPATIBLE

yes

This feature enables support for separate allocation classes.

This feature becomes active when a dedicated allocation class vdev (dedup or special) is created with the zpool create or zpool add commands. With device removal, it can be returned to the enabled state if all the dedicated allocation class vdevs are removed.

async_destroy

GUID

com.delphix:async_destroy

READ-ONLY COMPATIBLE

yes

Destroying a file system requires traversing all of its data in order to return its used space to the pool. Without async_destroy, the file system is not fully removed until all space has been reclaimed. If the destroy operation is interrupted by a reboot or power outage, the next attempt to open the pool will need to complete the destroy operation synchronously.

When async_destroy is enabled, the file system's data will be reclaimed by a background process, allowing the destroy operation to complete without traversing the entire file system. The background process is able to resume interrupted destroys after the pool has been opened, eliminating the need to finish interrupted destroys as part of the open operation. The amount of space remaining to be reclaimed by the background process is available through the freeing property.

This feature is only active while freeing is non-zero.

bookmarks

GUID

com.delphix:bookmarks

DEPENDENCIES

extensible_dataset

READ-ONLY COMPATIBLE

yes

This feature enables use of the zfs bookmark command.

This feature is active while any bookmarks exist in the pool. All bookmarks in the pool can be listed by running zfs list -t bookmark -r poolname.

bookmark_v2

GUID

com.datto:bookmark_v2

DEPENDENCIES

bookmark, extensible_dataset

READ-ONLY COMPATIBLE

no

This feature enables the creation and management of larger bookmarks which are needed for other features in ZFS.

This feature becomes active when a v2 bookmark is created and will be returned to the enabled state when all v2 bookmarks are destroyed.

bookmark_written

GUID

com.delphix:bookmark_written

DEPENDENCIES

bookmark, extensible_dataset, bookmark_v2

READ-ONLY COMPATIBLE

no

This feature enables additional bookmark accounting fields, enabling the written#bookmark property (space written since a bookmark) and estimates of send stream sizes for incrementals from bookmarks.

This feature becomes active when a bookmark is created and will be returned to the enabled state when all bookmarks with these fields are destroyed.

device_rebuild

GUID

org.openzfs:device_rebuild

READ-ONLY COMPATIBLE

yes

This feature enables the ability for the zpool attach and zpool replace commands to perform sequential reconstruction (instead of healing reconstruction) when resilvering.

Sequential reconstruction resilvers a device in LBA order without immediately verifying the checksums. Once complete, a scrub is started, which then verifies the checksums. This approach allows full redundancy to be restored to the pool in the minimum amount of time. This two-phase approach will take longer than a healing resilver when the time to verify the checksums is included. However, unless there is additional pool damage, no checksum errors should be reported by the scrub. This feature is incompatible with raidz configurations. This feature becomes active while a sequential resilver is in progress, and returns to enabled when the resilver completes.

device_removal

GUID

com.delphix:device_removal

READ-ONLY COMPATIBLE

no

This feature enables the zpool remove command to remove top-level vdevs, evacuating them to reduce the total size of the pool.

This feature becomes active when the zpool remove command is used on a top-level vdev, and will never return to being enabled.

draid

GUID

org.openzfs:draid

READ-ONLY COMPATIBLE

no

This feature enables use of the draid vdev type. dRAID is a variant of raidz which provides integrated distributed hot spares that allow faster resilvering while retaining the benefits of raidz. Data, parity, and spare space are organized in redundancy groups and distributed evenly over all of the devices.

This feature becomes active when creating a pool which uses the draid vdev type, or when adding a new draid vdev to an existing pool.

edonr

GUID

org.illumos:edonr

DEPENDENCIES

extensible_dataset

READ-ONLY COMPATIBLE

no

This feature enables the use of the Edon-R hash algorithm for checksum, including for nopwrite (if compression is also enabled, an overwrite of a block whose checksum matches the data being written will be ignored). In an abundance of caution, Edon-R requires verification when used with dedup: zfs set dedup=edonr,verify (see zfs-set(8)).

Edon-R is a very high-performance hash algorithm that was part of the NIST SHA-3 competition. It provides extremely high hash performance (over 350% faster than SHA-256), but was not selected because of its unsuitability as a general purpose secure hash algorithm. This implementation utilizes the new salted checksumming functionality in ZFS, which means that the checksum is pre-seeded with a secret 256-bit random key (stored on the pool) before being fed the data block to be checksummed. Thus the produced checksums are unique to a given pool, preventing hash collision attacks on systems with dedup.

When the edonr feature is set to enabled, the administrator can turn on the edonr checksum on any dataset using zfs set checksum=edonr dset (see zfs-set(8)). This feature becomes active once a checksum property has been set to edonr, and will return to being enabled once all filesystems that have ever had their checksum set to edonr are destroyed.

FreeBSD does not support the edonr feature.

embedded_data

GUID

com.delphix:embedded_data

READ-ONLY COMPATIBLE

no

This feature improves the performance and compression ratio of highly-compressible blocks. Blocks whose contents can compress to 112 bytes or smaller can take advantage of this feature.

When this feature is enabled, the contents of highly-compressible blocks are stored in the block "pointer" itself (a misnomer in this case, as it contains the compressed data, rather than a pointer to its location on disk). Thus the space of the block (one sector, typically 512B or 4kB) is saved, and no additional I/O is needed to read and write the data block. This feature becomes active as soon as it is enabled and will never return to being enabled.

empty_bpobj

GUID

com.delphix:empty_bpobj

READ-ONLY COMPATIBLE

yes

This feature increases the performance of creating and using a large number of snapshots of a single filesystem or volume, and also reduces the disk space required.

When there are many snapshots, each snapshot uses many Block Pointer Objects (bpobjs) to track blocks associated with that snapshot. However, in common use cases, most of these bpobjs are empty. This feature allows us to create each bpobj on-demand, thus eliminating the empty bpobjs.

This feature is active while there are any filesystems, volumes, or snapshots which were created after enabling this feature.

enabled_txg

GUID

com.delphix:enabled_txg

READ-ONLY COMPATIBLE

yes

Once this feature is enabled, ZFS records the transaction group number in which new features are enabled. This has no user-visible impact, but other features may depend on this feature.

This feature becomes active

as soon as it is enabled and will never return to being enabled.

encryption

GUID

com.datto:encryption

DEPENDENCIES

bookmark_v2, extensible_dataset

READ-ONLY COMPATIBLE

no

This feature enables the creation and management of natively encrypted datasets.

This feature becomes active when an encrypted dataset is created and will be returned to the enabled state when all datasets that use this feature are destroyed.

extensible_dataset

GUID

com.delphix:extensible_dataset

READ-ONLY COMPATIBLE

no

This feature allows more flexible use of internal ZFS data structures, and exists for other features to depend on.

This feature will be active when the first dependent feature uses it, and will be returned to the enabled state when all datasets that use this feature are destroyed.

filesystem_limits

GUID

com.joyent:filesystem_limits

DEPENDENCIES

extensible_dataset

READ-ONLY COMPATIBLE

yes

This feature enables filesystem and snapshot limits. These limits can be used to control how many filesystems and/or snapshots can be created at the point in the tree on which the limits are set.

This feature is active once either of the limit properties has been set on a dataset. Once activated the feature is never deactivated.

hole_birth

GUID

com.delphix:hole_birth

DEPENDENCIES

enabled_txg

READ-ONLY COMPATIBLE

no

This feature has/had bugs, the result of which is that, if you do a zfs send -i (or -R, since it uses -i) from an affected dataset, the receiving party will not see any checksum or other errors, but the resulting destination snapshot will not match the source. Its use by zfs send -i has been disabled by default (see send_holes_without_birth_time in zfs(4)).

This feature improves performance of incremental sends (zfs send -i) and receives for objects with many holes. The most common case of hole-filled objects is zvols.

An incremental send stream from snapshot A to snapshot B contains information about every block that changed between A and B. Blocks which did not change between those snapshots can be identified and omitted from the stream using a piece of metadata called the "block birth time", but birth times are not recorded for holes (blocks filled only with zeroes). Since holes created after A cannot be distinguished from holes created before A, information about every hole in the entire filesystem or zvol is included in the send stream.

For workloads where holes are rare this is not a problem. However, when incrementally replicating filesystems or zvols with many holes (for example a zvol formatted with another filesystem) a lot of time will be spent sending and receiving unnecessary information about holes that already exist on the receiving side.

Once the hole_birth feature has been enabled the block birth times of all new holes will be recorded. Incremental sends between snapshots created after this feature is enabled will use this new metadata to avoid sending information about holes that already exist on the receiving side.

This feature becomes active as soon as it is enabled and will never return to being enabled.

large_blocks

GUID

org.open-zfs:large_blocks

DEPENDENCIES

extensible_dataset

READ-ONLY COMPATIBLE

no

This feature allows the record size on a dataset to be set larger than 128kB.

This feature becomes active once a dataset contains a file with a block size larger than 128kB, and will return to being enabled once all filesystems that have ever had their recordsize larger than 128kB are destroyed.

large_dnode

GUID

org.zfsonlinux:large_dnode

DEPENDENCIES

extensible_dataset

READ-ONLY COMPATIBLE

no

This feature allows the size of dnodes in a dataset to be set larger than 512B. This feature becomes active once a dataset contains an object with a dnode larger than 512B, which occurs as a result of setting the dnodesize dataset property to a value other than legacy. The feature will return to being enabled once all filesystems that have ever contained a dnode larger than 512B are destroyed. Large dnodes allow more data to be stored in the bonus buffer, thus potentially improving performance by avoiding the use of spill blocks.

livelist

GUID

com.delphix:livelist

READ-ONLY COMPATIBLE

yes

This feature allows clones to be deleted faster than the traditional method when a large number of random/sparse writes have been made to the clone. All blocks allocated and freed after a clone is created are tracked by the the clone's livelist which is referenced during the deletion of the clone. The feature is activated when a clone is created and remains active until all clones have been destroyed.

log_spacemap

GUID

com.delphix:log_spacemap

DEPENDENCIES

com.delphix:spacemap_v2

READ-ONLY COMPATIBLE

yes

This feature improves performance for heavily-fragmented pools, especially when workloads are heavy in random-writes. It does so by logging all the metaslab changes on a single spacemap every TXG instead of scattering multiple writes to all the metaslab spacemaps.

This feature becomes active as soon as it is enabled and will never return to being enabled.

lz4_compress

GUID

org.illumos:lz4_compress

READ-ONLY COMPATIBLE

no

lz4 is a high-performance real-time compression algorithm that features significantly faster compression and decompression as well as a higher compression ratio than the older lzjb compression. Typically, lz4 compression is approximately 50% faster on compressible data and 200% faster on incompressible data than lzjb. It is also approximately 80% faster on decompression, while giving approximately a 10% better compression ratio.

When the lz4_compress feature is set to enabled, the administrator can turn on lz4 compression on any dataset on the pool using the zfs-set(8) command. All newly written metadata will be compressed with the lz4 algorithm.

This feature becomes active as soon as it is enabled and will never return to being enabled.

multi_vdev_crash_dump

GUID

com.joyent:multi_vdev_crash_dump

READ-ONLY COMPATIBLE

no

This feature allows a dump device to be configured with a pool comprised of multiple vdevs. Those vdevs may be arranged in any mirrored or raidz configuration.

When the multi_vdev_crash_dump feature is set to enabled, the administrator can use dumpadm(1M) to configure a dump device on a pool comprised of multiple vdevs.

Under FreeBSD and Linux this feature is unused, but registered for compatibility. New pools created on these systems will have the feature enabled but will never transition to active, as this functionality is not required for crash dump support. Existing pools where this feature is active can be imported.

obsolete_counts

GUID

com.delphix:obsolete_counts

DEPENDENCIES

device_removal

READ-ONLY COMPATIBLE

yes

This feature is an enhancement of device_removal, which will over time reduce the memory used to track removed devices. When indirect blocks are freed or remapped, we note that their part of the indirect mapping is "obsolete" – no longer needed.

This feature becomes active when the zpool remove command is used on a top-level vdev, and will never return to being enabled.

project_quota

GUID

org.zfsonlinux:project_quota

DEPENDENCIES

extensible_dataset

READ-ONLY COMPATIBLE

yes

This feature allows administrators to account the spaces and objects usage information against the project identifier (ID).

The project ID is an object-based attribute. When upgrading an existing filesystem, objects without a project ID will be assigned a zero project ID. When this feature is enabled, newly created objects inherit their parent directories' project ID if the parent's inherit flag is set (via chattr [+-]P or zfs project -s|-C). Otherwise, the new object's project ID will be zero. An object's project ID can be changed at any time by the owner (or privileged user) via chattr -p prjid or zfs project -p prjid.

This feature will become active as soon as it is enabled and will never return to being disabled. Each filesystem will be upgraded automatically when remounted, or when a new file is created under that filesystem. The upgrade can also be triggered on filesystems via zfs set version=current fs. The upgrade process runs in the background and may take a while to complete for filesystems containing large amounts of files.

redaction_bookmarks

GUID

com.delphix:redaction_bookmarks

DEPENDENCIES

bookmarks, extensible_dataset

READ-ONLY COMPATIBLE

no

This feature enables the use of redacted zfs sends, which create redaction bookmarks storing the list of blocks redacted by the send that created them. For more information about redacted sends, see zfs-send(8).

redacted_datasets

GUID

com.delphix:redacted_datasets

DEPENDENCIES

extensible_dataset

READ-ONLY COMPATIBLE

no

This feature enables the receiving of redacted zfs sendstreams. which create redacted datasets when received. These datasets are missing some of their blocks, and so cannot be safely mounted, and their contents cannot be safely read. For more information about redacted receives, see zfs-send(8).

resilver_defer

GUID

com.datto:resilver_defer

READ-ONLY COMPATIBLE

yes

This feature allows ZFS to postpone new resilvers if an existing one is already in progress. Without this feature, any new resilvers will cause the currently running one to be immediately restarted from the beginning.

This feature becomes active once a resilver has been deferred, and returns to being enabled when the deferred resilver begins.

sha512

GUID

org.illumos:sha512

DEPENDENCIES

extensible_dataset

READ-ONLY COMPATIBLE

no

This feature enables the use of the SHA-512/256 truncated hash algorithm (FIPS 180-4) for checksum and dedup. The native 64-bit arithmetic of SHA-512 provides an approximate 50% performance boost over SHA-256 on 64-bit hardware and is thus a good minimum-change replacement candidate for systems where hash performance is important, but these systems cannot for whatever reason utilize the faster skein and edonr algorithms.

When the sha512 feature is set to enabled, the administrator can turn on the sha512 checksum on any dataset using zfs set checksum=sha512 dset (see zfs-set(8)). This feature becomes active once a checksum property has been set to sha512, and will return to being enabled once all filesystems that have ever had their checksum set to sha512 are destroyed.

skein

GUID

org.illumos:skein

DEPENDENCIES

extensible_dataset

READ-ONLY COMPATIBLE

no

This feature enables the use of the Skein hash algorithm for checksum and dedup. Skein is a high-performance secure hash algorithm that was a finalist in the NIST SHA-3 competition. It provides a very high security margin and high performance on 64-bit hardware (80% faster than SHA-256). This implementation also utilizes the new salted checksumming functionality in ZFS, which means that the checksum is pre-seeded with a secret 256-bit random key (stored on the pool) before being fed the data block to be checksummed. Thus the produced checksums are unique to a given pool, preventing hash collision attacks on systems with dedup.

When the skein feature is set to enabled, the administrator can turn on the skein checksum on any dataset using zfs set checksum=skein dset (see zfs-set(8)). This feature becomes active once a checksum property has been set to skein, and will return to being enabled once all filesystems that have ever had their checksum set to skein are destroyed.

spacemap_histogram

GUID

com.delphix:spacemap_histogram

READ-ONLY COMPATIBLE

yes

This features allows ZFS to maintain more information about how free space is organized within the pool. If this feature is enabled, it will be activated when a new space map object is created, or an existing space map is upgraded to the new format, and never returns back to being enabled.

spacemap_v2

GUID

com.delphix:spacemap_v2

READ-ONLY COMPATIBLE

yes

This feature enables the use of the new space map encoding which consists of two words (instead of one) whenever it is advantageous. The new encoding allows space maps to represent large regions of space more efficiently on-disk while also increasing their maximum addressable offset.

This feature becomes active once it is enabled, and never returns back to being enabled.

userobj_accounting

GUID

org.zfsonlinux:userobj_accounting

DEPENDENCIES

extensible_dataset

READ-ONLY COMPATIBLE

yes

This feature allows administrators to account the object usage information by user and group.

This feature becomes active as soon as it is enabled and will never return to being enabled. Each filesystem will be upgraded automatically when remounted, or when a new file is created under that filesystem. The upgrade can also be triggered on filesystems via zfs set version=current fs. The upgrade process runs in the background and may take a while to complete for filesystems containing large amounts of files.

zpool_checkpoint

GUID

com.delphix:zpool_checkpoint

READ-ONLY COMPATIBLE

yes

This feature enables the zpool checkpoint command that can checkpoint the state of the pool at the time it was issued and later rewind back to it or discard it.

This feature becomes active when the zpool checkpoint command is used to checkpoint the pool. The feature will only return back to being enabled when the pool is rewound or the checkpoint has been discarded.

zstd_compress

GUID

org.freebsd:zstd_compress

DEPENDENCIES

extensible_dataset

READ-ONLY COMPATIBLE

no

zstd is a high-performance compression algorithm that features a combination of high compression ratios and high speed. Compared to gzip, zstd offers slightly better compression at much higher speeds. Compared to lz4, zstd offers much better compression while being only modestly slower. Typically, zstd compression speed ranges from 250 to 500 MB/s per thread and decompression speed is over 1 GB/s per thread.

When the zstd feature is set to enabled, the administrator can turn on zstd compression of any dataset using zfs set compress=zstd dset (see zfs-set(8)). This feature becomes active once a compress property has been set to zstd, and will return to being enabled once all filesystems that have ever had their compress property set to zstd are destroyed.

## SEE ALSO
zpool(8)