zgenhostid

title: zgenhostid(8)

summary: generate host ID into /etc/hostid

text:
# zgenhostid(8)

## NAME
zgenhostid — generate host ID into /etc/hostid

## SYNOPSIS
zgenhostid

[-f] [-o filename] [hostid]

## DESCRIPTION
Creates /etc/hostid file and stores the host ID in it. If hostid was provided, validate and store that value. Otherwise, randomly generate an ID.

## OPTIONS
-h

Display a summary of the command-line options.

-f

Allow output overwrite.

-o filename

Write to filename instead of the default /etc/hostid.

hostid

Specifies the value to be placed in /etc/hostid. It should be a number with a value between 1 and 2^32-1. If 0, generate a random ID. This value must be unique among your systems. It must be an 8-digit-long hexadecimal number, optionally prefixed by "0x".

## FILES
/etc/hostid

## EXAMPLES
Generate a random hostid and store it

\# zgenhostid

Record the libc-generated hostid in /etc/hostid

\# zgenhostid "$(hostid)"

Record a custom hostid (0xdeadbeef) in /etc/hostid

\# zgenhostid deadbeef

Record a custom hostid (0x01234567) in /tmp/hostid and overwrite the file if it exists

\# zgenhostid -f -o /tmp/hostid 0x01234567

## SEE ALSO
genhostid(1), hostid(1), sethostid(3), spl(4)

## HISTORY
zgenhostid emulates the genhostid(1) utility and is provided for use on systems which do not include the utility or do not provide the sethostid(3) function.


zinject

package: zfs-utils

title: zinject(8)

summary: ZFS Fault Injector

text:
# zinject(8)

## NAME
zinject — ZFS Fault Injector

## DESCRIPTION
zinject creates artificial problems in a ZFS pool by simulating data corruption or device failures. This program is dangerous.

## SYNOPSIS
zinject

List injection records.

zinject

-b objset:object:level:start:end [-f frequency] -amu [pool]

Force an error into the pool at a bookmark.

zinject

-c id|all

Cancel injection records.

zinject

-d vdev -A degrade|fault pool

Force a vdev into the DEGRADED or FAULTED state.

zinject

-d vdev -D latency:lanes [-T read|write] pool

Add an artificial delay to I/O requests on a particular device, such that the requests take a minimum of latency milliseconds to complete. Each delay has an associated number of lanes which defines the number of concurrent I/O requests that can be processed. For example, with a single lane delay of 10 ms (-D 10:1), the device will only be able to service a single I/O request at a time with each request taking 10 ms to complete. So, if only a single request is submitted every 10 ms, the average latency will be 10 ms; but if more than one request is submitted every 10 ms, the average latency will be more than 10 ms.

Similarly, if a delay of 10 ms is specified to have two lanes (-D 10:2), then the device will be able to service two requests at a time, each with a minimum latency of 10 ms. So, if two requests are submitted every 10 ms, then the average latency will be 10 ms; but if more than two requests are submitted every 10 ms, the average latency will be more than 10 ms.

Also note, these delays are additive. So two invocations of -D 10:1 are roughly equivalent to a single invocation of -D 10:2. This also means, that one can specify multiple lanes with differing target latencies. For example, an invocation of -D 10:1 followed by -D 25:2 will create 3 lanes on the device: one lane with a latency of 10 ms and two lanes with a 25 ms latency.

zinject

-d vdev [-e device_error] [-L label_error] [-T failure] [-f frequency] [-F] pool

Force a vdev error.

zinject

-i seconds pool

Add an artificial delay during the future import of a pool. This injector is automatically cleared after the import is finished.

zinject

-E delay [-a] [-m] [-f freq] [-l level] [-r range] [-T iotype] [-t type|-b bookmark]

Inject pipeline ready stage delays for the given object or bookmark. The delay is specified in milliseconds.

zinject

-I [-s seconds|-g txgs] pool

Simulate a hardware failure that fails to honor a cache flush.

zinject

-p function pool

Panic inside the specified function.

zinject

-t data -C dvas [-e device_error] [-f frequency] [-l level] [-r range] [-amq] path

Force an error into the contents of a file.

zinject

-t dnode -C dvas [-e device_error] [-f frequency] [-l level] [-amq] path

Force an error into the metadnode for a file or directory.

zinject

-t mos_type -C dvas [-e device_error] [-f frequency] [-l level] [-r range] [-amqu] pool

Force an error into the MOS of a pool.

## OPTIONS
-a

Flush the ARC before injection.

-b objset:object:level:start:end

Force an error into the pool at this bookmark tuple. Each number is in hexadecimal, and only one block can be specified.

-C dvas

Inject the given error only into specific DVAs. The mask should be specified as a list of 0-indexed DVAs separated by commas (e.g. 0,2). This option is not applicable to logical data errors such as decompress and decrypt.

-d vdev

A vdev specified by path or GUID.

-e device_error

Specify checksum

for an ECKSUM error,

decompress

for a data decompression error,

decrypt

for a data decryption error,

corrupt

to flip a bit in the data after a read,

dtl

for an ECHILD error,

io

for an EIO error where reopening the device will succeed,

nxio

for an ENXIO error where reopening the device will fail, or

noop

to drop the IO without executing it, and return success.

For EIO and ENXIO, the "failed" reads or writes still occur. The probe simply sets the error value reported by the I/O pipeline so it appears the read or write failed. Decryption errors only currently work with file data.

-f frequency

Only inject errors a fraction of the time. Expressed as a real number percentage between 0.0001 and 100.

-F

Fail faster. Do fewer checks.

-f txgs

Run for this many transaction groups before reporting failure.

-h

Print the usage message.

-l level

Inject an error at a particular block level. The default is 0.

-L label_error

Set the label error region to one of nvlist, pad1, pad2, or uber.

-m

Automatically remount the underlying filesystem.

-q

Quiet mode. Only print the handler number added.

-r range

Inject an error over a particular logical range of an object, which will be translated to the appropriate blkid range according to the object's properties.

-s seconds

Run for this many seconds before reporting failure.

-T type

Inject the error into I/O of this type. read, write, flush, claim, free

Fundamental I/O types

all

All fundamental I/O types

probe

Device probe I/O

-t mos_type

Set this to mos

for any data in the MOS,

mosdir

for an object directory,

config

for the pool configuration,

bpobj

for the block pointer list,

spacemap

for the space map,

metaslab

for the metaslab, or

errlog

for the persistent error log.

-u

Unload the pool after injection.

## ENVIRONMENT VARIABLES
ZFS_HOSTID

Run zinject in debug mode.

## SEE ALSO
zfs(8), zpool(8)

branch: sisyphus


zpool

title: zpool(8)

summary: configure ZFS storage pools

text:
# zpool(8)

## NAME
zpool — configure ZFS storage pools

## SYNOPSIS
zpool

-?V

zpool

version [-j]

zpool

subcommand [arguments]

## DESCRIPTION
The zpool command configures ZFS storage pools. A storage pool is a collection of devices that provides physical storage and data replication for ZFS datasets. All datasets within a storage pool share the same space. See zfs(8) for information on managing datasets.

For an overview of creating and managing ZFS storage pools see the zpoolconcepts(7) manual page.

## SUBCOMMANDS
All subcommands that modify state are logged persistently to the pool in their original form.

The zpool command provides subcommands to create and destroy storage pools, add capacity to storage pools, and provide information about the storage pools. The following subcommands are supported:

zpool -?

Displays a help message.

zpool -V, --version

zpool version [-j]

Displays the software version of the zpool userland utility and the ZFS kernel module. Use -j option to output in JSON format.

### Creation
zpool-create(8)

Creates a new storage pool containing the virtual devices specified on the command line.

zpool-initialize(8)

Begins initializing by writing to all unallocated regions on the specified devices, or all eligible devices in the pool if no individual devices are specified.

### Destruction
zpool-destroy(8)

Destroys the given pool, freeing up any devices for other use.

zpool-labelclear(8)

Removes ZFS label information from the specified device.

### Virtual Devices
zpool-attach(8)/zpool-detach(8)

Converts a non-redundant disk into a mirror, or increases the redundancy level of an existing mirror (attach), or performs the inverse operation (detach).

zpool-add(8)/zpool-remove(8)

Adds the specified virtual devices to the given pool, or removes the specified device from the pool.

zpool-replace(8)

Replaces an existing device (which may be faulted) with a new one.

zpool-split(8)

Creates a new pool by splitting all mirrors in an existing pool (which decreases its redundancy).

### Properties
Available pool properties listed in the zpoolprops(7) manual page.

zpool-list(8)

Lists the given pools along with a health status and space usage.

zpool-get(8)/zpool-set(8)

Retrieves the given list of properties (or all properties if all is used) for the specified storage pool(s).

### Monitoring
zpool-status(8)

Displays the detailed health status for the given pools.

zpool-iostat(8)

Displays logical I/O statistics for the given pools/vdevs. Physical I/O operations may be observed via iostat(1).

zpool-events(8)

Lists all recent events generated by the ZFS kernel modules. These events are consumed by the zed(8) and used to automate administrative tasks such as replacing a failed device with a hot spare. That manual page also describes the subclasses and event payloads that can be generated.

zpool-history(8)

Displays the command history of the specified pool(s) or all pools if no pool is specified.

### Maintenance
zpool-prefetch(8)

Prefetches specific types of pool data.

zpool-scrub(8)

Begins a scrub or resumes a paused scrub.

zpool-checkpoint(8)

Checkpoints the current state of pool, which can be later restored by zpool import --rewind-to-checkpoint.

zpool-trim(8)

Initiates an immediate on-demand TRIM operation for all of the free space in a pool. This operation informs the underlying storage devices of all blocks in the pool which are no longer allocated and allows thinly provisioned devices to reclaim the space.

zpool-sync(8)

This command forces all in-core dirty data to be written to the primary pool storage and not the ZIL. It will also update administrative information including quota reporting. Without arguments, zpool sync will sync all pools on the system. Otherwise, it will sync only the specified pool(s).

zpool-upgrade(8)

Manage the on-disk format version of storage pools.

zpool-wait(8)

Waits until all background activity of the given types has ceased in the given pool.

### Fault Resolution
zpool-offline(8)/zpool-online(8)

Takes the specified physical device offline or brings it online.

zpool-resilver(8)

Starts a resilver. If an existing resilver is already running it will be restarted from the beginning.

zpool-reopen(8)

Reopen all the vdevs associated with the pool.

zpool-clear(8)

Clears device errors in a pool.

### Import & Export
zpool-import(8)

Make disks containing ZFS storage pools available for use on the system.

zpool-export(8)

Exports the given pools from the system.

zpool-reguid(8)

Generates a new unique identifier for the pool.

## EXIT STATUS
The following exit values are returned:

0

Successful completion.

1

An error occurred.

2

Invalid command line options were specified.

## EXAMPLES

### Example 1: Creating a RAID-Z Storage Pool
The following command creates a pool with a single raidz root vdev that consists of six disks:

\# zpool create tank raidz sda sdb sdc sdd sde sdf

### Example 2: Creating a Mirrored Storage Pool
The following command creates a pool with two mirrors, where each mirror contains two disks:

\# zpool create tank mirror sda sdb mirror sdc sdd

### Example 3: Creating a ZFS Storage Pool by Using Partitions
The following command creates a non-redundant pool using two disk partitions:

\# zpool create tank sda1 sdb2

### Example 4: Creating a ZFS Storage Pool by Using Files
The following command creates a non-redundant pool using files. While not recommended, a pool based on files can be useful for experimental purposes.

\# zpool create tank /path/to/file/a /path/to/file/b

### Example 5: Making a non-mirrored ZFS Storage Pool mirrored
The following command converts an existing single device sda into a mirror by attaching a second device to it, sdb.

\# zpool attach tank sda sdb

### Example 6: Adding a Mirror to a ZFS Storage Pool
The following command adds two mirrored disks to the pool tank, assuming the pool is already made up of two-way mirrors. The additional space is immediately available to any datasets within the pool.

\# zpool add tank mirror sda sdb

### Example 7: Listing Available ZFS Storage Pools
The following command lists all available pools on the system. In this case, the pool zion is faulted due to a missing device. The results from this command are similar to the following:

```
# zpool list
NAME    SIZE  ALLOC   FREE  EXPANDSZ   FRAG    CAP  DEDUP  HEALTH  ALTROOT
rpool  19.9G  8.43G  11.4G         -    33%    42%  1.00x  ONLINE  -
tank   61.5G  20.0G  41.5G         -    48%    32%  1.00x  ONLINE  -
zion       -      -      -         -      -      -      -  FAULTED -
```

### Example 8: Destroying a ZFS Storage Pool
The following command destroys the pool tank and any datasets contained within:

\# zpool destroy -f tank

### Example 9: Exporting a ZFS Storage Pool
The following command exports the devices in pool tank so that they can be relocated or later imported:

\# zpool export tank

### Example 10: Importing a ZFS Storage Pool
The following command displays available pools, and then imports the pool tank for use on the system. The results from this command are similar to the following:

```
# zpool import
  pool: tank
    id: 15451357997522795478
 state: ONLINE
action: The pool can be imported using its name or numeric identifier.
config:

        tank        ONLINE
          mirror    ONLINE
            sda     ONLINE
            sdb     ONLINE

# zpool import tank
```

### Example 11: Upgrading All ZFS Storage Pools to the Current Version
The following command upgrades all ZFS Storage pools to the current version of the software:

```
# zpool upgrade -a
This system is currently running ZFS version 2.
```

### Example 12: Managing Hot Spares
The following command creates a new pool with an available hot spare:

\# zpool create tank mirror sda sdb spare sdc

If one of the disks were to fail, the pool would be reduced to the degraded state. The failed device can be replaced using the following command:

\# zpool replace tank sda sdd

Once the data has been resilvered, the spare is automatically removed and is made available for use should another device fail. The hot spare can be permanently removed from the pool using the following command:

\# zpool remove tank sdc

### Example 13: Creating a ZFS Pool with Mirrored Separate Intent Logs
The following command creates a ZFS storage pool consisting of two, two-way mirrors and mirrored log devices:

\# zpool create pool mirror sda sdb mirror sdc sdd log mirror sde sdf

### Example 14: Adding Cache Devices to a ZFS Pool
The following command adds two disks for use as cache devices to a ZFS storage pool:

\# zpool add pool cache sdc sdd

Once added, the cache devices gradually fill with content from main memory. Depending on the size of your cache devices, it could take over an hour for them to fill. Capacity and reads can be monitored using the iostat subcommand as follows:

\# zpool iostat -v pool 5

### Example 15: Removing a Mirrored top-level (Log or Data) Device
The following commands remove the mirrored log device mirror-2 and mirrored top-level data device mirror-1.

Given this configuration:

```
  pool: tank
 state: ONLINE
 scrub: none requested
config:

         NAME        STATE     READ WRITE CKSUM
         tank        ONLINE       0     0     0
           mirror-0  ONLINE       0     0     0
             sda     ONLINE       0     0     0
             sdb     ONLINE       0     0     0
           mirror-1  ONLINE       0     0     0
             sdc     ONLINE       0     0     0
             sdd     ONLINE       0     0     0
         logs
           mirror-2  ONLINE       0     0     0
             sde     ONLINE       0     0     0
             sdf     ONLINE       0     0     0
```

The command to remove the mirrored log mirror-2 is:

\# zpool remove tank mirror-2

At this point, the log device no longer exists (both sides of the mirror have been removed):

```
  pool: tank
 state: ONLINE
  scan: none requested
config:

        NAME        STATE     READ WRITE CKSUM
        tank        ONLINE       0     0     0
          mirror-0  ONLINE       0     0     0
            sda     ONLINE       0     0     0
            sdb     ONLINE       0     0     0
          mirror-1  ONLINE       0     0     0
            sdc     ONLINE       0     0     0
            sdd     ONLINE       0     0     0
```

The command to remove the mirrored data mirror-1 is:

\# zpool remove tank mirror-1

After mirror-1 has been evacuated, the pool remains redundant, but the total amount of space is reduced:

```
  pool: tank
 state: ONLINE
  scan: none requested
config:

        NAME        STATE     READ WRITE CKSUM
        tank        ONLINE       0     0     0
          mirror-0  ONLINE       0     0     0
            sda     ONLINE       0     0     0
            sdb     ONLINE       0     0     0
```

### Example 16: Displaying expanded space on a device
The following command displays the detailed information for the pool data. This pool is comprised of a single raidz vdev where one of its devices increased its capacity by 10 GiB. In this example, the pool will not be able to utilize this extra capacity until all the devices under the raidz vdev have been expanded.

```
# zpool list -v data
NAME         SIZE  ALLOC   FREE  EXPANDSZ   FRAG    CAP  DEDUP  HEALTH  ALTROOT
data        23.9G  14.6G  9.30G         -    48%    61%  1.00x  ONLINE  -
  raidz1    23.9G  14.6G  9.30G         -    48%
    sda         -      -      -         -      -
    sdb         -      -      -       10G      -
    sdc         -      -      -         -      -
```

### Example 17: Adding output columns
Additional columns can be added to the zpool status and zpool iostat output with -c.

```
# zpool status -c vendor,model,size
   NAME     STATE  READ WRITE CKSUM vendor  model        size
   tank     ONLINE 0    0     0
   mirror-0 ONLINE 0    0     0
   U1       ONLINE 0    0     0     SEAGATE ST8000NM0075 7.3T
   U10      ONLINE 0    0     0     SEAGATE ST8000NM0075 7.3T
   U11      ONLINE 0    0     0     SEAGATE ST8000NM0075 7.3T
   U12      ONLINE 0    0     0     SEAGATE ST8000NM0075 7.3T
   U13      ONLINE 0    0     0     SEAGATE ST8000NM0075 7.3T
   U14      ONLINE 0    0     0     SEAGATE ST8000NM0075 7.3T

# zpool iostat -vc size
              capacity     operations     bandwidth
pool        alloc   free   read  write   read  write  size
----------  -----  -----  -----  -----  -----  -----  ----
rpool       14.6G  54.9G      4     55   250K  2.69M
  sda1      14.6G  54.9G      4     55   250K  2.69M   70G
----------  -----  -----  -----  -----  -----  -----  ----
```

## ENVIRONMENT VARIABLES
ZFS_ABORT

Cause zpool to dump core on exit for the purposes of running ::findleaks.

ZFS_COLOR

Use ANSI color in zpool status and zpool iostat output.

ZPOOL_AUTO_POWER_ON_SLOT

Automatically attempt to turn on the drives enclosure slot power to a drive when running the zpool online or zpool clear commands. This has the same effect as passing the --power option to those commands.

ZPOOL_POWER_ON_SLOT_TIMEOUT_MS

The maximum time in milliseconds to wait for a slot power sysfs value to return the correct value after writing it. For example, after writing "on" to the sysfs enclosure slot power_control file, it can take some time for the enclosure to power down the slot and return "on" if you read back the 'power_control' value. Defaults to 30 seconds (30000ms) if not set.

ZPOOL_IMPORT_PATH

The search path for devices or files to use with the pool. This is a colon-separated list of directories in which zpool looks for device nodes and files. Similar to the -d option in zpool import.

ZPOOL_IMPORT_UDEV_TIMEOUT_MS

The maximum time in milliseconds that zpool import will wait for an expected device to be available.

ZPOOL_STATUS_NON_NATIVE_ASHIFT_IGNORE

If set, suppress warning about non-native vdev ashift in zpool status. The value is not used, only the presence or absence of the variable matters.

ZPOOL_VDEV_NAME_GUID

Cause zpool subcommands to output vdev GUIDs by default. This behavior is identical to the zpool status -g command line option.

ZPOOL_VDEV_NAME_FOLLOW_LINKS

Cause zpool subcommands to follow links for vdev names by default. This behavior is identical to the zpool status -L command line option.

ZPOOL_VDEV_NAME_PATH

Cause zpool subcommands to output full vdev path names by default. This behavior is identical to the zpool status -P command line option.

ZFS_VDEV_DEVID_OPT_OUT

Older OpenZFS implementations had issues when attempting to display pool config vdev names if a devid NVP value is present in the pool's config. For example, a pool that originated on illumos platform would have a devid value in the config and zpool status would fail when listing the config. This would also be true for future Linux-based pools.

A pool can be stripped of any devid values on import or prevented from adding them on zpool create or zpool add by setting ZFS_VDEV_DEVID_OPT_OUT.

ZPOOL_SCRIPTS_AS_ROOT

Allow a privileged user to run zpool status/iostat -c. Normally, only unprivileged users are allowed to run -c.

ZPOOL_SCRIPTS_PATH

The search path for scripts when running zpool status/iostat -c. This is a colon-separated list of directories and overrides the default ~/.zpool.d and /etc/zfs/zpool.d search paths.

ZPOOL_SCRIPTS_ENABLED

Allow a user to run zpool status/iostat -c. If ZPOOL_SCRIPTS_ENABLED is not set, it is assumed that the user is allowed to run zpool status/iostat -c.

ZFS_MODULE_TIMEOUT

Time, in seconds, to wait for /dev/zfs to appear. Defaults to 10, max 600 (10 minutes). If <0, wait forever; if 0, don't wait.

## INTERFACE STABILITY
Evolving

## SEE ALSO
zfs(4), zpool-features(7), zpoolconcepts(7), zpoolprops(7), zed(8), zfs(8), zpool-add(8), zpool-attach(8), zpool-checkpoint(8), zpool-clear(8), zpool-create(8), zpool-ddtprune(8), zpool-destroy(8), zpool-detach(8), zpool-events(8), zpool-export(8), zpool-get(8), zpool-history(8), zpool-import(8), zpool-initialize(8), zpool-iostat(8), zpool-labelclear(8), zpool-list(8), zpool-offline(8), zpool-online(8), zpool-prefetch(8), zpool-reguid(8), zpool-remove(8), zpool-reopen(8), zpool-replace(8), zpool-resilver(8), zpool-scrub(8), zpool-set(8), zpool-split(8), zpool-status(8), zpool-sync(8), zpool-trim(8), zpool-upgrade(8), zpool-wait(8)


zpool-add

title: zpool-add(8)

summary: add vdevs to ZFS storage pool

text:
# zpool-add(8)

## NAME
zpool-add — add vdevs to ZFS storage pool

## SYNOPSIS
zpool

add [-fgLnP] [--allow-in-use --allow-replication-mismatch --allow-ashift-mismatch] [-o property=value] pool vdev…

## DESCRIPTION
Adds the specified virtual devices to the given pool. The vdev specification is described in the Virtual Devices section of zpoolconcepts(7). The behavior of the -f option, and the device checks performed are described in the zpool create subcommand.

-f

Forces use of vdevs, even if they appear in use, have conflicting ashift values, or specify a conflicting replication level. Not all devices can be overridden in this manner.

-g

Display vdev, GUIDs instead of the normal device names. These GUIDs can be used in place of device names for the zpool detach/offline/remove/replace commands.

-L

Display real paths for vdevs resolving all symbolic links. This can be used to look up the current block device name regardless of the /dev/disk path used to open it.

-n

Displays the configuration that would be used without actually adding the vdevs. The actual pool creation can still fail due to insufficient privileges or device sharing.

-P

Display real paths for vdevs instead of only the last component of the path. This can be used in conjunction with the -L flag.

-o property=value

Sets the given pool properties. See the zpoolprops(7) manual page for a list of valid properties that can be set. The only property supported at the moment is ashift.

--allow-ashift-mismatch

Disable the ashift validation which allows mismatched ashift values in the pool. Adding top-level vdevs with different sector sizes will prohibit future device removal operations, see zpool-remove(8).

--allow-in-use

Allow vdevs to be added even if they might be in use in another pool.

--allow-replication-mismatch

Allow vdevs with conflicting replication levels to be added to the pool.

## EXAMPLES

### Example 1: Adding a Mirror to a ZFS Storage Pool
The following command adds two mirrored disks to the pool tank, assuming the pool is already made up of two-way mirrors. The additional space is immediately available to any datasets within the pool.

\# zpool add tank mirror sda sdb

### Example 2: Adding Cache Devices to a ZFS Pool
The following command adds two disks for use as cache devices to a ZFS storage pool:

\# zpool add pool cache sdc sdd

Once added, the cache devices gradually fill with content from main memory. Depending on the size of your cache devices, it could take over an hour for them to fill. Capacity and reads can be monitored using the iostat subcommand as follows:

\# zpool iostat -v pool 5

## SEE ALSO
zpool-attach(8), zpool-import(8), zpool-initialize(8), zpool-online(8), zpool-remove(8)


zpool-attach

title: zpool-attach(8)

summary: attach new device to existing ZFS vdev

text:
# zpool-attach(8)

## NAME
zpool-attach — attach new device to existing ZFS vdev

## SYNOPSIS
zpool

attach [-fsw] [-o property=value] pool vdev new_device

## DESCRIPTION
Attaches new_device to the existing vdev. The behavior differs depending on if the existing vdev is a RAID-Z device, or a mirror/plain device.

If the existing vdev is a mirror or plain device (e.g. specified as "sda" or "mirror-7"), the new device will be mirrored with the existing vdev, a resilver will be initiated, and the new device will contribute to additional redundancy once the resilver completes. If vdev is not currently part of a mirrored configuration, device automatically transforms into a two-way mirror of device and new_device. If vdev is part of a two-way mirror, attaching new_device creates a three-way mirror, and so on. In either case, new_device begins to resilver immediately and any running scrub is canceled.

If the existing vdev is a RAID-Z device (e.g. specified as "raidz2-0"), the new device will become part of that RAID-Z group. A "raidz expansion" will be initiated, and once the expansion completes, the new device will contribute additional space to the RAID-Z group. The expansion entails reading all allocated space from existing disks in the RAID-Z group, and rewriting it to the new disks in the RAID-Z group (including the newly added device). Its progress can be monitored with zpool status.

Data redundancy is maintained during and after the expansion. If a disk fails while the expansion is in progress, the expansion pauses until the health of the RAID-Z vdev is restored (e.g. by replacing the failed disk and waiting for reconstruction to complete). Expansion does not change the number of failures that can be tolerated without data loss (e.g. a RAID-Z2 is still a RAID-Z2 even after expansion). A RAID-Z vdev can be expanded multiple times.

After the expansion completes, old blocks retain their old data-to-parity ratio (e.g. 5-wide RAID-Z2 has 3 data and 2 parity) but distributed among the larger set of disks. New blocks will be written with the new data-to-parity ratio (e.g. a 5-wide RAID-Z2 which has been expanded once to 6-wide, has 4 data and 2 parity). However, the vdev's assumed parity ratio does not change, so slightly less space than is expected may be reported for newly-written blocks, according to zfs list, df, ls -s, and similar tools.

A pool-wide scrub is initiated at the end of the expansion in order to verify the checksums of all blocks which have been copied during the expansion.

-f

Forces use of new_device, even if it appears to be in use. Not all vdevs can be overridden in this manner.

-o property=value

Sets the given pool properties. See the zpoolprops(7) manual page for a list of valid properties that can be set. The only property supported at the moment is ashift.

-s

When attaching to a mirror or plain vdev, the new_device is reconstructed sequentially to restore redundancy as quickly as possible. Checksums are not verified during sequential reconstruction so a scrub is started when the resilver completes.

-w

Waits until new_device has finished resilvering or expanding before returning.

## SEE ALSO
zpool-add(8), zpool-detach(8), zpool-import(8), zpool-initialize(8), zpool-online(8), zpool-replace(8), zpool-resilver(8)


zpool-checkpoint

title: zpool-checkpoint(8)

text:
# zpool-checkpoint(8)

## NAME
zpool-checkpoint — check-point current ZFS storage pool state

## SYNOPSIS
zpool

checkpoint [-d [-w]] pool

## DESCRIPTION
Checkpoints the current state of pool , which can be later restored by zpool import --rewind-to-checkpoint. The existence of a checkpoint in a pool prohibits the following zpool subcommands: remove, attach, detach, split, and reguid. In addition, it may break reservation boundaries if the pool lacks free space. The zpool status command indicates the existence of a checkpoint or the progress of discarding a checkpoint from a pool. zpool list can be used to check how much space the checkpoint takes from the pool.

## OPTIONS
-d, --discard

Discards an existing checkpoint from pool.

-w, --wait

Waits until the checkpoint has finished being discarded before returning.

## SEE ALSO
zfs-snapshot(8), zpool-import(8), zpool-status(8)


zpool-clear

title: zpool-clear(8)

summary: clear device errors in ZFS storage pool

text:
# zpool-clear(8)

## NAME
zpool-clear — clear device errors in ZFS storage pool

## SYNOPSIS
zpool

clear [--power] pool [device]…

## DESCRIPTION
Clears device errors in a pool. If no arguments are specified, all device errors within the pool are cleared. If one or more devices is specified, only those errors associated with the specified device or devices are cleared.

If the pool was suspended it will be brought back online provided the devices can be accessed. Pools with multihost enabled which have been suspended cannot be resumed when there is evidence that the pool was imported by another host. The same checks performed during an import will be applied before the clear proceeds.

--power

Power on the devices's slot in the storage enclosure and wait for the device to show up before attempting to clear errors. This is done on all the devices specified. Alternatively, you can set the ZPOOL_AUTO_POWER_ON_SLOT environment variable to always enable this behavior. Note: This flag currently works on Linux only.

## SEE ALSO
zdb(8), zpool-reopen(8), zpool-status(8)


zpool-create

title: zpool-create(8)

summary: create ZFS storage pool

text:
# zpool-create(8)

## NAME
zpool-create — create ZFS storage pool

## SYNOPSIS
zpool

create [-dfn] [-m mountpoint] [-o property=value]… [-o feature@feature=value] [-o compatibility=off|legacy|file[,file]…] [-O file-system-property=value]… [-R root] [-t tname] pool vdev…

## DESCRIPTION
Creates a new storage pool containing the virtual devices specified on the command line. The pool name must begin with a letter, and can only contain alphanumeric characters as well as the underscore ("_"), dash ("-"), colon (":"), space (" "), and period ("."). The pool names mirror, raidz, draid, spare and log are reserved, as are names beginning with mirror, raidz, draid, and spare. The vdev specification is described in the Virtual Devices section of zpoolconcepts(7).

The command attempts to verify that each device specified is accessible and not currently in use by another subsystem. However this check is not robust enough to detect simultaneous attempts to use a new device in different pools, even if multihost= enabled. The administrator must ensure that simultaneous invocations of any combination of zpool replace, zpool create, zpool add, or zpool labelclear do not refer to the same device. Using the same device in two pools will result in pool corruption.

There are some uses, such as being currently mounted, or specified as the dedicated dump device, that prevents a device from ever being used by ZFS. Other uses, such as having a preexisting UFS file system, can be overridden with -f.

The command also checks that the replication strategy for the pool is consistent. An attempt to combine redundant and non-redundant storage in a single pool, or to mix disks and files, results in an error unless -f is specified. The use of differently-sized devices within a single raidz or mirror group is also flagged as an error unless -f is specified.

Unless the -R option is specified, the default mount point is /pool. The mount point must not exist or must be empty, or else the root dataset will not be able to be be mounted. This can be overridden with the -m option.

By default all supported features are enabled on the new pool. The -d option and the -o compatibility property (e.g -o compatibility=2020) can be used to restrict the features that are enabled, so that the pool can be imported on other releases of ZFS.

-d

Do not enable any features on the new pool. Individual features can be enabled by setting their corresponding properties to enabled with -o. See zpool-features(7) for details about feature properties.

-f

Forces use of vdevs, even if they appear in use or specify a conflicting replication level. Not all devices can be overridden in this manner.

-m mountpoint

Sets the mount point for the root dataset. The default mount point is /pool or altroot/pool if altroot is specified. The mount point must be an absolute path, legacy, or none. For more information on dataset mount points, see zfsprops(7).

-n

Displays the configuration that would be used without actually creating the pool. The actual pool creation can still fail due to insufficient privileges or device sharing.

-o property=value

Sets the given pool properties. See zpoolprops(7) for a list of valid properties that can be set.

-o compatibility=off|legacy|file[,file]…

Specifies compatibility feature sets. See zpool-features(7) for more information about compatibility feature sets.

-o feature@feature=value

Sets the given pool feature. See the zpool-features(7) section for a list of valid features that can be set. Value can be either disabled or enabled.

-O file-system-property=value

Sets the given file system properties in the root file system of the pool. See zfsprops(7) for a list of valid properties that can be set.

-R root

Equivalent to -o cachefile=none -o altroot=root

-t tname

Sets the in-core pool name to tname while the on-disk name will be the name specified as pool. This will set the default of the cachefile property to none. This is intended to handle name space collisions when creating pools for other systems, such as virtual machines or physical machines whose pools live on network block devices.

## EXAMPLES

### Example 1: Creating a RAID-Z Storage Pool
The following command creates a pool with a single raidz root vdev that consists of six disks:

\# zpool create tank raidz sda sdb sdc sdd sde sdf

### Example 2: Creating a Mirrored Storage Pool
The following command creates a pool with two mirrors, where each mirror contains two disks:

\# zpool create tank mirror sda sdb mirror sdc sdd

### Example 3: Creating a ZFS Storage Pool by Using Partitions
The following command creates a non-redundant pool using two disk partitions:

\# zpool create tank sda1 sdb2

### Example 4: Creating a ZFS Storage Pool by Using Files
The following command creates a non-redundant pool using files. While not recommended, a pool based on files can be useful for experimental purposes.

\# zpool create tank /path/to/file/a /path/to/file/b

### Example 5: Managing Hot Spares
The following command creates a new pool with an available hot spare:

\# zpool create tank mirror sda sdb spare sdc

### Example 6: Creating a ZFS Pool with Mirrored Separate Intent Logs
The following command creates a ZFS storage pool consisting of two, two-way mirrors and mirrored log devices:

\# zpool create pool mirror sda sdb mirror sdc sdd log mirror sde sdf

## SEE ALSO
zpool-destroy(8), zpool-export(8), zpool-import(8)


zpool-ddtprune

title: zpool-ddtprune(8)

text:
# zpool-ddtprune(8)

## NAME
zpool-ddtprune — Prunes the oldest entries from the single reference dedup table(s)

## SYNOPSIS
zpool

ddtprune -d days | -p percentage pool

## DESCRIPTION
This command prunes older unique entries from the dedup table. As a complement to the dedup quota feature, ddtprune allows removal of older non-duplicate entries to make room for newer duplicate entries.

The amount to prune can be based on a target percentage of the unique entries or based on the age (i.e., every unique entry older than N days).

## SEE ALSO
zdb(8), zpool-status(8)


zpool-destroy

title: zpool-destroy(8)

summary: destroy ZFS storage pool

text:
# zpool-destroy(8)

## NAME
zpool-destroy — destroy ZFS storage pool

## SYNOPSIS
zpool

destroy [-f] pool

## DESCRIPTION
Destroys the given pool, freeing up any devices for other use. This command tries to unmount any active datasets before destroying the pool.

-f

Forcefully unmount all active datasets.

## EXAMPLES

### Example 1: Destroying a ZFS Storage Pool
The following command destroys the pool tank and any datasets contained within:

\# zpool destroy -f tank


zpool-detach

title: zpool-detach(8)

summary: detach device from ZFS mirror

text:
# zpool-detach(8)

## NAME
zpool-detach — detach device from ZFS mirror

## SYNOPSIS
zpool

detach pool device

## DESCRIPTION
Detaches device from a mirror. The operation is refused if there are no other valid replicas of the data. If device may be re-added to the pool later on then consider the zpool offline command instead.

## SEE ALSO
zpool-attach(8), zpool-labelclear(8), zpool-offline(8), zpool-remove(8), zpool-replace(8), zpool-split(8)


zpool-events

title: zpool-events(8)

summary: list recent events generated by kernel

text:
# zpool-events(8)

## NAME
zpool-events — list recent events generated by kernel

## SYNOPSIS
zpool

events [-vHf] [pool]

zpool

events -c

## DESCRIPTION
Lists all recent events generated by the ZFS kernel modules. These events are consumed by the zed(8) and used to automate administrative tasks such as replacing a failed device with a hot spare. For more information about the subclasses and event payloads that can be generated see EVENTS and the following sections.

## OPTIONS
-c

Clear all previous events.

-f

Follow mode.

-H

Scripted mode. Do not display headers, and separate fields by a single tab instead of arbitrary space.

-v

Print the entire payload for each event.

## EVENTS
These are the different event subclasses. The full event name would be ereport.fs.zfs.SUBCLASS, but only the last part is listed here.

checksum

Issued when a checksum error has been detected.

io

Issued when there is an I/O error in a vdev in the pool.

data

Issued when there have been data errors in the pool.

deadman

Issued when an I/O request is determined to be "hung", this can be caused by lost completion events due to flaky hardware or drivers. See zfs_deadman_failmode in zfs(4) for additional information regarding "hung" I/O detection and configuration.

delay

Issued when a completed I/O request exceeds the maximum allowed time specified by the zio_slow_io_ms module parameter. This can be an indicator of problems with the underlying storage device. The number of delay events is ratelimited by the zfs_slow_io_events_per_second module parameter.

dio_verify_rd

Issued when there was a checksum verify error after a Direct I/O read has been issued.

dio_verify_wr

Issued when there was a checksum verify error after a Direct I/O write has been issued. This event can only take place if the module parameter zfs_vdev_direct_write_verify is not set to zero. See zfs(4) for more details on the zfs_vdev_direct_write_verify module parameter.

config_sync

Issued every time a vdev change have been done to the pool.

zpool

Issued when a pool cannot be imported.

pool_create

Issued when a pool is created.

pool_destroy

Issued when a pool is destroyed.

pool_export

Issued when a pool is exported.

pool_import

Issued when a pool is imported.

pool_reguid

Issued when a REGUID (new unique identifier for the pool have been regenerated) have been detected.

vdev.unknown

Issued when the vdev is unknown. Such as trying to clear device errors on a vdev that have failed/been kicked from the system/pool and is no longer available.

vdev.open_failed

Issued when a vdev could not be opened (because it didn't exist for example).

vdev.corrupt_data

Issued when corrupt data have been detected on a vdev.

vdev.no_replicas

Issued when there are no more replicas to sustain the pool. This would lead to the pool being DEGRADED.

vdev.bad_guid_sum

Issued when a missing device in the pool have been detected.

vdev.too_small

Issued when the system (kernel) have removed a device, and ZFS notices that the device isn't there any more. This is usually followed by a probe_failure event.

vdev.bad_label

Issued when the label is OK but invalid.

vdev.bad_ashift

Issued when the ashift alignment requirement has increased.

vdev_remove

Issued when a vdev is detached from a mirror (or a spare detached from a vdev where it have been used to replace a failed drive - only works if the original drive have been re-added).

vdev_remove_aux

Issued when an auxiliary vdev is removed.

vdev_remove_dev

Issued when a specific device is removed from a vdev.

vdev_clear

Issued when clearing device errors in a pool. Such as running zpool clear on a device in the pool.

vdev_check

Issued when a check to see if a given vdev could be opened is started.

vdev_spare

Issued when a spare have kicked in to replace a failed device.

vdev_autoexpand

Issued when a vdev can be automatically expanded.

vdev_add

Issued when a vdev is added to a pool.

vdev_attach

Issued when a vdev is attached to a mirror or raidz vdev type.

vdev_online

Issued when an offline vdev is brought online

io_failure

Issued when there is an I/O failure in a vdev in the pool.

probe_failure

Issued when a probe fails on a vdev. This would occur if a vdev have been kicked from the system outside of ZFS (such as the kernel have removed the device).

log_replay

Issued when the intent log cannot be replayed. The can occur in the case of a missing or damaged log device.

resilver_start

Issued when a resilver is started.

resilver_finish

Issued when the running resilver have finished.

scrub_start

Issued when a scrub is started on a pool.

scrub_finish

Issued when a pool has finished scrubbing.

scrub_abort

Issued when a scrub is aborted on a pool.

scrub_resume

Issued when a scrub is resumed on a pool.

scrub_paused

Issued when a scrub is paused on a pool.

errorscrub_start

Issued when a errorscrub is started on a pool.

errorscrub_finish

Issued when a pool has finished errorscrubbing.

errorscrub_abort

Issued when a errorscrub is aborted on a pool.

errorscrub_resume

Issued when a errorscrub is resumed on a pool.

errorscrub_paused

Issued when a errorscrub is paused on a pool.

trim_start

Issued when a trim is started on a pool.

trim_finish

Issued when a pool has finished trimbing.

trim_cancel

Issued when a trim is canceled on a pool.

trim_resume

Issued when a trim is resumed on a pool.

trim_suspend

Issued when a trim is suspend on a pool.

authentication

Issued when there is a decryption / authentication error.

config_cache_write

Issued when the config cache file cannot be written.

bootfs_vdev_attach

Issued when a vdev is attached to a root pool with the bootfs property set.

sitout

Issued when a RAIDZ or DRAID vdev triggers the autosit logic. This logic detects when a disk in such a vdev is significantly slower than its peers, and sits them out temporarily to preserve the performance of the pool.

## PAYLOADS
This is the payload (data, information) that accompanies an event.

For zed(8), these are set to uppercase and prefixed with ZEVENT_.

pool

Pool name.

pool_failmode

Failmode - wait, continue, or panic. See the failmode property in zpoolprops(7) for more information.

pool_guid

The GUID of the pool.

pool_context

The load state for the pool (0=none, 1=open, 2=import, 3=tryimport, 4=recover 5=error).

vdev_guid

The GUID of the vdev in question (the vdev failing or operated upon with zpool clear, etc.).

vdev_type

Type of vdev - disk, file, mirror, etc. See the Virtual Devices section of zpoolconcepts(7) for more information on possible values.

vdev_path

Full path of the vdev, including any -partX.

vdev_devid

ID of vdev (if any).

vdev_fru

Physical FRU location.

vdev_state

State of vdev (0=uninitialized, 1=closed, 2=offline, 3=removed, 4=failed to open, 5=faulted, 6=degraded, 7=healthy).

vdev_ashift

The ashift value of the vdev.

vdev_complete_ts

The time the last I/O request completed for the specified vdev.

vdev_delta_ts

The time since the last I/O request completed for the specified vdev.

vdev_spare_paths

List of spares, including full path and any -partX.

vdev_spare_guids

GUID(s) of spares.

vdev_read_errors

How many read errors that have been detected on the vdev.

vdev_write_errors

How many write errors that have been detected on the vdev.

vdev_cksum_errors

How many checksum errors that have been detected on the vdev.

parent_guid

GUID of the vdev parent.

parent_type

Type of parent. See vdev_type.

parent_path

Path of the vdev parent (if any).

parent_devid

ID of the vdev parent (if any).

zio_objset

The object set number for a given I/O request.

zio_object

The object number for a given I/O request.

zio_level

The indirect level for the block. Level 0 is the lowest level and includes data blocks. Values > 0 indicate metadata blocks at the appropriate level.

zio_blkid

The block ID for a given I/O request.

zio_err

The error number for a failure when handling a given I/O request, compatible with errno(3) with the value of EBADE used to indicate a ZFS checksum error.

zio_offset

The offset in bytes of where to write the I/O request for the specified vdev.

zio_size

The size in bytes of the I/O request.

zio_flags

The current flags describing how the I/O request should be handled. See the I/O FLAGS section for the full list of I/O flags.

zio_stage

The current stage of the I/O in the pipeline. See the I/O STAGES section for a full list of all the I/O stages.

zio_pipeline

The valid pipeline stages for the I/O. See the I/O STAGES section for a full list of all the I/O stages.

zio_priority

The queue priority of the I/O request. See the I/O PRIORITIES section for a full list of all the I/O priorities.

zio_tyoe

The type of the I/O request. See the I/O TYPES section for a full list of all the I/O types.

zio_delay

The time elapsed (in nanoseconds) waiting for the block layer to complete the I/O request. Unlike zio_delta, this does not include any vdev queuing time and is therefore solely a measure of the block layer performance.

zio_timestamp

The time when a given I/O request was submitted.

zio_delta

The time required to service a given I/O request.

prev_state

The previous state of the vdev.

cksum_algorithm

Checksum algorithm used. See zfsprops(7) for more information on the available checksum algorithms.

cksum_byteswap

Whether or not the data is byteswapped.

bad_ranges

[start, end) pairs of corruption offsets. Offsets are always aligned on a 64-bit boundary, and can include some gaps of non-corruption. (See bad_ranges_min_gap)

bad_ranges_min_gap

In order to bound the size of the bad_ranges array, gaps of non-corruption less than or equal to bad_ranges_min_gap bytes have been merged with adjacent corruption. Always at least 8 bytes, since corruption is detected on a 64-bit word basis.

bad_range_sets

This array has one element per range in bad_ranges. Each element contains the count of bits in that range which were clear in the good data and set in the bad data.

bad_range_clears

This array has one element per range in bad_ranges. Each element contains the count of bits for that range which were set in the good data and clear in the bad data.

bad_set_bits

If this field exists, it is an array of (bad data & ~(good data)); that is, the bits set in the bad data which are cleared in the good data. Each element corresponds a byte whose offset is in a range in bad_ranges, and the array is ordered by offset. Thus, the first element is the first byte in the first bad_ranges range, and the last element is the last byte in the last bad_ranges range.

bad_cleared_bits

Like bad_set_bits, but contains (good data & ~(bad data)); that is, the bits set in the good data which are cleared in the bad data.

## I/O STAGES
The ZFS I/O pipeline is comprised of various stages which are defined below. The individual stages are used to construct these basic I/O operations: Read, Write, Free, Claim, Flush and Trim. These stages may be set on an event to describe the life cycle of a given I/O request.

Stage

Bit Mask

Operations

ZIO_STAGE_OPEN

0x00000001

RWFCXT

ZIO_STAGE_READ_BP_INIT

0x00000002

R-----

ZIO_STAGE_WRITE_BP_INIT

0x00000004

-W----

ZIO_STAGE_FREE_BP_INIT

0x00000008

--F---

ZIO_STAGE_ISSUE_ASYNC

0x00000010

-WF--T

ZIO_STAGE_WRITE_COMPRESS

0x00000020

-W----

ZIO_STAGE_ENCRYPT

0x00000040

-W----

ZIO_STAGE_CHECKSUM_GENERATE

0x00000080

-W----

ZIO_STAGE_NOP_WRITE

0x00000100

-W----

ZIO_STAGE_BRT_FREE

0x00000200

--F---

ZIO_STAGE_DDT_READ_START

0x00000400

R-----

ZIO_STAGE_DDT_READ_DONE

0x00000800

R-----

ZIO_STAGE_DDT_WRITE

0x00001000

-W----

ZIO_STAGE_DDT_FREE

0x00002000

--F---

ZIO_STAGE_GANG_ASSEMBLE

0x00004000

RWFC--

ZIO_STAGE_GANG_ISSUE

0x00008000

RWFC--

ZIO_STAGE_DVA_THROTTLE

0x00010000

-W----

ZIO_STAGE_DVA_ALLOCATE

0x00020000

-W----

ZIO_STAGE_DVA_FREE

0x00040000

--F---

ZIO_STAGE_DVA_CLAIM

0x00080000

---C--

ZIO_STAGE_READY

0x00100000

RWFCIT

ZIO_STAGE_VDEV_IO_START

0x00200000

RW--XT

ZIO_STAGE_VDEV_IO_DONE

0x00400000

RW--XT

ZIO_STAGE_VDEV_IO_ASSESS

0x00800000

RW--XT

ZIO_STAGE_CHECKSUM_VERIFY

0x01000000

R-----

ZIO_STAGE_DIO_CHECKSUM_VERIFY

0x02000000

-W----

ZIO_STAGE_DONE

0x04000000

RWFCXT

## I/O FLAGS
Every I/O request in the pipeline contains a set of flags which describe its function and are used to govern its behavior. These flags will be set in an event as a zio_flags payload entry.

Flag

Bit Mask

ZIO_FLAG_DONT_AGGREGATE

0x00000001

ZIO_FLAG_IO_REPAIR

0x00000002

ZIO_FLAG_SELF_HEAL

0x00000004

ZIO_FLAG_RESILVER

0x00000008

ZIO_FLAG_SCRUB

0x00000010

ZIO_FLAG_SCAN_THREAD

0x00000020

ZIO_FLAG_PHYSICAL

0x00000040

ZIO_FLAG_CANFAIL

0x00000080

ZIO_FLAG_SPECULATIVE

0x00000100

ZIO_FLAG_CONFIG_WRITER

0x00000200

ZIO_FLAG_DONT_RETRY

0x00000400

ZIO_FLAG_NODATA

0x00001000

ZIO_FLAG_INDUCE_DAMAGE

0x00002000

ZIO_FLAG_ALLOC_THROTTLED

0x00004000

ZIO_FLAG_IO_RETRY

0x00008000

ZIO_FLAG_PROBE

0x00010000

ZIO_FLAG_TRYHARD

0x00020000

ZIO_FLAG_OPTIONAL

0x00040000

ZIO_FLAG_DONT_QUEUE

0x00080000

ZIO_FLAG_DONT_PROPAGATE

0x00100000

ZIO_FLAG_IO_BYPASS

0x00200000

ZIO_FLAG_IO_REWRITE

0x00400000

ZIO_FLAG_RAW_COMPRESS

0x00800000

ZIO_FLAG_RAW_ENCRYPT

0x01000000

ZIO_FLAG_GANG_CHILD

0x02000000

ZIO_FLAG_DDT_CHILD

0x04000000

ZIO_FLAG_GODFATHER

0x08000000

ZIO_FLAG_NOPWRITE

0x10000000

ZIO_FLAG_REEXECUTED

0x20000000

ZIO_FLAG_DELEGATED

0x40000000

ZIO_FLAG_FASTWRITE

0x80000000

## I/O TYPES
Every I/O request in the pipeline has a single type value. This value describes the kind of low-level work the I/O represents. This value will be set in an event as a zio_type payload entry.

Type

Value

Description

ZIO_TYPE_NULL

0x0

internal I/O sync point

ZIO_TYPE_READ

0x1

data read

ZIO_TYPE_WRITE

0x2

data write

ZIO_TYPE_FREE

0x3

block free

ZIO_TYPE_CLAIM

0x4

block claim (ZIL replay)

ZIO_TYPE_FLUSH

0x5

disk cache flush request

ZIO_TYPE_TRIM

0x6

trim (discard)

## I/O PRIORITIES
Every I/O request in the pipeline has a single priority value. This value is used by the queuing code to decide which I/O to issue next. This value will be set in an event as a zio_priority payload entry.

Type

Value

Description

ZIO_PRIORITY_SYNC_READ

0x0

ZIO_PRIORITY_SYNC_WRITE

0x1

ZIL

ZIO_PRIORITY_ASYNC_READ

0x2

prefetch

ZIO_PRIORITY_ASYNC_WRITE

0x3

spa_sync()

ZIO_PRIORITY_SCRUB

0x4

asynchronous scrub/resilver reads

ZIO_PRIORITY_REMOVAL

0x5

reads/writes for vdev removal

ZIO_PRIORITY_INITIALIZING

0x6

initializing I/O

ZIO_PRIORITY_TRIM

0x7

trim I/O (discard)

ZIO_PRIORITY_REBUILD

0x8

reads/writes for vdev rebuild

ZIO_PRIORITY_NOW

0xa

non-queued i/os (e.g. free)

## SEE ALSO
zfs(4), zed(8), zpool-wait(8)


zpool-export

title: zpool-export(8)

summary: export ZFS storage pools

text:
# zpool-export(8)

## NAME
zpool-export — export ZFS storage pools

## SYNOPSIS
zpool

export [-f] -a|pool…

## DESCRIPTION
Exports the given pools from the system. All devices are marked as exported, but are still considered in use by other subsystems. The devices can be moved between systems (even those of different endianness) and imported as long as a sufficient number of devices are present.

Before exporting the pool, all datasets within the pool are unmounted. A pool can not be exported if it has a shared spare that is currently being used.

For pools to be portable, you must give the zpool command whole disks, not just partitions, so that ZFS can label the disks with portable EFI labels. Otherwise, disk drivers on platforms of different endianness will not recognize the disks.

-a

Exports all pools imported on the system.

-f

Forcefully unmount all datasets, and allow export of pools with active shared spares. This command will forcefully export the pool even if it has a shared spare that is currently being used. This may lead to potential data corruption.

## EXAMPLES

### Example 1: Exporting a ZFS Storage Pool
The following command exports the devices in pool tank so that they can be relocated or later imported:

\# zpool export tank

## SEE ALSO
zpool-import(8)


zpool-get

title: zpool-get(8)

text:
# zpool-get(8)

## NAME
zpool-get — retrieve properties of ZFS storage pools

## SYNOPSIS
zpool

get [-Hp] [-j [--json-int, --json-pool-key-guid]] [-o field[,field]…] all|property[,property]… [pool]…

zpool

get [-Hp] [-j [--json-int]] [-o field[,field]…] all|property[,property]… pool [all-vdevs|vdev]…

zpool

set property=value pool

zpool

set property=value pool vdev

## DESCRIPTION
zpool get [-Hp] [-j [--json-int, --json-pool-key-guid]] [-o field[,field]…] all|property[,property]… [pool]…

Retrieves the given list of properties (or all properties if all is used) for the specified storage pool(s). These properties are displayed with the following fields: name

Name of storage pool.

property

Property name.

value

Property value.

source

Property source, either default or local.

See the zpoolprops(7) manual page for more information on the available pool properties.

-j, --json [--json-int, --json-pool-key-guid]

Display the list of properties in JSON format. Specify --json-int to display the numbers in integer format instead of strings in JSON output. Specify --json-pool-key-guid to set pool GUID as key for pool objects instead of pool name.

-H

Scripted mode. Do not display headers, and separate fields by a single tab instead of arbitrary space.

-o field

A comma-separated list of columns to display, defaults to name,property,value,source.

-p

Display numbers in parsable (exact) values.

zpool get [-j [--json-int]] [-Hp] [-o field[,field]…] all|property[,property]… pool [all-vdevs|vdev]…

Retrieves the given list of properties (or all properties if all is used) for the specified vdevs (or all vdevs if all-vdevs is used) in the specified pool. These properties are displayed with the following fields: name

Name of vdev.

property

Property name.

value

Property value.

source

Property source, either default or local.

See the vdevprops(7) manual page for more information on the available pool properties.

-j, --json [--json-int]

Display the list of properties in JSON format. Specify --json-int to display the numbers in integer format instead of strings in JSON output.

-H

Scripted mode. Do not display headers, and separate fields by a single tab instead of arbitrary space.

-o field

A comma-separated list of columns to display, defaults to name,property,value,source.

-p

Display numbers in parsable (exact) values.

zpool set property=value pool

Sets the given property on the specified pool. See the zpoolprops(7) manual page for more information on what properties can be set and acceptable values.

zpool set property=value pool vdev

Sets the given property on the specified vdev in the specified pool. See the vdevprops(7) manual page for more information on what properties can be set and acceptable values.

## SEE ALSO
vdevprops(7), zpool-features(7), zpoolprops(7), zpool-list(8)


zpool-history

title: zpool-history(8)

text:
# zpool-history(8)

## NAME
zpool-history — inspect command history of ZFS storage pools

## SYNOPSIS
zpool

history [-il] [pool]…

## DESCRIPTION
Displays the command history of the specified pool(s) or all pools if no pool is specified.

-i

Displays internally logged ZFS events in addition to user initiated events.

-l

Displays log records in long format, which in addition to standard format includes, the user name, the hostname, and the zone in which the operation was performed.

## SEE ALSO
zpool-checkpoint(8), zpool-events(8), zpool-status(8), zpool-wait(8)


zpool-import

title: zpool-import(8)

text:
# zpool-import(8)

## NAME
zpool-import — import ZFS storage pools or list available pools

## SYNOPSIS
zpool

import [-D] [-d dir|device]…

zpool

import -a [-DflmN] [-F [-nTX]] [--rewind-to-checkpoint] [-c cachefile|-d dir|device] [-o mntopts] [-o property=value]… [-R root]

zpool

import [-Dflmt] [-F [-nTX]] [--rewind-to-checkpoint] [-c cachefile|-d dir|device] [-o mntopts] [-o property=value]… [-R root] [-s] pool|id [newpool]

## DESCRIPTION
zpool import [-D] [-d dir|device]…

Lists pools available to import. If the -d or -c options are not specified, this command searches for devices using libblkid on Linux and geom on FreeBSD. The -d option can be specified multiple times, and all directories are searched. If the device appears to be part of an exported pool, this command displays a summary of the pool with the name of the pool, a numeric identifier, as well as the vdev layout and current health of the device for each device or file. Destroyed pools, pools that were previously destroyed with the zpool destroy command, are not listed unless the -D option is specified. The numeric identifier is unique, and can be used instead of the pool name when multiple exported pools of the same name are available.

-c cachefile

Reads configuration from the given cachefile that was created with the cachefile pool property. This cachefile is used instead of searching for devices.

-d dir|device

Uses device or searches for devices or files in dir. The -d option can be specified multiple times.

-D

Lists destroyed pools only.

zpool import -a [-DflmN] [-F [-nTX]] [-c cachefile|-d dir|device] [-o mntopts] [-o property=value]… [-R root] [-s]

Imports all pools found in the search directories. Identical to the previous command, except that all pools with a sufficient number of devices available are imported. Destroyed pools, pools that were previously destroyed with the zpool destroy command, will not be imported unless the -D option is specified. -a

Searches for and imports all pools found.

-c cachefile

Reads configuration from the given cachefile that was created with the cachefile pool property. This cachefile is used instead of searching for devices.

-d dir|device

Uses device or searches for devices or files in dir. The -d option can be specified multiple times. This option is incompatible with the -c option.

-D

Imports destroyed pools only. The -f option is also required.

-f

Forces import, even if the pool appears to be potentially active.

-F

Recovery mode for a non-importable pool. Attempt to return the pool to an importable state by discarding the last few transactions. Not all damaged pools can be recovered by using this option. If successful, the data from the discarded transactions is irretrievably lost. This option is ignored if the pool is importable or already imported.

-l

Indicates that this command will request encryption keys for all encrypted datasets it attempts to mount as it is bringing the pool online. Note that if any datasets have a keylocation of prompt this command will block waiting for the keys to be entered. Without this flag encrypted datasets will be left unavailable until the keys are loaded.

-m

Allows a pool to import when there is a missing log device. Recent transactions can be lost because the log device will be discarded.

-n

Used with the -F recovery option. Determines whether a non-importable pool can be made importable again, but does not actually perform the pool recovery. For more details about pool recovery mode, see the -F option, above.

-N

Import the pool without mounting any file systems.

-o mntopts

Comma-separated list of mount options to use when mounting datasets within the pool. See zfs(8) for a description of dataset properties and mount options.

-o property=value

Sets the specified property on the imported pool. See the zpoolprops(7) manual page for more information on the available pool properties.

-R root

Sets the cachefile property to none and the altroot property to root.

--rewind-to-checkpoint

Rewinds pool to the checkpointed state. Once the pool is imported with this flag there is no way to undo the rewind. All changes and data that were written after the checkpoint are lost! The only exception is when the readonly mounting option is enabled. In this case, the checkpointed state of the pool is opened and an administrator can see how the pool would look like if they were to fully rewind.

-s

Scan using the default search path, the libblkid cache will not be consulted. A custom search path may be specified by setting the ZPOOL_IMPORT_PATH environment variable.

-X

Used with the -F recovery option. Determines whether extreme measures to find a valid txg should take place. This allows the pool to be rolled back to a txg which is no longer guaranteed to be consistent. Pools imported at an inconsistent txg may contain uncorrectable checksum errors. For more details about pool recovery mode, see the -F option, above. WARNING: This option can be extremely hazardous to the health of your pool and should only be used as a last resort.

-T

Specify the txg to use for rollback. Implies -FX. For more details about pool recovery mode, see the -X option, above. WARNING: This option can be extremely hazardous to the health of your pool and should only be used as a last resort.

zpool import [-Dflmt] [-F [-nTX]] [-c cachefile|-d dir|device] [-o mntopts] [-o property=value]… [-R root] [-s] pool|id [newpool]

Imports a specific pool. A pool can be identified by its name or the numeric identifier. If newpool is specified, the pool is imported using the name newpool. Otherwise, it is imported with the same name as its exported name. If a device is removed from a system without running zpool export first, the device appears as potentially active. It cannot be determined if this was a failed export, or whether the device is really in use from another host. To import a pool in this state, the -f option is required.

-c cachefile

Reads configuration from the given cachefile that was created with the cachefile pool property. This cachefile is used instead of searching for devices.

-d dir|device

Uses device or searches for devices or files in dir. The -d option can be specified multiple times. This option is incompatible with the -c option.

-D

Imports destroyed pool. The -f option is also required.

-f

Forces import, even if the pool appears to be potentially active.

-F

Recovery mode for a non-importable pool. Attempt to return the pool to an importable state by discarding the last few transactions. Not all damaged pools can be recovered by using this option. If successful, the data from the discarded transactions is irretrievably lost. This option is ignored if the pool is importable or already imported.

-l

Indicates that this command will request encryption keys for all encrypted datasets it attempts to mount as it is bringing the pool online. Note that if any datasets have a keylocation of prompt this command will block waiting for the keys to be entered. Without this flag encrypted datasets will be left unavailable until the keys are loaded.

-m

Allows a pool to import when there is a missing log device. Recent transactions can be lost because the log device will be discarded.

-n

Used with the -F recovery option. Determines whether a non-importable pool can be made importable again, but does not actually perform the pool recovery. For more details about pool recovery mode, see the -F option, above.

-o mntopts

Comma-separated list of mount options to use when mounting datasets within the pool. See zfs(8) for a description of dataset properties and mount options.

-o property=value

Sets the specified property on the imported pool. See the zpoolprops(7) manual page for more information on the available pool properties.

-R root

Sets the cachefile property to none and the altroot property to root.

-s

Scan using the default search path, the libblkid cache will not be consulted. A custom search path may be specified by setting the ZPOOL_IMPORT_PATH environment variable.

-X

Used with the -F recovery option. Determines whether extreme measures to find a valid txg should take place. This allows the pool to be rolled back to a txg which is no longer guaranteed to be consistent. Pools imported at an inconsistent txg may contain uncorrectable checksum errors. For more details about pool recovery mode, see the -F option, above. WARNING: This option can be extremely hazardous to the health of your pool and should only be used as a last resort.

-T

Specify the txg to use for rollback. Implies -FX. For more details about pool recovery mode, see the -X option, above. WARNING: This option can be extremely hazardous to the health of your pool and should only be used as a last resort.

-t

Used with newpool. Specifies that newpool is temporary. Temporary pool names last until export. Ensures that the original pool name will be used in all label updates and therefore is retained upon export. Will also set -o cachefile=none when not explicitly specified.

## EXAMPLES

### Example 9: Importing a ZFS Storage Pool
The following command displays available pools, and then imports the pool tank for use on the system. The results from this command are similar to the following:

```
# zpool import
  pool: tank
    id: 15451357997522795478
 state: ONLINE
action: The pool can be imported using its name or numeric identifier.
config:

        tank        ONLINE
          mirror    ONLINE
            sda     ONLINE
            sdb     ONLINE

# zpool import tank
```

## SEE ALSO
zpool-export(8), zpool-list(8), zpool-status(8)


zpool-initialize

title: zpool-initialize(8)

text:
# zpool-initialize(8)

## NAME
zpool-initialize — write to unallocated regions of ZFS storage pool

## SYNOPSIS
zpool

initialize [-c|-s |-u] [-w] -a|pool [device]…

## DESCRIPTION
Begins initializing by writing to all unallocated regions on the specified devices, or all eligible devices in the pool if no individual devices are specified. Only leaf data or log devices may be initialized.

-a, --all

Begin, cancel, suspend initializing on all pools.

-c, --cancel

Cancel initializing on the specified devices, or all eligible devices if none are specified. If one or more target devices are invalid or are not currently being initialized, the command will fail and no cancellation will occur on any device.

-s, --suspend

Suspend initializing on the specified devices, or all eligible devices if none are specified. If one or more target devices are invalid or are not currently being initialized, the command will fail and no suspension will occur on any device. Initializing can then be resumed by running zpool initialize with no flags on the relevant target devices.

-u, --uninit

Clears the initialization state on the specified devices, or all eligible devices if none are specified. If the devices are being actively initialized the command will fail. After being cleared zpool initialize with no flags can be used to re-initialize all unallocated regions on the relevant target devices.

-w, --wait

Wait until the devices have finished initializing before returning.

## SEE ALSO
zpool-add(8), zpool-attach(8), zpool-create(8), zpool-online(8), zpool-replace(8), zpool-trim(8)


zpool-iostat

title: zpool-iostat(8)

text:
# zpool-iostat(8)

## NAME
zpool-iostat — display logical I/O statistics for ZFS storage pools

## SYNOPSIS
zpool

iostat [[[-c SCRIPT] [-lq]]|-rw] [-T u|d] [-ghHLnpPvy] [pool…|[pool vdev…]|vdev…] [interval [count]]

## DESCRIPTION
Displays logical I/O statistics for the given pools/vdevs. Physical I/O statistics may be observed via iostat(1). If writes are located nearby, they may be merged into a single larger operation. Additional I/O may be generated depending on the level of vdev redundancy. To filter output, you may pass in a list of pools, a pool and list of vdevs in that pool, or a list of any vdevs from any pool. If no items are specified, statistics for every pool in the system are shown. When given an interval, the statistics are printed every interval seconds until killed. If -n flag is specified the headers are displayed only once, otherwise they are displayed periodically. If count is specified, the command exits after count reports are printed. The first report printed is always the statistics since boot regardless of whether interval and count are passed. However, this behavior can be suppressed with the -y flag. Also note that the units of K, M, G… that are printed in the report are in base 1024. To get the raw values, use the -p flag.

-c [SCRIPT1[,SCRIPT2]…]

Run a script (or scripts) on each vdev and include the output as a new column in the zpool iostat output. Users can run any script found in their ~/.zpool.d directory or from the system /etc/zfs/zpool.d directory. Script names containing the slash (/) character are not allowed. The default search path can be overridden by setting the ZPOOL_SCRIPTS_PATH environment variable. A privileged user can only run -c if they have the ZPOOL_SCRIPTS_AS_ROOT environment variable set. If a script requires the use of a privileged command, like smartctl(8), then it's recommended you allow the user access to it in /etc/sudoers or add the user to the /etc/sudoers.d/zfs file. If -c is passed without a script name, it prints a list of all scripts. -c also sets verbose mode (-v).

Script output should be in the form of "name=value". The column name is set to "name" and the value is set to "value". Multiple lines can be used to output multiple columns. The first line of output not in the "name=value" format is displayed without a column title, and no more output after that is displayed. This can be useful for printing error messages. Blank or NULL values are printed as a '-' to make output AWKable.

The following environment variables are set before running each script:

VDEV_PATH

Full path to the vdev

VDEV_UPATH

Underlying path to the vdev (/dev/sd*). For use with device mapper, multipath, or partitioned vdevs.

VDEV_ENC_SYSFS_PATH

The sysfs path to the enclosure for the vdev (if any).

-T u|d

Display a time stamp. Specify u for a printed representation of the internal representation of time. See time(1). Specify d for standard date format. See date(1).

-g

Display vdev GUIDs instead of the normal device names. These GUIDs can be used in place of device names for the zpool detach/offline/remove/replace commands.

-H

Scripted mode. Do not display headers, and separate fields by a single tab instead of arbitrary space.

-L

Display real paths for vdevs resolving all symbolic links. This can be used to look up the current block device name regardless of the /dev/disk/ path used to open it.

-n

Print headers only once when passed

-p

Display numbers in parsable (exact) values. Time values are in nanoseconds.

-P

Display full paths for vdevs instead of only the last component of the path. This can be used in conjunction with the -L flag.

-r

Print request size histograms for the leaf vdev's I/O. This includes histograms of individual I/O (ind) and aggregate I/O (agg). These stats can be useful for observing how well I/O aggregation is working. Note that TRIM I/O may exceed 16M, but will be counted as 16M.

-v

Verbose statistics Reports usage statistics for individual vdevs within the pool, in addition to the pool-wide statistics.

-y

Normally the first line of output reports the statistics since boot: suppress it.

-w

Display latency histograms: total_wait

Total I/O time (queuing + disk I/O time).

disk_wait

Disk I/O time (time reading/writing the disk).

syncq_wait

Amount of time I/O spent in synchronous priority queues. Does not include disk time.

asyncq_wait

Amount of time I/O spent in asynchronous priority queues. Does not include disk time.

scrub

Amount of time I/O spent in scrub queue. Does not include disk time.

rebuild

Amount of time I/O spent in rebuild queue. Does not include disk time.

-l

Include average latency statistics: total_wait

Average total I/O time (queuing + disk I/O time).

disk_wait

Average disk I/O time (time reading/writing the disk).

syncq_wait

Average amount of time I/O spent in synchronous priority queues. Does not include disk time.

asyncq_wait

Average amount of time I/O spent in asynchronous priority queues. Does not include disk time.

scrub

Average queuing time in scrub queue. Does not include disk time.

trim

Average queuing time in trim queue. Does not include disk time.

rebuild

Average queuing time in rebuild queue. Does not include disk time.

-q

Include active queue statistics. Each priority queue has both pending (pend) and active (activ) I/O requests. Pending requests are waiting to be issued to the disk, and active requests have been issued to disk and are waiting for completion. These stats are broken out by priority queue: syncq_read/write

Current number of entries in synchronous priority queues.

asyncq_read/write

Current number of entries in asynchronous priority queues.

scrubq_read

Current number of entries in scrub queue.

trimq_write

Current number of entries in trim queue.

rebuildq_write

Current number of entries in rebuild queue.

All queue statistics are instantaneous measurements of the number of entries in the queues. If you specify an interval, the measurements will be sampled from the end of the interval.

## EXAMPLES

### Example 13: Adding Cache Devices to a ZFS Pool
The following command adds two disks for use as cache devices to a ZFS storage pool:

\# zpool add pool cache sdc sdd

Once added, the cache devices gradually fill with content from main memory. Depending on the size of your cache devices, it could take over an hour for them to fill. Capacity and reads can be monitored using the iostat subcommand as follows:

\# zpool iostat -v pool 5

### Example 16: Adding output columns
Additional columns can be added to the zpool status and zpool iostat output with -c.

```
# zpool status -c vendor,model,size
   NAME     STATE  READ WRITE CKSUM vendor  model        size
   tank     ONLINE 0    0     0
   mirror-0 ONLINE 0    0     0
   U1       ONLINE 0    0     0     SEAGATE ST8000NM0075 7.3T
   U10      ONLINE 0    0     0     SEAGATE ST8000NM0075 7.3T
   U11      ONLINE 0    0     0     SEAGATE ST8000NM0075 7.3T
   U12      ONLINE 0    0     0     SEAGATE ST8000NM0075 7.3T
   U13      ONLINE 0    0     0     SEAGATE ST8000NM0075 7.3T
   U14      ONLINE 0    0     0     SEAGATE ST8000NM0075 7.3T

# zpool iostat -vc size
              capacity     operations     bandwidth
pool        alloc   free   read  write   read  write  size
----------  -----  -----  -----  -----  -----  -----  ----
rpool       14.6G  54.9G      4     55   250K  2.69M
  sda1      14.6G  54.9G      4     55   250K  2.69M   70G
----------  -----  -----  -----  -----  -----  -----  ----
```

## SEE ALSO
iostat(1), smartctl(8), zpool-list(8), zpool-status(8)


zpool-labelclear

title: zpool-labelclear(8)

text:
# zpool-labelclear(8)

## NAME
zpool-labelclear — remove ZFS label information from device

## SYNOPSIS
zpool

labelclear [-f] device

## DESCRIPTION
Removes ZFS label information from the specified device. If the device is a cache device, it also removes the L2ARC header (persistent L2ARC). The device must not be part of an active pool configuration.

-f

Treat exported or foreign devices as inactive.

## SEE ALSO
zpool-destroy(8), zpool-detach(8), zpool-remove(8), zpool-replace(8)


zpool-list

title: zpool-list(8)

text:
# zpool-list(8)

## NAME
zpool-list — list information about ZFS storage pools

## SYNOPSIS
zpool

list [-HgLpPv] [-j [--json-int, --json-pool-key-guid]] [-o property[,property]…] [-T u|d] [pool]… [interval [count]]

## DESCRIPTION
Lists the given pools along with a health status and space usage. If no pools are specified, all pools in the system are listed. When given an interval, the information is printed every interval seconds until killed. If count is specified, the command exits after count reports are printed.

-j, --json [--json-int, --json-pool-key-guid]

Display the list of pools in JSON format. Specify --json-int to display the numbers in integer format instead of strings. Specify --json-pool-key-guid to set pool GUID as key for pool objects instead of pool names.

-g

Display vdev GUIDs instead of the normal device names. These GUIDs can be used in place of device names for the zpool detach/offline/remove/replace commands.

-H

Scripted mode. Do not display headers, and separate fields by a single tab instead of arbitrary space.

-o property

Comma-separated list of properties to display. See the zpoolprops(7) manual page for a list of valid properties. The default list is name, size, allocated, free, checkpoint, expandsize, fragmentation, capacity, dedupratio, health, altroot.

-L

Display real paths for vdevs resolving all symbolic links. This can be used to look up the current block device name regardless of the /dev/disk path used to open it.

-p

Display numbers in parsable (exact) values.

-P

Display full paths for vdevs instead of only the last component of the path. This can be used in conjunction with the -L flag.

-T u|d

Display a time stamp. Specify u for a printed representation of the internal representation of time. See time(1). Specify d for standard date format. See date(1).

-v

Verbose statistics. Reports usage statistics for individual vdevs within the pool, in addition to the pool-wide statistics.

## EXAMPLES

### Example 1: Listing Available ZFS Storage Pools
The following command lists all available pools on the system. In this case, the pool zion is faulted due to a missing device. The results from this command are similar to the following:

```
# zpool list
NAME    SIZE  ALLOC   FREE  EXPANDSZ   FRAG    CAP  DEDUP  HEALTH  ALTROOT
rpool  19.9G  8.43G  11.4G         -    33%    42%  1.00x  ONLINE  -
tank   61.5G  20.0G  41.5G         -    48%    32%  1.00x  ONLINE  -
zion       -      -      -         -      -      -      -  FAULTED -
```

### Example 2: Displaying expanded space on a device
The following command displays the detailed information for the pool data. This pool is comprised of a single raidz vdev where one of its devices increased its capacity by 10 GiB. In this example, the pool will not be able to utilize this extra capacity until all the devices under the raidz vdev have been expanded.

```
# zpool list -v data
NAME         SIZE  ALLOC   FREE  EXPANDSZ   FRAG    CAP  DEDUP  HEALTH  ALTROOT
data        23.9G  14.6G  9.30G         -    48%    61%  1.00x  ONLINE  -
  raidz1    23.9G  14.6G  9.30G         -    48%
    sda         -      -      -         -      -
    sdb         -      -      -       10G      -
    sdc         -      -      -         -      -
```

### Example 3: Displaying expanded space on a device
The following command lists all available pools on the system in JSON format.

```
# zpool list -j | jq
{
  "output_version": {
    "command": "zpool list",
    "vers_major": 0,
    "vers_minor": 1
  },
  "pools": {
    "tank": {
      "name": "tank",
      "type": "POOL",
      "state": "ONLINE",
      "guid": "15220353080205405147",
      "txg": "2671",
      "spa_version": "5000",
      "zpl_version": "5",
      "properties": {
        "size": {
          "value": "111G",
          "source": {
            "type": "NONE",
            "data": "-"
          }
        },
        "allocated": {
          "value": "30.8G",
          "source": {
            "type": "NONE",
            "data": "-"
          }
        },
        "free": {
          "value": "80.2G",
          "source": {
            "type": "NONE",
            "data": "-"
          }
        },
        "checkpoint": {
          "value": "-",
          "source": {
            "type": "NONE",
            "data": "-"
          }
        },
        "expandsize": {
          "value": "-",
          "source": {
            "type": "NONE",
            "data": "-"
          }
        },
        "fragmentation": {
          "value": "0%",
          "source": {
            "type": "NONE",
            "data": "-"
          }
        },
        "capacity": {
          "value": "27%",
          "source": {
            "type": "NONE",
            "data": "-"
          }
        },
        "dedupratio": {
          "value": "1.00x",
          "source": {
            "type": "NONE",
            "data": "-"
          }
        },
        "health": {
          "value": "ONLINE",
          "source": {
            "type": "NONE",
            "data": "-"
          }
        },
        "altroot": {
          "value": "-",
          "source": {
            "type": "DEFAULT",
            "data": "-"
          }
        }
      }
    }
  }
}
```

## SEE ALSO
zpool-import(8), zpool-status(8)


zpool-offline

title: zpool-offline(8)

text:
# zpool-offline(8)

## NAME
zpool-offline — take physical devices offline in ZFS storage pool

## SYNOPSIS
zpool

offline [--power|[-ft]] pool device…

zpool

online [--power] [-e] pool device…

## DESCRIPTION
zpool offline [--power|[-ft]] pool device…

Takes the specified physical device offline. While the device is offline, no attempt is made to read or write to the device. This command is not applicable to spares. --power

Power off the device's slot in the storage enclosure. This flag currently works on Linux only

-f

Force fault. Instead of offlining the disk, put it into a faulted state. The fault will persist across imports unless the -t flag was specified.

-t

Temporary. Upon reboot, the specified physical device reverts to its previous state.

zpool online [--power] [-e] pool device…

Brings the specified physical device online. This command is not applicable to spares. --power

Power on the device's slot in the storage enclosure and wait for the device to show up before attempting to online it. Alternatively, you can set the ZPOOL_AUTO_POWER_ON_SLOT environment variable to always enable this behavior. This flag currently works on Linux only

-e

Expand the device to use all available space. If the device is part of a mirror or raidz then all devices must be expanded before the new space will become available to the pool.

## SEE ALSO
zpool-detach(8), zpool-remove(8), zpool-reopen(8), zpool-resilver(8)


zpool-online

title: zpool-online(8)

text:
# zpool-online(8)

## NAME
zpool-offline — take physical devices offline in ZFS storage pool

## SYNOPSIS
zpool

offline [--power|[-ft]] pool device…

zpool

online [--power] [-e] pool device…

## DESCRIPTION
zpool offline [--power|[-ft]] pool device…

Takes the specified physical device offline. While the device is offline, no attempt is made to read or write to the device. This command is not applicable to spares. --power

Power off the device's slot in the storage enclosure. This flag currently works on Linux only

-f

Force fault. Instead of offlining the disk, put it into a faulted state. The fault will persist across imports unless the -t flag was specified.

-t

Temporary. Upon reboot, the specified physical device reverts to its previous state.

zpool online [--power] [-e] pool device…

Brings the specified physical device online. This command is not applicable to spares. --power

Power on the device's slot in the storage enclosure and wait for the device to show up before attempting to online it. Alternatively, you can set the ZPOOL_AUTO_POWER_ON_SLOT environment variable to always enable this behavior. This flag currently works on Linux only

-e

Expand the device to use all available space. If the device is part of a mirror or raidz then all devices must be expanded before the new space will become available to the pool.

## SEE ALSO
zpool-detach(8), zpool-remove(8), zpool-reopen(8), zpool-resilver(8)


zpool-prefetch

title: zpool-prefetch(8)

summary: Prefetches pool metadata into ARC

text:
# zpool-prefetch(8)

## NAME
zpool-prefetch — Prefetches pool metadata into ARC

## SYNOPSIS
zpool

prefetch [-t type] pool

## DESCRIPTION
Massively prefetch metadata of a specific type for the given pool into the ARC to reduce latency of some operations later. If no type is specified, all types are prefetched.

The following types are supported:

brt

Prefetch the BRT (block reference table). This may improve performance for block cloning operations, and frees for earlier cloned blocks.

ddt

Prefetch the DDT (deduplication table). This may improve performance of writes when deduplication is enabled, and frees for earlier deduplicated blocks.


zpool-reguid

title: zpool-reguid(8)

text:
# zpool-reguid(8)

## NAME
zpool-reguid — generate new unique identifier for ZFS storage pool

## SYNOPSIS
zpool

reguid [-g guid] pool

## DESCRIPTION
Generates a new unique identifier for the pool. You must ensure that all devices in this pool are online and healthy before performing this action.

-g guid

Set the pool GUID to the provided value. The GUID can be any 64-bit value accepted by strtoull(3) in base 10. zpool-reguid will return an error if the provided GUID is already in use.

## SEE ALSO
zpool-export(8), zpool-import(8)


zpool-remove

title: zpool-remove(8)

summary: remove devices from ZFS storage pool

text:
# zpool-remove(8)

## NAME
zpool-remove — remove devices from ZFS storage pool

## SYNOPSIS
zpool

remove [-npw] pool device…

zpool

remove -s pool

## DESCRIPTION
zpool remove [-npw] pool device…

Removes the specified device from the pool. This command supports removing hot spare, cache, log, and both mirrored and non-redundant primary top-level vdevs, including dedup and special vdevs. Top-level vdevs can only be removed if the primary pool storage does not contain a top-level raidz or draid vdev, all top-level vdevs have the same ashift size, and the keys for all encrypted datasets are loaded.

Removing a top-level vdev reduces the total amount of space in the storage pool. The specified device will be evacuated by copying all allocated space from it to the other devices in the pool. In this case, the zpool remove command initiates the removal and returns, while the evacuation continues in the background. The removal progress can be monitored with zpool status. If an I/O error is encountered during the removal process it will be canceled. The device_removal feature flag must be enabled to remove a top-level vdev, see zpool-features(7).

A mirrored top-level device (log or data) can be removed by specifying the top- level mirror for the same. Non-log devices or data devices that are part of a mirrored configuration can be removed using the zpool detach command.

-n

Do not actually perform the removal ("No-op"). Instead, print the estimated amount of memory that will be used by the mapping table after the removal completes. This is nonzero only for top-level vdevs.

-p

Used in conjunction with the -n flag, displays numbers as parsable (exact) values.

-w

Waits until the removal has completed before returning.

zpool remove -s pool

Stops and cancels an in-progress removal of a top-level vdev.

## EXAMPLES

### Example 1: Removing a Mirrored top-level (Log or Data) Device
The following commands remove the mirrored log device mirror-2 and mirrored top-level data device mirror-1.

Given this configuration:

```
  pool: tank
 state: ONLINE
 scrub: none requested
config:

         NAME        STATE     READ WRITE CKSUM
         tank        ONLINE       0     0     0
           mirror-0  ONLINE       0     0     0
             sda     ONLINE       0     0     0
             sdb     ONLINE       0     0     0
           mirror-1  ONLINE       0     0     0
             sdc     ONLINE       0     0     0
             sdd     ONLINE       0     0     0
         logs
           mirror-2  ONLINE       0     0     0
             sde     ONLINE       0     0     0
             sdf     ONLINE       0     0     0
```

The command to remove the mirrored log mirror-2 is:

\# zpool remove tank mirror-2

At this point, the log device no longer exists (both sides of the mirror have been removed):

```
  pool: tank
 state: ONLINE
  scan: none requested
config:

        NAME        STATE     READ WRITE CKSUM
        tank        ONLINE       0     0     0
          mirror-0  ONLINE       0     0     0
            sda     ONLINE       0     0     0
            sdb     ONLINE       0     0     0
          mirror-1  ONLINE       0     0     0
            sdc     ONLINE       0     0     0
            sdd     ONLINE       0     0     0
```

The command to remove the mirrored data mirror-1 is:

\# zpool remove tank mirror-1

After mirror-1 has been evacuated, the pool remains redundant, but the total amount of space is reduced:

```
  pool: tank
 state: ONLINE
  scan: none requested
config:

        NAME        STATE     READ WRITE CKSUM
        tank        ONLINE       0     0     0
          mirror-0  ONLINE       0     0     0
            sda     ONLINE       0     0     0
            sdb     ONLINE       0     0     0
```

## SEE ALSO
zpool-add(8), zpool-detach(8), zpool-labelclear(8), zpool-offline(8), zpool-replace(8), zpool-split(8)


zpool-reopen

title: zpool-reopen(8)

text:
# zpool-reopen(8)

## NAME
zpool-reopen — reopen vdevs associated with ZFS storage pools

## SYNOPSIS
zpool

reopen [-n] [pool]…

## DESCRIPTION
Reopen all vdevs associated with the specified pools, or all pools if none specified.

## OPTIONS
-n

Do not restart an in-progress scrub operation. This is not recommended and can result in partially resilvered devices unless a second scrub is performed.


zpool-replace

title: zpool-replace(8)

text:
# zpool-replace(8)

## NAME
zpool-replace — replace one device with another in ZFS storage pool

## SYNOPSIS
zpool

replace [-fsw] [-o property=value] pool device [new-device]

## DESCRIPTION
Replaces device with new-device. This is equivalent to attaching new-device, waiting for it to resilver, and then detaching device. Any in progress scrub will be canceled.

The size of new-device must be greater than or equal to the minimum size of all the devices in a mirror or raidz configuration.

new-device is required if the pool is not redundant. If new-device is not specified, it defaults to device. This form of replacement is useful after an existing disk has failed and has been physically replaced. In this case, the new disk may have the same /dev path as the old device, even though it is actually a different disk. ZFS recognizes this.

-f

Forces use of new-device, even if it appears to be in use. Not all devices can be overridden in this manner.

-o property=value

Sets the given pool properties. See the zpoolprops(7) manual page for a list of valid properties that can be set. The only property supported at the moment is ashift.

-s

The new-device is reconstructed sequentially to restore redundancy as quickly as possible. Checksums are not verified during sequential reconstruction so a scrub is started when the resilver completes. Sequential reconstruction is not supported for raidz configurations.

-w

Waits until the replacement has completed before returning.

## SEE ALSO
zpool-detach(8), zpool-initialize(8), zpool-online(8), zpool-resilver(8)


zpool-resilver

title: zpool-resilver(8)

summary: resilver devices in ZFS storage pools

text:
# zpool-resilver(8)

## NAME
zpool-resilver — resilver devices in ZFS storage pools

## SYNOPSIS
zpool

resilver pool…

## DESCRIPTION
Starts a resilver of the specified pools. If an existing resilver is already running it will be restarted from the beginning. Any drives that were scheduled for a deferred resilver will be added to the new one. This requires the resilver_defer pool feature.

## SEE ALSO
zpool-iostat(8), zpool-online(8), zpool-reopen(8), zpool-replace(8), zpool-scrub(8), zpool-status(8)


zpool-scrub

title: zpool-scrub(8)

text:
# zpool-scrub(8)

## NAME
zpool-scrub — begin or resume scrub of ZFS storage pools

## SYNOPSIS
zpool

scrub [-e |-p | -s| -C][-w] [-S date] [-E date] -a|pool…

## DESCRIPTION
Begins a scrub or resumes a paused scrub. The scrub examines all data in the specified pools to verify that it checksums correctly. For replicated (mirror, raidz, or draid) devices, ZFS automatically repairs any damage discovered during the scrub. The zpool status command reports the progress of the scrub and summarizes the results of the scrub upon completion.

Scrubbing and resilvering are very similar operations. The difference is that resilvering only examines data that ZFS knows to be out of date (for example, when attaching a new device to a mirror or replacing an existing device), whereas scrubbing examines all data to discover silent errors due to hardware faults or disk failure.

When scrubbing a pool with encrypted filesystems the keys do not need to be loaded. However, if the keys are not loaded and an unrepairable checksum error is detected the file name cannot be included in the zpool status -v verbose error report.

Because scrubbing and resilvering are I/O-intensive operations, ZFS only allows one at a time.

A scrub is split into two parts: metadata scanning and block scrubbing. The metadata scanning sorts blocks into large sequential ranges which can then be read much more efficiently from disk when issuing the scrub I/O.

If a scrub is paused, the zpool scrub resumes it. If a resilver is in progress, ZFS does not allow a scrub to be started until the resilver completes.

Note that, due to changes in pool data on a live system, it is possible for scrubs to progress slightly beyond 100% completion. During this period, no completion time estimate will be provided.

## OPTIONS
-a, --all

Begin, pause, stop scrub on all pools. Initiating scrubs on multiple pools can put considerable load and memory pressure on the system, so this operation should be performed with caution.

-s

Stop scrubbing.

-p

Pause scrubbing. Scrub pause state and progress are periodically synced to disk. If the system is restarted or pool is exported during a paused scrub, even after import, scrub will remain paused until it is resumed. Once resumed the scrub will pick up from the place where it was last checkpointed to disk. To resume a paused scrub issue zpool scrub or zpool scrub -e again.

-w

Wait until scrub has completed before returning.

-e

Only scrub files with known data errors as reported by zpool status -v. The pool must have been scrubbed at least once with the head_errlog feature enabled to use this option. Error scrubbing cannot be run simultaneously with regular scrubbing or resilvering, nor can it be run when a regular scrub is paused.

-C

Continue scrub from last saved txg (see zpool last_scrubbed_txg property).

-S date, -E date

Allows specifying the date range for blocks created between these dates.

- -S Defines a start date. If not specified, scrubbing begins from the start of the pool's existence.

- -E Defines an end date. If not specified, scrubbing continues up to the most recent data.

The provided date should be in the format: “YYYY-MM-DD HH:MM”. Where:

- “YYYY” is the year.

- “MM” is the numeric representation of the month.

- “DD” is the day of the month.

- “HH” is the hour.

- “MM” is the minutes.

The hour and minutes parameters can be omitted. The time should be provided in machine local time zone. Specifying dates prior to enabling this feature will result in scrubbing starting from the date the pool was created. If the time was moved backward manually the data range may become inaccurate.

## EXAMPLES

### Example 1
Status of pool with ongoing scrub:

```
# zpool status
  ...
  scan: scrub in progress since Sun Jul 25 16:07:49 2021
        403M / 405M scanned at 100M/s, 68.4M / 405M issued at 10.0M/s
        0B repaired, 16.91% done, 00:00:04 to go
  ...
```

Where metadata which references 403M of file data has been scanned at 100M/s, and 68.4M of that file data has been scrubbed sequentially at 10.0M/s.

## PERIODIC SCRUB
On machines using systemd, scrub timers can be enabled on per-pool basis. weekly and monthly timer units are provided.

systemctl enable zfs-scrub-weekly@rpool.timer --now

systemctl enable zfs-scrub-monthly@otherpool.timer --now

## SEE ALSO
systemd.timer(5), zpool-iostat(8), zpool-resilver(8), zpool-status(8)