lvmthin

title: lvmthin(7)

summary: LVM thin provisioning

text:
# lvmthin(7)

## NAME
lvmthin — LVM thin provisioning

## DESCRIPTION
Blocks in a standard lvm(8) Logical Volume (LV) are allocated when the LV is created, but blocks in a thin provisioned LV are allocated as they are written. Because of this, a thin provisioned LV is given a virtual size, and can then be much larger than physically available storage. The amount of physical storage provided for thin provisioned LVs can be increased later as the need arises.

Blocks in a standard LV are allocated (during creation) from the Volume Group (VG), but blocks in a thin LV are allocated (during use) from a special "thin pool LV". The thin pool LV contains blocks of physical storage, and blocks in thin LVs just reference blocks in the thin pool LV.

A thin pool LV must be created before thin LVs can be created within it. A thin pool LV is created by combining two standard LVs: a large data LV that will hold blocks for thin LVs, and a metadata LV that will hold metadata. The metadata tracks which data blocks belong to each thin LV.

Snapshots of thin LVs are efficient because the data blocks common to a thin LV and any of its snapshots are shared. Snapshots may be taken of thin LVs or of other thin snapshots. Blocks common to recursive snapshots are also shared in the thin pool. There is no limit to or degradation from sequences of snapshots.

As thin LVs or snapshot LVs are written to, they consume data blocks in the thin pool. As free data blocks in the pool decrease, more free blocks may need to be supplied. This is done by extending the thin pool data LV with additional physical space from the VG. Removing thin LVs or snapshots from the thin pool can also free blocks in the thin pool. However, removing LVs is not always an effective way of freeing space in a thin pool because the amount is limited to the number of blocks not shared with other LVs in the pool.

Incremental block allocation from thin pools can cause thin LVs to become fragmented. Standard LVs generally avoid this problem by allocating all the blocks at once during creation.

## THIN TERMS
ThinDataLV

thin data LV

large LV created in a VG

used by thin pool to store ThinLV blocks

ThinMetaLV

thin metadata LV

small LV created in a VG

used by thin pool to track data block usage

ThinPoolLV

thin pool LV

combination of ThinDataLV and ThinMetaLV

contains ThinLVs and SnapLVs

ThinLV

thin LV

created from ThinPoolLV

appears blank after creation

SnapLV

snapshot LV

created from ThinPoolLV

appears as a snapshot of another LV after creation

## THIN USAGE
The primary method for using lvm thin provisioning:

### 1. Create ThinDataLV
Create an LV that will hold thin pool data.

lvcreate -n ThinDataLV -L LargeSize VG

Example

\# lvcreate -n pool0 -L 10G vg

### 2. Create ThinMetaLV
Create an LV that will hold thin pool metadata.

lvcreate -n ThinMetaLV -L SmallSize VG

Example

\# lvcreate -n pool0meta -L 1G vg

\# lvs

LV VG Attr LSize

pool0 vg -wi-a----- 10.00g

pool0meta vg -wi-a----- 1.00g

### 3. Create ThinPoolLV

```
Combine the data and metadata LVs into a thin pool LV.
ThinDataLV is renamed to hidden ThinPoolLV_tdata.
ThinMetaLV is renamed to hidden ThinPoolLV_tmeta.
The new ThinPoolLV takes the previous name of ThinDataLV.
```

lvconvert --type thin-pool --poolmetadata VG/ThinMetaLV VG/ThinDataLV

Example

\# lvconvert --type thin-pool --poolmetadata vg/pool0meta vg/pool0

\# lvs vg/pool0

LV VG Attr LSize Pool Origin Data% Meta%

pool0 vg twi-a-tz-- 10.00g 0.00 0.00

\# lvs -a

LV VG Attr LSize

pool0 vg twi-a-tz-- 10.00g

[pool0_tdata] vg Twi-ao---- 10.00g

[pool0_tmeta] vg ewi-ao---- 1.00g

### 4. Create ThinLV

```
Create a new thin LV from the thin pool LV.
The thin LV is created with a virtual size.
Multiple new thin LVs may be created in the thin pool.
Thin LV names must be unique in the VG.
The '--type thin' option is inferred from the virtual size option.
The --thinpool argument specifies which thin pool will
contain the ThinLV.
```

lvcreate -n ThinLV -V VirtualSize --thinpool ThinPoolLV VG

Example

Create a thin LV in a thin pool:

\# lvcreate -n thin1 -V 1T --thinpool pool0 vg

Create another thin LV in the same thin pool:

\# lvcreate -n thin2 -V 1T --thinpool pool0 vg

\# lvs vg/thin1 vg/thin2

LV VG Attr LSize Pool Origin Data%

thin1 vg Vwi-a-tz-- 1.00t pool0 0.00

thin2 vg Vwi-a-tz-- 1.00t pool0 0.00

### 5. Create SnapLV
Create snapshots of an existing ThinLV or SnapLV.

Do not specify -L, --size when creating a thin snapshot.

A size argument will cause an old COW snapshot to be created.

lvcreate -n SnapLV --snapshot VG/ThinLV

lvcreate -n SnapLV --snapshot VG/PrevSnapLV

Example

Create first snapshot of an existing ThinLV:

\# lvcreate -n thin1s1 -s vg/thin1

Create second snapshot of the same ThinLV:

\# lvcreate -n thin1s2 -s vg/thin1

Create a snapshot of the first snapshot:

\# lvcreate -n thin1s1s1 -s vg/thin1s1

\# lvs vg/thin1s1 vg/thin1s2 vg/thin1s1s1

LV VG Attr LSize Pool Origin

thin1s1 vg Vwi---tz-k 1.00t pool0 thin1

thin1s2 vg Vwi---tz-k 1.00t pool0 thin1

thin1s1s1 vg Vwi---tz-k 1.00t pool0 thin1s1

### 6. Activate SnapLV
Thin snapshots are created with the persistent "activation skip" flag, indicated by the "k" attribute. Use -K with lvchange or vgchange to activate thin snapshots with the "k" attribute.

lvchange -ay -K VG/SnapLV

Example

\# lvchange -ay -K vg/thin1s1

\# lvs vg/thin1s1

LV VG Attr LSize Pool Origin

thin1s1 vg Vwi-a-tz-k 1.00t pool0 thin1

## THIN TOPICS
Automatic pool metadata LV

Specify devices for data and metadata LVs

Tolerate device failures using raid

Spare metadata LV

Metadata check and repair

Activation of thin snapshots

Removing thin pool LVs, thin LVs and snapshots

Manually manage free data space of thin pool LV

Manually manage free metadata space of a thin pool LV

Using fstrim to increase free space in a thin pool LV

Automatically extend thin pool LV

Data space exhaustion

Metadata space exhaustion

Automatic extend settings

Zeroing

Discard

Chunk size

Size of pool metadata LV

Create a thin snapshot of an external, read only LV

Convert a standard LV to a thin LV with an external origin

Single step thin pool LV creation

Single step thin pool LV and thin LV creation

Merge thin snapshots

XFS on snapshots

### Automatic pool metadata LV
A thin data LV can be converted to a thin pool LV without specifying a thin pool metadata LV. LVM automatically creates a metadata LV from the same VG.

lvcreate -n ThinDataLV -L LargeSize VG

lvconvert --type thin-pool VG/ThinDataLV

Example

```
# lvcreate -n pool0 -L 10G vg
# lvconvert --type thin-pool vg/pool0
```

```
# lvs -a

  pool0           vg          twi-a-tz--  10.00g

  [pool0_tdata]   vg          Twi-ao----  10.00g

  [pool0_tmeta]   vg          ewi-ao----  16.00m
```

### Specify devices for data and metadata LVs
The data and metadata LVs in a thin pool are best created on separate physical devices. To do that, specify the device name(s) at the end of the lvcreate line. It can be especially helpful to use fast devices for the metadata LV.

lvcreate -n ThinDataLV -L LargeSize VG LargePV

lvcreate -n ThinMetaLV -L SmallSize VG SmallPV

lvconvert --type thin-pool --poolmetadata VG/ThinMetaLV VG/ThinDataLV

Example

```
# lvcreate -n pool0 -L 10G vg /dev/sdA
# lvcreate -n pool0meta -L 1G vg /dev/sdB
# lvconvert --type thin-pool --poolmetadata vg/pool0meta vg/pool0
```

lvm.conf(5) thin_pool_metadata_require_separate_pvs

controls the default PV usage for thin pool creation.

### Tolerate device failures using raid
To tolerate device failures, use raid for the pool data LV and pool metadata LV. This is especially recommended for pool metadata LVs.

lvcreate --type raid1 -m 1 -n ThinMetaLV -L SmallSize VG PVA PVB

lvcreate --type raid1 -m 1 -n ThinDataLV -L LargeSize VG PVC PVD

lvconvert --type thin-pool --poolmetadata VG/ThinMetaLV VG/ThinDataLV

Example

```
# lvcreate --type raid1 -m 1 -n pool0 -L 10G vg /dev/sdA /dev/sdB
# lvcreate --type raid1 -m 1 -n pool0meta -L 1G vg /dev/sdC /dev/sdD
# lvconvert --type thin-pool --poolmetadata vg/pool0meta vg/pool0
```

### Spare metadata LV
The first time a thin pool LV is created, lvm will create a spare metadata LV in the VG. This behavior can be controlled with the option --poolmetadataspare y|n. (Future thin pool creations will also attempt to create the pmspare LV if none exists.)

To create the pmspare ("pool metadata spare") LV, lvm first creates an LV with a default name, e.g. lvol0, and then converts this LV to a hidden LV with the _pmspare suffix, e.g. lvol0_pmspare.

One pmspare LV is kept in a VG to be used for any thin pool.

The pmspare LV cannot be created explicitly, but may be removed explicitly.

Example

```
# lvcreate -n pool0 -L 10G vg
# lvcreate -n pool0meta -L 1G vg
# lvconvert --type thin-pool --poolmetadata vg/pool0meta vg/pool0
```

```
# lvs -a

  [lvol0_pmspare] vg          ewi-------

  pool0           vg          twi---tz--

  [pool0_tdata]   vg          Twi-------

  [pool0_tmeta]   vg          ewi-------
```

The "Metadata check and repair" section describes the use of the pmspare LV.

### Metadata check and repair
If thin pool metadata is damaged, it may be repairable. Checking and repairing thin pool metadata is analogous to running fsck/repair on a file system.

When a thin pool LV is activated, lvm runs the thin_check command to check the correctness of the metadata on the pool metadata LV.

lvm.conf(5) thin_check_executable

can be set to an empty string ("") to disable the thin_check step. This is not recommended.

lvm.conf(5) thin_check_options

controls the command options used for the thin_check command.

If the thin_check command finds a problem with the metadata, the thin pool LV is not activated, and the thin pool metadata needs to be repaired.

Simple repair commands are not always successful. Advanced repair may require editing thin pool metadata and lvm metadata. Newer versions of the kernel and lvm tools may be more successful at repair. Report the details of damaged thin metadata to get the best advice on recovery.

Command to repair a thin pool:

lvconvert --repair VG/ThinPoolLV

Repair performs the following steps:

1

Creates a new, repaired copy of the metadata.

lvconvert runs the thin_repair command to read damaged metadata from the existing pool metadata LV, and writes a new repaired copy to the VG's pmspare LV.

2

Replaces the thin pool metadata LV.

If step 1 is successful, the thin pool metadata LV is replaced with the pmspare LV containing the corrected metadata. The previous thin pool metadata LV, containing the damaged metadata, becomes visible with the new name ThinPoolLV_metaN (where N is 0,1,...).

If the repair works, the thin pool LV and its thin LVs can be activated. User should manually check if repaired thin pool kernel metadata has all data for all lvm2 known LVs by individual activation of every thin LV. When all works, user should continue with fsck of all filesystems present on these volumes. Once the thin pool is considered fully functional user may remove ThinPoolLV_metaN (the LV containing the damaged thin pool metadata) for possible space reuse. For a better performance it may be useful to pvmove the new repaired metadata LV (written to previous pmspare volume) to a faster PV, e.g. SSD.

If the repair operation fails, the thin pool LV and its thin LVs are not accessible and it may be necessary to restore their content from a backup. In such case the content of unmodified original damaged ThinPoolLV_metaN volume can be used by your support for more advanced recovery methods.

If metadata is manually restored with thin_repair directly, the pool metadata LV can be manually swapped with another LV containing new metadata:

lvconvert --thinpool VG/ThinPoolLV --poolmetadata VG/NewThinMetaLV

Note: Thin pool metadata is compact so even small corruptions in them may result in significant portions of mappings to be lost. It is recommended to use fast resilient storage for them.

### Activation of thin snapshots
When a thin snapshot LV is created, it is by default given the "activation skip" flag. This flag is indicated by the "k" attribute displayed by lvs:

```
# lvs vg/thin1s1

  LV         VG  Attr       LSize Pool  Origin

  thin1s1    vg  Vwi---tz-k 1.00t pool0 thin1
```

This flag causes the snapshot LV to be skipped, i.e. not activated, by normal activation commands. The skipping behavior does not apply to deactivation commands.

A snapshot LV with the "k" attribute can be activated using the -K (or --ignoreactivationskip) option in addition to the standard -ay (or --activate y) option.

Command to activate a thin snapshot LV:

lvchange -ay -K VG/SnapLV

The persistent "activation skip" flag can be turned off during lvcreate, or later with lvchange using the -kn (or --setactivationskip n) option. It can be turned on again with -ky (or --setactivationskip y).

When the "activation skip" flag is removed, normal activation commands will activate the LV, and the -K activation option is not needed.

Command to create snapshot LV without the activation skip flag:

lvcreate -kn -n SnapLV -s VG/ThinLV

Command to remove the activation skip flag from a snapshot LV:

lvchange -kn VG/SnapLV

lvm.conf(5) auto_set_activation_skip

controls the default activation skip setting used by lvcreate.

### Removing thin pool LVs, thin LVs and snapshots
Removing a thin LV and its related snapshots returns the blocks it used to the thin pool LV. These blocks will be reused for other thin LVs and snapshots.

Removing a thin pool LV removes both the data LV and metadata LV and returns the space to the VG.

lvremove of thin pool LVs, thin LVs and snapshots cannot be reversed with vgcfgrestore.

vgcfgbackup does not back up thin pool metadata.

### Manually manage free data space of thin pool LV
The available free space in a thin pool LV can be displayed with the lvs command. Free space can be added by extending the thin pool LV.

Command to extend thin pool data space:

lvextend -L Size VG/ThinPoolLV

Example

```
1. A thin pool LV is using 26.96% of its data blocks.
# lvs

  LV    VG           Attr       LSize   Pool  Origin Data%

  pool0 vg           twi-a-tz--  10.00g               26.96
```

```
2. Double the amount of physical space in the thin pool LV.
# lvextend -L+10G vg/pool0
```

```
3. The percentage of used data blocks is half the previous value.
# lvs

  LV    VG           Attr       LSize   Pool  Origin Data%

  pool0 vg           twi-a-tz--  20.00g               13.48
```

Other methods of increasing free data space in a thin pool LV include removing a thin LV and its related snapshots, or running fstrim on the file system using a thin LV.

### Manually manage free metadata space of a thin pool LV
The available metadata space in a thin pool LV can be displayed with the lvs -o+metadata_percent command.

Command to extend thin pool metadata space:

lvextend --poolmetadatasize Size VG/ThinPoolLV

Example

1. A thin pool LV is using 12.40% of its metadata blocks.

```
# lvs -oname,size,data_percent,metadata_percent vg/pool0

  LV    LSize   Data%  Meta%

  pool0  20.00g  13.48  12.40
```

2. Display a thin pool LV with its component thin data LV and thin metadata LV.

```
# lvs -a -oname,attr,size vg

  LV              Attr       LSize

  pool0           twi-a-tz--  20.00g

  [pool0_tdata]   Twi-ao----  20.00g

  [pool0_tmeta]   ewi-ao----  12.00m
```

3. Double the amount of physical space in the thin metadata LV.

```
# lvextend --poolmetadatasize +12M vg/pool0
```

4. The percentage of used metadata blocks is half the previous value.

```
# lvs -a -oname,size,data_percent,metadata_percent vg

  LV              LSize   Data%  Meta%

  pool0            20.00g  13.48   6.20

  [pool0_tdata]    20.00g

  [pool0_tmeta]    24.00m
```

### Using fstrim to increase free space in a thin pool LV
Removing files in a file system on top of a thin LV does not generally add free space back to the thin pool. Manually running the fstrim command can return space back to the thin pool that had been used by removed files. fstrim uses discards and will not work if the thin pool LV has discards mode set to ignore.

Example

A thin pool has 10G of physical data space, and a thin LV has a virtual size of 100G. Writing a 1G file to the file system reduces the free space in the thin pool by 10% and increases the virtual usage of the file system by 1%. Removing the 1G file restores the virtual 1% to the file system, but does not restore the physical 10% to the thin pool. The fstrim command restores the physical space to the thin pool.

```
# lvs -a -oname,attr,size,pool_lv,origin,data_percent,metadata_percent vg

  LV            Attr       LSize   Pool  Origin Data%  Meta%

  pool0         twi-a-tz--  10.00g              47.01  21.03

  thin1         Vwi-aotz-- 100.00g pool0         2.70
```

```
# df -h /mnt/X
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/vg-thin1   99G  1.1G   93G   2% /mnt/X
```

```
# dd if=/dev/zero of=/mnt/X/1Gfile bs=4096 count=262144; sync
```

```
# lvs

  pool0         vg   twi-a-tz--  10.00g         57.01  25.26

  thin1         vg   Vwi-aotz-- 100.00g pool0    3.70
```

```
# df -h /mnt/X
/dev/mapper/vg-thin1   99G  2.1G   92G   3% /mnt/X
```

```
# rm /mnt/X/1Gfile
```

```
# lvs

  pool0         vg   twi-a-tz--  10.00g         57.01  25.26

  thin1         vg   Vwi-aotz-- 100.00g pool0    3.70
```

```
# df -h /mnt/X
/dev/mapper/vg-thin1   99G  1.1G   93G   2% /mnt/X
```

```
# fstrim -v /mnt/X
```

```
# lvs

  pool0         vg   twi-a-tz--  10.00g         47.01  21.03

  thin1         vg   Vwi-aotz-- 100.00g pool0    2.70
```

The "Discard" section covers an option for automatically freeing data space in a thin pool.

### Automatically extend thin pool LV
The lvm daemon dmeventd (lvm2-monitor) monitors the data usage of thin pool LVs and extends them when the usage reaches a certain level. The necessary free space must exist in the VG to extend thin pool LVs. Monitoring and extension of thin pool LVs are controlled independently.

— Monitoring —

When a thin pool LV is activated, dmeventd will begin monitoring it by default.

Command to start or stop dmeventd monitoring a thin pool LV:

lvchange --monitor y|n VG/ThinPoolLV

The current dmeventd monitoring status of a thin pool LV can be displayed with the command lvs -o+seg_monitor.

— Autoextending —

dmeventd should be configured to extend thin pool LVs before all data space is used. Warnings are emitted through syslog when the use of a thin pool reaches 80%, 85%, 90% and 95%. (See the section "Data space exhaustion" for the effects of not extending a thin pool LV.) The point at which dmeventd extends thin pool LVs, and the amount are controlled with two configuration settings:

lvm.conf(5) thin_pool_autoextend_threshold

is a percentage full value that defines when the thin pool LV should be extended. Setting this to 100 disables automatic extension. The minimum value is 50.

lvm.conf(5) thin_pool_autoextend_percent

defines how much extra data space should be added to the thin pool LV from the VG, in percent of its current size.

— Disabling —

There are multiple ways that extension of thin pools could be prevented:

•

If the dmeventd daemon is not running, no monitoring or automatic extension will occur.

•

Even when dmeventd is running, all monitoring can be disabled with the lvm.conf monitoring setting.

•

To activate or create a thin pool LV without interacting with dmeventd, the --ignoremonitoring option can be used. With this option, the command will not ask dmeventd to monitor the thin pool LV.

•

Setting thin_pool_autoextend_threshold to 100 disables automatic extension of thin pool LVs, even if they are being monitored by dmeventd.

Example

If thin_pool_autoextend_threshold is 70 and thin_pool_autoextend_percent is 20, whenever a pool exceeds 70% usage, it will be extended by another 20%. For a 1G pool, using 700M will trigger a resize to 1.2G. When the usage exceeds 840M, the pool will be extended to 1.44G, and so on.

### Data space exhaustion
When properly managed, thin pool data space should be extended before it is all used (see the section "Automatically extend thin pool LV"). If thin pool data space is already exhausted, it can still be extended (see the section "Manually manage free data space of thin pool LV".)

The behavior of a full thin pool is configurable with the --errorwhenfull y|n option to lvcreate or lvchange. The errorwhenfull setting applies only to writes; reading thin LVs can continue even when data space is exhausted.

Command to change the handling of a full thin pool:

lvchange --errorwhenfull y|n VG/ThinPoolLV

lvm.conf(5) error_when_full

controls the default error when full behavior.

The current setting of a thin pool LV can be displayed with the command: lvs -o+lv_when_full.

The errorwhenfull setting does not effect the monitoring and autoextend settings, and the monitoring/autoextend settings do not effect the errorwhenfull setting. It is only when monitoring/autoextend are not effective that the thin pool becomes full and the errorwhenfull setting is applied.

— errorwhenfull n —

This is the default. Writes to thin LVs are accepted and queued, with the expectation that pool data space will be extended soon. Once data space is extended, the queued writes will be processed, and the thin pool will return to normal operation.

While waiting to be extended, the thin pool will queue writes for up to 60 seconds (the default). If data space has not been extended after this time, the queued writes will return an error to the caller, e.g. the file system. This can result in file system corruption for non-journaled file systems that may require repair. When a thin pool returns errors for writes to a thin LV, any file system is subject to losing unsynced user data.

The 60 second timeout can be changed or disabled with the dm-thin-pool kernel module option no_space_timeout. This option sets the number of seconds that thin pools will queue writes. If set to 0, writes will not time out. Disabling timeouts can result in the system running out of resources, memory exhaustion, hung tasks, and deadlocks. (The timeout applies to all thin pools on the system.)

— errorwhenfull y —

Writes to thin LVs immediately return an error, and no writes are queued. In the case of a file system, this can result in corruption that may require fs repair (the specific consequences depend on the thin LV user.)

— data percent —

When data space is exhausted, the lvs command displays 100 under Data% for the thin pool LV:

```
# lvs vg/pool0

  LV     VG           Attr       LSize   Pool  Origin Data%

  pool0  vg           twi-a-tz-- 512.00m              100.00
```

— causes —

A thin pool may run out of data space for any of the following reasons:

•

Automatic extension of the thin pool is disabled, and the thin pool is not manually extended. (Disabling automatic extension is not recommended.)

•

The dmeventd daemon is not running and the thin pool is not manually extended. (Disabling dmeventd is not recommended.)

•

Automatic extension of the thin pool is too slow given the rate of writes to thin LVs in the pool. (This can be addressed by tuning the thin_pool_autoextend_threshold and thin_pool_autoextend_percent. See "Automatic extend settings".)

•

The VG does not have enough free blocks to extend the thin pool.

### Metadata space exhaustion
If thin pool metadata space is exhausted (or a thin pool metadata operation fails), errors will be returned for IO operations on thin LVs.

When metadata space is exhausted, the lvs command displays 100 under Meta% for the thin pool LV:

```
# lvs -o lv_name,size,data_percent,metadata_percent vg/pool0

  LV    LSize Data%  Meta%

  pool0              100.00
```

The same reasons for thin pool data space exhaustion apply to thin pool metadata space.

Metadata space exhaustion can lead to inconsistent thin pool metadata and inconsistent file systems, so the response requires offline checking and repair.

1.

Deactivate the thin pool LV, or reboot the system if this is not possible.

2.

Repair thin pool with lvconvert --repair.

See "Metadata check and repair".

3.

Extend pool metadata space with lvextend --poolmetadatasize.

See "Manually manage free metadata space of a thin pool LV".

4.

Check and repair file system.

### Automatic extend settings
Thin pool LVs can be extended according to preset values. The presets determine if the LV should be extended based on how full it is, and if so by how much. When dmeventd monitors thin pool LVs, it uses lvextend with these presets. (See "Automatically extend thin pool LV".)

Command to extend a thin pool data LV using presets:

lvextend --use-policies VG/ThinPoolLV

The command uses these settings:

lvm.conf(5) thin_pool_autoextend_threshold

autoextend the LV when its usage exceeds this percent.

lvm.conf(5) thin_pool_autoextend_percent

autoextend the LV by this much additional space.

To see the default values of these settings, run:

lvmconfig --type default --withcomment

activation/thin_pool_autoextend_threshold

lvmconfig --type default --withcomment

activation/thin_pool_autoextend_percent

To change these values globally, edit lvm.conf(5).

To change these values on a per-VG or per-LV basis, attach a "profile" to the VG or LV. A profile is a collection of config settings, saved in a local text file (using the lvm.conf format). lvm looks for profiles in the profile_dir directory, e.g. /etc/lvm/profile/. Once attached to a VG or LV, lvm will process the VG or LV using the settings from the attached profile. A profile is named and referenced by its file name.

To use a profile to customize the lvextend settings for an LV:

•

Create a file containing settings, saved in profile_dir.

For the profile_dir location, run:

lvmconfig config/profile_dir

•

Attach the profile to an LV, using the command:

lvchange --metadataprofile ProfileName VG/ThinPoolLV

•

Extend the LV using the profile settings:

lvextend --use-policies VG/ThinPoolLV

Example

```
# lvmconfig config/profile_dir
profile_dir="/etc/lvm/profile"
```

```
# cat /etc/lvm/profile/pool0extend.profile
activation {
```

```
thin_pool_autoextend_threshold=50
thin_pool_autoextend_percent=10
```

```
}
```

```
# lvchange --metadataprofile pool0extend vg/pool0
```

```
# lvextend --use-policies vg/pool0
```

Notes

•

A profile is attached to a VG or LV by name, where the name references a local file in profile_dir. If the VG is moved to another machine, the file with the profile also needs to be moved.

•

Only certain settings can be used in a VG or LV profile, see:

lvmconfig --type profilable-metadata.

•

An LV without a profile of its own will inherit the VG profile.

•

Remove a profile from an LV using the command:

lvchange --detachprofile VG/ThinPoolLV.

•

Commands can also have profiles applied to them. The settings that can be applied to a command are different than the settings that can be applied to a VG or LV. See lvmconfig --type profilable-command. To apply a profile to a command, write a profile, save it in the profile directory, and run the command using the option: --commandprofile ProfileName.

### Zeroing
When a thin pool provisions a new data block for a thin LV, the new block is first overwritten with zeros. The zeroing mode is indicated by the "z" attribute displayed by lvs. The option -Z (or --zero) can be added to commands to specify the zeroing mode.

Command to set the zeroing mode when creating a thin pool LV:

lvconvert --type thin-pool -Z y|n

--poolmetadata VG/ThinMetaLV VG/ThinDataLV

Command to change the zeroing mode of an existing thin pool LV:

lvchange -Z y|n VG/ThinPoolLV

If zeroing mode is changed from "n" to "y", previously provisioned blocks are not zeroed.

Provisioning of large zeroed chunks impacts performance.

lvm.conf(5) thin_pool_zero

controls the default zeroing mode used when creating a thin pool.

### Discard
The discard behavior of a thin pool LV determines how discard requests are handled. Enabling discard under a file system may adversely affect the file system performance (see the section on fstrim for an alternative.) Possible discard behaviors:

ignore: Ignore any discards that are received.

nopassdown: Process any discards in the thin pool itself and allow the no longer needed extents to be overwritten by new data.

passdown: Process discards in the thin pool (as with nopassdown), and pass the discards down the the underlying device. This is the default mode.

Command to display the current discard mode of a thin pool LV:

lvs -o+discards VG/ThinPoolLV

Command to set the discard mode when creating a thin pool LV:

lvconvert --discards ignore|nopassdown|passdown

--type thin-pool --poolmetadata VG/ThinMetaLV VG/ThinDataLV

Command to change the discard mode of an existing thin pool LV:

lvchange --discards ignore|nopassdown|passdown VG/ThinPoolLV

Example

```
# lvs -o name,discards vg/pool0

  pool0 passdown
```

```
# lvchange --discards ignore vg/pool0
```

lvm.conf(5) thin_pool_discards

controls the default discards mode used when creating a thin pool.

### Chunk size
The size of data blocks managed by a thin pool can be specified with the --chunksize option when the thin pool LV is created. The default unit is KiB. The value must be a multiple of 64KiB between 64KiB and 1GiB.

When a thin pool is used primarily for the thin provisioning feature, a larger value is optimal. To optimize for many snapshots, a smaller value reduces copying time and consumes less space.

Command to display the thin pool LV chunk size:

lvs -o+chunksize VG/ThinPoolLV

Example

```
# lvs -o name,chunksize

  pool0 64.00k
```

lvm.conf(5) thin_pool_chunk_size

controls the default chunk size used when creating a thin pool.

The default value is shown by:

lvmconfig --type default allocation/thin_pool_chunk_size

### Size of pool metadata LV
The amount of thin metadata depends on how many blocks are shared between thin LVs (i.e. through snapshots). A thin pool with many snapshots may need a larger metadata LV. Thin pool metadata LV sizes can be from 2MiB to approximately 16GiB.

When using lvcreate to create what will become a thin metadata LV, the size is specified with the -L|--size option.

When an LVM command automatically creates a thin metadata LV, the size is specified with the --poolmetadatasize option. When this option is not given, LVM automatically chooses a size based on the data size and chunk size.

It can be hard to predict the amount of metadata space that will be needed, so it is recommended to start with a size of 1GiB which should be enough for all practical purposes. A thin pool metadata LV can later be manually or automatically extended if needed.

Configurable setting lvm.conf(5) allocation/thin_pool_crop_metadata gives control over cropping to 15.81GiB to stay backward compatible with older versions of lvm2. With enabled cropping there can be observed some problems when using volumes above this size with thin tools (i.e. thin_repair). Cropping should be enabled only when compatibility is required.

### Create a thin snapshot of an external, read only LV
Thin snapshots are typically taken of other thin LVs or other thin snapshot LVs within the same thin pool. It is also possible to take thin snapshots of external, read only LVs. Writes to the snapshot are stored in the thin pool, and the external LV is used to read unwritten parts of the thin snapshot.

lvcreate -n SnapLV -s VG/ExternalOriginLV --thinpool VG/ThinPoolLV

Example

```
# lvchange -an vg/lve
# lvchange --permission r vg/lve
# lvcreate -n snaplve -s vg/lve --thinpool vg/pool0
```

```
# lvs vg/lve vg/snaplve

  LV      VG  Attr       LSize  Pool  Origin Data%

  lve     vg  ori------- 10.00g

  snaplve vg  Vwi-a-tz-- 10.00g pool0 lve      0.00
```

### Convert a standard LV to a thin LV with an external origin
A new thin LV can be created and given the name of an existing standard LV. At the same time, the existing LV is converted to a read only external LV with a new name. Unwritten portions of the thin LV are read from the external LV. The new name given to the existing LV can be specified with --originname, otherwise the existing LV will be given a default name, e.g. lvol#.

Convert ExampleLV into a read only external LV with the new name NewExternalOriginLV, and create a new thin LV that is given the previous name of ExampleLV.

lvconvert --type thin --thinpool VG/ThinPoolLV

--originname NewExternalOriginLV VG/ExampleLV

Example

```
# lvcreate -n lv_example -L 10G vg
```

```
# lvs

  lv_example      vg          -wi-a-----  10.00g
```

```
# lvconvert --type thin --thinpool vg/pool0

          --originname lv_external --thin vg/lv_example
```

```
# lvs

  LV              VG          Attr       LSize   Pool  Origin

  lv_example      vg          Vwi-a-tz--  10.00g pool0 lv_external

  lv_external     vg          ori-------  10.00g
```

### Single step thin pool LV creation
A thin pool LV can be created with a single lvcreate command, rather than using lvconvert on existing LVs. This one command creates a thin data LV, a thin metadata LV, and combines the two into a thin pool LV.

lvcreate --type thin-pool -L LargeSize -n ThinPoolLV VG

Example

```
# lvcreate --type thin-pool -L8M -n pool0 vg
```

```
# lvs vg/pool0

  LV    VG  Attr       LSize Pool Origin Data%

  pool0 vg  twi-a-tz-- 8.00m               0.00
```

```
# lvs -a

  pool0           vg          twi-a-tz--   8.00m

  [pool0_tdata]   vg          Twi-ao----   8.00m

  [pool0_tmeta]   vg          ewi-ao----   8.00m
```

### Single step thin pool LV and thin LV creation
A thin pool LV and a thin LV can be created with a single lvcreate command. This one command creates a thin data LV, a thin metadata LV, combines the two into a thin pool LV, and creates a thin LV in the new pool.

-L LargeSize specifies the physical size of the thin pool LV.

-V VirtualSize specifies the virtual size of the thin LV.

lvcreate --type thin -V VirtualSize -L LargeSize

-n ThinLV --thinpool VG/ThinPoolLV

Equivalent to:

lvcreate --type thin-pool -L LargeSize VG/ThinPoolLV

lvcreate -n ThinLV -V VirtualSize --thinpool VG/ThinPoolLV

Example

```
# lvcreate -L8M -V2G -n thin1 --thinpool vg/pool0
```

```
# lvs -a

  pool0           vg          twi-a-tz--   8.00m

  [pool0_tdata]   vg          Twi-ao----   8.00m

  [pool0_tmeta]   vg          ewi-ao----   8.00m

  thin1           vg          Vwi-a-tz--   2.00g pool0
```

### Merge thin snapshots
A thin snapshot can be merged into its origin thin LV using the lvconvert --merge command. The result of a snapshot merge is that the origin thin LV takes the content of the snapshot LV, and the snapshot LV is removed. Any content that was unique to the origin thin LV is lost after the merge.

Because a merge changes the content of an LV, it cannot be done while the LVs are open, e.g. mounted. If a merge is initiated while the LVs are open, the effect of the merge is delayed until the origin thin LV is next activated.

lvconvert --merge VG/SnapLV

Example

```
# lvs vg

  LV      VG Attr       LSize   Pool  Origin

  pool0   vg twi-a-tz--  10.00g

  thin1   vg Vwi-a-tz-- 100.00g pool0

  thin1s1 vg Vwi-a-tz-k 100.00g pool0 thin1
```

```
# lvconvert --merge vg/thin1s1
```

```
# lvs vg

  LV      VG Attr       LSize   Pool  Origin

  pool0   vg twi-a-tz--  10.00g

  thin1   vg Vwi-a-tz-- 100.00g pool0
```

Example

```
Delayed merging of open LVs.
```

```
# lvs vg

  LV      VG Attr       LSize   Pool  Origin

  pool0   vg twi-a-tz--  10.00g

  thin1   vg Vwi-aotz-- 100.00g pool0

  thin1s1 vg Vwi-aotz-k 100.00g pool0 thin1
```

```
# df
/dev/mapper/vg-thin1            100G   33M  100G   1% /mnt/X
/dev/mapper/vg-thin1s1          100G   33M  100G   1% /mnt/Xs
```

```
# ls /mnt/X
file1 file2 file3
# ls /mnt/Xs
file3 file4 file5
```

```
# lvconvert --merge vg/thin1s1
Logical volume vg/thin1s1 contains a filesystem in use.
Delaying merge since snapshot is open.
Merging of thin snapshot thin1s1 will occur on next activation.
```

```
# umount /mnt/X
# umount /mnt/Xs
```

```
# lvs -a vg

  LV              VG   Attr       LSize   Pool  Origin

  pool0           vg   twi-a-tz--  10.00g

  [pool0_tdata]   vg   Twi-ao----  10.00g

  [pool0_tmeta]   vg   ewi-ao----   1.00g

  thin1           vg   Owi-a-tz-- 100.00g pool0

  [thin1s1]       vg   Swi-a-tz-k 100.00g pool0 thin1
```

```
# lvchange -an vg/thin1
# lvchange -ay vg/thin1
```

```
# mount /dev/vg/thin1 /mnt/X
```

```
# ls /mnt/X
file3 file4 file5
```

### XFS on snapshots
Mounting an XFS file system on a new snapshot LV requires attention to the file system's log state and uuid. On the snapshot LV, the xfs log will contain a dummy transaction, and the xfs uuid will match the uuid from the file system on the origin LV.

If the snapshot LV is writable, mounting will recover the log to clear the dummy transaction, but will require skipping the uuid check:

\# mount /dev/VG/SnapLV /mnt -o nouuid

After the first mount with the above approach, the UUID can subsequently be changed using:

\# xfs_admin -U generate /dev/VG/SnapLV

\# mount /dev/VG/SnapLV /mnt

Once the UUID has been changed, the mount command will no longer require the nouuid option.

If the snapshot LV is readonly, the log recovery and uuid check need to be skipped while mounting readonly:

\# mount /dev/VG/SnapLV /mnt -o ro,nouuid,norecovery

## SEE ALSO
lvm(8), lvm.conf(5), lvmconfig(8), lvcreate(8), lvconvert(8), lvchange(8), lvextend(8), lvremove(8), lvs(8),

thin_dump(8), thin_repair(8), thin_restore(8)


lvmvdo

title: lvmvdo(7)

text:
# lvmvdo(7)

## NAME
lvmvdo — Support for Virtual Data Optimizer in LVM

## DESCRIPTION
VDO is software that provides inline block-level deduplication, compression, and thin provisioning capabilities for primary storage.

Deduplication is a technique for reducing the consumption of storage resources by eliminating multiple copies of duplicate blocks. Compression takes the individual unique blocks and shrinks them. These reduced blocks are then efficiently packed together into physical blocks. Thin provisioning manages the mapping from logical blocks presented by VDO to where the data has actually been physically stored, and also eliminates any blocks of all zeroes.

With deduplication, instead of writing the same data more than once, VDO detects and records each duplicate block as a reference to the original block. VDO maintains a mapping from Logical Block Addresses (LBA) (used by the storage layer above VDO) to physical block addresses (used by the storage layer under VDO). After deduplication, multiple logical block addresses may be mapped to the same physical block address; these are called shared blocks and are reference-counted by the software.

With compression, VDO compresses multiple blocks (or shared blocks) with the fast LZ4 algorithm, and bins them together where possible so that multiple compressed blocks fit within a 4 KB block on the underlying storage. Mapping from LBA is to a physical block address and index within it for the desired compressed data. All compressed blocks are individually reference counted for correctness.

Block sharing and block compression are invisible to applications using the storage, which read and write blocks as they would if VDO were not present. When a shared block is overwritten, a new physical block is allocated for storing the new block data to ensure that other logical block addresses that are mapped to the shared physical block are not modified.

To use VDO with lvm(8), you must install the standard VDO user-space tools vdoformat(8) and the currently non-standard kernel VDO module "kvdo".

The "kvdo" module implements fine-grained storage virtualization, thin provisioning, block sharing, and compression. The "uds" module provides memory-efficient duplicate identification. The user-space tools include vdostats(8) for extracting statistics from VDO volumes.

## VDO TERMS
VDODataLV

VDO data LV

A large hidden LV with the _vdata suffix. It is created in a VG

used by the VDO kernel target to store all data and metadata blocks.

VDOPoolLV

VDO pool LV

A pool for virtual VDOLV(s), which are the size of used VDODataLV.

Only a single VDOLV is currently supported.

VDOLV

VDO LV

Created from VDOPoolLV.

Appears blank after creation.

## VDO USAGE
The primary methods for using VDO with lvm2:

### 1. Create a VDOPoolLV and a VDOLV
Create a VDOPoolLV that will hold VDO data, and a virtual size VDOLV that the user can use. If you do not specify the virtual size, then the VDOLV is created with the maximum size that always fits into data volume even if no deduplication or compression can happen (i.e. it can hold the incompressible content of /dev/urandom). If you do not specify the name of VDOPoolLV, it is taken from the sequence of vpool0, vpool1 ...

Note: The performance of TRIM/Discard operations is slow for large volumes of VDO type. Please try to avoid sending discard requests unless necessary because it might take considerable amount of time to finish the discard operation.

```
lvcreate --type vdo -n VDOLV -L DataSize -V LargeVirtualSize VG/VDOPoolLV
lvcreate --vdo -L DataSize VG
```

Example

```
# lvcreate --type vdo -n vdo0 -L 10G -V 100G vg/vdopool0
# mkfs.ext4 -E nodiscard /dev/vg/vdo0
```

### 2. Convert an existing LV into VDOPoolLV
Convert an already created or existing LV into a VDOPoolLV, which is a volume that can hold data and metadata. You will be prompted to confirm such conversion because it IRREVERSIBLY DESTROYS the content of such volume and the volume is immediately formatted by vdoformat(8) as a VDO pool data volume. You can specify the virtual size of the VDOLV associated with this VDOPoolLV. If you do not specify the virtual size, it will be set to the maximum size that can keep 100% incompressible data there.

```
lvconvert --type vdo-pool -n VDOLV -V VirtualSize VG/VDOPoolLV
lvconvert --vdopool VG/VDOPoolLV
```

Example

```
# lvconvert --type vdo-pool -n vdo0 -V10G vg/ExistingLV
```

### 3. Change the compression and deduplication of a VDOPoolLV
Disable or enable the compression and deduplication for VDOPoolLV (the volume that maintains all VDO LV(s) associated with it).

lvchange --compression y|n --deduplication y|n VG/VDOPoolLV

Example

```
# lvchange --compression n  vg/vdopool0
# lvchange --deduplication y vg/vdopool1
```

### 4. Change the default settings used for creating a VDOPoolLV
VDO allows to set a large variety of options. Lots of these settings can be specified in lvm.conf or profile settings. You can prepare a number of different profiles in the /etc/lvm/profile directory and just specify the profile file name. Check the output of lvmconfig --type default --withcomments for a detailed description of all individual VDO settings.

Example

```
# cat <<EOF > /etc/lvm/profile/vdo_create.profile
allocation {
```

```
vdo_use_compression=1
vdo_use_deduplication=1
vdo_use_metadata_hints=1
vdo_minimum_io_size=4096
vdo_block_map_cache_size_mb=128
vdo_block_map_period=16380
vdo_check_point_frequency=0
vdo_use_sparse_index=0
vdo_index_memory_size_mb=256
vdo_slab_size_mb=2048
vdo_ack_threads=1
vdo_bio_threads=1
vdo_bio_rotation=64
vdo_cpu_threads=2
vdo_hash_zone_threads=1
vdo_logical_threads=1
vdo_physical_threads=1
vdo_write_policy="auto"
vdo_max_discard=1
```

```
}
EOF
```

```
# lvcreate --vdo -L10G --metadataprofile vdo_create vg/vdopool0
# lvcreate --vdo -L10G --config 'allocation/vdo_cpu_threads=4' vg/vdopool1
```

### 5. Set or change VDO settings with option --vdosettings
Use the form 'option=value' or 'option1=value option2=value', or repeat --vdosettings for each option being set. Options are listed in the Example section above, for the full description see lvm.conf(5). Options can omit 'vdo_' and 'vdo_use_' prefixes and all its underscores. So i.e. vdo_use_metadata_hints=1 and metadatahints=1 are equivalent. To change the option for an already existing VDOPoolLV use lvchange(8) command. However not all option can be changed. Only compression and deduplication options can be also changed for an active VDO LV. Lowest priority options are specified with configuration file, then with --vdosettings and highest are expliction option --compression and --deduplication.

Example

```
# lvcreate --vdo -L10G --vdosettings 'ack_threads=1 hash_zone_threads=2' vg/vdopool0
# lvchange --vdosettings 'bio_threads=2 deduplication=1' vg/vdopool0
```

### 6. Checking the usage of VDOPoolLV
To quickly check how much data on a VDOPoolLV is already consumed, use lvs(8). The Data% field reports how much data is occupied in the content of the virtual data for the VDOLV and how much space is already consumed with all the data and metadata blocks in the VDOPoolLV. For a detailed description, use the vdostats(8) command.

Note: vdostats(8) currently understands only /dev/mapper device names.

Example

```
# lvcreate --type vdo -L10G -V20G -n vdo0 vg/vdopool0
# mkfs.ext4 -E nodiscard /dev/vg/vdo0
# lvs -a vg
```

```
  LV               VG Attr       LSize  Pool     Origin Data%

  vdo0             vg vwi-a-v--- 20.00g vdopool0        0.01

  vdopool0         vg dwi-ao---- 10.00g                 30.16

  [vdopool0_vdata] vg Dwi-ao---- 10.00g
```

```
# vdostats --all /dev/mapper/vg-vdopool0-vpool
/dev/mapper/vg-vdopool0 :

  version                             : 30

  release version                     : 133524

  data blocks used                    : 79

  ...
```

### 7. Extending the VDOPoolLV size
You can add more space to hold VDO data and metadata by extending the VDODataLV using the commands lvresize(8) and lvextend(8). The extension needs to add at least one new VDO slab. You can configure the slab size with the allocation/vdo_slab_size_mb setting.

You can also enable automatic size extension of a monitored VDOPoolLV with the activation/vdo_pool_autoextend_percent and activation/vdo_pool_autoextend_threshold settings.

Note: You cannot reduce the size of a VDOPoolLV.

lvextend -L+AddingSize VG/VDOPoolLV

Example

```
# lvextend -L+50G vg/vdopool0
# lvresize -L300G vg/vdopool1
```

### 8. Extending or reducing the VDOLV size
You can extend or reduce a virtual VDO LV as a standard LV with the lvresize(8), lvextend(8), and lvreduce(8) commands.

Note: The reduction needs to process TRIM for reduced disk area to unmap used data blocks from the VDOPoolLV, which might take a long time.

lvextend -L+AddingSize VG/VDOLV

lvreduce -L-ReducingSize VG/VDOLV

Example

```
# lvextend -L+50G vg/vdo0
# lvreduce -L-50G vg/vdo1
# lvresize -L200G vg/vdo2
```

### 9. Component activation of a VDODataLV
You can activate a VDODataLV separately as a component LV for examination purposes. The activation of the VDODataLV activates the data LV in read-only mode, and the data LV cannot be modified. If the VDODataLV is active as a component, any upper LV using this volume CANNOT be activated. You have to deactivate the VDODataLV first to continue to use the VDOPoolLV.

Example

```
# lvchange -ay vg/vpool0_vdata
# lvchange -an vg/vpool0_vdata
```

## VDO TOPICS

### 1. Stacking VDO
You can convert or stack a VDOPooLV with these currently supported volume types: linear, stripe, raid, and cache with cachepool.

### 2. VDOPoolLV on top of raid
Using a raid type LV for a VDODataLV.

Example

```
# lvcreate --type raid1 -L 5G -n vdopool vg
# lvconvert --type vdo-pool -V 10G vg/vdopool
```

### 3. Caching a VDOPoolLV
VDOPoolLV (accepts also VDODataLV volume name) caching provides a mechanism to accelerate reads and writes of already compressed and deduplicated data blocks together with VDO metadata.

Example

```
# lvcreate --type vdo -L 5G -V 10G -n vdo1 vg/vdopool
# lvcreate --type cache-pool -L 1G -n cachepool vg
# lvconvert --cache --cachepool vg/cachepool vg/vdopool
# lvconvert --uncache vg/vdopool
```

### 4. Caching a VDOLV
VDO LV cache allow you to 'cache' a device for better performance before it hits the processing of the VDO Pool LV layer.

Example

```
# lvcreate --type vdo -L 5G -V 10G -n vdo1 vg/vdopool
# lvcreate --type cache-pool -L 1G -n cachepool vg
# lvconvert --cache --cachepool vg/cachepool vg/vdo1
# lvconvert --uncache vg/vdo1
```

### 5. Usage of Discard/TRIM with a VDOLV
You can discard data on a VDO LV and reduce used blocks on a VDOPoolLV. However, the current performance of discard operations is still not optimal and takes a considerable amount of time and CPU. Unless you really need it, you should avoid using discard.

When a block device is going to be rewritten, its blocks will be automatically reused for new data. Discard is useful in situations when user knows that the given portion of a VDO LV is not going to be used and the discarded space can be used for block provisioning in other regions of the VDO LV. For the same reason, you should avoid using mkfs with discard for a freshly created VDO LV to save a lot of time that this operation would take otherwise as device is already expected to be empty.

### 6. Memory usage
The VDO target requires 370 MiB of RAM plus an additional 268 MiB per each 1 TiB of physical storage managed by the volume.

UDS requires a minimum of 250 MiB of RAM, which is also the default amount that deduplication uses.

The memory required for the UDS index is determined by the index type and the required size of the deduplication window and is controlled by the allocation/vdo_use_sparse_index setting.

With enabled UDS sparse indexing, it relies on the temporal locality of data and attempts to retain only the most relevant index entries in memory and can maintain a deduplication window that is ten times larger than with dense while using the same amount of memory.

Although the sparse index provides the greatest coverage, the dense index provides more deduplication advice. For most workloads, given the same amount of memory, the difference in deduplication rates between dense and sparse indexes is negligible.

A dense index with 1 GiB of RAM maintains a 1 TiB deduplication window, while a sparse index with 1 GiB of RAM maintains a 10 TiB deduplication window. In general, 1 GiB is sufficient for 4 TiB of physical space with a dense index and 40 TiB with a sparse index.

### 7. Storage space requirements
You can configure a VDOPoolLV to use up to 256 TiB of physical storage. Only a certain part of the physical storage is usable to store data. This section provides the calculations to determine the usable size of a VDO-managed volume.

The VDO target requires storage for two types of VDO metadata and for the UDS index:

- The first type of VDO metadata uses approximately 1 MiB for each 4 GiB of physical storage plus an additional 1 MiB per slab.

- The second type of VDO metadata consumes approximately 1.25 MiB for each 1 GiB of logical storage, rounded up to the nearest slab.

- The amount of storage required for the UDS index depends on the type of index and the amount of RAM allocated to the index. For each 1 GiB of RAM, a dense UDS index uses 17 GiB of storage and a sparse UDS index will use 170 GiB of storage.

## SEE ALSO
lvm(8), lvm.conf(5), lvmconfig(8), lvcreate(8), lvconvert(8), lvchange(8), lvextend(8), lvreduce(8), lvresize(8), lvremove(8), lvs(8),

vdo(8), vdoformat(8), vdostats(8),

mkfs(8)


lxc

package: lxc-core

title: lxc(7)

summary: linux containers

text:
# lxc(7)

## NAME
lxc - linux containers

## OVERVIEW
The container technology is actively being pushed into the mainstream Linux kernel. It provides resource management through control groups and resource isolation via namespaces.

lxc, aims to use these new functionalities to provide a userspace container object which provides full resource isolation and resource control for an applications or a full system.

lxc is small enough to easily manage a container with simple command lines and complete enough to be used for other purposes.

## REQUIREMENTS
The kernel version >= 3.10 shipped with the distros, will work with lxc, this one will have less functionalities but enough to be interesting.

lxc relies on a set of functionalities provided by the kernel. The helper script lxc-checkconfig will give you information about your kernel configuration, required, and missing features.

## FUNCTIONAL SPECIFICATION
A container is an object isolating some resources of the host, for the application or system running in it.

The application / system will be launched inside a container specified by a configuration that is either initially created or passed as a parameter of the commands.

How to run an application in a container

Before running an application, you should know what are the resources you want to isolate. The default configuration is to isolate PIDs, the sysv IPC and mount points. If you want to run a simple shell inside a container, a basic configuration is needed, especially if you want to share the rootfs. If you want to run an application like sshd, you should provide a new network stack and a new hostname. If you want to avoid conflicts with some files eg. /var/run/httpd.pid, you should remount /var/run with an empty directory. If you want to avoid the conflicts in all the cases, you can specify a rootfs for the container. The rootfs can be a directory tree, previously bind mounted with the initial rootfs, so you can still use your distro but with your own /etc and /home

Here is an example of directory tree for sshd:

```

[root@lxc sshd]$ tree -d rootfs

rootfs
|-- bin
|-- dev
|   |-- pts
|   `-- shm
|       `-- network
|-- etc
|   `-- ssh
|-- lib
|-- proc
|-- root
|-- sbin
|-- sys
|-- usr
`-- var

    |-- empty

    |   `-- sshd

    |-- lib

    |   `-- empty

    |       `-- sshd

    `-- run

        `-- sshd

```

and the mount points file associated with it:

```
	[root@lxc sshd]$ cat fstab
	/lib /home/root/sshd/rootfs/lib none ro,bind 0 0
	/bin /home/root/sshd/rootfs/bin none ro,bind 0 0
	/usr /home/root/sshd/rootfs/usr none ro,bind 0 0
	/sbin /home/root/sshd/rootfs/sbin none ro,bind 0 0

```

How to run a system in a container

Running a system inside a container is paradoxically easier than running an application. Why? Because you don't have to care about the resources to be isolated, everything needs to be isolated, the other resources are specified as being isolated but without configuration because the container will set them up. eg. the ipv4 address will be setup by the system container init scripts. Here is an example of the mount points file:

```
	[root@lxc debian]$ cat fstab
	/dev	/home/root/debian/rootfs/dev none bind 0 0
	/dev/pts /home/root/debian/rootfs/dev/pts  none bind 0 0

```

### CONTAINER LIFE CYCLE
When the container is created, it contains the configuration information. When a process is launched, the container will be starting and running. When the last process running inside the container exits, the container is stopped.

In case of failure when the container is initialized, it will pass through the aborting state.

```
   ---------

  | STOPPED |<---------------

   ---------                 |

       |                     |

     start                   |

       |                     |

       V                     |

   ----------                |

  | STARTING |--error-       |

   ----------         |      |

       |              |      |

       V              V      |

   ---------    ----------   |

  | RUNNING |  | ABORTING |  |

   ---------    ----------   |

       |              |      |

  no process          |      |

       |              |      |

       V              |      |

   ----------         |      |

  | STOPPING |<-------       |

   ----------                |

       |                     |

        ---------------------

```

### CONFIGURATION
The container is configured through a configuration file, the format of the configuration file is described in lxc.conf(5)

### CREATING / DESTROYING CONTAINERS
A persistent container object can be created via the lxc-create command. It takes a container name as parameter and optional configuration file and template. The name is used by the different commands to refer to this container. The lxc-destroy command will destroy the container object.

```
	  lxc-create -n foo
	  lxc-destroy -n foo

```

### VOLATILE CONTAINER
It is not mandatory to create a container object before starting it. The container can be directly started with a configuration file as parameter.

### STARTING / STOPPING CONTAINER
When the container has been created, it is ready to run an application / system. This is the purpose of the lxc-execute and lxc-start commands. If the container was not created before starting the application, the container will use the configuration file passed as parameter to the command, and if there is no such parameter either, then it will use a default isolation. If the application ended, the container will be stopped, but if needed the lxc-stop command can be used to stop the container.

Running an application inside a container is not exactly the same thing as running a system. For this reason, there are two different commands to run an application into a container:

```
	  lxc-execute -n foo [-f config] /bin/bash
	  lxc-start -n foo [-f config] [/bin/bash]

```

The lxc-execute command will run the specified command into a container via an intermediate process, lxc-init. This lxc-init after launching the specified command, will wait for its end and all other reparented processes. (to support daemons in the container). In other words, in the container, lxc-init has PID 1 and the first process of the application has PID 2.

The lxc-start command will directly run the specified command in the container. The PID of the first process is 1. If no command is specified lxc-start will run the command defined in lxc.init.cmd or if not set, /sbin/init .

To summarize, lxc-execute is for running an application and lxc-start is better suited for running a system.

If the application is no longer responding, is inaccessible or is not able to finish by itself, a wild lxc-stop command will kill all the processes in the container without pity.

```
	  lxc-stop -n foo -k

```

### CONNECT TO AN AVAILABLE TTY
If the container is configured with ttys, it is possible to access it through them. It is up to the container to provide a set of available ttys to be used by the following command. When the tty is lost, it is possible to reconnect to it without login again.

```
	  lxc-console -n foo -t 3

```

### FREEZE / UNFREEZE CONTAINER
Sometime, it is useful to stop all the processes belonging to a container, eg. for job scheduling. The commands:

```
	  lxc-freeze -n foo

```

will put all the processes in an uninteruptible state and

```
	  lxc-unfreeze -n foo

```

will resume them.

This feature is enabled if the freezer cgroup v1 controller is enabled in the kernel.

### GETTING INFORMATION ABOUT CONTAINER
When there are a lot of containers, it is hard to follow what has been created or destroyed, what is running or what are the PIDs running in a specific container. For this reason, the following commands may be useful:

```
	  lxc-ls -f
	  lxc-info -n foo

```

lxc-ls lists containers.

lxc-info gives information for a specific container.

Here is an example on how the combination of these commands allows one to list all the containers and retrieve their state.

```
	  for i in $(lxc-ls -1); do
	    lxc-info -n $i
	  done

```

### MONITORING CONTAINER
It is sometime useful to track the states of a container, for example to monitor it or just to wait for a specific state in a script.

lxc-monitor command will monitor one or several containers. The parameter of this command accepts a regular expression for example:

```
	  lxc-monitor -n "foo|bar"

```

will monitor the states of containers named 'foo' and 'bar', and:

```
	  lxc-monitor -n ".*"

```

will monitor all the containers.

For a container 'foo' starting, doing some work and exiting, the output will be in the form:

```
	  'foo' changed state to [STARTING]
	  'foo' changed state to [RUNNING]
	  'foo' changed state to [STOPPING]
	  'foo' changed state to [STOPPED]

```

lxc-wait command will wait for a specific state change and exit. This is useful for scripting to synchronize the launch of a container or the end. The parameter is an ORed combination of different states. The following example shows how to wait for a container if it successfully started as a daemon.

```
	  # launch lxc-wait in background
	  lxc-wait -n foo -s STOPPED &
	  LXC_WAIT_PID=$!
	  # this command goes in background
	  lxc-execute -n foo mydaemon &
	  # block until the lxc-wait exits
	  # and lxc-wait exits when the container
	  # is STOPPED
	  wait $LXC_WAIT_PID
	  echo "'foo' is finished"

```

### CGROUP SETTINGS FOR CONTAINERS
The container is tied with the control groups, when a container is started a control group is created and associated with it. The control group properties can be read and modified when the container is running by using the lxc-cgroup command.

lxc-cgroup command is used to set or get a control group subsystem which is associated with a container. The subsystem name is handled by the user, the command won't do any syntax checking on the subsystem name, if the subsystem name does not exists, the command will fail.

```
	  lxc-cgroup -n foo cpuset.cpus

```

will display the content of this subsystem.

```
	  lxc-cgroup -n foo cpu.shares 512

```

will set the subsystem to the specified value.

## SEE ALSO
lxc(7), lxc-create(1), lxc-copy(1), lxc-destroy(1), lxc-start(1), lxc-stop(1), lxc-execute(1), lxc-console(1), lxc-monitor(1), lxc-wait(1), lxc-cgroup(1), lxc-ls(1), lxc-info(1), lxc-freeze(1), lxc-unfreeze(1), lxc-attach(1), lxc.conf(5)

## AUTHOR
Daniel Lezcano <daniel.lezcano@free.fr>

Christian Brauner <christian.brauner@ubuntu.com>

Serge Hallyn <serge@hallyn.com>

Stéphane Graber <stgraber@ubuntu.com>


package: man-pages

title: address_families(7)

summary: socket address families (domains)

text:
# address_families(7)

## NAME
address_families - socket address families (domains)

## SYNOPSIS

```
#include <sys/types.h>          /* See NOTES */
#include <sys/socket.h>
```

```
int socket(int domain, int type, int protocol);
```

## DESCRIPTION
The domain argument of the socket(2) specifies a communication domain; this selects the protocol family which will be used for communication. These families are defined in <sys/socket.h>. The formats currently understood by the Linux kernel include:

AF_UNIX, AF_LOCAL

Local communication For further information, see unix(7).

AF_INET

IPv4 Internet protocols. For further information, see ip(7).

AF_AX25

Amateur radio AX.25 protocol. For further information, see ax25(4).

AF_IPX

IPX - Novell protocols.

AF_APPLETALK

AppleTalk For further information, see ddp(7).

AF_NETROM

AX.25 packet layer protocol. For further information, see netrom(4), The Packet Radio Protocols and Linux and the AX.25, NET/ROM, and ROSE network programming chapters of the Linux Amateur Radio AX.25 HOWTO.

AF_BRIDGE

Can't be used for creating sockets; mostly used for bridge links in rtnetlink(7) protocol commands.

AF_ATMPVC

Access to raw ATM Permanent Virtual Circuits (PVCs). For further information, see the ATM on Linux HOWTO.

AF_X25

ITU-T X.25 / ISO-8208 protocol. For further information, see x25(7).

AF_INET6

IPv6 Internet protocols. For further information, see ipv6(7).

AF_ROSE

RATS (Radio Amateur Telecommunications Society) Open Systems environment (ROSE) AX.25 packet layer protocol. For further information, see the resources listed for AF_NETROM.

AF_DECnet

DECet protocol sockets. See Documentation/networking/decnet.txt in the Linux kernel source tree for details.

AF_NETBEUI

Reserved for "802.2LLC project"; never used.

AF_SECURITY

This was a short-lived (between Linux 2.1.30 and 2.1.99pre2) protocol family for firewall upcalls.

AF_KEY

Key management protocol, originally developed for usage with IPsec (since Linux 2.1.38). This has no relation to keyctl(2) and the in-kernel key storage facility. See RFC 2367 PF_KEY Key Management API, Version 2 for details.

AF_NETLINK

Kernel user interface device For further information, see netlink(7).

AF_PACKET

Low-level packet interface. For further information, see packet(7).

AF_ECONET

Acorn Econet protocol (removed in Linux 3.5). See the Econet documentation for details.

AF_ATMSVC

Access to ATM Switched Virtual Circuits (SVCs) See the ATM on Linux HOWTO for details.

AF_RDS

Reliable Datagram Sockets (RDS) protocol (since Linux 2.6.30). RDS over RDMA has no relation to AF_SMC or AF_XDP. For further information see rds(7), rds-rdma(7), and Documentation/networking/rds.txt in the Linux kernel source tree.

AF_IRDA

Socket interface over IrDA (moved to staging in Linux 4.14, removed in Linux 4.17). For further information see irda(7).

AF_PPPOX

Generic PPP transport layer, for setting up L2 tunnels (L2TP and PPPoE). See Documentation/networking/l2tp.txt in the Linux kernel source tree for details.

AF_WANPIPE

Legacy protocol for wide area network (WAN) connectivity that was used by Sangoma WAN cards (called "WANPIPE"); removed in Linux 2.6.21.

AF_LLC

Logical link control (IEEE 802.2 LLC) protocol, upper part of data link layer of ISO/OSI networking protocol stack (since Linux 2.4); has no relation to AF_PACKET. See chapter 13.5.3. Logical Link Control in Understanding Linux Kernel Internals (O'Reilly Media, 2006) and IEEE Standards for Local Area Networks: Logical Link Control (The Institute of Electronics and Electronics Engineers, Inc., New York, New York, 1985) for details. See also some historical notes regarding its development.

AF_IB

InfiniBand native addressing (since Linux 3.11).

AF_MPLS

Multiprotocol Label Switching (since Linux 4.1); mostly used for configuring MPLS routing via netlink(7), as it doesn't expose ability to create sockets to user space.

AF_CAN

Controller Area Network automotive bus protocol (since Linux 2.6.25). See Documentation/networking/can.rst in the Linux kernel source tree for details.

AF_TIPC

TIPC, "cluster domain sockets" protocol (since Linux 2.6.16). See TIPC Programmer's Guide and the protocol description for details.

AF_BLUETOOTH

Bluetooth low-level socket protocol (since Linux 3.11). See Bluetooth Management API overview and An Introduction to Bluetooth Programming by Albert Huang for details.

AF_IUCV

IUCV (inter-user communication vehicle) z/VM protocol for hypervisor-guest interaction (since Linux 2.6.21); has no relation to AF_VSOCK and/or AF_SMC See IUCV protocol overview for details.

AF_RXRPC

Rx, Andrew File System remote procedure call protocol (since Linux 2.6.22). See Documentation/networking/rxrpc.txt in the Linux kernel source tree for details.

AF_ISDN

New "modular ISDN" driver interface protocol (since Linux 2.6.27). See the mISDN wiki for details.

AF_PHONET

Nokia cellular modem IPC/RPC interface (since Linux 2.6.31). See Documentation/networking/phonet.txt in the Linux kernel source tree for details.

AF_IEEE802154

IEEE 802.15.4 WPAN (wireless personal area network) raw packet protocol (since Linux 2.6.31). See Documentation/networking/ieee802154.txt in the Linux kernel source tree for details.

AF_CAIF

Ericsson's Communication CPU to Application CPU interface (CAIF) protocol (since Linux 2.6.36). See Documentation/networking/caif/Linux-CAIF.txt in the Linux kernel source tree for details.

AF_ALG

Interface to kernel crypto API (since Linux 2.6.38). See Documentation/crypto/userspace-if.rst in the Linux kernel source tree for details.

AF_VSOCK

VMWare VSockets protocol for hypervisor-guest interaction (since Linux 3.9); has no relation to AF_IUCV and AF_SMC. For further information, see vsock(7).

AF_KCM

KCM (kernel connection multiplexer) interface (since Linux 4.6). See Documentation/networking/kcm.txt in the Linux kernel source tree for details.

AF_QIPCRTR

Qualcomm IPC router interface protocol (since Linux 4.7).

AF_SMC

SMC-R (shared memory communications over RDMA) protocol (since Linux 4.11), and SMC-D (shared memory communications, direct memory access) protocol for intra-node z/VM quest interaction (since Linux 4.19); has no relation to AF_RDS, AF_IUCV or AF_VSOCK. See RFC 7609 IBM's Shared Memory Communications over RDMA (SMC-R) Protocol for details regarding SMC-R. See SMC-D Reference Information for details regarding SMC-D.

AF_XDP

XDP (express data path) interface (since Linux 4.18). See Documentation/networking/af_xdp.rst in the Linux kernel source tree for details.

## SEE ALSO
socket(2), socket(7)


aio

title: aio(7)

summary: POSIX asynchronous I/O overview

text:
# aio(7)

## NAME
aio - POSIX asynchronous I/O overview

## DESCRIPTION
The POSIX asynchronous I/O (AIO) interface allows applications to initiate one or more I/O operations that are performed asynchronously (i.e., in the background). The application can elect to be notified of completion of the I/O operation in a variety of ways: by delivery of a signal, by instantiation of a thread, or no notification at all.

The POSIX AIO interface consists of the following functions:

aio_read(3)

Enqueue a read request. This is the asynchronous analog of read(2).

aio_write(3)

Enqueue a write request. This is the asynchronous analog of write(2).

aio_fsync(3)

Enqueue a sync request for the I/O operations on a file descriptor. This is the asynchronous analog of fsync(2) and fdatasync(2).

aio_error(3)

Obtain the error status of an enqueued I/O request.

aio_return(3)

Obtain the return status of a completed I/O request.

aio_suspend(3)

Suspend the caller until one or more of a specified set of I/O requests completes.

aio_cancel(3)

Attempt to cancel outstanding I/O requests on a specified file descriptor.

lio_listio(3)

Enqueue multiple I/O requests using a single function call.

The aiocb ("asynchronous I/O control block") structure defines parameters that control an I/O operation. An argument of this type is employed with all of the functions listed above. This structure has the following form:

```
#include <aiocb.h>
struct aiocb {

    /* The order of these fields is implementation-dependent */

    int             aio_fildes;     /* File descriptor */

    off_t           aio_offset;     /* File offset */

    volatile void  *aio_buf;        /* Location of buffer */

    size_t          aio_nbytes;     /* Length of transfer */

    int             aio_reqprio;    /* Request priority */

    struct sigevent aio_sigevent;   /* Notification method */

    int             aio_lio_opcode; /* Operation to be performed;

                                       lio_listio() only */

    /* Various implementation-internal fields not shown */
};
/* Operation codes for 'aio_lio_opcode': */
enum { LIO_READ, LIO_WRITE, LIO_NOP };
```

The fields of this structure are as follows:

aio_fildes

The file descriptor on which the I/O operation is to be performed.

aio_offset

This is the file offset at which the I/O operation is to be performed.

aio_buf

This is the buffer used to transfer data for a read or write operation.

aio_nbytes

This is the size of the buffer pointed to by aio_buf.

aio_reqprio

This field specifies a value that is subtracted from the calling thread's real-time priority in order to determine the priority for execution of this I/O request (see pthread_setschedparam(3)). The specified value must be between 0 and the value returned by sysconf(_SC_AIO_PRIO_DELTA_MAX). This field is ignored for file synchronization operations.

aio_sigevent

This field is a structure that specifies how the caller is to be notified when the asynchronous I/O operation completes. Possible values for aio_sigevent.sigev_notify are SIGEV_NONE, SIGEV_SIGNAL, and SIGEV_THREAD. See sigevent(7) for further details.

aio_lio_opcode

The type of operation to be performed; used only for lio_listio(3).

In addition to the standard functions listed above, the GNU C library provides the following extension to the POSIX AIO API:

aio_init(3)

Set parameters for tuning the behavior of the glibc POSIX AIO implementation.

## ERRORS
EINVAL

The aio_reqprio field of the aiocb structure was less than 0, or was greater than the limit returned by the call sysconf(_SC_AIO_PRIO_DELTA_MAX).

## VERSIONS
The POSIX AIO interfaces are provided by glibc since version 2.1.

## CONFORMING TO
POSIX.1-2001, POSIX.1-2008.

## NOTES
It is a good idea to zero out the control block buffer before use (see memset(3)). The control block buffer and the buffer pointed to by aio_buf must not be changed while the I/O operation is in progress. These buffers must remain valid until the I/O operation completes.

Simultaneous asynchronous read or write operations using the same aiocb structure yield undefined results.

The current Linux POSIX AIO implementation is provided in user space by glibc. This has a number of limitations, most notably that maintaining multiple threads to perform I/O operations is expensive and scales poorly. Work has been in progress for some time on a kernel state-machine-based implementation of asynchronous I/O (see io_submit(2), io_setup(2), io_cancel(2), io_destroy(2), io_getevents(2)), but this implementation hasn't yet matured to the point where the POSIX AIO implementation can be completely reimplemented using the kernel system calls.

## EXAMPLES
The program below opens each of the files named in its command-line arguments and queues a request on the resulting file descriptor using aio_read(3). The program then loops, periodically monitoring each of the I/O operations that is still in progress using aio_error(3). Each of the I/O requests is set up to provide notification by delivery of a signal. After all I/O requests have completed, the program retrieves their status using aio_return(3).

The SIGQUIT signal (generated by typing control-\) causes the program to request cancellation of each of the outstanding requests using aio_cancel(3).

Here is an example of what we might see when running this program. In this example, the program queues two requests to standard input, and these are satisfied by two lines of input containing "abc" and "x".

```
$ ./a.out /dev/stdin /dev/stdin
opened /dev/stdin on descriptor 3
opened /dev/stdin on descriptor 4
aio_error():

    for request 0 (descriptor 3): In progress

    for request 1 (descriptor 4): In progress
abc
I/O completion signal received
aio_error():

    for request 0 (descriptor 3): I/O succeeded

    for request 1 (descriptor 4): In progress
aio_error():

    for request 1 (descriptor 4): In progress
x
I/O completion signal received
aio_error():

    for request 1 (descriptor 4): I/O succeeded
All I/O requests completed
aio_return():

    for request 0 (descriptor 3): 4

    for request 1 (descriptor 4): 2
```

### Program source

```
#include <fcntl.h>
#include <stdlib.h>
#include <unistd.h>
#include <stdio.h>
#include <errno.h>
#include <aio.h>
#include <signal.h>
#define BUF_SIZE 20     /* Size of buffers for read operations */
#define errExit(msg) do { perror(msg); exit(EXIT_FAILURE); } while (0)
struct ioRequest {      /* Application-defined structure for tracking

                           I/O requests */

    int           reqNum;

    int           status;

    struct aiocb *aiocbp;
};
static volatile sig_atomic_t gotSIGQUIT = 0;

                        /* On delivery of SIGQUIT, we attempt to

                           cancel all outstanding I/O requests */
static void             /* Handler for SIGQUIT */
quitHandler(int sig)
{

    gotSIGQUIT = 1;
}
#define IO_SIGNAL SIGUSR1   /* Signal used to notify I/O completion */
static void                 /* Handler for I/O completion signal */
aioSigHandler(int sig, siginfo_t *si, void *ucontext)
{

    if (si->si_code == SI_ASYNCIO) {

        write(STDOUT_FILENO, "I/O completion signal received\n", 31);

        /* The corresponding ioRequest structure would be available as

               struct ioRequest *ioReq = si->si_value.sival_ptr;

           and the file descriptor would then be available via

               ioReq->aiocbp->aio_fildes */

    }
}
int
main(int argc, char *argv[])
{

    struct sigaction sa;

    int s;

    int numReqs;        /* Total number of queued I/O requests */

    int openReqs;       /* Number of I/O requests still in progress */

    if (argc < 2) {

        fprintf(stderr, "Usage: %s <pathname> <pathname>...\n",

                argv[0]);

        exit(EXIT_FAILURE);

    }

    numReqs = argc - 1;

    /* Allocate our arrays. */

    struct ioRequest *ioList = calloc(numReqs, sizeof(*ioList));

    if (ioList == NULL)

        errExit("calloc");

    struct aiocb *aiocbList = calloc(numReqs, sizeof(*aiocbList));

    if (aiocbList == NULL)

        errExit("calloc");

    /* Establish handlers for SIGQUIT and the I/O completion signal. */

    sa.sa_flags = SA_RESTART;

    sigemptyset(&sa.sa_mask);

    sa.sa_handler = quitHandler;

    if (sigaction(SIGQUIT, &sa, NULL) == -1)

        errExit("sigaction");

    sa.sa_flags = SA_RESTART | SA_SIGINFO;

    sa.sa_sigaction = aioSigHandler;

    if (sigaction(IO_SIGNAL, &sa, NULL) == -1)

        errExit("sigaction");

    /* Open each file specified on the command line, and queue

       a read request on the resulting file descriptor. */

    for (int j = 0; j < numReqs; j++) {

        ioList[j].reqNum = j;

        ioList[j].status = EINPROGRESS;

        ioList[j].aiocbp = &aiocbList[j];

        ioList[j].aiocbp->aio_fildes = open(argv[j + 1], O_RDONLY);

        if (ioList[j].aiocbp->aio_fildes == -1)

            errExit("open");

        printf("opened %s on descriptor %d\n", argv[j + 1],

                ioList[j].aiocbp->aio_fildes);

        ioList[j].aiocbp->aio_buf = malloc(BUF_SIZE);

        if (ioList[j].aiocbp->aio_buf == NULL)

            errExit("malloc");

        ioList[j].aiocbp->aio_nbytes = BUF_SIZE;

        ioList[j].aiocbp->aio_reqprio = 0;

        ioList[j].aiocbp->aio_offset = 0;

        ioList[j].aiocbp->aio_sigevent.sigev_notify = SIGEV_SIGNAL;

        ioList[j].aiocbp->aio_sigevent.sigev_signo = IO_SIGNAL;

        ioList[j].aiocbp->aio_sigevent.sigev_value.sival_ptr =

                                &ioList[j];

        s = aio_read(ioList[j].aiocbp);

        if (s == -1)

            errExit("aio_read");

    }

    openReqs = numReqs;

    /* Loop, monitoring status of I/O requests. */

    while (openReqs > 0) {

        sleep(3);       /* Delay between each monitoring step */

        if (gotSIGQUIT) {

            /* On receipt of SIGQUIT, attempt to cancel each of the

               outstanding I/O requests, and display status returned

               from the cancellation requests. */

            printf("got SIGQUIT; canceling I/O requests: \n");

            for (int j = 0; j < numReqs; j++) {

                if (ioList[j].status == EINPROGRESS) {

                    printf("    Request %d on descriptor %d:", j,

                            ioList[j].aiocbp->aio_fildes);

                    s = aio_cancel(ioList[j].aiocbp->aio_fildes,

                            ioList[j].aiocbp);

                    if (s == AIO_CANCELED)

                        printf("I/O canceled\n");

                    else if (s == AIO_NOTCANCELED)

                        printf("I/O not canceled\n");

                    else if (s == AIO_ALLDONE)

                        printf("I/O all done\n");

                    else

                        perror("aio_cancel");

                }

            }

            gotSIGQUIT = 0;

        }

        /* Check the status of each I/O request that is still

           in progress. */

        printf("aio_error():\n");

        for (int j = 0; j < numReqs; j++) {

            if (ioList[j].status == EINPROGRESS) {

                printf("    for request %d (descriptor %d): ",

                        j, ioList[j].aiocbp->aio_fildes);

                ioList[j].status = aio_error(ioList[j].aiocbp);

                switch (ioList[j].status) {

                case 0:

                    printf("I/O succeeded\n");

                    break;

                case EINPROGRESS:

                    printf("In progress\n");

                    break;

                case ECANCELED:

                    printf("Canceled\n");

                    break;

                default:

                    perror("aio_error");

                    break;

                }

                if (ioList[j].status != EINPROGRESS)

                    openReqs--;

            }

        }

    }

    printf("All I/O requests completed\n");

    /* Check status return of all I/O requests. */

    printf("aio_return():\n");

    for (int j = 0; j < numReqs; j++) {

        ssize_t s;

        s = aio_return(ioList[j].aiocbp);

        printf("    for request %d (descriptor %d): %zd\n",

                j, ioList[j].aiocbp->aio_fildes, s);

    }

    exit(EXIT_SUCCESS);
}
```

## SEE ALSO
io_cancel(2), io_destroy(2), io_getevents(2), io_setup(2), io_submit(2), aio_cancel(3), aio_error(3), aio_init(3), aio_read(3), aio_return(3), aio_write(3), lio_listio(3)

"Asynchronous I/O Support in Linux 2.5", Bhattacharya, Pratt, Pulavarty, and Morgan, Proceedings of the Linux Symposium, 2003, https://www.kernel.org/doc/ols/2003/ols2003-pages-351-366.pdf


title: armscii-8(7)

text:
# armscii-8(7)

## NAME
armscii-8 - Armenian character set encoded in octal, decimal, and hexadecimal

## DESCRIPTION
The Armenian Standard Code for Information Interchange, 8-bit coded character set.

### ArmSCII-8 characters
The following table displays the characters in ArmSCII-8 that are printable and unlisted in the ascii(7) manual page.

Oct

Dec

Hex

Char

Description

240

160

A0

NO-BREAK SPACE

242

162

A2

և

ARMENIAN SMALL LIGATURE ECH YIWN

243

163

A3

։

ARMENIAN FULL STOP

244

164

A4

)

RIGHT PARENTHESIS

245

165

A5

(

LEFT PARENTHESIS

246

166

A6

»

RIGHT-POINTING DOUBLE ANGLE QUOTATION MARK

247

167

A7

«

LEFT-POINTING DOUBLE ANGLE QUOTATION MARK

250

168

A8

—

EM DASH

251

169

A9

.

FULL STOP

252

170

AA

՝

ARMENIAN COMMA

253

171

AB

,

COMMA

254

172

AC

-

HYPHEN-MINUS

255

173

AD

֊

ARMENIAN HYPHEN

256

174

AE

…

HORIZONTAL ELLIPSIS

257

175

AF

՜

ARMENIAN EXCLAMATION MARK

260

176

B0

՛

ARMENIAN EMPHASIS MARK

261

177

B1

՞

ARMENIAN QUESTION MARK

262

178

B2

Ա

ARMENIAN CAPITAL LETTER AYB

263

179

B3

ա

ARMENIAN SMALL LETTER AYB

264

180

B4

Բ

ARMENIAN CAPITAL LETTER BEN

265

181

B5

բ

ARMENIAN SMALL LETTER BEN

266

182

B6

Գ

ARMENIAN CAPITAL LETTER GIM

267

183

B7

գ

ARMENIAN SMALL LETTER GIM

270

184

B8

Դ

ARMENIAN CAPITAL LETTER DA

271

185

B9

դ

ARMENIAN SMALL LETTER DA

272

186

BA

Ե

ARMENIAN CAPITAL LETTER ECH

273

187

BB

ե

ARMENIAN SMALL LETTER ECH

274

188

BC

Զ

ARMENIAN CAPITAL LETTER ZA

275

189

BD

զ

ARMENIAN SMALL LETTER ZA

276

190

BE

Է

ARMENIAN CAPITAL LETTER EH

277

191

BF

է

ARMENIAN SMALL LETTER EH

300

192

C0

Ը

ARMENIAN CAPITAL LETTER ET

301

193

C1

ը

ARMENIAN SMALL LETTER ET

302

194

C2

Թ

ARMENIAN CAPITAL LETTER TO

303

195

C3

թ

ARMENIAN SMALL LETTER TO

304

196

C4

Ժ

ARMENIAN CAPITAL LETTER ZHE

305

197

C5

ժ

ARMENIAN SMALL LETTER ZHE

306

198

C6

Ի

ARMENIAN CAPITAL LETTER INI

307

199

C7

ի

ARMENIAN SMALL LETTER INI

310

200

C8

Լ

ARMENIAN CAPITAL LETTER LIWN

311

201

C9

լ

ARMENIAN SMALL LETTER LIWN

312

202

CA

Խ

ARMENIAN CAPITAL LETTER XEH

313

203

CB

խ

ARMENIAN SMALL LETTER XEH

314

204

CC

Ծ

ARMENIAN CAPITAL LETTER CA

315

205

CD

ծ

ARMENIAN SMALL LETTER CA

316

206

CE

Կ

ARMENIAN CAPITAL LETTER KEN

317

207

CF

կ

ARMENIAN SMALL LETTER KEN

320

208

D0

Հ

ARMENIAN CAPITAL LETTER HO

321

209

D1

հ

ARMENIAN SMALL LETTER HO

322

210

D2

Ձ

ARMENIAN CAPITAL LETTER JA

323

211

D3

ձ

ARMENIAN SMALL LETTER JA

324

212

D4

Ղ

ARMENIAN CAPITAL LETTER GHAD

325

213

D5

ղ

ARMENIAN SMALL LETTER GHAD

326

214

D6

Ճ

ARMENIAN CAPITAL LETTER CHEH

327

215

D7

ճ

ARMENIAN SMALL LETTER CHEH

330

216

D8

Մ

ARMENIAN CAPITAL LETTER MEN

331

217

D9

մ

ARMENIAN SMALL LETTER MEN

332

218

DA

Յ

ARMENIAN CAPITAL LETTER YI

333

219

DB

յ

ARMENIAN SMALL LETTER YI

334

220

DC

Ն

ARMENIAN CAPITAL LETTER NOW

335

221

DD

ն

ARMENIAN SMALL LETTER NOW

336

222

DE

Շ

ARMENIAN CAPITAL LETTER SHA

337

223

DF

շ

ARMENIAN SMALL LETTER SHA

340

224

E0

Ո

ARMENIAN CAPITAL LETTER VO

341

225

E1

ո

ARMENIAN SMALL LETTER VO

342

226

E2

Չ

ARMENIAN CAPITAL LETTER CHA

343

227

E3

չ

ARMENIAN SMALL LETTER CHA

344

228

E4

Պ

ARMENIAN CAPITAL LETTER PEH

345

229

E5

պ

ARMENIAN SMALL LETTER PEH

346

230

E6

Ջ

ARMENIAN CAPITAL LETTER JHEH

347

231

E7

ջ

ARMENIAN SMALL LETTER JHEH

350

232

E8

Ռ

ARMENIAN CAPITAL LETTER RA

351

233

E9

ռ

ARMENIAN SMALL LETTER RA

352

234

EA

Ս

ARMENIAN CAPITAL LETTER SEH

353

235

EB

ս

ARMENIAN SMALL LETTER SEH

354

236

EC

Վ

ARMENIAN CAPITAL LETTER VEW

355

237

ED

վ

ARMENIAN SMALL LETTER VEW

356

238

EE

Տ

ARMENIAN CAPITAL LETTER TIWN

357

239

EF

տ

ARMENIAN SMALL LETTER TIWN

360

240

F0

Ր

ARMENIAN CAPITAL LETTER REH

361

241

F1

ր

ARMENIAN SMALL LETTER REH

362

242

F2

Ց

ARMENIAN CAPITAL LETTER CO

363

243

F3

ց

ARMENIAN SMALL LETTER CO

364

244

F4

Ւ

ARMENIAN CAPITAL LETTER YIWN

365

245

F5

ւ

ARMENIAN SMALL LETTER YIWN

366

246

F6

Փ

ARMENIAN CAPITAL LETTER PIWR

367

247

F7

փ

ARMENIAN SMALL LETTER PIWR

370

248

F8

Ք

ARMENIAN CAPITAL LETTER KEH

371

249

F9

ք

ARMENIAN SMALL LETTER KEH

372

250

FA

Օ

ARMENIAN CAPITAL LETTER OH

373

251

FB

օ

ARMENIAN SMALL LETTER OH

374

252

FC

Ֆ

ARMENIAN CAPITAL LETTER FEH

375

253

FD

ֆ

ARMENIAN SMALL LETTER FEH

376

254

FE

՚

ARMENIAN APOSTROPHE

## SEE ALSO
ascii(7), charsets(7), utf-8(7)


arp

title: arp(7)

summary: Linux ARP kernel module.

text:
# arp(7)

## NAME
arp - Linux ARP kernel module.

## DESCRIPTION
This kernel protocol module implements the Address Resolution Protocol defined in RFC 826. It is used to convert between Layer2 hardware addresses and IPv4 protocol addresses on directly connected networks. The user normally doesn't interact directly with this module except to configure it; instead it provides a service for other protocols in the kernel.

A user process can receive ARP packets by using packet(7) sockets. There is also a mechanism for managing the ARP cache in user-space by using netlink(7) sockets. The ARP table can also be controlled via ioctl(2) on any AF_INET socket.

The ARP module maintains a cache of mappings between hardware addresses and protocol addresses. The cache has a limited size so old and less frequently used entries are garbage-collected. Entries which are marked as permanent are never deleted by the garbage-collector. The cache can be directly manipulated by the use of ioctls and its behavior can be tuned by the /proc interfaces described below.

When there is no positive feedback for an existing mapping after some time (see the /proc interfaces below), a neighbor cache entry is considered stale. Positive feedback can be gotten from a higher layer; for example from a successful TCP ACK. Other protocols can signal forward progress using the MSG_CONFIRM flag to sendmsg(2). When there is no forward progress, ARP tries to reprobe. It first tries to ask a local arp daemon app_solicit times for an updated MAC address. If that fails and an old MAC address is known, a unicast probe is sent ucast_solicit times. If that fails too, it will broadcast a new ARP request to the network. Requests are sent only when there is data queued for sending.

Linux will automatically add a nonpermanent proxy arp entry when it receives a request for an address it forwards to and proxy arp is enabled on the receiving interface. When there is a reject route for the target, no proxy arp entry is added.

### Ioctls
Three ioctls are available on all AF_INET sockets. They take a pointer to a struct arpreq as their argument.

```
struct arpreq {

    struct sockaddr arp_pa;      /* protocol address */

    struct sockaddr arp_ha;      /* hardware address */

    int             arp_flags;   /* flags */

    struct sockaddr arp_netmask; /* netmask of protocol address */

    char            arp_dev[16];
};
```

SIOCSARP, SIOCDARP and SIOCGARP respectively set, delete, and get an ARP mapping. Setting and deleting ARP maps are privileged operations and may be performed only by a process with the CAP_NET_ADMIN capability or an effective UID of 0.

arp_pa must be an AF_INET address and arp_ha must have the same type as the device which is specified in arp_dev. arp_dev is a zero-terminated string which names a device.

arp_flags

flag

meaning

ATF_COM

Lookup complete

ATF_PERM

Permanent entry

ATF_PUBL

Publish entry

ATF_USETRAILERS

Trailers requested

ATF_NETMASK

Use a netmask

ATF_DONTPUB

Don't answer

If the ATF_NETMASK flag is set, then arp_netmask should be valid. Linux 2.2 does not support proxy network ARP entries, so this should be set to 0xffffffff, or 0 to remove an existing proxy arp entry. ATF_USETRAILERS is obsolete and should not be used.

### /proc interfaces
ARP supports a range of /proc interfaces to configure parameters on a global or per-interface basis. The interfaces can be accessed by reading or writing the /proc/sys/net/ipv4/neigh/*/* files. Each interface in the system has its own directory in /proc/sys/net/ipv4/neigh/. The setting in the "default" directory is used for all newly created devices. Unless otherwise specified, time-related interfaces are specified in seconds.

anycast_delay (since Linux 2.2)

The maximum number of jiffies to delay before replying to a IPv6 neighbor solicitation message. Anycast support is not yet implemented. Defaults to 1 second.

app_solicit (since Linux 2.2)

The maximum number of probes to send to the user space ARP daemon via netlink before dropping back to multicast probes (see mcast_solicit). Defaults to 0.

base_reachable_time (since Linux 2.2)

Once a neighbor has been found, the entry is considered to be valid for at least a random value between base_reachable_time/2 and 3*base_reachable_time/2. An entry's validity will be extended if it receives positive feedback from higher level protocols. Defaults to 30 seconds. This file is now obsolete in favor of base_reachable_time_ms.

base_reachable_time_ms (since Linux 2.6.12)

As for base_reachable_time, but measures time in milliseconds. Defaults to 30000 milliseconds.

delay_first_probe_time (since Linux 2.2)

Delay before first probe after it has been decided that a neighbor is stale. Defaults to 5 seconds.

gc_interval (since Linux 2.2)

How frequently the garbage collector for neighbor entries should attempt to run. Defaults to 30 seconds.

gc_stale_time (since Linux 2.2)

Determines how often to check for stale neighbor entries. When a neighbor entry is considered stale, it is resolved again before sending data to it. Defaults to 60 seconds.

gc_thresh1 (since Linux 2.2)

The minimum number of entries to keep in the ARP cache. The garbage collector will not run if there are fewer than this number of entries in the cache. Defaults to 128.

gc_thresh2 (since Linux 2.2)

The soft maximum number of entries to keep in the ARP cache. The garbage collector will allow the number of entries to exceed this for 5 seconds before collection will be performed. Defaults to 512.

gc_thresh3 (since Linux 2.2)

The hard maximum number of entries to keep in the ARP cache. The garbage collector will always run if there are more than this number of entries in the cache. Defaults to 1024.

locktime (since Linux 2.2)

The minimum number of jiffies to keep an ARP entry in the cache. This prevents ARP cache thrashing if there is more than one potential mapping (generally due to network misconfiguration). Defaults to 1 second.

mcast_solicit (since Linux 2.2)

The maximum number of attempts to resolve an address by multicast/broadcast before marking the entry as unreachable. Defaults to 3.

proxy_delay (since Linux 2.2)

When an ARP request for a known proxy-ARP address is received, delay up to proxy_delay jiffies before replying. This is used to prevent network flooding in some cases. Defaults to 0.8 seconds.

proxy_qlen (since Linux 2.2)

The maximum number of packets which may be queued to proxy-ARP addresses. Defaults to 64.

retrans_time (since Linux 2.2)

The number of jiffies to delay before retransmitting a request. Defaults to 1 second. This file is now obsolete in favor of retrans_time_ms.

retrans_time_ms (since Linux 2.6.12)

The number of milliseconds to delay before retransmitting a request. Defaults to 1000 milliseconds.

ucast_solicit (since Linux 2.2)

The maximum number of attempts to send unicast probes before asking the ARP daemon (see app_solicit). Defaults to 3.

unres_qlen (since Linux 2.2)

The maximum number of packets which may be queued for each unresolved address by other network layers. Defaults to 3.

## VERSIONS
The struct arpreq changed in Linux 2.0 to include the arp_dev member and the ioctl numbers changed at the same time. Support for the old ioctls was dropped in Linux 2.2.

Support for proxy arp entries for networks (netmask not equal 0xffffffff) was dropped in Linux 2.2. It is replaced by automatic proxy arp setup by the kernel for all reachable hosts on other interfaces (when forwarding and proxy arp is enabled for the interface).

The neigh/* interfaces did not exist before Linux 2.2.

## BUGS
Some timer settings are specified in jiffies, which is architecture- and kernel version-dependent; see time(7).

There is no way to signal positive feedback from user space. This means connection-oriented protocols implemented in user space will generate excessive ARP traffic, because ndisc will regularly reprobe the MAC address. The same problem applies for some kernel protocols (e.g., NFS over UDP).

This man page mashes together functionality that is IPv4-specific with functionality that is shared between IPv4 and IPv6.

## SEE ALSO
capabilities(7), ip(7), arpd(8)

RFC 826 for a description of ARP. RFC 2461 for a description of IPv6 neighbor discovery and the base algorithms used. Linux 2.2+ IPv4 ARP uses the IPv6 algorithms when applicable.


ascii

title: ascii(7)

text:
# ascii(7)

## NAME
ascii - ASCII character set encoded in octal, decimal, and hexadecimal

## DESCRIPTION
ASCII is the American Standard Code for Information Interchange. It is a 7-bit code. Many 8-bit codes (e.g., ISO 8859-1) contain ASCII as their lower half. The international counterpart of ASCII is known as ISO 646-IRV.

The following table contains the 128 ASCII characters.

C program '\X' escapes are noted.

Oct

Dec

Hex

Char

Oct

Dec

Hex

Char

000

0

00

NUL '\0' (null character)

100

64

40

@

001

1

01

SOH (start of heading)

101

65

41

A

002

2

02

STX (start of text)

102

66

42

B

003

3

03

ETX (end of text)

103

67

43

C

004

4

04

EOT (end of transmission)

104

68

44

D

005

5

05

ENQ (enquiry)

105

69

45

E

006

6

06

ACK (acknowledge)

106

70

46

F

007

7

07

BEL '\a' (bell)

107

71

47

G

010

8

08

BS '\b' (backspace)

110

72

48

H

011

9

09

HT '\t' (horizontal tab)

111

73

49

I

012

10

0A

LF '\n' (new line)

112

74

4A

J

013

11

0B

VT '\v' (vertical tab)

113

75

4B

K

014

12

0C

FF '\f' (form feed)

114

76

4C

L

015

13

0D

CR '\r' (carriage ret)

115

77

4D

M

016

14

0E

SO (shift out)

116

78

4E

N

017

15

0F

SI (shift in)

117

79

4F

O

020

16

10

DLE (data link escape)

120

80

50

P

021

17

11

DC1 (device control 1)

121

81

51

Q

022

18

12

DC2 (device control 2)

122

82

52

R

023

19

13

DC3 (device control 3)

123

83

53

S

024

20

14

DC4 (device control 4)

124

84

54

T

025

21

15

NAK (negative ack.)

125

85

55

U

026

22

16

SYN (synchronous idle)

126

86

56

V

027

23

17

ETB (end of trans. blk)

127

87

57

W

030

24

18

CAN (cancel)

130

88

58

X

031

25

19

EM (end of medium)

131

89

59

Y

032

26

1A

SUB (substitute)

132

90

5A

Z

033

27

1B

ESC (escape)

133

91

5B

[

034

28

1C

FS (file separator)

134

92

5C

\ '\\'

035

29

1D

GS (group separator)

135

93

5D

]

036

30

1E

RS (record separator)

136

94

5E

^

037

31

1F

US (unit separator)

137

95

5F

_

040

32

20

SPACE

140

96

60

`

041

33

21

!

141

97

61

a

042

34

22

"

142

98

62

b

043

35

23

\#

143

99

63

c

044

36

24

$

144

100

64

d

045

37

25

%

145

101

65

e

046

38

26

&

146

102

66

f

047

39

27

'

147

103

67

g

050

40

28

(

150

104

68

h

051

41

29

)

151

105

69

i

052

42

2A

*

152

106

6A

j

053

43

2B

+

153

107

6B

k

054

44

2C

,

154

108

6C

l

055

45

2D

-

155

109

6D

m

056

46

2E

.

156

110

6E

n

057

47

2F

/

157

111

6F

o

060

48

30

0

160

112

70

p

061

49

31

1

161

113

71

q

062

50

32

2

162

114

72

r

063

51

33

3

163

115

73

s

064

52

34

4

164

116

74

t

065

53

35

5

165

117

75

u

066

54

36

6

166

118

76

v

067

55

37

7

167

119

77

w

070

56

38

8

170

120

78

x

071

57

39

9

171

121

79

y

072

58

3A

:

172

122

7A

z

073

59

3B

;

173

123

7B

{

074

60

3C

<

174

124

7C

|

075

61

3D

=

175

125

7D

}

076

62

3E

>

176

126

7E

~

077

63

3F

?

177

127

7F

DEL

### Tables
For convenience, below are more compact tables in hex and decimal.

```
   2 3 4 5 6 7       30 40 50 60 70 80 90 100 110 120

 -------------      ---------------------------------
0:   0 @ P ` p     0:    (  2  <  F  P  Z  d   n   x
1: ! 1 A Q a q     1:    )  3  =  G  Q  [  e   o   y
2: " 2 B R b r     2:    *  4  >  H  R  \  f   p   z
3: # 3 C S c s     3: !  +  5  ?  I  S  ]  g   q   {
4: $ 4 D T d t     4: "  ,  6  @  J  T  ^  h   r   |
5: % 5 E U e u     5: #  -  7  A  K  U  _  i   s   }
6: & 6 F V f v     6: $  .  8  B  L  V  `  j   t   ~
7: ' 7 G W g w     7: %  /  9  C  M  W  a  k   u  DEL
8: ( 8 H X h x     8: &  0  :  D  N  X  b  l   v
9: ) 9 I Y i y     9: '  1  ;  E  O  Y  c  m   w
A: * : J Z j z
B: + ; K [ k {
C: , < L \ l |
D: - = M ] m }
E: . > N ^ n ~
F: / ? O _ o DEL
```

## NOTES

### History
An ascii manual page appeared in Version 7 of AT&T UNIX.

On older terminals, the underscore code is displayed as a left arrow, called backarrow, the caret is displayed as an up-arrow and the vertical bar has a hole in the middle.

Uppercase and lowercase characters differ by just one bit and the ASCII character 2 differs from the double quote by just one bit, too. That made it much easier to encode characters mechanically or with a non-microcontroller-based electronic keyboard and that pairing was found on old teletypes.

The ASCII standard was published by the United States of America Standards Institute (USASI) in 1968.

## SEE ALSO
charsets(7), iso_8859-1(7), iso_8859-2(7), iso_8859-3(7), iso_8859-4(7), iso_8859-5(7), iso_8859-6(7), iso_8859-7(7), iso_8859-8(7), iso_8859-9(7), iso_8859-10(7), iso_8859-11(7), iso_8859-13(7), iso_8859-14(7), iso_8859-15(7), iso_8859-16(7), utf-8(7)