mount.ceph

title: mount.ceph(8)

summary: mount a Ceph file system

text:
# mount.ceph(8)

## NAME
mount.ceph - mount a Ceph file system

## SYNOPSIS

```
mount.ceph name*@*fsid.*fs_name*=/[subdir] dir [-o options ]
```

## DESCRIPTION
mount.ceph is a helper for mounting the Ceph file system on a Linux host. It serves to resolve monitor hostname(s) into IP addresses and read authentication keys from disk; the Linux kernel client component does most of the real work. To mount a Ceph file system use:

```
mount.ceph name@07fe3187-00d9-42a3-814b-72a4d5e7d5be.fs_name=/ /mnt/mycephfs -o mon_addr=1.2.3.4
```

where "name" is the RADOS client name (referred to hereafter as "RADOS user", and meaning any individual or system actor such as an application).

Mount helper can fill in the cluster FSID by reading the ceph configuration file. Its recommended to call the mount helper via mount(8) as per:

```
mount -t ceph name@.fs_name=/ /mnt/mycephfs -o mon_addr=1.2.3.4
```

Note that the dot . still needs to be a part of the device string in this case.

The first argument is the device part of the mount command. It includes the RADOS user for authentication, the file system name and a path within CephFS that will be mounted at the mount point.

Monitor addresses can be passed using mon_addr mount option. Multiple monitor addresses can be passed by separating addresses with a slash (/). Only one monitor is needed to mount successfully; the client will learn about all monitors from any responsive monitor. However, it is a good idea to specify more than one in case the one happens to be down at the time of mount. Monitor addresses takes the form ip_address[:port]. If the port is not specified, the Ceph default of 6789 is assumed.

If monitor addresses are not specified, then mount.ceph will attempt to determine monitor addresses using local configuration files and/or DNS SRV records. In similar way, if authentication is enabled on Ceph cluster (which is done using CephX) and options secret and secretfile are not specified in the command, the mount helper will spawn a child process that will use the standard Ceph library routines to find a keyring and fetch the secret from it (including the monitor address and FSID if those not specified).

A sub-directory of the file system can be mounted by specifying the (absolute) path to the sub-directory right after "=" in the device part of the mount command.

Mount helper application conventions dictate that the first two options are device to be mounted and the mountpoint for that device. Options must be passed only after these fixed arguments.

## OPTIONS

### Basic
conf

Path to a ceph.conf file. This is used to initialize the Ceph context for autodiscovery of monitor addresses and auth secrets. The default is to use the standard search path for ceph.conf files.

mount_timeout

int (seconds), Default: 60

ms_mode=<legacy|crc|secure|prefer-crc|prefer-secure>

Set the connection mode that the client uses for transport. The available modes are:

- legacy: use messenger v1 protocol to talk to the cluster

- crc: use messenger v2, without on-the-wire encryption

- secure: use messenger v2, with on-the-wire encryption

- prefer-crc: crc mode, if denied agree to secure mode

- prefer-secure: secure mode, if denied agree to crc mode

mon_addr

Monitor address of the cluster in the form of ip_address[:port]

fsid

Cluster FSID. This can be found using ceph fsid command.

secret

secret key for use with CephX. This option is insecure because it exposes the secret on the command line. To avoid this, use the secretfile option.

secretfile

path to file containing the secret key to use with CephX

recover_session=<no|clean>

Set auto reconnect mode in the case where the client is blocklisted. The available modes are no and clean. The default is no.

- no: never attempt to reconnect when client detects that it has been blocklisted. Blocklisted clients will not attempt to reconnect and their operations will fail too.

- clean: client reconnects to the Ceph cluster automatically when it detects that it has been blocklisted. During reconnect, client drops dirty data/metadata, invalidates page caches and writable file handles. After reconnect, file locks become stale because the MDS loses track of them. If an inode contains any stale file locks, read/write on the inode is not allowed until applications release all stale file locks.

command

fs=<fs-name> Specify the non-default file system to be mounted, when using the old syntax.

command

mds_namespace=<fs-name> A synonym of "fs=" (Deprecated).

### Advanced
cap_release_safety

int, Default: calculated

caps_wanted_delay_max

int, cap release delay, Default: 60

caps_wanted_delay_min

int, cap release delay, Default: 5

dirstat

funky cat dirname for stats, Default: off

nodirstat

no funky cat dirname for stats

ip

my ip

noasyncreaddir

no dcache readdir

nocrc

no data crc on writes

noshare

create a new client instance, instead of sharing an existing instance of a client mounting the same cluster

osdkeepalive

int, Default: 5

osd_idle_ttl

int (seconds), Default: 60

rasize

int (bytes), max readahead. Default: 8388608 (8192*1024)

rbytes

Report the recursive size of the directory contents for st_size on directories. Default: off

norbytes

Do not report the recursive size of the directory contents for st_size on directories.

readdir_max_bytes

int, Default: 524288 (512*1024)

readdir_max_entries

int, Default: 1024

rsize

int (bytes), max read size. Default: 16777216 (16*1024*1024)

snapdirname

string, set the name of the hidden snapdir. Default: .snap

write_congestion_kb

int (kb), max writeback in flight. scale with available memory. Default: calculated from available memory

wsize

int (bytes), max write size. Default: 16777216 (16*1024*1024) (writeback uses smaller of wsize and stripe unit)

wsync

Execute all namespace operations synchronously. This ensures that the namespace operation will only complete after receiving a reply from the MDS. This is the default.

nowsync

Allow the client to do namespace operations asynchronously. When this option is enabled, a namespace operation may complete before the MDS replies, if it has sufficient capabilities to do so.

## EXAMPLES
Mount the full file system:

```
mount -t ceph fs_user@.mycephfs2=/ /mnt/mycephfs
```

Mount only part of the namespace/file system:

```
mount.ceph fs_user@.mycephfs2=/some/directory/in/cephfs /mnt/mycephfs
```

Pass the monitor host's IP address, optionally:

```
mount.ceph fs_user@.mycephfs2=/ /mnt/mycephfs -o mon_addr=192.168.0.1
```

Pass the port along with IP address if it's running on a non-standard port:

```
mount.ceph fs_user@.mycephfs2=/ /mnt/mycephfs -o mon_addr=192.168.0.1:7000
```

If there are multiple monitors, pass each address separated by a /:

```
mount.ceph fs_user@.mycephfs2=/ /mnt/mycephfs -o mon_addr=192.168.0.1/192.168.0.2/192.168.0.3
```

Pass secret key for CephX user optionally:

```
mount.ceph fs_user@.mycephfs2=/ /mnt/mycephfs -o secret=AQATSKdNGBnwLhAAnNDKnH65FmVKpXZJVasUeQ==
```

Pass file containing secret key to avoid leaving secret key in shell's command history:

```
mount.ceph fs_user@.mycephfs2=/ /mnt/mycephfs -o secretfile=/etc/ceph/fs_username.secret
```

If authentication is disabled on Ceph cluster, omit the credential related option:

```
mount.ceph fs_user@.mycephfs2=/ /mnt/mycephfs
```

To mount using the old syntax:

```
mount -t ceph 192.168.0.1:/ /mnt/mycephfs
```

## AVAILABILITY
mount.ceph is part of Ceph, a massively scalable, open-source, distributed storage system. Please refer to the Ceph documentation at https://docs.ceph.com for more information.

## FEATURE AVAILABILITY
The recover_session= option was added to mainline Linux kernels in v5.4. wsync and nowsync were added in v5.7.

## SEE ALSO
ceph-fuse(8), ceph(8)

## COPYRIGHT
2010-2024, Inktank Storage, Inc. and contributors. Licensed under Creative Commons Attribution Share Alike 3.0 (CC-BY-SA-3.0)


rados

package: ceph-common

title: rados(8)

summary: rados object storage utility

text:
# rados(8)

## NAME
rados - rados object storage utility

## SYNOPSIS

```
rados [ options ] [ command ]
```

## DESCRIPTION
rados is a utility for interacting with a Ceph object storage cluster (RADOS), part of the Ceph distributed storage system.

## GLOBAL OPTIONS
--object-locator object_locator

Set object_locator for operation.

-p pool, --pool pool

Interact with the given pool. Required by most commands.

--target-pool pool

Select target pool by name.

--pgid

As an alternative to --pool, --pgid also allow users to specify the PG id to which the command will be directed. With this option, certain commands like ls allow users to limit the scope of the command to the given PG.

-N namespace, --namespace namespace

Specify the rados namespace to use for the object.

--all

Use with ls to list objects in all namespaces. Put in CEPH_ARGS environment variable to make this the default.

--default

Use with ls to list objects in default namespace. Takes precedence over --all in case --all is in environment.

-s snap, --snap snap

Read from the given pool snapshot. Valid for all pool-specific read operations.

--create

Create the pool or directory that was specified.

-i infile

will specify an input file to be passed along as a payload with the command to the monitor cluster. This is only used for specific monitor commands.

-m monaddress[:port]

Connect to specified monitor (instead of looking through ceph.conf).

-b block_size

Set the block size for put/get/append ops and for write benchmarking.

--striper

Uses the striping API of rados rather than the default one. Available for stat, stat2, get, put, append, truncate, rm, ls and all xattr related operation.

-O object_size, --object-size object_size

Set the object size for put/get ops and for write benchmarking.

--max-objects

Set the max number of objects for write benchmarking.

--lock-cookie locker-cookie

Will set the lock cookie for acquiring advisory lock (lock get command). If the cookie is not empty, this option must be passed to lock break command to find the correct lock when releasing lock.

--target-locator

Use with cp to specify the locator of the new object.

--target-nspace

Use with cp to specify the namespace of the new object.

## BENCH OPTIONS
-t N, --concurrent-ios=N

Set number of concurrent I/O operations.

--show-time

Prefix output with date/time.

--no-verify

Do not verify contents of read objects.

--write-object

Write contents to the objects.

--write-omap

Write contents to the omap.

--write-xattr

Write contents to the extended attributes.

## LOAD GEN OPTIONS
--num-objects

Total number of objects.

--min-object-size

Min object size.

--max-object-size

Max object size.

--min-op-len

Min io size of operations.

--max-op-len

Max io size of operations.

--max-ops

Max number of operations.

--max-backlog

Max backlog size.

--read-percent

Percent of operations that are read.

--target-throughput

Target throughput (in bytes).

--run-length

Total time (in seconds).

--offset-align

At what boundary to align random op offsets.

## CACHE POOLS OPTIONS
--with-clones

Include clones when doing flush or evict.

## OMAP OPTIONS
--omap-key-file file

Read the omap key from a file.

## GENERIC OPTIONS
-c FILE, --conf FILE

Read configuration from the given configuration file.

--id ID

Set ID portion of my name.

-n TYPE.ID, --name TYPE.ID

Set cephx user name.

--cluster NAME

Set cluster name (default: ceph).

--setuser USER

Set uid to user or uid (and gid to user's gid).

--setgroup GROUP

Set gid to group or gid.

--version

Show version and quit.

## GLOBAL COMMANDS
lspools

List object pools

df

Show utilization statistics, including disk usage (bytes) and object counts, over the entire system and broken down by pool.

list-inconsistent-pg pool

List inconsistent PGs in given pool.

list-inconsistent-obj pgid

List inconsistent objects in given PG.

list-inconsistent-snapset pgid

List inconsistent snapsets in given PG.

## POOL SPECIFIC COMMANDS
get name outfile

Read object name from the cluster and write it to outfile.

put name infile [--offset offset]

Write object name with start offset (default:0) to the cluster with contents from infile. Warning: The put command creates a single RADOS object, sized just as large as your input file. Unless your objects are of reasonable and consistent sizes, that is probably not what you want -- consider using RGW/S3, CephFS, or RBD instead.

append name infile

Append object name to the cluster with contents from infile.

rm [--force-full] name ...

Remove object(s) with name(s). With --force-full will remove when cluster is marked full.

listwatchers name

List the watchers of object name.

ls outfile

List objects in the given pool and write to outfile. Instead of --pool if --pgid will be specified, ls will only list the objects in the given PG.

lssnap

List snapshots for given pool.

clonedata srcname dstname --object-locator key

Clone object byte data from srcname to dstname. Both objects must be stored with the locator key key (usually either srcname or dstname). Object attributes and omap keys are not copied or cloned.

mksnap foo

Create pool snapshot named foo.

rmsnap foo

Remove pool snapshot named foo.

bench seconds mode [ -b objsize ] [ -t threads ]

Benchmark for seconds. The mode can be write, seq, or rand. seq and rand are read benchmarks, either sequential or random. Before running one of the reading benchmarks, run a write benchmark with the --no-cleanup option. The default object size is 4 MB, and the default number of simulated threads (parallel writes) is 16. The --run-name <label> option is useful for benchmarking a workload test from multiple clients. The <label> is an arbitrary object name. It is "benchmark_last_metadata" by default, and is used as the underlying object name for "read" and "write" ops. Note: -b objsize option is valid only in write mode. Note: write and seq must be run on the same host otherwise the objects created by write will have names that will fail seq.

cleanup [ --run-name run_name ] [ --prefix prefix ]

Clean up a previous benchmark operation. Note: the default run-name is "benchmark_last_metadata"

listxattr name

List all extended attributes of an object.

getxattr name attr

Dump the extended attribute value of attr of an object.

setxattr name attr value

Set the value of attr in the extended attributes of an object.

rmxattr name attr

Remove attr from the extended attributes of an object.

stat name

Get stat (ie. mtime, size) of given object

stat2 name

Get stat (similar to stat, but with high precision time) of given object

listomapkeys name

List all the keys stored in the object map of object name.

listomapvals name

List all key/value pairs stored in the object map of object name. The values are dumped in hexadecimal.

getomapval [ --omap-key-file file ] name key [ out-file ]

Dump the hexadecimal value of key in the object map of object name. If the optional out-file argument is not provided, the value will be written to standard output.

setomapval [ --omap-key-file file ] name key [ value ]

Set the value of key in the object map of object name. If the optional value argument is not provided, the value will be read from standard input.

rmomapkey [ --omap-key-file file ] name key

Remove key from the object map of object name.

getomapheader name

Dump the hexadecimal value of the object map header of object name.

setomapheader name value

Set the value of the object map header of object name.

export filename

Serialize pool contents to a file or standard output.n"

import [--dry-run] [--no-overwrite] < filename | - >

Load pool contents from a file or standard input

## EXAMPLES
To view cluster utilization:

```
rados df
```

To get a list object in pool foo sent to stdout:

```
rados -p foo ls -
```

To get a list of objects in PG 0.6:

```
rados --pgid 0.6 ls
```

To write an object:

```
rados -p foo put myobject blah.txt
```

To create a snapshot:

```
rados -p foo mksnap mysnap
```

To delete the object:

```
rados -p foo rm myobject
```

To read a previously snapshotted version of an object:

```
rados -p foo -s mysnap get myobject blah.txt.old
```

To list inconsistent objects in PG 0.6:

```
rados list-inconsistent-obj 0.6 --format=json-pretty
```

## AVAILABILITY
rados is part of Ceph, a massively scalable, open-source, distributed storage system. Please refer to the Ceph documentation at https://docs.ceph.com for more information.

## SEE ALSO
ceph(8)

## COPYRIGHT
2010-2024, Inktank Storage, Inc. and contributors. Licensed under Creative Commons Attribution Share Alike 3.0 (CC-BY-SA-3.0)


radosgw-admin

title: radosgw-admin(8)

text:
# radosgw-admin(8)

## NAME
radosgw-admin - rados REST gateway user administration utility

## SYNOPSIS

```
radosgw-admin command [ options ... ]
```

## DESCRIPTION
radosgw-admin is a Ceph Object Gateway user administration utility. It is used to create and modify users.

## COMMANDS
radosgw-admin utility provides commands for administration purposes as follows:

user create

Create a new user.

user modify

Modify a user.

user info

Display information for a user including any subusers and keys.

user rename

Renames a user.

user rm

Remove a user.

user suspend

Suspend a user.

user enable

Re-enable user after suspension.

user check

Check user info.

user stats

Show user stats as accounted by the quota subsystem.

user list

List all users.

caps add

Add user capabilities.

caps rm

Remove user capabilities.

subuser create

Create a new subuser (primarily useful for clients using the Swift API).

subuser modify

Modify a subuser.

subuser rm

Remove a subuser.

key create

Create access key.

key rm

Remove access key.

bucket list

List buckets, or, if a bucket is specified with --bucket=<bucket>, list its objects. Adding --allow-unordered removes the ordering requirement, possibly generating results more quickly for buckets with large number of objects.

bucket limit check

Show bucket sharding stats.

bucket link

Link bucket to specified user.

bucket unlink

Unlink bucket from specified user.

bucket chown

Change bucket ownership to the specified user and update object ACLs. Invoke with --marker to resume if the command is interrupted.

bucket stats

Returns bucket statistics.

bucket rm

Remove a bucket.

bucket check

Check bucket index.

bucket rewrite

Rewrite all objects in the specified bucket.

bucket radoslist

List the RADOS objects that contain the data for all objects in the designated bucket, if --bucket=<bucket> is specified. Otherwise, list the RADOS objects that contain data for all buckets.

bucket reshard

Reshard a bucket's index.

bucket sync disable

Disable bucket sync.

bucket sync enable

Enable bucket sync.

bi get

Retrieve bucket index object entries.

bi put

Store bucket index object entries.

bi list

List raw bucket index entries.

bi purge

Purge bucket index entries.

object rm

Remove an object.

object stat

Stat an object for its metadata.

object unlink

Unlink object from bucket index.

object rewrite

Rewrite the specified object.

objects expire

Run expired objects cleanup.

period rm

Remove a period.

period get

Get the period info.

period get-current

Get the current period info.

period pull

Pull a period.

period push

Push a period.

period list

List all periods.

period update

Update the staging period.

period commit

Commit the staging period.

quota set

Set quota params.

quota enable

Enable quota.

quota disable

Disable quota.

global quota get

View global quota parameters.

global quota set

Set global quota parameters.

global quota enable

Enable a global quota.

global quota disable

Disable a global quota.

realm create

Create a new realm.

realm rm

Remove a realm.

realm get

Show the realm info.

realm get-default

Get the default realm name.

realm list

List all realms.

realm list-periods

List all realm periods.

realm rename

Rename a realm.

realm set

Set the realm info (requires infile).

realm default

Set the realm as default.

realm pull

Pull a realm and its current period.

zonegroup add

Add a zone to a zonegroup.

zonegroup create

Create a new zone group info.

zonegroup default

Set the default zone group.

zonegroup rm

Remove a zone group info.

zonegroup get

Show the zone group info.

zonegroup modify

Modify an existing zonegroup.

zonegroup set

Set the zone group info (requires infile).

zonegroup remove

Remove a zone from a zonegroup.

zonegroup rename

Rename a zone group.

zonegroup list

List all zone groups set on this cluster.

zonegroup placement list

List zonegroup's placement targets.

zonegroup placement add

Add a placement target id to a zonegroup.

zonegroup placement modify

Modify a placement target of a specific zonegroup.

zonegroup placement rm

Remove a placement target from a zonegroup.

zonegroup placement default

Set a zonegroup's default placement target.

zone create

Create a new zone.

zone rm

Remove a zone.

zone get

Show zone cluster params.

zone set

Set zone cluster params (requires infile).

zone modify

Modify an existing zone.

zone list

List all zones set on this cluster.

metadata sync status

Get metadata sync status.

metadata sync init

Init metadata sync.

metadata sync run

Run metadata sync.

data sync status

Get data sync status of the specified source zone.

data sync init

Init data sync for the specified source zone.

data sync run

Run data sync for the specified source zone.

sync error list

List sync errors.

sync error trim

Trim sync errors.

zone rename

Rename a zone.

zone placement list

List a zone's placement targets.

zone placement add

Add a zone placement target.

zone placement modify

Modify a zone placement target.

zone placement rm

Remove a zone placement target.

pool add

Add an existing pool for data placement.

pool rm

Remove an existing pool from data placement set.

pools list

List placement active set.

policy

Display bucket/object policy.

log list

List log objects.

log show

Dump a log from specific object or (bucket + date + bucket-id). (NOTE: required to specify formatting of date to "YYYY-MM-DD-hh")

log rm

Remove log object.

usage show

Show the usage information (with optional user and date range).

usage trim

Trim usage information (with optional user and date range).

gc list

Dump expired garbage collection objects (specify --include-all to list all entries, including unexpired).

gc process

Manually process garbage.

lc list

List all bucket lifecycle progress.

lc process

Manually process lifecycle transitions. If a bucket is specified (e.g., via --bucket_id or via --bucket and optional --tenant), only that bucket is processed.

metadata get

Get metadata info.

metadata put

Put metadata info.

metadata rm

Remove metadata info.

metadata list

List metadata info.

mdlog list

List metadata log which is needed for multi-site deployments.

mdlog trim

Trim metadata log manually instead of relying on the gateway's integrated log sync. Before trimming, compare the listings and make sure the last sync was complete, otherwise it can reinitiate a sync.

mdlog status

Read metadata log status.

bilog list

List bucket index log which is needed for multi-site deployments.

bilog trim

Trim bucket index log (use start-marker, end-marker) manually instead of relying on the gateway's integrated log sync. Before trimming, compare the listings and make sure the last sync was complete, otherwise it can reinitiate a sync.

datalog list

List data log which is needed for multi-site deployments.

datalog trim

Trim data log manually instead of relying on the gateway's integrated log sync. Before trimming, compare the listings and make sure the last sync was complete, otherwise it can reinitiate a sync.

datalog status

Read data log status.

orphans find

Init and run search for leaked RADOS objects. DEPRECATED. See the "rgw-orphan-list" tool.

orphans finish

Clean up search for leaked RADOS objects. DEPRECATED. See the "rgw-orphan-list" tool.

orphans list-jobs

List the current orphans search job IDs. DEPRECATED. See the "rgw-orphan-list" tool.

role create

Create a new role for use with STS (Security Token Service).

role rm

Remove a role.

role get

Get a role.

role list

List the roles with specified path prefix.

role modify

Modify the assume role policy of an existing role.

role-policy put

Add/update permission policy to role.

role-policy list

List the policies attached to a role.

role-policy get

Get the specified inline policy document embedded with the given role.

role-policy rm

Remove the policy attached to a role

reshard add

Schedule a resharding of a bucket

reshard list

List all bucket resharding or scheduled to be resharded

reshard process

Process of scheduled reshard jobs

reshard status

Resharding status of a bucket

reshard cancel

Cancel resharding a bucket

topic list

List bucket notifications/pubsub topics

topic get

Get a bucket notifications/pubsub topic

topic rm

Remove a bucket notifications/pubsub topic

subscription get

Get a pubsub subscription definition

subscription rm

Remove a pubsub subscription

subscription pull

Show events in a pubsub subscription

subscription ack

Acknowledge (remove) events in a pubsub subscription

## OPTIONS
-c ceph.conf, --conf=ceph.conf

Use ceph.conf configuration file instead of the default /etc/ceph/ceph.conf to determine monitor addresses during startup.

-m monaddress[:port]

Connect to specified monitor (instead of selecting one from ceph.conf).

--tenant=<tenant>

Name of the tenant.

--uid=uid

The user on which to operate.

--new-uid=uid

The new ID of the user. Used with 'user rename' command.

--subuser=<name>

Name of the subuser.

--access-key=<key>

S3 access key.

--email=email

The e-mail address of the user.

--secret/--secret-key=<key>

The secret key.

--gen-access-key

Generate random access key (for S3).

--gen-secret

Generate random secret key.

--key-type=<type>

Key type, options are: swift, s3.

--temp-url-key[-2]=<key>

Temporary URL key.

--max-buckets

Maximum number of buckets for a user (0 for no limit, negative value to disable bucket creation). Default is 1000.

--access=<access>

Set the access permissions for the subuser. Available access permissions are read, write, readwrite and full.

--display-name=<name>

The display name of the user.

--admin

Set the admin flag on the user.

--system

Set the system flag on the user.

--bucket=[tenant-id/]bucket

Specify the bucket name. If tenant-id is not specified, the tenant-id of the user (--uid) is used.

--pool=<pool>

Specify the pool name. Also used with orphans find as data pool to scan for leaked rados objects.

--object=object

Specify the object name.

--date=yyyy-mm-dd

The date in the format yyyy-mm-dd.

--start-date=yyyy-mm-dd

The start date in the format yyyy-mm-dd.

--end-date=yyyy-mm-dd

The end date in the format yyyy-mm-dd.

--bucket-id=<bucket-id>

Specify the bucket id.

--bucket-new-name=[tenant-id/]<bucket>

Optional for bucket link; use to rename a bucket. While the tenant-id can be specified, this is not necessary in normal operation.

--shard-id=<shard-id>

Optional for mdlog list, bi list, data sync status. Required for mdlog trim.

--max-entries=<entries>

Optional for listing operations to specify the max entries.

--purge-data

When specified, user removal will also purge the user's data.

--purge-keys

When specified, subuser removal will also purge the subuser' keys.

--purge-objects

When specified, the bucket removal will also purge all objects in it.

--metadata-key=<key>

Key from which to retrieve metadata, used with metadata get.

--remote=<remote>

Zone or zonegroup id of remote gateway.

--period=<id>

Period ID.

--url=<url>

URL for pushing/pulling period or realm.

--epoch=<number>

Period epoch.

--commit

Commit the period during 'period update'.

--staging

Get the staging period info.

--master

Set as master.

--master-zone=<id>

Master zone ID.

--rgw-realm=<name>

The realm name.

--realm-id=<id>

The realm ID.

--realm-new-name=<name>

New name for the realm.

--rgw-zonegroup=<name>

The zonegroup name.

--zonegroup-id=<id>

The zonegroup ID.

--zonegroup-new-name=<name>

The new name of the zonegroup.

--rgw-zone=<zone>

Zone in which the gateway is running.

--zone-id=<id>

The zone ID.

--zone-new-name=<name>

The new name of the zone.

--source-zone

The source zone for data sync.

--default

Set the entity (realm, zonegroup, zone) as default.

--read-only

Set the zone as read-only when adding to the zonegroup.

--placement-id

Placement ID for the zonegroup placement commands.

--tags=<list>

The list of tags for zonegroup placement add and modify commands.

--tags-add=<list>

The list of tags to add for zonegroup placement modify command.

--tags-rm=<list>

The list of tags to remove for zonegroup placement modify command.

--endpoints=<list>

The zone endpoints.

--index-pool=<pool>

The placement target index pool.

--data-pool=<pool>

The placement target data pool.

--data-extra-pool=<pool>

The placement target data extra (non-EC) pool.

--placement-index-type=<type>

The placement target index type (normal, indexless, or #id).

--placement-inline-data=<true>

Whether the placement target is configured to store a data chunk inline in head objects.

--tier-type=<type>

The zone tier type.

--tier-config=<k>=<v>[,...]

Set zone tier config keys, values.

--tier-config-rm=<k>[,...]

Unset zone tier config keys.

--sync-from-all[=false]

Set/reset whether zone syncs from all zonegroup peers.

--sync-from=[zone-name][,...]

Set the list of zones from which to sync.

--sync-from-rm=[zone-name][,...]

Remove zone(s) from list of zones from which to sync.

--bucket-index-max-shards

Override a zone's or zonegroup's default number of bucket index shards. This option is accepted by the 'zone create', 'zone modify', 'zonegroup add', and 'zonegroup modify' commands, and applies to buckets that are created after the zone/zonegroup changes take effect.

--fix

Fix the bucket index in addition to checking it.

--check-objects

Bucket check: Rebuilds the bucket index according to actual object state.

--format=<format>

Specify output format for certain operations. Supported formats: xml, json.

--sync-stats

Option for the 'user stats' command. When specified, it will update user stats with the current stats reported by the user's buckets indexes.

--show-config

Show configuration.

--show-log-entries=<flag>

Enable/disable dumping of log entries on log show.

--show-log-sum=<flag>

Enable/disable dump of log summation on log show.

--skip-zero-entries

Log show only dumps entries that don't have zero value in one of the numeric field.

--infile

Specify a file to read when setting data.

--categories=<list>

Comma separated list of categories, used in usage show.

--caps=<caps>

List of capabilities (e.g., "usage=read, write; user=read").

--compression=<compression-algorithm>

Placement target compression algorithm (lz4|snappy|zlib|zstd).

--yes-i-really-mean-it

Required as a guardrail for certain destructive operations.

--min-rewrite-size

Specify the minimum object size for bucket rewrite (default 4M).

--max-rewrite-size

Specify the maximum object size for bucket rewrite (default ULLONG_MAX).

--min-rewrite-stripe-size

Specify the minimum stripe size for object rewrite (default 0). If the value is set to 0, then the specified object will always be rewritten when restriping.

--warnings-only

When specified with bucket limit check, list only buckets nearing or over the current max objects per shard value.

--bypass-gc

When specified with bucket deletion, triggers object deletion without involving GC.

--inconsistent-index

When specified with bucket deletion and bypass-gc set to true, ignores bucket index consistency.

--max-concurrent-ios

Maximum concurrent bucket operations. Affects operations that scan the bucket index, e.g., listing, deletion, and all scan/search operations such as finding orphans or checking the bucket index. The default is 32.

## QUOTA OPTIONS
--max-objects

Specify the maximum number of objects (negative value to disable).

--max-size

Specify the maximum object size (in B/K/M/G/T, negative value to disable).

--quota-scope

The scope of quota (bucket, user).

## ORPHANS SEARCH OPTIONS
--num-shards

Number of shards to use for temporary scan info

--orphan-stale-secs

Number of seconds to wait before declaring an object to be an orphan. The efault is 86400 (24 hours).

--job-id

Set the job id (for orphans find)

## ORPHANS LIST-JOBS OPTIONS
--extra-info

Provide extra info in the job list.

## ROLE OPTIONS
--role-name

The name of the role to create.

--path

The path to the role.

--assume-role-policy-doc

The trust relationship policy document that grants an entity permission to assume the role.

--policy-name

The name of the policy document.

--policy-doc

The permission policy document.

--path-prefix

The path prefix for filtering the roles.

## BUCKET NOTIFICATIONS/PUBSUB OPTIONS
--topic

The bucket notifications/pubsub topic name.

--subscription

The pubsub subscription name.

--event-id

The event id in a pubsub subscription.

## EXAMPLES
Generate a new user:

```
$ radosgw-admin user create --display-name="johnny rotten" --uid=johnny
{ "user_id": "johnny",

  "rados_uid": 0,

  "display_name": "johnny rotten",

  "email": "",

  "suspended": 0,

  "subusers": [],

  "keys": [

        { "user": "johnny",

          "access_key": "TCICW53D9BQ2VGC46I44",

          "secret_key": "tfm9aHMI8X76L3UdgE+ZQaJag1vJQmE6HDb5Lbrz"}],

  "swift_keys": []}
```

Remove a user:

```
$ radosgw-admin user rm --uid=johnny
```

Rename a user:

```
$ radosgw-admin user rename --uid=johny --new-uid=joe
```

Remove a user and all associated buckets with their contents:

```
$ radosgw-admin user rm --uid=johnny --purge-data
```

Remove a bucket:

```
$ radosgw-admin bucket rm --bucket=foo
```

Link bucket to specified user:

```
$ radosgw-admin bucket link --bucket=foo --bucket_id=<bucket id> --uid=johnny
```

Unlink bucket from specified user:

```
$ radosgw-admin bucket unlink --bucket=foo --uid=johnny
```

Rename a bucket:

```
$ radosgw-admin bucket link --bucket=foo --bucket-new-name=bar --uid=johnny
```

Move a bucket from the old global tenant space to a specified tenant:

```
$ radosgw-admin bucket link --bucket=/foo --uid=12345678$12345678'
```

Link bucket to specified user and change object ACLs:

```
$ radosgw-admin bucket chown --bucket=/foo --uid=12345678$12345678'
```

Show the logs of a bucket from April 1st, 2012:

```
$ radosgw-admin log show --bucket=foo --date=2012-04-01-01 --bucket-id=default.14193.1
```

Show usage information for user from March 1st to (but not including) April 1st, 2012:

```
$ radosgw-admin usage show --uid=johnny \

                --start-date=2012-03-01 --end-date=2012-04-01
```

Show only summary of usage information for all users:

```
$ radosgw-admin usage show --show-log-entries=false
```

Trim usage information for user until March 1st, 2012:

```
$ radosgw-admin usage trim --uid=johnny --end-date=2012-04-01
```

## AVAILABILITY
radosgw-admin is part of Ceph, a massively scalable, open-source, distributed storage system. Please refer to the Ceph documentation at https://docs.ceph.com for more information.

## SEE ALSO
ceph(8) radosgw(8)

## COPYRIGHT
2010-2024, Inktank Storage, Inc. and contributors. Licensed under Creative Commons Attribution Share Alike 3.0 (CC-BY-SA-3.0)


rbd

title: rbd(8)

summary: manage rados block device (RBD) images

text:
# rbd(8)

## NAME
rbd - manage rados block device (RBD) images

## SYNOPSIS

```
rbd [ -c ceph.conf ] [ -m monaddr ] [--cluster cluster-name]
[ -p | --pool pool ] [ command ... ]
```

## DESCRIPTION
rbd is a utility for manipulating rados block device (RBD) images, used by the Linux rbd driver and the rbd storage driver for QEMU/KVM. RBD images are simple block devices that are striped over objects and stored in a RADOS object store. The size of the objects the image is striped over must be a power of two.

## OPTIONS
-c ceph.conf, --conf ceph.conf

Use ceph.conf configuration file instead of the default /etc/ceph/ceph.conf to determine monitor addresses during startup.

-m monaddress[:port]

Connect to specified monitor (instead of looking through ceph.conf).

--cluster cluster-name

Use different cluster name as compared to default cluster name ceph.

-p pool-name, --pool pool-name

Interact with the given pool. Required by most commands.

--namespace namespace-name

Use a pre-defined image namespace within a pool

--no-progress

Do not output progress information (goes to standard error by default for some commands).

## PARAMETERS
--image-format format-id

Specifies which object layout to use. The default is 2.

- format 1 - (deprecated) Use the original format for a new rbd image. This format is understood by all versions of librbd and the kernel rbd module, but does not support newer features like cloning.

- format 2 - Use the second rbd format, which is supported by librbd since the Bobtail release and the kernel rbd module since kernel 3.10 (except for "fancy" striping, which is supported since kernel 4.17). This adds support for cloning and is more easily extensible to allow more features in the future.

-s size-in-M/G/T, --size size-in-M/G/T

Specifies the size of the new rbd image or the new size of the existing rbd image in M/G/T. If no suffix is given, unit M is assumed.

--object-size size-in-B/K/M

Specifies the object size in B/K/M. Object size will be rounded up the nearest power of two; if no suffix is given, unit B is assumed. The default object size is 4M, smallest is 4K and maximum is 32M. The default value can be changed with the configuration option rbd_default_order, which takes a power of two (default object size is 2 ^ rbd_default_order).

--stripe-unit size-in-B/K/M

Specifies the stripe unit size in B/K/M. If no suffix is given, unit B is assumed. See striping section (below) for more details.

--stripe-count num

Specifies the number of objects to stripe over before looping back to the first object. See striping section (below) for more details.

--snap snap

Specifies the snapshot name for the specific operation.

--id username

Specifies the username (without the client. prefix) to use with the map command.

--keyring filename

Specifies a keyring file containing a secret for the specified user to use with the map command. If not specified, the default keyring locations will be searched.

--keyfile filename

Specifies a file containing the secret key of --id user to use with the map command. This option is overridden by --keyring if the latter is also specified.

--shared lock-tag

Option for lock add that allows multiple clients to lock the same image if they use the same tag. The tag is an arbitrary string. This is useful for situations where an image must be open from more than one client at once, like during live migration of a virtual machine, or for use underneath a clustered file system.

--format format

Specifies output formatting (default: plain, json, xml)

--pretty-format

Make json or xml formatted output more human-readable.

-o krbd-options, --options krbd-options

Specifies which options to use when mapping or unmapping an image via the rbd kernel driver. krbd-options is a comma-separated list of options (similar to mount(8) mount options). See kernel rbd (krbd) options section below for more details.

--read-only

Map the image read-only. Equivalent to -o ro.

--image-feature feature-name

Specifies which RBD format 2 feature should be enabled when creating an image. Multiple features can be enabled by repeating this option multiple times. The following features are supported:

- layering: layering support

- striping: striping v2 support

- exclusive-lock: exclusive locking support

- object-map: object map support (requires exclusive-lock)

- fast-diff: fast diff calculations (requires object-map)

- deep-flatten: snapshot flatten support

- journaling: journaled IO support (requires exclusive-lock)

- data-pool: erasure coded pool support

--image-shared

Specifies that the image will be used concurrently by multiple clients. This will disable features that are dependent upon exclusive ownership of the image.

--whole-object

Specifies that the diff should be limited to the extents of a full object instead of showing intra-object deltas. When the object map feature is enabled on an image, limiting the diff to the object extents will dramatically improve performance since the differences can be computed by examining the in-memory object map instead of querying RADOS for each object within the image.

--limit

Specifies the limit for the number of snapshots permitted.

## COMMANDS
bench --io-type <read | write | readwrite | rw> [--io-size size-in-B/K/M/G/T] [--io-threads num-ios-in-flight] [--io-total size-in-B/K/M/G/T] [--io-pattern seq | rand] [--rw-mix-read read proportion in readwrite] image-spec

Generate a series of IOs to the image and measure the IO throughput and latency. If no suffix is given, unit B is assumed for both --io-size and --io-total. Defaults are: --io-size 4096, --io-threads 16, --io-total 1G, --io-pattern seq, --rw-mix-read 50.

children snap-spec

List the clones of the image at the given snapshot. This checks every pool, and outputs the resulting poolname/imagename. This requires image format 2.

clone [--object-size size-in-B/K/M] [--stripe-unit size-in-B/K/M --stripe-count num] [--image-feature feature-name] [--image-shared] parent-snap-spec child-image-spec

Will create a clone (copy-on-write child) of the parent snapshot. Object size will be identical to that of the parent image unless specified. Size will be the same as the parent snapshot. The --stripe-unit and --stripe-count arguments are optional, but must be used together. The parent snapshot must be protected (see rbd snap protect). This requires image format 2.

config global get config-entity key

Get a global-level configuration override.

config global list [--format plain | json | xml] [--pretty-format] config-entity

List global-level configuration overrides.

config global set config-entity key value

Set a global-level configuration override.

config global remove config-entity key

Remove a global-level configuration override.

config image get image-spec key

Get an image-level configuration override.

config image list [--format plain | json | xml] [--pretty-format] image-spec

List image-level configuration overrides.

config image set image-spec key value

Set an image-level configuration override.

config image remove image-spec key

Remove an image-level configuration override.

config pool get pool-name key

Get a pool-level configuration override.

config pool list [--format plain | json | xml] [--pretty-format] pool-name

List pool-level configuration overrides.

config pool set pool-name key value

Set a pool-level configuration override.

config pool remove pool-name key

Remove a pool-level configuration override.

cp (src-image-spec | src-snap-spec) dest-image-spec

Copy the content of a src-image into the newly created dest-image. dest-image will have the same size, object size, and image format as src-image. Note: snapshots are not copied, use deep cp command to include snapshots.

create (-s | --size size-in-M/G/T) [--image-format format-id] [--object-size size-in-B/K/M] [--stripe-unit size-in-B/K/M --stripe-count num] [--thick-provision] [--no-progress] [--image-feature feature-name]... [--image-shared] image-spec

Will create a new rbd image. You must also specify the size via --size. The --stripe-unit and --stripe-count arguments are optional, but must be used together. If the --thick-provision is enabled, it will fully allocate storage for the image at creation time. It will take a long time to do. Note: thick provisioning requires zeroing the contents of the entire image.

deep cp (src-image-spec | src-snap-spec) dest-image-spec

Deep copy the content of a src-image into the newly created dest-image. Dest-image will have the same size, object size, image format, and snapshots as src-image.

device list [-t | --device-type device-type] [--format plain | json | xml] --pretty-format

Show the rbd images that are mapped via the rbd kernel module (default) or other supported device.

device map [-t | --device-type device-type] [--cookie device-cookie] [--show-cookie] [--snap-id snap-id] [--read-only] [--exclusive] [-o | --options device-options] image-spec | snap-spec

Map the specified image to a block device via the rbd kernel module (default) or other supported device (nbd on Linux or ggate on FreeBSD). The --options argument is a comma separated list of device type specific options (opt1,opt2=val,...).

device unmap [-t | --device-type device-type] [-o | --options device-options] [--snap-id snap-id] image-spec | snap-spec | device-path

Unmap the block device that was mapped via the rbd kernel module (default) or other supported device. The --options argument is a comma separated list of device type specific options (opt1,opt2=val,...).

device attach [-t | --device-type device-type] --device device-path [--cookie device-cookie] [--show-cookie] [--snap-id snap-id] [--read-only] [--exclusive] [--force] [-o | --options device-options] image-spec | snap-spec

Attach the specified image to the specified block device (currently only nbd on Linux). This operation is unsafe and should not be normally used. In particular, specifying the wrong image or the wrong block device may lead to data corruption as no validation is performed by nbd kernel driver. The --options argument is a comma separated list of device type specific options (opt1,opt2=val,...).

device detach [-t | --device-type device-type] [-o | --options device-options] [--snap-id snap-id] image-spec | snap-spec | device-path

Detach the block device that was mapped or attached (currently only nbd on Linux). This operation is unsafe and should not be normally used. The --options argument is a comma separated list of device type specific options (opt1,opt2=val,...).

diff [--from-snap snap-name] [--whole-object] image-spec | snap-spec

Dump a list of byte extents in the image that have changed since the specified start snapshot, or since the image was created. Each output line includes the starting offset (in bytes), the length of the region (in bytes), and either 'zero' or 'data' to indicate whether the region is known to be zeros or may contain other data.

du [-p | --pool pool-name] [image-spec | snap-spec] [--merge-snapshots]

Will calculate the provisioned and actual disk usage of all images and associated snapshots within the specified pool. It can also be used against individual images and snapshots. If the RBD fast-diff feature is not enabled on images, this operation will require querying the OSDs for every potential object within the image.

The --merge-snapshots will merge snapshots used space into their parent images.

encryption format image-spec format passphrase-file [--cipher-alg alg]

Formats image to an encrypted format. All data previously written to the image will become unreadable. A cloned image cannot be formatted, although encrypted images can be cloned. Supported formats: luks1, luks2. Supported cipher algorithms: aes-128, aes-256 (default).

export [--export-format format (1 or 2)] (image-spec | snap-spec) [dest-path]

Export image to dest path (use - for stdout). The --export-format accepts '1' or '2' currently. Format 2 allow us to export not only the content of image, but also the snapshots and other properties, such as image_order, features.

export-diff [--from-snap snap-name] [--whole-object] (image-spec | snap-spec) dest-path

Export an incremental diff for an image to dest path (use - for stdout). If an initial snapshot is specified, only changes since that snapshot are included; otherwise, any regions of the image that contain data are included. The end snapshot is specified using the standard --snap option or @snap syntax (see below). The image diff format includes metadata about image size changes, and the start and end snapshots. It efficiently represents discarded or 'zero' regions of the image.

feature disable image-spec feature-name...

Disable the specified feature on the specified image. Multiple features can be specified.

feature enable image-spec feature-name...

Enable the specified feature on the specified image. Multiple features can be specified.

flatten image-spec

If image is a clone, copy all shared blocks from the parent snapshot and make the child independent of the parent, severing the link between parent snap and child. The parent snapshot can be unprotected and deleted if it has no further dependent clones. This requires image format 2.

group create group-spec

Create a group.

group image add group-spec image-spec

Add an image to a group.

group image list group-spec

List images in a group.

group image remove group-spec image-spec

Remove an image from a group.

group ls [-p | --pool pool-name]

List rbd groups.

group rename src-group-spec dest-group-spec

Rename a group. Note: rename across pools is not supported.

group rm group-spec

Delete a group.

group snap create group-snap-spec

Make a snapshot of a group.

group snap list group-spec

List snapshots of a group.

group snap rm group-snap-spec

Remove a snapshot from a group.

group snap rename group-snap-spec snap-name

Rename group's snapshot.

group snap rollback group-snap-spec

Rollback group to snapshot.

image-meta get image-spec key

Get metadata value with the key.

image-meta list image-spec

Show metadata held on the image. The first column is the key and the second column is the value.

image-meta remove image-spec key

Remove metadata key with the value.

image-meta set image-spec key value

Set metadata key with the value. They will displayed in image-meta list.

import [--export-format format (1 or 2)] [--image-format format-id] [--object-size size-in-B/K/M] [--stripe-unit size-in-B/K/M --stripe-count num] [--image-feature feature-name]... [--image-shared] src-path [image-spec]

Create a new image and imports its data from path (use - for stdin). The import operation will try to create sparse rbd images if possible. For import from stdin, the sparsification unit is the data block size of the destination image (object size). The --stripe-unit and --stripe-count arguments are optional, but must be used together.

The --export-format accepts '1' or '2' currently. Format 2 allow us to import not only the content of image, but also the snapshots and other properties, such as image_order, features.

import-diff src-path image-spec

Import an incremental diff of an image and applies it to the current image. If the diff was generated relative to a start snapshot, we verify that snapshot already exists before continuing. If there was an end snapshot we verify it does not already exist before applying the changes, and create the snapshot when we are done.

info image-spec | snap-spec

Will dump information (such as size and object size) about a specific rbd image. If image is a clone, information about its parent is also displayed. If a snapshot is specified, whether it is protected is shown as well.

journal client disconnect journal-spec

Flag image journal client as disconnected.

journal export [--verbose] [--no-error] src-journal-spec path-name

Export image journal to path (use - for stdout). It can be make a backup of the image journal especially before attempting dangerous operations. Note that this command may not always work if the journal is badly corrupted.

journal import [--verbose] [--no-error] path-name dest-journal-spec

Import image journal from path (use - for stdin).

journal info journal-spec

Show information about image journal.

journal inspect [--verbose] journal-spec

Inspect and report image journal for structural errors.

journal reset journal-spec

Reset image journal.

journal status journal-spec

Show status of image journal.

lock add [--shared lock-tag] image-spec lock-id

Lock an image. The lock-id is an arbitrary name for the user's convenience. By default, this is an exclusive lock, meaning it will fail if the image is already locked. The --shared option changes this behavior. Note that locking does not affect any operation other than adding a lock. It does not protect an image from being deleted.

lock ls image-spec

Show locks held on the image. The first column is the locker to use with the lock remove command.

lock rm image-spec lock-id locker

Release a lock on an image. The lock id and locker are as output by lock ls.

ls [-l | --long] [pool-name]

Will list all rbd images listed in the rbd_directory object. With -l, also show snapshots, and use longer-format output including size, parent (if clone), format, etc.

merge-diff first-diff-path second-diff-path merged-diff-path

Merge two continuous incremental diffs of an image into one single diff. The first diff's end snapshot must be equal with the second diff's start snapshot. The first diff could be - for stdin, and merged diff could be - for stdout, which enables multiple diff files to be merged using something like 'rbd merge-diff first second - | rbd merge-diff - third result'. Note this command currently only support the source incremental diff with stripe_count == 1

migration abort image-spec

Cancel image migration. This step may be run after successful or failed migration prepare or migration execute steps and returns the image to its initial (before migration) state. All modifications to the destination image are lost.

migration commit image-spec

Commit image migration. This step is run after a successful migration prepare and migration execute steps and removes the source image data.

migration execute image-spec

Execute image migration. This step is run after a successful migration prepare step and copies image data to the destination.

migration prepare [--order order] [--object-size object-size] [--image-feature image-feature] [--image-shared] [--stripe-unit stripe-unit] [--stripe-count stripe-count] [--data-pool data-pool] [--import-only] [--source-spec json] [--source-spec-path path] src-image-spec [dest-image-spec]

Prepare image migration. This is the first step when migrating an image, i.e. changing the image location, format or other parameters that can't be changed dynamically. The destination can match the source, and in this case dest-image-spec can be omitted. After this step the source image is set as a parent of the destination image, and the image is accessible in copy-on-write mode by its destination spec. An image can also be migrated from a read-only import source by adding the --import-only optional and providing a JSON-encoded --source-spec or a path to a JSON-encoded source-spec file using the --source-spec-path optionals.

mirror image demote image-spec

Demote a primary image to non-primary for RBD mirroring.

mirror image disable [--force] image-spec

Disable RBD mirroring for an image. If the mirroring is configured in image mode for the image's pool, then it can be explicitly disabled mirroring for each image within the pool.

mirror image enable image-spec mode

Enable RBD mirroring for an image. If the mirroring is configured in image mode for the image's pool, then it can be explicitly enabled mirroring for each image within the pool. The mirror image mode can either be journal (default) or snapshot. The journal mode requires the RBD journaling feature.

mirror image promote [--force] image-spec

Promote a non-primary image to primary for RBD mirroring.

mirror image resync image-spec

Force resync to primary image for RBD mirroring.

mirror image status image-spec

Show RBD mirroring status for an image.

mirror pool demote [pool-name]

Demote all primary images within a pool to non-primary. Every mirroring enabled image will demoted in the pool.

mirror pool disable [pool-name]

Disable RBD mirroring by default within a pool. When mirroring is disabled on a pool in this way, mirroring will also be disabled on any images (within the pool) for which mirroring was enabled explicitly.

mirror pool enable [pool-name] mode

Enable RBD mirroring by default within a pool. The mirroring mode can either be pool or image. If configured in pool mode, all images in the pool with the journaling feature enabled are mirrored. If configured in image mode, mirroring needs to be explicitly enabled (by mirror image enable command) on each image.

mirror pool info [pool-name]

Show information about the pool mirroring configuration. It includes mirroring mode, peer UUID, remote cluster name, and remote client name.

mirror pool peer add [pool-name] remote-cluster-spec

Add a mirroring peer to a pool. remote-cluster-spec is [remote client name@]remote cluster name. The default for remote client name is "client.admin".

This requires mirroring mode is enabled.

mirror pool peer remove [pool-name] uuid

Remove a mirroring peer from a pool. The peer uuid is available from mirror pool info command.

mirror pool peer set [pool-name] uuid key value

Update mirroring peer settings. The key can be either client or cluster, and the value is corresponding to remote client name or remote cluster name.

mirror pool promote [--force] [pool-name]

Promote all non-primary images within a pool to primary. Every mirroring enabled image will promoted in the pool.

mirror pool status [--verbose] [pool-name]

Show status for all mirrored images in the pool. With --verbose, also show additionally output status details for every mirroring image in the pool.

mirror snapshot schedule add [-p | --pool pool] [--namespace namespace] [--image image] interval [start-time]

Add mirror snapshot schedule.

mirror snapshot schedule list [-R | --recursive] [--format format] [--pretty-format] [-p | --pool pool] [--namespace namespace] [--image image]

List mirror snapshot schedule.

mirror snapshot schedule remove [-p | --pool pool] [--namespace namespace] [--image image] interval [start-time]

Remove mirror snapshot schedule.

mirror snapshot schedule status [-p | --pool pool] [--format format] [--pretty-format] [--namespace namespace] [--image image]

Show mirror snapshot schedule status.

mv src-image-spec dest-image-spec

Rename an image. Note: rename across pools is not supported.

namespace create pool-name/namespace-name

Create a new image namespace within the pool.

namespace list pool-name

List image namespaces defined within the pool.

namespace remove pool-name/namespace-name

Remove an empty image namespace from the pool.

object-map check image-spec | snap-spec

Verify the object map is correct.

object-map rebuild image-spec | snap-spec

Rebuild an invalid object map for the specified image. An image snapshot can be specified to rebuild an invalid object map for a snapshot.

pool init [pool-name] [--force]

Initialize pool for use by RBD. Newly created pools must initialized prior to use.

resize (-s | --size size-in-M/G/T) [--allow-shrink] image-spec

Resize rbd image. The size parameter also needs to be specified. The --allow-shrink option lets the size be reduced.

rm image-spec

Delete an rbd image (including all data blocks). If the image has snapshots, this fails and nothing is deleted.

snap create snap-spec

Create a new snapshot. Requires the snapshot name parameter specified.

snap limit clear image-spec

Remove any previously set limit on the number of snapshots allowed on an image.

snap limit set [--limit] limit image-spec

Set a limit for the number of snapshots allowed on an image.

snap ls image-spec

Dump the list of snapshots inside a specific image.

snap protect snap-spec

Protect a snapshot from deletion, so that clones can be made of it (see rbd clone). Snapshots must be protected before clones are made; protection implies that there exist dependent cloned children that refer to this snapshot. rbd clone will fail on a nonprotected snapshot. This requires image format 2.

snap purge image-spec

Remove all unprotected snapshots from an image.

snap rename src-snap-spec dest-snap-spec

Rename a snapshot. Note: rename across pools and images is not supported.

snap rm [--force] snap-spec

Remove the specified snapshot.

snap rollback snap-spec

Rollback image content to snapshot. This will iterate through the entire blocks array and update the data head content to the snapshotted version.

snap unprotect snap-spec

Unprotect a snapshot from deletion (undo snap protect). If cloned children remain, snap unprotect fails. (Note that clones may exist in different pools than the parent snapshot.) This requires image format 2.

sparsify [--sparse-size sparse-size] image-spec

Reclaim space for zeroed image extents. The default sparse size is 4096 bytes and can be changed via --sparse-size option with the following restrictions: it should be power of two, not less than 4096, and not larger than image object size.

status image-spec

Show the status of the image, including which clients have it open.

trash ls [pool-name]

List all entries from trash.

trash mv image-spec

Move an image to the trash. Images, even ones actively in-use by clones, can be moved to the trash and deleted at a later time.

trash purge [pool-name]

Remove all expired images from trash.

trash restore image-id

Restore an image from trash.

trash rm image-id

Delete an image from trash. If image deferment time has not expired you can not removed it unless use force. But an actively in-use by clones or has snapshots can not be removed.

trash purge schedule add [-p | --pool pool] [--namespace namespace] interval [start-time]

Add trash purge schedule.

trash purge schedule list [-R | --recursive] [--format format] [--pretty-format] [-p | --pool pool] [--namespace namespace]

List trash purge schedule.

trash purge schedule remove [-p | --pool pool] [--namespace namespace] interval [start-time]

Remove trash purge schedule.

trash purge schedule status [-p | --pool pool] [--format format] [--pretty-format] [--namespace namespace]

Show trash purge schedule status.

watch image-spec

Watch events on image.

## IMAGE, SNAP, GROUP AND JOURNAL SPECS

```
image-spec      is [pool-name/[namespace-name/]]image-name
snap-spec       is [pool-name/[namespace-name/]]image-name@snap-name
group-spec      is [pool-name/[namespace-name/]]group-name
group-snap-spec is [pool-name/[namespace-name/]]group-name@snap-name
journal-spec    is [pool-name/[namespace-name/]]journal-name
```

The default for pool-name is "rbd" and namespace-name is "". If an image name contains a slash character ('/'), pool-name is required.

The journal-name is image-id.

You may specify each name individually, using --pool, --namespace, --image, and --snap options, but this is discouraged in favor of the above spec syntax.

## STRIPING
RBD images are striped over many objects, which are then stored by the Ceph distributed object store (RADOS). As a result, read and write requests for the image are distributed across many nodes in the cluster, generally preventing any single node from becoming a bottleneck when individual images get large or busy.

The striping is controlled by three parameters:

object-size

The size of objects we stripe over is a power of two. It will be rounded up the nearest power of two. The default object size is 4 MB, smallest is 4K and maximum is 32M.

stripe_unit

Each [stripe_unit] contiguous bytes are stored adjacently in the same object, before we move on to the next object.

stripe_count

After we write [stripe_unit] bytes to [stripe_count] objects, we loop back to the initial object and write another stripe, until the object reaches its maximum size. At that point, we move on to the next [stripe_count] objects.

By default, [stripe_unit] is the same as the object size and [stripe_count] is 1. Specifying a different [stripe_unit] and/or [stripe_count] is often referred to as using "fancy" striping and requires format 2.

## KERNEL RBD (KRBD) OPTIONS
Most of these options are useful mainly for debugging and benchmarking. The default values are set in the kernel and may therefore depend on the version of the running kernel.

Per client instance rbd device map options:

- fsid=aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee - FSID that should be assumed by the client.

- ip=a.b.c.d[:p] - IP and, optionally, port the client should use.

- share - Enable sharing of client instances with other mappings (default).

- noshare - Disable sharing of client instances with other mappings.

- crc - Enable CRC32C checksumming for msgr1 on-the-wire protocol (default). For msgr2.1 protocol this option is ignored: full checksumming is always on in 'crc' mode and always off in 'secure' mode.

- nocrc - Disable CRC32C checksumming for msgr1 on-the-wire protocol. Note that only payload checksumming is disabled, header checksumming is always on. For msgr2.1 protocol this option is ignored.

- cephx_require_signatures - Require msgr1 message signing feature (since 3.19, default). This option is deprecated and will be removed in the future as the feature has been supported since the Bobtail release.

- nocephx_require_signatures - Don't require msgr1 message signing feature (since 3.19). This option is deprecated and will be removed in the future.

- tcp_nodelay - Disable Nagle's algorithm on client sockets (since 4.0, default).

- notcp_nodelay - Enable Nagle's algorithm on client sockets (since 4.0).

- cephx_sign_messages - Enable message signing for msgr1 on-the-wire protocol (since 4.4, default). For msgr2.1 protocol this option is ignored: message signing is built into 'secure' mode and not offered in 'crc' mode.

- nocephx_sign_messages - Disable message signing for msgr1 on-the-wire protocol (since 4.4). For msgr2.1 protocol this option is ignored.

- mount_timeout=x - A timeout on various steps in rbd device map and rbd device unmap sequences (default is 60 seconds). In particular, since 4.2 this can be used to ensure that rbd device unmap eventually times out when there is no network connection to a cluster.

- osdkeepalive=x - OSD keepalive timeout (default is 5 seconds).

- osd_idle_ttl=x - OSD idle TTL (default is 60 seconds).

Per mapping (block device) rbd device map options:

- rw - Map the image read-write (default). Overridden by --read-only.

- ro - Map the image read-only. Equivalent to --read-only.

- queue_depth=x - queue depth (since 4.2, default is 128 requests).

- lock_on_read - Acquire exclusive lock on reads, in addition to writes and discards (since 4.9).

- exclusive - Disable automatic exclusive lock transitions (since 4.12). Equivalent to --exclusive.

- lock_timeout=x - A timeout on waiting for the acquisition of exclusive lock (since 4.17, default is 0 seconds, meaning no timeout).

- notrim - Turn off discard and write zeroes offload support to avoid deprovisioning a fully provisioned image (since 4.17). When enabled, discard requests will fail with -EOPNOTSUPP, write zeroes requests will fall back to manually zeroing.

- abort_on_full - Fail write requests with -ENOSPC when the cluster is full or the data pool reaches its quota (since 5.0). The default behaviour is to block until the full condition is cleared.

- alloc_size - Minimum allocation unit of the underlying OSD object store backend (since 5.1, default is 64K bytes). This is used to round off and drop discards that are too small. For bluestore, the recommended setting is bluestore_min_alloc_size (currently set to 4K for all types of drives, previously used to be set to 64K for hard disk drives and 16K for solid-state drives). For filestore with filestore_punch_hole = false, the recommended setting is image object size (typically 4M).

- crush_location=x - Specify the location of the client in terms of CRUSH hierarchy (since 5.8). This is a set of key-value pairs separated from each other by '|', with keys separated from values by ':'. Note that '|' may need to be quoted or escaped to avoid it being interpreted as a pipe by the shell. The key is the bucket type name (e.g. rack, datacenter or region with default bucket types) and the value is the bucket name. For example, to indicate that the client is local to rack "myrack", data center "mydc" and region "myregion":

```
crush_location=rack:myrack|datacenter:mydc|region:myregion
```

Each key-value pair stands on its own: "myrack" doesn't need to reside in "mydc", which in turn doesn't need to reside in "myregion". The location is not a path to the root of the hierarchy but rather a set of nodes that are matched independently, owning to the fact that bucket names are unique within a CRUSH map. "Multipath" locations are supported, so it is possible to indicate locality for multiple parallel hierarchies:

```
crush_location=rack:myrack1|rack:myrack2|datacenter:mydc
```

- read_from_replica=no - Disable replica reads, always pick the primary OSD (since 5.8, default).

- read_from_replica=balance - When issued a read on a replicated pool, pick a random OSD for serving it (since 5.8). This mode is safe for general use only since Octopus (i.e. after "ceph osd require-osd-release octopus"). Otherwise it should be limited to read-only workloads such as images mapped read-only everywhere or snapshots.

- read_from_replica=localize - When issued a read on a replicated pool, pick the most local OSD for serving it (since 5.8). The locality metric is calculated against the location of the client given with crush_location; a match with the lowest-valued bucket type wins. For example, with default bucket types, an OSD in a matching rack is closer than an OSD in a matching data center, which in turn is closer than an OSD in a matching region. This mode is safe for general use only since Octopus (i.e. after "ceph osd require-osd-release octopus"). Otherwise it should be limited to read-only workloads such as images mapped read-only everywhere or snapshots.

- compression_hint=none - Don't set compression hints (since 5.8, default).

- compression_hint=compressible - Hint to the underlying OSD object store backend that the data is compressible, enabling compression in passive mode (since 5.8).

- compression_hint=incompressible - Hint to the underlying OSD object store backend that the data is incompressible, disabling compression in aggressive mode (since 5.8).

- ms_mode=legacy - Use msgr1 on-the-wire protocol (since 5.11, default).

- ms_mode=crc - Use msgr2.1 on-the-wire protocol, select 'crc' mode, also referred to as plain mode (since 5.11). If the daemon denies 'crc' mode, fail the connection.

- ms_mode=secure - Use msgr2.1 on-the-wire protocol, select 'secure' mode (since 5.11). 'secure' mode provides full in-transit encryption ensuring both confidentiality and authenticity. If the daemon denies 'secure' mode, fail the connection.

- ms_mode=prefer-crc - Use msgr2.1 on-the-wire protocol, select 'crc' mode (since 5.11). If the daemon denies 'crc' mode in favor of 'secure' mode, agree to 'secure' mode.

- ms_mode=prefer-secure - Use msgr2.1 on-the-wire protocol, select 'secure' mode (since 5.11). If the daemon denies 'secure' mode in favor of 'crc' mode, agree to 'crc' mode.

- rxbounce - Use a bounce buffer when receiving data (since 5.17). The default behaviour is to read directly into the destination buffer. A bounce buffer is needed if the destination buffer isn't guaranteed to be stable (i.e. remain unchanged while it is being read to). In particular this is the case for Windows where a system-wide "dummy" (throwaway) page may be mapped into the destination buffer in order to generate a single large I/O. Otherwise, "libceph: ... bad crc/signature" or "libceph: ... integrity error, bad crc" errors and associated performance degradation are expected.

- udev - Wait for udev device manager to finish executing all matching "add" rules and release the device before exiting (default). This option is not passed to the kernel.

- noudev - Don't wait for udev device manager. When enabled, the device may not be fully usable immediately on exit.

rbd device unmap options:

- force - Force the unmapping of a block device that is open (since 4.9). The driver will wait for running requests to complete and then unmap; requests sent to the driver after initiating the unmap will be failed.

- udev - Wait for udev device manager to finish executing all matching "remove" rules and clean up after the device before exiting (default). This option is not passed to the kernel.

- noudev - Don't wait for udev device manager.

## EXAMPLES
To create a new rbd image that is 100 GB:

```
rbd create mypool/myimage --size 102400
```

To use a non-default object size (8 MB):

```
rbd create mypool/myimage --size 102400 --object-size 8M
```

To delete an rbd image (be careful!):

```
rbd rm mypool/myimage
```

To create a new snapshot:

```
rbd snap create mypool/myimage@mysnap
```

To create a copy-on-write clone of a protected snapshot:

```
rbd clone mypool/myimage@mysnap otherpool/cloneimage
```

To see which clones of a snapshot exist:

```
rbd children mypool/myimage@mysnap
```

To delete a snapshot:

```
rbd snap rm mypool/myimage@mysnap
```

To map an image via the kernel with cephx enabled:

```
rbd device map mypool/myimage --id admin --keyfile secretfile
```

To map an image via the kernel with different cluster name other than default ceph:

```
rbd device map mypool/myimage --cluster cluster-name
```

To unmap an image:

```
rbd device unmap /dev/rbd0
```

To create an image and a clone from it:

```
rbd import --image-format 2 image mypool/parent
rbd snap create mypool/parent@snap
rbd snap protect mypool/parent@snap
rbd clone mypool/parent@snap otherpool/child
```

To create an image with a smaller stripe_unit (to better distribute small writes in some workloads):

```
rbd create mypool/myimage --size 102400 --stripe-unit 65536B --stripe-count 16
```

To change an image from one image format to another, export it and then import it as the desired image format:

```
rbd export mypool/myimage@snap /tmp/img
rbd import --image-format 2 /tmp/img mypool/myimage2
```

To lock an image for exclusive use:

```
rbd lock add mypool/myimage mylockid
```

To release a lock:

```
rbd lock remove mypool/myimage mylockid client.2485
```

To list images from trash:

```
rbd trash ls mypool
```

To defer delete an image (use --expires-at to set expiration time, default is now):

```
rbd trash mv mypool/myimage --expires-at "tomorrow"
```

To delete an image from trash (be careful!):

```
rbd trash rm mypool/myimage-id
```

To force delete an image from trash (be careful!):

```
rbd trash rm mypool/myimage-id  --force
```

To restore an image from trash:

```
rbd trash restore mypool/myimage-id
```

To restore an image from trash and rename it:

```
rbd trash restore mypool/myimage-id --image mynewimage
```

## AVAILABILITY
rbd is part of Ceph, a massively scalable, open-source, distributed storage system. Please refer to the Ceph documentation at https://docs.ceph.com for more information.

## SEE ALSO
ceph(8), rados(8)

## COPYRIGHT
2010-2024, Inktank Storage, Inc. and contributors. Licensed under Creative Commons Attribution Share Alike 3.0 (CC-BY-SA-3.0)


rbd-replay

title: rbd-replay(8)

text:
# rbd-replay(8)

## NAME
rbd-replay - replay rados block device (RBD) workloads

## SYNOPSIS

```
rbd-replay [ options ] replay_file
```

## DESCRIPTION
rbd-replay is a utility for replaying rados block device (RBD) workloads.

## OPTIONS
-c ceph.conf, --conf ceph.conf

Use ceph.conf configuration file instead of the default /etc/ceph/ceph.conf to determine monitor addresses during startup.

-p pool, --pool pool

Interact with the given pool. Defaults to 'rbd'.

--latency-multiplier

Multiplies inter-request latencies. Default: 1.

--read-only

Only replay non-destructive requests.

--map-image rule

Add a rule to map image names in the trace to image names in the replay cluster. A rule of image1@snap1=image2@snap2 would map snap1 of image1 to snap2 of image2.

--dump-perf-counters

Experimental Dump performance counters to standard out before an image is closed. Performance counters may be dumped multiple times if multiple images are closed, or if the same image is opened and closed multiple times. Performance counters and their meaning may change between versions.

## EXAMPLES
To replay workload1 as fast as possible:

```
rbd-replay --latency-multiplier=0 workload1
```

To replay workload1 but use test_image instead of prod_image:

```
rbd-replay --map-image=prod_image=test_image workload1
```

## AVAILABILITY
rbd-replay is part of Ceph, a massively scalable, open-source, distributed storage system. Please refer to the Ceph documentation at https://docs.ceph.com for more information.

## SEE ALSO
rbd-replay-prep(8), rbd(8)

## COPYRIGHT
2010-2024, Inktank Storage, Inc. and contributors. Licensed under Creative Commons Attribution Share Alike 3.0 (CC-BY-SA-3.0)


rbd-replay-many

title: rbd-replay-many(8)

text:
# rbd-replay-many(8)

## NAME
rbd-replay-many - replay a rados block device (RBD) workload on several clients

## SYNOPSIS

```
rbd-replay-many [ options ] --original-image name host1 [ host2 [ ... ] ] -- rbd_replay_args
```

## DESCRIPTION
rbd-replay-many is a utility for replaying a rados block device (RBD) workload on several clients. Although all clients use the same workload, they replay against separate images. This matches normal use of librbd, where each original client is a VM with its own image.

Configuration and replay files are not automatically copied to clients. Replay images must already exist.

## OPTIONS
--original-image name

Specifies the name (and snap) of the originally traced image. Necessary for correct name mapping.

--image-prefix prefix

Prefix of image names to replay against. Specifying --image-prefix=foo results in clients replaying against foo-0, foo-1, etc. Defaults to the original image name.

--exec program

Path to the rbd-replay executable.

--delay seconds

Delay between starting each client. Defaults to 0.

## EXAMPLES
Typical usage:

```
rbd-replay-many host-0 host-1 --original-image=image -- -c ceph.conf replay.bin
```

This results in the following commands being executed:

```
ssh host-0 'rbd-replay' --map-image 'image=image-0' -c ceph.conf replay.bin
ssh host-1 'rbd-replay' --map-image 'image=image-1' -c ceph.conf replay.bin
```

## AVAILABILITY
rbd-replay-many is part of Ceph, a massively scalable, open-source, distributed storage system. Please refer to the Ceph documentation at https://docs.ceph.com for more information.

## SEE ALSO
rbd-replay(8), rbd(8)

## COPYRIGHT
2010-2024, Inktank Storage, Inc. and contributors. Licensed under Creative Commons Attribution Share Alike 3.0 (CC-BY-SA-3.0)


rbd-replay-prep

title: rbd-replay-prep(8)

text:
# rbd-replay-prep(8)

## NAME
rbd-replay-prep - prepare captured rados block device (RBD) workloads for replay

## SYNOPSIS

```
rbd-replay-prep [ --window seconds ] [ --anonymize ] trace_dir replay_file
```

## DESCRIPTION
rbd-replay-prep processes raw rados block device (RBD) traces to prepare them for rbd-replay.

## OPTIONS
--window seconds

Requests further apart than 'seconds' seconds are assumed to be independent.

--anonymize

Anonymizes image and snap names.

--verbose

Print all processed events to console

## EXAMPLES
To prepare workload1-trace for replay:

```
rbd-replay-prep workload1-trace/ust/uid/1000/64-bit workload1
```

## AVAILABILITY
rbd-replay-prep is part of Ceph, a massively scalable, open-source, distributed storage system. Please refer to the Ceph documentation at https://docs.ceph.com for more information.

## SEE ALSO
rbd-replay(8), rbd(8)

## COPYRIGHT
2010-2024, Inktank Storage, Inc. and contributors. Licensed under Creative Commons Attribution Share Alike 3.0 (CC-BY-SA-3.0)


rbdmap

title: rbdmap(8)

summary: map RBD devices at boot time

text:
# rbdmap(8)

## NAME
rbdmap - map RBD devices at boot time

## SYNOPSIS

```
rbdmap map
rbdmap unmap
```

## DESCRIPTION
rbdmap is a shell script that automates rbd map and rbd unmap operations on one or more RBD (RADOS Block Device) images. While the script can be run manually by the system administrator at any time, the principal use case is automatic mapping/mounting of RBD images at boot time (and unmounting/unmapping at shutdown), as triggered by the init system (a systemd unit file, rbdmap.service is included with the ceph-common package for this purpose).

The script takes a single argument, which can be either "map" or "unmap". In either case, the script parses a configuration file (defaults to /etc/ceph/rbdmap, but can be overridden via an environment variable RBDMAPFILE). Each line of the configuration file corresponds to an RBD image which is to be mapped, or unmapped.

The configuration file format is:

```
IMAGESPEC RBDOPTS
```

where IMAGESPEC should be specified as POOLNAME/IMAGENAME (the pool name, a forward slash, and the image name), or merely IMAGENAME, in which case the POOLNAME defaults to "rbd". RBDOPTS is an optional list of parameters to be passed to the underlying rbd map command. These parameters and their values should be specified as a comma-separated string:

```
PARAM1=VAL1,PARAM2=VAL2,...,PARAMN=VALN
```

This will cause the script to issue an rbd map command like the following:

```
rbd map POOLNAME/IMAGENAME --PARAM1 VAL1 --PARAM2 VAL2
```

(See the rbd manpage for a full list of possible options.) For parameters and values which contain commas or equality signs, a simple apostrophe can be used to prevent replacing them.

When run as rbdmap map, the script parses the configuration file, and for each RBD image specified attempts to first map the image (using the rbd map command) and, second, to mount the image.

When run as rbdmap unmap, images listed in the configuration file will be unmounted and unmapped.

rbdmap unmap-all attempts to unmount and subsequently unmap all currently mapped RBD images, regardless of whether or not they are listed in the configuration file.

If successful, the rbd map operation maps the image to a /dev/rbdX device, at which point a udev rule is triggered to create a friendly device name symlink, /dev/rbd/POOLNAME/IMAGENAME, pointing to the real mapped device.

In order for mounting/unmounting to succeed, the friendly device name must have a corresponding entry in /etc/fstab.

When writing /etc/fstab entries for RBD images, it's a good idea to specify the "noauto" (or "nofail") mount option. This prevents the init system from trying to mount the device too early - before the device in question even exists. (Since rbdmap.service executes a shell script, it is typically triggered quite late in the boot sequence.)

## EXAMPLES
Example /etc/ceph/rbdmap for three RBD images called "bar1", "bar2" and "bar3", which are in pool "foopool":

```
foopool/bar1    id=admin,keyring=/etc/ceph/ceph.client.admin.keyring
foopool/bar2    id=admin,keyring=/etc/ceph/ceph.client.admin.keyring
foopool/bar3    id=admin,keyring=/etc/ceph/ceph.client.admin.keyring,options='lock_on_read,queue_depth=1024'
```

Each line in the file contains two strings: the image spec and the options to be passed to rbd map. These two lines get transformed into the following commands:

```
rbd map foopool/bar1 --id admin --keyring /etc/ceph/ceph.client.admin.keyring
rbd map foopool/bar2 --id admin --keyring /etc/ceph/ceph.client.admin.keyring
rbd map foopool/bar2 --id admin --keyring /etc/ceph/ceph.client.admin.keyring --options lock_on_read,queue_depth=1024
```

If the images had XFS file systems on them, the corresponding /etc/fstab entries might look like this:

```
/dev/rbd/foopool/bar1 /mnt/bar1 xfs noauto 0 0
/dev/rbd/foopool/bar2 /mnt/bar2 xfs noauto 0 0
/dev/rbd/foopool/bar3 /mnt/bar3 xfs noauto 0 0
```

After creating the images and populating the /etc/ceph/rbdmap file, making the images get automatically mapped and mounted at boot is just a matter of enabling that unit:

```
systemctl enable rbdmap.service
```

## OPTIONS
None

## AVAILABILITY
rbdmap is part of Ceph, a massively scalable, open-source, distributed storage system. Please refer to the Ceph documentation at https://docs.ceph.com for more information.

## SEE ALSO
rbd(8),

## COPYRIGHT
2010-2024, Inktank Storage, Inc. and contributors. Licensed under Creative Commons Attribution Share Alike 3.0 (CC-BY-SA-3.0)


ceph-fuse

package: ceph-fuse

title: ceph-fuse(8)

summary: FUSE-based client for ceph

text:
# ceph-fuse(8)

## NAME
ceph-fuse - FUSE-based client for ceph

## SYNOPSIS

```
ceph-fuse [-n client.username] [ -m monaddr:port ] mountpoint [ fuse options ]
```

## DESCRIPTION
ceph-fuse is a FUSE ("Filesystem in USErspace") client for Ceph distributed file system. It will mount a ceph file system specified via the -m option or described by ceph.conf (see below) at the specific mount point. See Mount CephFS using FUSE for detailed information.

The file system can be unmounted with:

```
fusermount -u mountpoint
```

or by sending SIGINT to the ceph-fuse process.

## OPTIONS
Any options not recognized by ceph-fuse will be passed on to libfuse.

-o opt,[opt...]

Mount options.

-c ceph.conf, --conf=ceph.conf

Use ceph.conf configuration file instead of the default /etc/ceph/ceph.conf to determine monitor addresses during startup.

-m monaddress[:port]

Connect to specified monitor (instead of looking through ceph.conf).

-n client.{cephx-username}

Pass the name of CephX user whose secret key is be to used for mounting.

--id <client-id>

Pass the name of CephX user whose secret key is be to used for mounting. --id takes just the ID of the client in contrast to -n. For example, --id 0 for using client.0.

-k <path-to-keyring>

Provide path to keyring; useful when it's absent in standard locations.

--client_mountpoint/-r root_directory

Use root_directory as the mounted root, rather than the full Ceph tree.

-f

Foreground: do not daemonize after startup (run in foreground). Do not generate a pid file.

-d

Run in foreground, send all log output to stderr and enable FUSE debugging (-o debug).

-s

Disable multi-threaded operation.

--client_fs

Pass the name of Ceph FS to be mounted. Not passing this option mounts the default Ceph FS on the Ceph cluster.

## AVAILABILITY
ceph-fuse is part of Ceph, a massively scalable, open-source, distributed storage system. Please refer to the Ceph documentation at https://docs.ceph.com for more information.

## SEE ALSO
fusermount(8), ceph(8)

## COPYRIGHT
2010-2024, Inktank Storage, Inc. and contributors. Licensed under Creative Commons Attribution Share Alike 3.0 (CC-BY-SA-3.0)


mount.fuse.ceph

title: mount.fuse.ceph(8)

summary: mount ceph-fuse from /etc/fstab.

text:
# mount.fuse.ceph(8)

## NAME
mount.fuse.ceph - mount ceph-fuse from /etc/fstab.

## SYNOPSIS

```
mount.fuse.ceph [-h] [-o OPTIONS [OPTIONS ...]]
device [device ...]
mountpoint [mountpoint ...]
```

## DESCRIPTION
mount.fuse.ceph is a helper for mounting ceph-fuse from /etc/fstab.

To use mount.fuse.ceph, add an entry in /etc/fstab like:

```
DEVICE    PATH        TYPE        OPTIONS
none      /mnt/ceph   fuse.ceph   ceph.id=admin,_netdev,defaults  0 0
none      /mnt/ceph   fuse.ceph   ceph.name=client.admin,_netdev,defaults  0 0
none      /mnt/ceph   fuse.ceph   ceph.id=myuser,ceph.conf=/etc/ceph/foo.conf,_netdev,defaults  0 0
```

ceph-fuse options are specified in the OPTIONS column and must begin with 'ceph.' prefix. This way ceph related fs options will be passed to ceph-fuse and others will be ignored by ceph-fuse.

## OPTIONS
ceph.id=<username>

Specify that the ceph-fuse will authenticate as the given user.

ceph.name=client.admin

Specify that the ceph-fuse will authenticate as client.admin

ceph.conf=/etc/ceph/foo.conf

Sets 'conf' option to /etc/ceph/foo.conf via ceph-fuse command line.

Any valid ceph-fuse options can be passed this way.

## ADDITIONAL INFO
The old format /etc/fstab entries are also supported:

```
DEVICE                              PATH        TYPE        OPTIONS
id=admin                            /mnt/ceph   fuse.ceph   defaults   0 0
id=myuser,conf=/etc/ceph/foo.conf   /mnt/ceph   fuse.ceph   defaults   0 0
```

## AVAILABILITY
mount.fuse.ceph is part of Ceph, a massively scalable, open-source, distributed storage system. Please refer to the Ceph documentation at https://docs.ceph.com for more information.

## SEE ALSO
ceph-fuse(8), ceph(8)

## COPYRIGHT
2010-2024, Inktank Storage, Inc. and contributors. Licensed under Creative Commons Attribution Share Alike 3.0 (CC-BY-SA-3.0)


ceph-immutable-object-cache

package: ceph-immutable-object-cache

title: ceph-immutable-object-cache(8)

summary: Ceph daemon for immutable object cache

text:
# ceph-immutable-object-cache(8)

## NAME
ceph-immutable-object-cache - Ceph daemon for immutable object cache

## SYNOPSIS

```
ceph-immutable-object-cache
```

## DESCRIPTION
ceph-immutable-object-cache is a daemon for object cache of RADOS objects among Ceph clusters. It will promote the objects to a local directory upon promote requests and future reads will be serviced from these cached objects.

It connects to local clusters via the RADOS protocol, relying on default search paths to find ceph.conf files, monitor addresses and authentication information for them, i.e. /etc/ceph/$cluster.conf, /etc/ceph/$cluster.keyring, and /etc/ceph/$cluster.$name.keyring, where $cluster is the human-friendly name of the cluster, and $name is the rados user to connect as, e.g. client.ceph-immutable-object-cache.

## OPTIONS
-c ceph.conf, --conf=ceph.conf

Use ceph.conf configuration file instead of the default /etc/ceph/ceph.conf to determine monitor addresses during startup.

-m monaddress[:port]

Connect to specified monitor (instead of looking through ceph.conf).

-i ID, --id ID

Set the ID portion of name for ceph-immutable-object-cache

-n TYPE.ID, --name TYPE.ID

Set the rados user name for the gateway (eg. client.ceph-immutable-object-cache)

--cluster NAME

Set the cluster name (default: ceph)

-d

Run in foreground, log to stderr

-f

Run in foreground, log to usual location

## AVAILABILITY
ceph-immutable-object-cache is part of Ceph, a massively scalable, open-source, distributed storage system. Please refer to the Ceph documentation at https://docs.ceph.com for more information.

## SEE ALSO
rbd(8)

## COPYRIGHT
2010-2024, Inktank Storage, Inc. and contributors. Licensed under Creative Commons Attribution Share Alike 3.0 (CC-BY-SA-3.0)


ceph-mds

package: ceph-mds

title: ceph-mds(8)

summary: ceph metadata server daemon

text:
# ceph-mds(8)

## NAME
ceph-mds - ceph metadata server daemon

## SYNOPSIS

```
ceph-mds -i <ID> [flags]
```

## DESCRIPTION
ceph-mds is the metadata server daemon for the Ceph distributed file system. One or more instances of ceph-mds collectively manage the file system namespace, coordinating access to the shared OSD cluster.

Each ceph-mds daemon instance should have a unique name. The name is used to identify daemon instances in the ceph.conf.

Once the daemon has started, the monitor cluster will normally assign it a logical rank, or put it in a standby pool to take over for another daemon that crashes. Some of the specified options can cause other behaviors.

## OPTIONS
-f, --foreground

Foreground: do not daemonize after startup (run in foreground). Do not generate a pid file. Useful when run via ceph-run(8).

-d

Debug mode: like -f, but also send all log output to stderr.

--setuser userorgid

Set uid after starting. If a username is specified, the user record is looked up to get a uid and a gid, and the gid is also set as well, unless --setgroup is also specified.

--setgroup grouporgid

Set gid after starting. If a group name is specified the group record is looked up to get a gid.

-c ceph.conf, --conf=ceph.conf

Use ceph.conf configuration file instead of the default /etc/ceph/ceph.conf to determine monitor addresses during startup.

-m monaddress[:port]

Connect to specified monitor (instead of looking through ceph.conf).

--id/-i ID

Set ID portion of the MDS name.

## AVAILABILITY
ceph-mds is part of Ceph, a massively scalable, open-source, distributed storage system. Please refer to the Ceph documentation at https://docs.ceph.com for more information.

## SEE ALSO
ceph(8), ceph-mon(8), ceph-osd(8)

## COPYRIGHT
2010-2024, Inktank Storage, Inc. and contributors. Licensed under Creative Commons Attribution Share Alike 3.0 (CC-BY-SA-3.0)


ceph-mon

package: ceph-mon

title: ceph-mon(8)

summary: ceph monitor daemon

text:
# ceph-mon(8)

## NAME
ceph-mon - ceph monitor daemon

## SYNOPSIS

```
ceph-mon -i monid [ --mon-data mondatapath ]
```

## DESCRIPTION
ceph-mon is the cluster monitor daemon for the Ceph distributed file system. One or more instances of ceph-mon form a Paxos part-time parliament cluster that provides extremely reliable and durable storage of cluster membership, configuration, and state.

The mondatapath refers to a directory on a local file system storing monitor data. It is normally specified via the mon data option in the configuration file.

## OPTIONS
-f, --foreground

Foreground: do not daemonize after startup (run in foreground). Do not generate a pid file. Useful when run via ceph-run(8).

-d

Debug mode: like -f, but also send all log output to stderr.

--setuser userorgid

Set uid after starting. If a username is specified, the user record is looked up to get a uid and a gid, and the gid is also set as well, unless --setgroup is also specified.

--setgroup grouporgid

Set gid after starting. If a group name is specified the group record is looked up to get a gid.

-c ceph.conf, --conf=ceph.conf

Use ceph.conf configuration file instead of the default /etc/ceph/ceph.conf to determine monitor addresses during startup.

--mkfs

Initialize the mon data directory with seed information to form and initial ceph file system or to join an existing monitor cluster. Three pieces of information must be provided:

- The cluster fsid. This can come from a monmap (--monmap <path>) or explicitly via --fsid <uuid>.

- A list of monitors and their addresses. This list of monitors can come from a monmap (--monmap <path>), the mon host configuration value (in ceph.conf or via -m host1,host2,...), or (for backward compatibility) the deprecated mon addr lines in ceph.conf. If this monitor is to be part of the initial monitor quorum for a new Ceph cluster, then it must be included in the initial list, matching either the name or address of a monitor in the list. When matching by address, either the public addr or public subnet options may be used.

- The monitor secret key mon.. This must be included in the keyring provided via --keyring <path>.

--keyring

Specify a keyring for use with --mkfs.

--no-config-file

Signal that we don't want to rely on a ceph.conf, either user provided or the default, to run the daemon. This will entail providing all necessary options to the daemon as arguments.

## AVAILABILITY
ceph-mon is part of Ceph, a massively scalable, open-source, distributed storage system. Please refer to the Ceph documentation at https://docs.ceph.com for more information.

## SEE ALSO
ceph(8), ceph-mds(8), ceph-osd(8)

## COPYRIGHT
2010-2024, Inktank Storage, Inc. and contributors. Licensed under Creative Commons Attribution Share Alike 3.0 (CC-BY-SA-3.0)