title: cciss_vol_status(8)

text:
# cciss_vol_status(8)

## NAME
cciss_vol_status - show status of logical drives attached to HP Smartarray controllers

## SYNOPSIS
cciss_vol_status [OPTION] [DEVICE]...

## DESCRIPTION
Shows the status of logical drives configured on HP Smartarray controllers.

## OPTIONS
-p, --persnickety

Without this option, device nodes which can't be opened, or which are not found to be of the correct device type are silently ignored. This lets you use wildcards, e.g.: cciss_vol_status /dev/sg* /dev/cciss/c*d0, and the program will not complain as long as all devices which are found to be of the correct type are found to be ok. However, you may wish to explicitly list the devices you expect to be there, and be notified if they are not there (e.g. perhaps a PCI slot has died, and the system has rebooted, so that what was once /dev/cciss/c1d0 is no longer there at all). This option will cause the program to complain about any device node listed which does not appear to be the right device type, or is not openable.

-C, --copyright

If stderr is a terminal, Print out a copyright message, and exit.

-q, --quiet

This option doesn't do anything. Previously, without this option and if stderr is a terminal, a copyright message precedes the normal program output. Now, the copyright message is only printed via the -C option.

-s

Query each physical drive for S.M.A.R.T data and report any drives in "predictive failure" state.

-u, --try-unknown-devices

If a device has an unrecognized board ID, normally the program will not attempt to communicate with it. In case you have some Smart Array controller which is newer than this program, the program may not recognize it. This option permits the program to attempt to interrogate the board even if it is unrecognized on the assumption that it is in fact a Smart Array of some kind.

-v, --version

Print the version number and exit.

-x, --exhaustive

Deprecated. Previously, it "exhaustively" searched for logical drives, as, under some circumstances some logical drives might otherwise be missed. This option no longer does anything, as the algorithm for finding logical drives was changed to obviate the need for it.

## DEVICE
The DEVICE argument indicates which RAID controller is to be queried. Note, that it indicates which RAID controller, not which logical drive.

For the cciss driver, the "d0" nodes matching "/dev/cciss/c*d0" are the nodes which correspond to the RAID controllers. (See note 1, below.) It is not necessary to invoke cciss_vol_status on each logical drive individually, though if you do this, each time it will report the status of ALL logical drives on the controller.

For the hpsa driver, or for fibre attached MSA1000 family devices, or for the hpahcisr sotware RAID driver which emulates Smart Arrays, the RAID controller is accessed via the scsi generic driver, and the device nodes will match "/dev/sg*" Some variants of the "lsscsi" tool will easily identify which device node corresponds to the RAID controller. Some variants may only report the SCSI nexus (controller/bus/target/lun tuple.) Some distros may not have the lsscsi tool.

Executing the following query to the /sys filesystem and correlating this with the contents of /proc/scsi/scsi or output of lsscsi can help in finding the right /dev/sg node to use with cciss_vol_status:

```
wumpus:/home/scameron # ls -l /sys/class/scsi_generic/*
lrwxrwxrwx 1 root root 0 2009-11-18 12:31 /sys/class/scsi_generic/sg0 -> ../../devices/pci0000:00/0000:00:02.0/0000:02:00.0/0000:03:03.0/host0/target0:0:0/0:0:0:0/scsi_generic/sg0
lrwxrwxrwx 1 root root 0 2009-11-18 12:31 /sys/class/scsi_generic/sg1 -> ../../devices/pci0000:00/0000:00:1f.1/host2/target2:0:0/2:0:0:0/scsi_generic/sg1
lrwxrwxrwx 1 root root 0 2009-11-19 07:47 /sys/class/scsi_generic/sg2 -> ../../devices/pci0000:00/0000:00:05.0/0000:0e:00.0/host4/target4:3:0/4:3:0:0/scsi_generic/sg2
wumpus:/home/scameron # cat /proc/scsi/scsi
Attached devices:
Host: scsi0 Channel: 00 Id: 00 Lun: 00

  Vendor: COMPAQ   Model: BD03685A24       Rev: HPB6

  Type:   Direct-Access                    ANSI  SCSI revision: 03
Host: scsi2 Channel: 00 Id: 00 Lun: 00

  Vendor: SAMSUNG  Model: CD-ROM SC-148A   Rev: B408

  Type:   CD-ROM                           ANSI  SCSI revision: 05
Host: scsi4 Channel: 03 Id: 00 Lun: 00

  Vendor: HP       Model: P800             Rev: 6.82

  Type:   RAID                             ANSI  SCSI revision: 00
wumpus:/home/scameron # lsscsi
[0:0:0:0]    disk    COMPAQ   BD03685A24       HPB6  /dev/sda
[2:0:0:0]    cd/dvd  SAMSUNG  CD-ROM SC-148A   B408  /dev/sr0
[4:3:0:0]    storage HP       P800             6.82  -
```

From the above you can see that /dev/sg2 corresponds to SCSI nexus 4:3:0:0, which corresponds to the HP P800 RAID controller listed in /proc/scsi/scsi.

## EXAMPLE

```
	[root@somehost]# cciss_vol_status -q /dev/cciss/c*d0
	/dev/cciss/c0d0: (Smart Array P800) RAID 0 Volume 0 status: OK.
	/dev/cciss/c0d0: (Smart Array P800) RAID 0 Volume 1 status: OK.
	/dev/cciss/c0d0: (Smart Array P800) RAID 1 Volume 2 status: OK.
	/dev/cciss/c0d0: (Smart Array P800) RAID 5 Volume 4 status: OK.
	/dev/cciss/c0d0: (Smart Array P800) RAID 5 Volume 5 status: OK.
	/dev/cciss/c0d0: (Smart Array P800) Enclosure MSA60 (S/N: USP6340B3F) on Bus 2, Physical Port 1E status: Power Supply Unit failed
	/dev/cciss/c1d0: (Smart Array P800) RAID 5 Volume 0 status: OK.
	/dev/cciss/c1d0: (Smart Array P800) RAID 5 Volume 1 status: OK.
	/dev/cciss/c1d0: (Smart Array P800) RAID 5 Volume 2 status: OK.
	/dev/cciss/c1d0: (Smart Array P800) RAID 5 Volume 3 status: OK.
	/dev/cciss/c1d0: (Smart Array P800) RAID 5 Volume 4 status: OK.
	/dev/cciss/c1d0: (Smart Array P800) RAID 5 Volume 5 status: OK.
	/dev/cciss/c1d0: (Smart Array P800) RAID 5 Volume 6 status: OK.
	/dev/cciss/c1d0: (Smart Array P800) RAID 5 Volume 7 status: OK.
	[root@someotherhost]# cciss_vol_status -q /dev/sg0 /dev/cciss/c*d0
	/dev/sg0: (MSA1000) RAID 1 Volume 0 status: OK.   At least one spare drive.
	/dev/sg0: (MSA1000) RAID 5 Volume 1 status: OK.
	/dev/cciss/c0d0: (Smart Array P800) RAID 0 Volume 0 status: OK.
	[root@localhost]# ./cciss_vol_status -s /dev/sg1
	/dev/sda: (Smart Array P410i) RAID 0 Volume 0 status: OK.
		 connector 1I box 1 bay 1                 HP      DG072A9BB7                               B365P6803PCP0633     HPD0 S.M.A.R.T. predictive failure.
	[root@localhost]# echo $?
	1
	[root@localhost]# ./cciss_vol_status -s /dev/cciss/c0d0
	/dev/cciss/c0d0: (Smart Array P800) RAID 0 Volume 0 status: OK.
		 connector 2E box 1 bay 8                 HP      DF300BB6C3                           3LM08AP700009713RXUT     HPD3 S.M.A.R.T. predictive failure.
	/dev/cciss/c0d0: (Smart Array P800) Enclosure MSA60 (S/N: USP6340B3F) on Bus 2, Physical Port 2E status: OK.
```

## DIAGNOSTICS
Normally, a logical drive in good working order should report a status of "OK." Possible status values are:

"OK." (0) - The logical drive is in good working order.

"FAILED." (1) - The logical drive has failed, and no i/o to it is poosible.

Additionally, failed drives will be identified by connector, box and bay, as well as vendor, model, serial number, and firmware revision.

"Using interim recovery mode." (3) - One or more drives has failed,

but not so many that the logical drive can no longer operate. The failed drives should be replaced as soon as possible.

"Ready for recovery operation." (4) - Failed drive(s) have been

replaced, and the controller is about to begin rebuilding redundant parity data.

"Currently recovering." (5) - Failed drive(s) have been replaced,

and the controller is currently rebuilding redundant parity information.

"Wrong physical drive was replaced." (6) - A drive has failed, and

another (working) drive was replaced.

"A physical drive is not properly connected." (7) - There is some

cabling or backplane problem in the drive enclosure.

(From fwspecwww.doc, see cpqarray project on sourceforge.net):

Note: If the unit_status value is 6 (Wrong physical drive was replaced) or 7 (A physical drive is not properly connected), the unit_status of all other configured logical drives will be marked as 1 (Logical drive failed). This is to force the user to correct the problem and to insure that once the problem is corrected, the data will not have been corrupted by any user action.

"Hardware is overheating." (8) - Hardware is too hot.

"Hardware was overheated." (9) - At some point in the past,

the hardware got too hot.

"Currently expannding." (10) - The controller is currently in the

process of expanding a logical drive.

"Not yet available." (11) - The logical drive is not yet finished

being configured.

"Queued for expansion." (12) - The logical drive will be expended

when the controller is able to begin working on it.

Additionally, the following messages may appear regarding spare drive status:

```
	"At least one spare drive designated"
	"At least one spare drive activated and currently rebuilding"
	"At least one activated on-line spare drive is completely rebuilt on this logical drive"
	"At least one spare drive has failed"
	"At least one spare drive activated"
	"At least one spare drive remains available"
Active spares will be identified by connector, box and bay, as well
as by vendor, model, serial number, and firmware revision.
```

For each logical drive, the total number of failed physical drives, if more than zero, will be reported as:

"Total of n failed physical drives detected on this logical drive."

```

```

with "n" replaced by the actual number, of course.

"Replacement" drives -- newly inserted drives that replace a previously failed drive but are not yet finished rebuilding -- are also identified by connector, box and bay, as well as by vendor, model, serial number, and firmware revision.

If the -s option is specified, each physical drive will be queried for S.M.A.R.T data, any any drives in predictive failure state will be reported, identified by connector, box and bay, as well as vendor, model, serial number, and firmware revision.

Additionally failure conditions of disk enclosure fans, power supplies, and temperature are reported as follows:

```
	"Fan failed"
	"Temperature problem"
	"Door alert"
	"Power Supply Unit failed"
```

## FILES
/dev/cciss/c*d0 (Smart Array PCI controllers using the cciss driver)

/dev/sg* (Fibre attached MSA1000 controllers and Smart Array controllers using the hpsa driver or hpahcisr software RAID driver.)

## EXIT CODES
0 - All configured logical drives queried have status of "OK."

1 - One or more configured logical drives queried have status other than "OK."

## AUTHOR
Written by Stephen M. Cameron

## REPORTING BUGS
MSA500 G1 logical drive numbers may not be reported correctly.

I've seen enclosure serial numbers contain garbage.

Report bugs to <steve.cameron@hp.com>

## COPYRIGHT
Copyright © 2007 Hewlett-Packard Development Company, L.P.

This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

## SEE ALSO
http://cciss.sourceforge.net

## note 1
The /dev/cciss/c*d0 device nodes of the cciss driver do double duty. They serve as an access point to both the RAID controllers, and to the first logical drive of each RAID controller. Notice that a /dev/cciss/c*d0 node will be present for each controller even if no logical drives are configured on that controller. It might be cleaner if the driver had a special device node just for the controller, instead of making these device nodes do double duty. It has been like that since the 2.2 linux kernel timeframe. At that time, device major and minor nodes were statically allocated at compile time, and were in short supply. Changing this behavior at this point would break lots of userland programs.


cdemu-daemon

package: cdemu-daemon

title: cdemu-daemon(8)

summary: The CDEmu daemon

text:
# cdemu-daemon(8)

## NAME
cdemud - The CDEmu daemon

## SYNOPSIS
cdemu-daemon [options]

## DESCRIPTION
This is CDEmu daemon, the userspace part of the cdemu suite, a free, GPL CD/DVD-ROM device emulator for linux.

It receives SCSI commands from kernel module and processes them, passing the requested data back to the kernel. Daemon implements the actual virtual device; one instance per each device registered by kernel module. It uses libMirage, an image access library that is part of the cdemu suite, for the image access (e.g. sector reading).

The daemon registers itself on D-BUS' system or session bus (depending on the options passed to it) where it exposes an interface that can be used by clients to control it.

## OPTIONS
-n --num-devices=N

Number of devices. Specifies the number of virtual devices the daemon instance should register with kernel module and control. By default, 1 virtual device is created.

-c --ctl-device=path

Control device. Specifies the control device path. Control device is a character device provided by kernel module; it is used for communication between kernel and userspace part of CDEmu suite. By default, /dev/vhba_ctl is used.

-a --audio-driver=driver

Audio driver. Specifies audio driver to be used by virtual devices' audio play. Since libao is used for audio playback, any valid libao driver can be specified. If default is used as a driver, then libao's default driver is used. Few examples of valid libao drivers are alsa, oss, pulse or null. For all drivers, default device is used. By default, if audio driver is not explicitly specified via this option, null is used.

-b --bus=bus_type

Bus type. Valid options are system or session. By default, session bus is used. (Please note that the running of CDEmu daemon on system bus is considered deprecated and is discouraged for security reasons).

-l --logfile=logfile

Log file to write logging output into. By default, use of log file is disabled and messages are written to stdout.

-? --help

Display the help message.

## EXAMPLES
Running daemon on default bus with PulseAudio audio driver and 4 virtual devices:

cdemu-daemon --num-devices=4 --audio-driver=pulse

Running daemon on system bus with 2 virtual devices and logging redirected to a file:

cdemu-daemon --num-devices=2 --bus=system --logfile=/tmp/cdemu-daemon.log

## AUTHORS
Rok Mandeljc <rok.mandeljc@gmail.com>

CDEmu project's web page: http://cdemu.sourceforge.net

CDEmu project's mailing list: cdemu-devel@lists.sourceforge.net


ceph-create-keys

package: ceph-base

title: ceph-create-keys(8)

summary: ceph keyring generate tool

text:
# ceph-create-keys(8)

## NAME
ceph-create-keys - ceph keyring generate tool

## SYNOPSIS

```
ceph-create-keys [-h] [-v] [-t seconds] [--cluster name] --id id
```

## DESCRIPTION
ceph-create-keys is a utility to generate bootstrap keyrings using the given monitor when it is ready.

It creates following auth entities (or users)

client.admin

and its key for your client host.

client.bootstrap-{osd, rgw, mds}

and their keys for bootstrapping corresponding services

To list all users in the cluster:

```
ceph auth ls
```

## OPTIONS
--cluster

name of the cluster (default 'ceph').

-t

time out after seconds (default: 600) waiting for a response from the monitor

-i, --id

id of a ceph-mon that is coming up. ceph-create-keys will wait until it joins quorum.

-v, --verbose

be more verbose.

## AVAILABILITY
ceph-create-keys is part of Ceph, a massively scalable, open-source, distributed storage system. Please refer to the Ceph documentation at https://docs.ceph.com for more information.

## SEE ALSO
ceph(8)

## COPYRIGHT
2010-2024, Inktank Storage, Inc. and contributors. Licensed under Creative Commons Attribution Share Alike 3.0 (CC-BY-SA-3.0)


ceph-kvstore-tool

title: ceph-kvstore-tool(8)

summary: ceph kvstore manipulation tool

text:
# ceph-kvstore-tool(8)

## NAME
ceph-kvstore-tool - ceph kvstore manipulation tool

## SYNOPSIS

```
ceph-kvstore-tool <leveldb|rocksdb|bluestore-kv> <store path> command [args...]
```

## DESCRIPTION
ceph-kvstore-tool is a kvstore manipulation tool. It allows users to manipulate leveldb/rocksdb's data (like OSD's omap) offline.

## COMMANDS
ceph-kvstore-tool utility uses many commands for debugging purpose which are as follows:

list [prefix]

Print key of all KV pairs stored with the URL encoded prefix.

list-crc [prefix]

Print CRC of all KV pairs stored with the URL encoded prefix.

dump [prefix]

Print key and value of all KV pairs stored with the URL encoded prefix.

exists <prefix> [key]

Check if there is any KV pair stored with the URL encoded prefix. If key is also specified, check for the key with the prefix instead.

get <prefix> <key> [out <file>]

Get the value of the KV pair stored with the URL encoded prefix and key. If file is also specified, write the value to the file.

crc <prefix> <key>

Get the CRC of the KV pair stored with the URL encoded prefix and key.

get-size [<prefix> <key>]

Get estimated store size or size of value specified by prefix and key.

set <prefix> <key> [ver <N>|in <file>]

Set the value of the KV pair stored with the URL encoded prefix and key. The value could be version_t or text.

rm <prefix> <key>

Remove the KV pair stored with the URL encoded prefix and key.

rm-prefix <prefix>

Remove all KV pairs stored with the URL encoded prefix.

store-copy <path> [num-keys-per-tx]

Copy all KV pairs to another directory specified by path. [num-keys-per-tx] is the number of KV pairs copied for a transaction.

store-crc <path>

Store CRC of all KV pairs to a file specified by path.

compact

Subcommand compact is used to compact all data of kvstore. It will open the database, and trigger a database's compaction. After compaction, some disk space may be released.

compact-prefix <prefix>

Compact all entries specified by the URL encoded prefix.

compact-range <prefix> <start> <end>

Compact some entries specified by the URL encoded prefix and range.

destructive-repair

Make a (potentially destructive) effort to recover a corrupted database. Note that in the case of rocksdb this may corrupt an otherwise uncorrupted database--use this only as a last resort!

stats

Prints statistics from underlying key-value database. This is only for informative purposes. Format and information content may vary between releases. For RocksDB information includes compactions stats, performance counters, memory usage and internal RocksDB stats.

histogram

Presents key-value sizes distribution statistics from the underlying KV database.

## AVAILABILITY
ceph-kvstore-tool is part of Ceph, a massively scalable, open-source, distributed storage system. Please refer to the Ceph documentation at https://docs.ceph.com for more information.

## SEE ALSO
ceph(8)

## COPYRIGHT
2010-2024, Inktank Storage, Inc. and contributors. Licensed under Creative Commons Attribution Share Alike 3.0 (CC-BY-SA-3.0)


ceph-run

title: ceph-run(8)

summary: restart daemon on core dump

text:
# ceph-run(8)

## NAME
ceph-run - restart daemon on core dump

## SYNOPSIS

```
ceph-run command ...
```

## DESCRIPTION
ceph-run is a simple wrapper that will restart a daemon if it exits with a signal indicating it crashed and possibly core dumped (that is, signals 3, 4, 5, 6, 8, or 11).

The command should run the daemon in the foreground. For Ceph daemons, that means the -f option.

## OPTIONS
None

## AVAILABILITY
ceph-run is part of Ceph, a massively scalable, open-source, distributed storage system. Please refer to the Ceph documentation at https://docs.ceph.com for more information.

## SEE ALSO
ceph(8), ceph-mon(8), ceph-mds(8), ceph-osd(8)

## COPYRIGHT
2010-2024, Inktank Storage, Inc. and contributors. Licensed under Creative Commons Attribution Share Alike 3.0 (CC-BY-SA-3.0)


crushtool

title: crushtool(8)

summary: CRUSH map manipulation tool

text:
# crushtool(8)

## NAME
crushtool - CRUSH map manipulation tool

## SYNOPSIS

```
crushtool ( -d map | -c map.txt | --build --num_osds numosds
layer1 ... | --test ) [ -o outfile ]
```

## DESCRIPTION
crushtool is a utility that lets you create, compile, decompile and test CRUSH map files.

CRUSH is a pseudo-random data distribution algorithm that efficiently maps input values (which, in the context of Ceph, correspond to Placement Groups) across a heterogeneous, hierarchically structured device map. The algorithm was originally described in detail in the following paper (although it has evolved some since then):

```
http://www.ssrc.ucsc.edu/Papers/weil-sc06.pdf
```

The tool has four modes of operation.

--compile|-c map.txt

will compile a plaintext map.txt into a binary map file.

--decompile|-d map

will take the compiled map and decompile it into a plaintext source file, suitable for editing.

--build --num_osds {num-osds} layer1 ...

will create map with the given layer structure. See below for a detailed explanation.

--test

will perform a dry run of a CRUSH mapping for a range of input values [--min-x,--max-x] (default [0,1023]) which can be thought of as simulated Placement Groups. See below for a more detailed explanation.

Unlike other Ceph tools, crushtool does not accept generic options such as --debug-crush from the command line. They can, however, be provided via the CEPH_ARGS environment variable. For instance, to silence all output from the CRUSH subsystem:

```
CEPH_ARGS="--debug-crush 0" crushtool ...
```

## RUNNING TESTS WITH --TEST
The test mode will use the input crush map ( as specified with -i map ) and perform a dry run of CRUSH mapping or random placement (if --simulate is set ). On completion, two kinds of reports can be created. 1) The --show-... option outputs human readable information on stderr. 2) The --output-csv option creates CSV files that are documented by the --help-output option.

Note: Each Placement Group (PG) has an integer ID which can be obtained from ceph pg dump (for example PG 2.2f means pool id 2, PG id 32). The pool and PG IDs are combined by a function to get a value which is given to CRUSH to map it to OSDs. crushtool does not know about PGs or pools; it only runs simulations by mapping values in the range [--min-x,--max-x].

--show-statistics

Displays a summary of the distribution. For instance:

```
rule 1 (metadata) num_rep 5 result size == 5:    1024/1024
```

shows that rule 1 which is named metadata successfully mapped 1024 values to result size == 5 devices when trying to map them to num_rep 5 replicas. When it fails to provide the required mapping, presumably because the number of tries must be increased, a breakdown of the failures is displayed. For instance:

```
rule 1 (metadata) num_rep 10 result size == 8:   4/1024
rule 1 (metadata) num_rep 10 result size == 9:   93/1024
rule 1 (metadata) num_rep 10 result size == 10:  927/1024
```

shows that although num_rep 10 replicas were required, 4 out of 1024 values ( 4/1024 ) were mapped to result size == 8 devices only.

--show-mappings

Displays the mapping of each value in the range [--min-x,--max-x]. For instance:

```
CRUSH rule 1 x 24 [11,6]
```

shows that value 24 is mapped to devices [11,6] by rule 1.

One of the following is required when using the --show-mappings option:

a.

--num-rep

b.

both --min-rep and --max-rep

--num-rep stands for "number of replicas, indicates the number of replicas in a pool, and is used to specify an exact number of replicas (for example --num-rep 5). --min-rep and --max-rep are used together to specify a range of replicas (for example, --min-rep 1 --max-rep 10).

--show-bad-mappings

Displays which value failed to be mapped to the required number of devices. For instance:

```
bad mapping rule 1 x 781 num_rep 7 result [8,10,2,11,6,9]
```

shows that when rule 1 was required to map 7 devices, it could map only six : [8,10,2,11,6,9].

--show-utilization

Displays the expected and actual utilization for each device, for each number of replicas. For instance:

```
device 0: stored : 951      expected : 853.333
device 1: stored : 963      expected : 853.333
...
```

shows that device 0 stored 951 values and was expected to store 853. Implies --show-statistics.

--show-utilization-all

Displays the same as --show-utilization but does not suppress output when the weight of a device is zero. Implies --show-statistics.

--show-choose-tries

Displays how many attempts were needed to find a device mapping. For instance:

```
0:     95224
1:      3745
2:      2225
..
```

shows that 95224 mappings succeeded without retries, 3745 mappings succeeded with one attempts, etc. There are as many rows as the value of the --set-choose-total-tries option.

--output-csv

Creates CSV files (in the current directory) containing information documented by --help-output. The files are named after the rule used when collecting the statistics. For instance, if the rule : 'metadata' is used, the CSV files will be:

```
metadata-absolute_weights.csv
metadata-device_utilization.csv
...
```

The first line of the file shortly explains the column layout. For instance:

```
metadata-absolute_weights.csv
Device ID, Absolute Weight
0,1
...
```

--output-name NAME

Prepend NAME to the file names generated when --output-csv is specified. For instance --output-name FOO will create files:

```
FOO-metadata-absolute_weights.csv
FOO-metadata-device_utilization.csv
...
```

The --set-... options can be used to modify the tunables of the input crush map. The input crush map is modified in memory. For example:

```
$ crushtool -i mymap --test --show-bad-mappings
bad mapping rule 1 x 781 num_rep 7 result [8,10,2,11,6,9]
```

could be fixed by increasing the choose-total-tries as follows:

$ crushtool -i mymap --test

--show-bad-mappings --set-choose-total-tries 500

## BUILDING A MAP WITH --BUILD
The build mode will generate hierarchical maps. The first argument specifies the number of devices (leaves) in the CRUSH hierarchy. Each layer describes how the layer (or devices) preceding it should be grouped.

Each layer consists of:

```
bucket ( uniform | list | tree | straw | straw2 ) size
```

The bucket is the type of the buckets in the layer (e.g. "rack"). Each bucket name will be built by appending a unique number to the bucket string (e.g. "rack0", "rack1"...).

The second component is the type of bucket: straw should be used most of the time.

The third component is the maximum size of the bucket. A size of zero means a bucket of infinite capacity.

## EXAMPLE
Suppose we have two rows with two racks each and 20 nodes per rack. Suppose each node contains 4 storage devices for Ceph OSD Daemons. This configuration allows us to deploy 320 Ceph OSD Daemons. Lets assume a 42U rack with 2U nodes, leaving an extra 2U for a rack switch.

To reflect our hierarchy of devices, nodes, racks and rows, we would execute the following:

```
$ crushtool -o crushmap --build --num_osds 320 \

       node straw 4 \

       rack straw 20 \

       row straw 2 \

       root straw 0
# id        weight  type name       reweight
-87 320     root root
-85 160             row row0
-81 80                      rack rack0
-1  4                               node node0
0   1                                       osd.0   1
1   1                                       osd.1   1
2   1                                       osd.2   1
3   1                                       osd.3   1
-2  4                               node node1
4   1                                       osd.4   1
5   1                                       osd.5   1
...
```

CRUSH rules are created so the generated crushmap can be tested. They are the same rules as the ones created by default when creating a new Ceph cluster. They can be further edited with:

```
# decompile
crushtool -d crushmap -o map.txt
# edit
emacs map.txt
# recompile
crushtool -c map.txt -o crushmap
```

## RECLASSIFY
The reclassify function allows users to transition from older maps that maintain parallel hierarchies for OSDs of different types to a modern CRUSH map that makes use of the device class feature. For more information, see https://docs.ceph.com/en/latest/rados/operations/crush-map-edits/#migrating-from-a-legacy-ssd-rule-to-device-classes.

## EXAMPLE OUTPUT FROM --TEST
See https://github.com/ceph/ceph/blob/master/src/test/cli/crushtool/set-choose.t for sample crushtool --test commands and output produced thereby.

## AVAILABILITY
crushtool is part of Ceph, a massively scalable, open-source, distributed storage system. Please refer to the Ceph documentation at https://docs.ceph.com for more information.

## SEE ALSO
ceph(8), osdmaptool(8),

## AUTHORS
John Wilkins, Sage Weil, Loic Dachary

## COPYRIGHT
2010-2024, Inktank Storage, Inc. and contributors. Licensed under Creative Commons Attribution Share Alike 3.0 (CC-BY-SA-3.0)


monmaptool

title: monmaptool(8)

text:
# monmaptool(8)

## NAME
monmaptool - ceph monitor cluster map manipulation tool

## SYNOPSIS

```
monmaptool <action> [options] mapfilename
```

## DESCRIPTION
monmaptool is a utility to create, view, and modify a monitor cluster map for the Ceph distributed storage system. The monitor map specifies the only fixed addresses in the Ceph distributed system. All other daemons bind to arbitrary addresses and register themselves with the monitors.

When creating a map with --create, a new monitor map with a new, random UUID will be created. It should be followed by one or more monitor addresses.

The default Ceph monitor port for messenger protocol v1 is 6789, and 3300 for protocol v2.

Multiple actions can be performed per invocation.

## OPTIONS
--print

print a plaintext dump of the map, after any modifications are made.

--feature-list [plain|parseable]

list the enabled features as well as the available ones. By default, a human readable output is produced.

--create

create a new monitor map with a new UUID (and with it, a new, empty Ceph cluster).

--clobber

allow monmaptool to create a new mapfilename in place of an existing map. Only useful when --create is used.

--generate

generate a new monmap based on the values on the command line or specified in the ceph configuration. This is, in order of preference,

1.

--monmap filename to specify a monmap to load

2.

--mon-host 'host1,ip2' to specify a list of hosts or ip addresses

3.

[mon.foo] sections containing mon addr settings in the config. Note that this method is not recommended and support will be removed in a future release.

--filter-initial-members

filter the initial monmap by applying the mon initial members setting. Monitors not present in that list will be removed, and initial members not present in the map will be added with dummy addresses.

--add name ip[:port]

add a monitor with the specified ip:port to the map. If the nautilus feature is set, and the port is not, the monitor will be added for both messenger protocols.

--addv name [protocol:ip:port[,...]]

add a monitor with the specified version:ip:port to the map.

--rm name

remove the monitor with the specified name from the map.

--fsid uuid

set the fsid to the given uuid. If not specified with --create, a random fsid will be generated.

--feature-set value [--optional|--persistent]

enable a feature.

--feature-unset value [--optional|--persistent]

disable a feature.

--enable-all-features

enable all supported features.

--set-min-mon-release release

set the min_mon_release.

## EXAMPLE
To create a new map with three monitors (for a fresh Ceph cluster):

```
monmaptool --create --add nodeA 192.168.0.10 --add nodeB 192.168.0.11 \

  --add nodeC 192.168.0.12 --enable-all-features --clobber monmap
```

To display the contents of the map:

```
monmaptool --print monmap
```

To replace one monitor:

```
monmaptool --rm nodeA monmap
monmaptool --add nodeA 192.168.0.9 monmap
```

## AVAILABILITY
monmaptool is part of Ceph, a massively scalable, open-source, distributed storage system. Please refer to the Ceph documentation at https://docs.ceph.com for more information.

## SEE ALSO
ceph(8), crushtool(8),

## COPYRIGHT
2010-2024, Inktank Storage, Inc. and contributors. Licensed under Creative Commons Attribution Share Alike 3.0 (CC-BY-SA-3.0)


osdmaptool

title: osdmaptool(8)

summary: ceph osd cluster map manipulation tool

text:
# osdmaptool(8)

## NAME
osdmaptool - ceph osd cluster map manipulation tool

## SYNOPSIS

```
osdmaptool mapfilename [--print] [--createsimple numosd
[--pgbits bitsperosd ] ] [--clobber]
osdmaptool mapfilename [--import-crush crushmap]
osdmaptool mapfilename [--export-crush crushmap]
osdmaptool mapfilename [--upmap file] [--upmap-max max-optimizations]
[--upmap-deviation max-deviation] [--upmap-pool poolname]
[--save] [--upmap-active]
osdmaptool mapfilename [--upmap-cleanup] [--upmap file]
```

## DESCRIPTION
osdmaptool is a utility that lets you create, view, and manipulate OSD cluster maps from the Ceph distributed storage system. Notably, it lets you extract the embedded CRUSH map or import a new CRUSH map. It can also simulate the upmap balancer mode so you can get a sense of what is needed to balance your PGs.

## OPTIONS
--print

will simply make the tool print a plaintext dump of the map, after any modifications are made.

--dump <format>

displays the map in plain text when <format> is 'plain', 'json' if specified format is not supported. This is an alternative to the print option.

--clobber

will allow osdmaptool to overwrite mapfilename if changes are made.

--import-crush mapfile

will load the CRUSH map from mapfile and embed it in the OSD map.

--export-crush mapfile

will extract the CRUSH map from the OSD map and write it to mapfile.

--createsimple numosd [--pg-bits bitsperosd] [--pgp-bits bits]

will create a relatively generic OSD map with the numosd devices. If --pg-bits is specified, the initial placement group counts will be set with bitsperosd bits per OSD. That is, the pg_num map attribute will be set to numosd shifted by bitsperosd. If --pgp-bits is specified, then the pgp_num map attribute will be set to numosd shifted by bits.

--create-from-conf

creates an osd map with default configurations.

--test-map-pgs [--pool poolid] [--range-first <first> --range-last <last>]

will print out the mappings from placement groups to OSDs. If range is specified, then it iterates from first to last in the directory specified by argument to osdmaptool. Eg: osdmaptool --test-map-pgs --range-first 0 --range-last 2 osdmap_dir. This will iterate through the files named 0,1,2 in osdmap_dir.

--test-map-pgs-dump [--pool poolid] [--range-first <first> --range-last <last>]

will print out the summary of all placement groups and the mappings from them to the mapped OSDs. If range is specified, then it iterates from first to last in the directory specified by argument to osdmaptool. Eg: osdmaptool --test-map-pgs-dump --range-first 0 --range-last 2 osdmap_dir. This will iterate through the files named 0,1,2 in osdmap_dir.

--test-map-pgs-dump-all [--pool poolid] [--range-first <first> --range-last <last>]

will print out the summary of all placement groups and the mappings from them to all the OSDs. If range is specified, then it iterates from first to last in the directory specified by argument to osdmaptool. Eg: osdmaptool --test-map-pgs-dump-all --range-first 0 --range-last 2 osdmap_dir. This will iterate through the files named 0,1,2 in osdmap_dir.

--test-random

does a random mapping of placement groups to the OSDs.

--test-map-pg <pgid>

map a particular placement group(specified by pgid) to the OSDs.

--test-map-object <objectname> [--pool <poolid>]

map a particular placement group(specified by objectname) to the OSDs.

--test-crush [--range-first <first> --range-last <last>]

map placement groups to acting OSDs. If range is specified, then it iterates from first to last in the directory specified by argument to osdmaptool. Eg: osdmaptool --test-crush --range-first 0 --range-last 2 osdmap_dir. This will iterate through the files named 0,1,2 in osdmap_dir.

--mark-up-in

mark osds up and in (but do not persist).

--mark-out

mark an osd as out (but do not persist)

--mark-up <osdid>

mark an osd as up (but do not persist)

--mark-in <osdid>

mark an osd as in (but do not persist)

--tree

Displays a hierarchical tree of the map.

--clear-temp

clears pg_temp and primary_temp variables.

--clean-temps

clean pg_temps.

--health

dump health checks

--with-default-pool

include default pool when creating map

--upmap-cleanup <file>

clean up pg_upmap[_items] entries, writing commands to <file> [default: - for stdout]

--upmap <file>

calculate pg upmap entries to balance pg layout writing commands to <file> [default: - for stdout]

--upmap-max <max-optimizations>

set max upmap entries to calculate [default: 10]

--upmap-deviation <max-deviation>

max deviation from target [default: 5]

--upmap-pool <poolname>

restrict upmap balancing to 1 pool or the option can be repeated for multiple pools

--upmap-active

Act like an active balancer, keep applying changes until balanced

--adjust-crush-weight <osdid:weight>[,<osdid:weight>,<...>]

Change CRUSH weight of <osdid>

--save

write modified osdmap with upmap or crush-adjust changes

## EXAMPLE
To create a simple map with 16 devices:

```
osdmaptool --createsimple 16 osdmap --clobber
```

To view the result:

```
osdmaptool --print osdmap
```

To view the mappings of placement groups for pool 1:

```
osdmaptool osdmap --test-map-pgs-dump --pool 1
pool 1 pg_num 8
1.0     [0,2,1] 0
1.1     [2,0,1] 2
1.2     [0,1,2] 0
1.3     [2,0,1] 2
1.4     [0,2,1] 0
1.5     [0,2,1] 0
1.6     [0,1,2] 0
1.7     [1,0,2] 1
#osd    count   first   primary c wt    wt
osd.0   8       5       5       1       1
osd.1   8       1       1       1       1
osd.2   8       2       2       1       1

 in 3

 avg 8 stddev 0 (0x) (expected 2.3094 0.288675x))

 min osd.0 8

 max osd.0 8
size 0  0
size 1  0
size 2  0
size 3  8
```

In which,

1.

pool 1 has 8 placement groups. And two tables follow:

2.

A table for placement groups. Each row presents a placement group. With columns of:

- placement group id,

- acting set, and

- primary OSD.

3.

A table for all OSDs. Each row presents an OSD. With columns of:

- count of placement groups being mapped to this OSD,

- count of placement groups where this OSD is the first one in their acting sets,

- count of placement groups where this OSD is the primary of them,

- the CRUSH weight of this OSD, and

- the weight of this OSD.

4.

Looking at the number of placement groups held by 3 OSDs. We have

- avarge, stddev, stddev/average, expected stddev, expected stddev / average

- min and max

5.

The number of placement groups mapping to n OSDs. In this case, all 8 placement groups are mapping to 3 different OSDs.

In a less-balanced cluster, we could have following output for the statistics of placement group distribution, whose standard deviation is 1.41421:

```
#osd    count   first   primary c wt    wt
osd.0   8       5       5       1       1
osd.1   8       1       1       1       1
osd.2   8       2       2       1       1
#osd    count   first    primary c wt    wt
osd.0   33      9        9       0.0145874     1
osd.1   34      14       14      0.0145874     1
osd.2   31      7        7       0.0145874     1
osd.3   31      13       13      0.0145874     1
osd.4   30      14       14      0.0145874     1
osd.5   33      7        7       0.0145874     1

 in 6

 avg 32 stddev 1.41421 (0.0441942x) (expected 5.16398 0.161374x))

 min osd.4 30

 max osd.1 34
size 00
size 10
size 20
size 364
```

To simulate the active balancer in upmap mode:

```
     osdmaptool --upmap upmaps.out --upmap-active --upmap-deviation 6 --upmap-max 11 osdmap
osdmaptool: osdmap file 'osdmap'
writing upmap command output to: upmaps.out
checking for upmap cleanups
upmap, max-count 11, max deviation 6
pools movies photos metadata data
prepared 11/11 changes
Time elapsed 0.00310404 secs
pools movies photos metadata data
prepared 11/11 changes
Time elapsed 0.00283402 secs
pools data metadata movies photos
prepared 11/11 changes
Time elapsed 0.003122 secs
pools photos metadata data movies
prepared 11/11 changes
Time elapsed 0.00324372 secs
pools movies metadata data photos
prepared 1/11 changes
Time elapsed 0.00222609 secs
pools data movies photos metadata
prepared 0/11 changes
Time elapsed 0.00209916 secs
Unable to find further optimization, or distribution is already perfect
osd.0 pgs 41
osd.1 pgs 42
osd.2 pgs 42
osd.3 pgs 41
osd.4 pgs 46
osd.5 pgs 39
osd.6 pgs 39
osd.7 pgs 43
osd.8 pgs 41
osd.9 pgs 46
osd.10 pgs 46
osd.11 pgs 46
osd.12 pgs 46
osd.13 pgs 41
osd.14 pgs 40
osd.15 pgs 40
osd.16 pgs 39
osd.17 pgs 46
osd.18 pgs 46
osd.19 pgs 39
osd.20 pgs 42
Total time elapsed 0.0167765 secs, 5 rounds
```

## AVAILABILITY
osdmaptool is part of Ceph, a massively scalable, open-source, distributed storage system. Please refer to the Ceph documentation at https://docs.ceph.com for more information.

## SEE ALSO
ceph(8), crushtool(8),

## COPYRIGHT
2010-2024, Inktank Storage, Inc. and contributors. Licensed under Creative Commons Attribution Share Alike 3.0 (CC-BY-SA-3.0)


ceph

package: ceph-common

title: ceph(8)

summary: ceph administration tool

text:
# ceph(8)

## NAME
ceph - ceph administration tool

## SYNOPSIS

```
ceph auth [ add | caps | del | export | get | get-key | get-or-create | get-or-create-key | import | list | print-key | print_key ] ...
```

```
ceph compact
```

```
ceph config [ dump | ls | help | get | show | show-with-defaults | set | rm | log | reset | assimilate-conf | generate-minimal-conf ] ...
```

```
ceph config-key [ rm | exists | get | ls | dump | set ] ...
```

```
ceph daemon <name> | <path> <command> ...
```

```
ceph daemonperf <name> | <path> [ interval [ count ] ]
```

```
ceph df {detail}
```

```
ceph fs [ ls | new | reset | rm | authorize ] ...
```

```
ceph fsid
```

```
ceph health {detail}
```

```
ceph injectargs <injectedargs> [ <injectedargs>... ]
```

```
ceph log <logtext> [ <logtext>... ]
```

```
ceph mds [ compat | fail | rm | rmfailed | set_state | stat | repaired ] ...
```

```
ceph mon [ add | dump | getmap | remove | stat ] ...
```

```
ceph osd [ blocklist | blocked-by | create | new | deep-scrub | df | down | dump | erasure-code-profile | find | getcrushmap | getmap | getmaxosd | in | ls | lspools | map | metadata | ok-to-stop | out | pause | perf | pg-temp | force-create-pg | primary-affinity | primary-temp | repair | reweight | reweight-by-pg | rm | destroy | purge | safe-to-destroy | scrub | set | setcrushmap | setmaxosd  | stat | tree | unpause | unset ] ...
```

```
ceph osd crush [ add | add-bucket | create-or-move | dump | get-tunable | link | move | remove | rename-bucket | reweight | reweight-all | reweight-subtree | rm | rule | set | set-tunable | show-tunables | tunables | unlink ] ...
```

```
ceph osd pool [ create | delete | get | get-quota | ls | mksnap | rename | rmsnap | set | set-quota | stats ] ...
```

```
ceph osd pool application [ disable | enable | get | rm | set ] ...
```

```
ceph osd tier [ add | add-cache | cache-mode | remove | remove-overlay | set-overlay ] ...
```

```
ceph pg [ debug | deep-scrub | dump | dump_json | dump_pools_json | dump_stuck | getmap | ls | ls-by-osd | ls-by-pool | ls-by-primary | map | repair | scrub | stat ] ...
```

```
ceph quorum_status
```

```
ceph report { <tags> [ <tags>... ] }
```

```
ceph status
```

```
ceph sync force {--yes-i-really-mean-it} {--i-know-what-i-am-doing}
```

```
ceph tell <name (type.id)> <command> [options...]
```

```
ceph version
```

## DESCRIPTION
ceph is a control utility which is used for manual deployment and maintenance of a Ceph cluster. It provides a diverse set of commands that allows deployment of monitors, OSDs, placement groups, MDS and overall maintenance, administration of the cluster.

## COMMANDS

### auth
Manage authentication keys. It is used for adding, removing, exporting or updating of authentication keys for a particular entity such as a monitor or OSD. It uses some additional subcommands.

Subcommand add adds authentication info for a particular entity from input file, or random key if no input is given and/or any caps specified in the command.

Usage:

```
ceph auth add <entity> {<caps> [<caps>...]}
```

Subcommand caps updates caps for name from caps specified in the command.

Usage:

```
ceph auth caps <entity> <caps> [<caps>...]
```

Subcommand del deletes all caps for name.

Usage:

```
ceph auth del <entity>
```

Subcommand export writes keyring for requested entity, or master keyring if none given.

Usage:

```
ceph auth export {<entity>}
```

Subcommand get writes keyring file with requested key.

Usage:

```
ceph auth get <entity>
```

Subcommand get-key displays requested key.

Usage:

```
ceph auth get-key <entity>
```

Subcommand get-or-create adds authentication info for a particular entity from input file, or random key if no input given and/or any caps specified in the command.

Usage:

```
ceph auth get-or-create <entity> {<caps> [<caps>...]}
```

Subcommand get-or-create-key gets or adds key for name from system/caps pairs specified in the command. If key already exists, any given caps must match the existing caps for that key.

Usage:

```
ceph auth get-or-create-key <entity> {<caps> [<caps>...]}
```

Subcommand import reads keyring from input file.

Usage:

```
ceph auth import
```

Subcommand ls lists authentication state.

Usage:

```
ceph auth ls
```

Subcommand print-key displays requested key.

Usage:

```
ceph auth print-key <entity>
```

Subcommand print_key displays requested key.

Usage:

```
ceph auth print_key <entity>
```

### compact
Causes compaction of monitor's leveldb storage.

Usage:

```
ceph compact
```

### config
Configure the cluster. By default, Ceph daemons and clients retrieve their configuration options from monitor when they start, and are updated if any of the tracked options is changed at run time. It uses following additional subcommand.

Subcommand dump to dump all options for the cluster

Usage:

```
ceph config dump
```

Subcommand ls to list all option names for the cluster

Usage:

```
ceph config ls
```

Subcommand help to describe the specified configuration option

Usage:

```
ceph config help <option>
```

Subcommand get to dump the option(s) for the specified entity.

Usage:

```
ceph config get <who> {<option>}
```

Subcommand show to display the running configuration of the specified entity. Please note, unlike get, which only shows the options managed by monitor, show displays all the configurations being actively used. These options are pulled from several sources, for instance, the compiled-in default value, the monitor's configuration database, ceph.conf file on the host. The options can even be overridden at runtime. So, there is chance that the configuration options in the output of show could be different from those in the output of get.

Usage:

```
ceph config show {<who>}
```

Subcommand show-with-defaults to display the running configuration along with the compiled-in defaults of the specified entity

Usage:

```
ceph config show {<who>}
```

Subcommand set to set an option for one or more specified entities

Usage:

```
ceph config set <who> <option> <value> {--force}
```

Subcommand rm to clear an option for one or more entities

Usage:

```
ceph config rm <who> <option>
```

Subcommand log to show recent history of config changes. If count option is omitted it defeaults to 10.

Usage:

```
ceph config log {<count>}
```

Subcommand reset to revert configuration to the specified historical version

Usage:

```
ceph config reset <version>
```

Subcommand assimilate-conf to assimilate options from stdin, and return a new, minimal conf file

Usage:

```
ceph config assimilate-conf -i <input-config-path> > <output-config-path>
ceph config assimilate-conf < <input-config-path>
```

Subcommand generate-minimal-conf to generate a minimal ceph.conf file, which can be used for bootstrapping a daemon or a client.

Usage:

```
ceph config generate-minimal-conf > <minimal-config-path>
```

### config-key
Manage configuration key. Config-key is a general purpose key/value service offered by the monitors. This service is mainly used by Ceph tools and daemons for persisting various settings. Among which, ceph-mgr modules uses it for storing their options. It uses some additional subcommands.

Subcommand rm deletes configuration key.

Usage:

```
ceph config-key rm <key>
```

Subcommand exists checks for configuration keys existence.

Usage:

```
ceph config-key exists <key>
```

Subcommand get gets the configuration key.

Usage:

```
ceph config-key get <key>
```

Subcommand ls lists configuration keys.

Usage:

```
ceph config-key ls
```

Subcommand dump dumps configuration keys and values.

Usage:

```
ceph config-key dump
```

Subcommand set puts configuration key and value.

Usage:

```
ceph config-key set <key> {<val>}
```

### daemon
Submit admin-socket commands.

Usage:

```
ceph daemon {daemon_name|socket_path} {command} ...
```

Example:

```
ceph daemon osd.0 help
```

### daemonperf
Watch performance counters from a Ceph daemon.

Usage:

```
ceph daemonperf {daemon_name|socket_path} [{interval} [{count}]]
```

### df
Show cluster's free space status.

Usage:

```
ceph df {detail}
```

### features
Show the releases and features of all connected daemons and clients connected to the cluster, along with the numbers of them in each bucket grouped by the corresponding features/releases. Each release of Ceph supports a different set of features, expressed by the features bitmask. New cluster features require that clients support the feature, or else they are not allowed to connect to these new features. As new features or capabilities are enabled after an upgrade, older clients are prevented from connecting.

Usage:

```
ceph features
```

### fs
Manage cephfs file systems. It uses some additional subcommands.

Subcommand ls to list file systems

Usage:

```
ceph fs ls
```

Subcommand new to make a new file system using named pools <metadata> and <data>

Usage:

```
ceph fs new <fs_name> <metadata> <data>
```

Subcommand reset is used for disaster recovery only: reset to a single-MDS map

Usage:

```
ceph fs reset <fs_name> {--yes-i-really-mean-it}
```

Subcommand rm to disable the named file system

Usage:

```
ceph fs rm <fs_name> {--yes-i-really-mean-it}
```

Subcommand authorize creates a new client that will be authorized for the given path in <fs_name>. Pass / to authorize for the entire FS. <perms> below can be r, rw or rwp.

Usage:

```
ceph fs authorize <fs_name> client.<client_id> <path> <perms> [<path> <perms>...]
```

### fsid
Show cluster's FSID/UUID.

Usage:

```
ceph fsid
```

### health
Show cluster's health.

Usage:

```
ceph health {detail}
```

### heap
Show heap usage info (available only if compiled with tcmalloc)

Usage:

```
ceph tell <name (type.id)> heap dump|start_profiler|stop_profiler|stats
```

Subcommand release to make TCMalloc to releases no-longer-used memory back to the kernel at once.

Usage:

```
ceph tell <name (type.id)> heap release
```

Subcommand (get|set)_release_rate get or set the TCMalloc memory release rate. TCMalloc releases no-longer-used memory back to the kernel gradually. the rate controls how quickly this happens. Increase this setting to make TCMalloc to return unused memory more frequently. 0 means never return memory to system, 1 means wait for 1000 pages after releasing a page to system. It is 1.0 by default..

Usage:

```
ceph tell <name (type.id)> heap get_release_rate|set_release_rate {<val>}
```

### injectargs
Inject configuration arguments into monitor.

Usage:

```
ceph injectargs <injected_args> [<injected_args>...]
```

### log
Log supplied text to the monitor log.

Usage:

```
ceph log <logtext> [<logtext>...]
```

### mds
Manage metadata server configuration and administration. It uses some additional subcommands.

Subcommand compat manages compatible features. It uses some additional subcommands.

Subcommand rm_compat removes compatible feature.

Usage:

```
ceph mds compat rm_compat <int[0-]>
```

Subcommand rm_incompat removes incompatible feature.

Usage:

```
ceph mds compat rm_incompat <int[0-]>
```

Subcommand show shows mds compatibility settings.

Usage:

```
ceph mds compat show
```

Subcommand fail forces mds to status fail.

Usage:

```
ceph mds fail <role|gid>
```

Subcommand rm removes inactive mds.

Usage:

```
ceph mds rm <int[0-]> <name> (type.id)>
```

Subcommand rmfailed removes failed mds.

Usage:

```
ceph mds rmfailed <int[0-]>
```

Subcommand set_state sets mds state of <gid> to <numeric-state>.

Usage:

```
ceph mds set_state <int[0-]> <int[0-20]>
```

Subcommand stat shows MDS status.

Usage:

```
ceph mds stat
```

Subcommand repaired mark a damaged MDS rank as no longer damaged.

Usage:

```
ceph mds repaired <role>
```

### mon
Manage monitor configuration and administration. It uses some additional subcommands.

Subcommand add adds new monitor named <name> at <addr>.

Usage:

```
ceph mon add <name> <IPaddr[:port]>
```

Subcommand dump dumps formatted monmap (optionally from epoch)

Usage:

```
ceph mon dump {<int[0-]>}
```

Subcommand getmap gets monmap.

Usage:

```
ceph mon getmap {<int[0-]>}
```

Subcommand remove removes monitor named <name>.

Usage:

```
ceph mon remove <name>
```

Subcommand stat summarizes monitor status.

Usage:

```
ceph mon stat
```

### mgr
Ceph manager daemon configuration and management.

Subcommand dump dumps the latest MgrMap, which describes the active and standby manager daemons.

Usage:

```
ceph mgr dump
```

Subcommand fail will mark a manager daemon as failed, removing it from the manager map. If it is the active manager daemon a standby will take its place.

Usage:

```
ceph mgr fail <name>
```

Subcommand module ls will list currently enabled manager modules (plugins).

Usage:

```
ceph mgr module ls
```

Subcommand module enable will enable a manager module. Available modules are included in MgrMap and visible via mgr dump.

Usage:

```
ceph mgr module enable <module>
```

Subcommand module disable will disable an active manager module.

Usage:

```
ceph mgr module disable <module>
```

Subcommand metadata will report metadata about all manager daemons or, if the name is specified, a single manager daemon.

Usage:

```
ceph mgr metadata [name]
```

Subcommand versions will report a count of running daemon versions.

Usage:

```
ceph mgr versions
```

Subcommand count-metadata will report a count of any daemon metadata field.

Usage:

```
ceph mgr count-metadata <field>
```

### osd
Manage OSD configuration and administration. It uses some additional subcommands.

Subcommand blocklist manage blocklisted clients. It uses some additional subcommands.

Subcommand add add <addr> to blocklist (optionally until <expire> seconds from now)

Usage:

```
ceph osd blocklist add <EntityAddr> {<float[0.0-]>}
```

Subcommand ls show blocklisted clients

Usage:

```
ceph osd blocklist ls
```

Subcommand rm remove <addr> from blocklist

Usage:

```
ceph osd blocklist rm <EntityAddr>
```

Subcommand blocked-by prints a histogram of which OSDs are blocking their peers

Usage:

```
ceph osd blocked-by
```

Subcommand create creates new osd (with optional UUID and ID).

This command is DEPRECATED as of the Luminous release, and will be removed in a future release.

Subcommand new should instead be used.

Usage:

```
ceph osd create {<uuid>} {<id>}
```

Subcommand new can be used to create a new OSD or to recreate a previously destroyed OSD with a specific id. The new OSD will have the specified uuid, and the command expects a JSON file containing the base64 cephx key for auth entity client.osd.<id>, as well as optional base64 cepx key for dm-crypt lockbox access and a dm-crypt key. Specifying a dm-crypt requires specifying the accompanying lockbox cephx key.

Usage:

```
ceph osd new {<uuid>} {<id>} -i {<params.json>}
```

The parameters JSON file is optional but if provided, is expected to maintain a form of the following format:

```
{

    "cephx_secret": "AQBWtwhZdBO5ExAAIDyjK2Bh16ZXylmzgYYEjg==",

    "crush_device_class": "myclass"
}
```

Or:

```
{

    "cephx_secret": "AQBWtwhZdBO5ExAAIDyjK2Bh16ZXylmzgYYEjg==",

    "cephx_lockbox_secret": "AQDNCglZuaeVCRAAYr76PzR1Anh7A0jswkODIQ==",

    "dmcrypt_key": "<dm-crypt key>",

    "crush_device_class": "myclass"
}
```

Or:

```
{

    "crush_device_class": "myclass"
}
```

The "crush_device_class" property is optional. If specified, it will set the initial CRUSH device class for the new OSD.

Subcommand crush is used for CRUSH management. It uses some additional subcommands.

Subcommand add adds or updates crushmap position and weight for <name> with <weight> and location <args>.

Usage:

```
ceph osd crush add <osdname (id|osd.id)> <float[0.0-]> <args> [<args>...]
```

Subcommand add-bucket adds no-parent (probably root) crush bucket <name> of type <type>.

Usage:

```
ceph osd crush add-bucket <name> <type>
```

Subcommand create-or-move creates entry or moves existing entry for <name> <weight> at/to location <args>.

Usage:

```
ceph osd crush create-or-move <osdname (id|osd.id)> <float[0.0-]> <args>
[<args>...]
```

Subcommand dump dumps crush map.

Usage:

```
ceph osd crush dump
```

Subcommand get-tunable get crush tunable straw_calc_version

Usage:

```
ceph osd crush get-tunable straw_calc_version
```

Subcommand link links existing entry for <name> under location <args>.

Usage:

```
ceph osd crush link <name> <args> [<args>...]
```

Subcommand move moves existing entry for <name> to location <args>.

Usage:

```
ceph osd crush move <name> <args> [<args>...]
```

Subcommand remove removes <name> from crush map (everywhere, or just at <ancestor>).

Usage:

```
ceph osd crush remove <name> {<ancestor>}
```

Subcommand rename-bucket renames bucket <srcname> to <dstname>

Usage:

```
ceph osd crush rename-bucket <srcname> <dstname>
```

Subcommand reweight change <name>'s weight to <weight> in crush map.

Usage:

```
ceph osd crush reweight <name> <float[0.0-]>
```

Subcommand reweight-all recalculate the weights for the tree to ensure they sum correctly

Usage:

```
ceph osd crush reweight-all
```

Subcommand reweight-subtree changes all leaf items beneath <name> to <weight> in crush map

Usage:

```
ceph osd crush reweight-subtree <name> <weight>
```

Subcommand rm removes <name> from crush map (everywhere, or just at <ancestor>).

Usage:

```
ceph osd crush rm <name> {<ancestor>}
```

Subcommand rule is used for creating crush rules. It uses some additional subcommands.

Subcommand create-erasure creates crush rule <name> for erasure coded pool created with <profile> (default default).

Usage:

```
ceph osd crush rule create-erasure <name> {<profile>}
```

Subcommand create-simple creates crush rule <name> to start from <root>, replicate across buckets of type <type>, using a choose mode of <firstn|indep> (default firstn; indep best for erasure pools).

Usage:

```
ceph osd crush rule create-simple <name> <root> <type> {firstn|indep}
```

Subcommand dump dumps crush rule <name> (default all).

Usage:

```
ceph osd crush rule dump {<name>}
```

Subcommand ls lists crush rules.

Usage:

```
ceph osd crush rule ls
```

Subcommand rm removes crush rule <name>.

Usage:

```
ceph osd crush rule rm <name>
```

Subcommand set used alone, sets crush map from input file.

Usage:

```
ceph osd crush set
```

Subcommand set with osdname/osd.id update crushmap position and weight for <name> to <weight> with location <args>.

Usage:

```
ceph osd crush set <osdname (id|osd.id)> <float[0.0-]> <args> [<args>...]
```

Subcommand set-tunable set crush tunable <tunable> to <value>. The only tunable that can be set is straw_calc_version.

Usage:

```
ceph osd crush set-tunable straw_calc_version <value>
```

Subcommand show-tunables shows current crush tunables.

Usage:

```
ceph osd crush show-tunables
```

Subcommand tree shows the crush buckets and items in a tree view.

Usage:

```
ceph osd crush tree
```

Subcommand tunables sets crush tunables values to <profile>.

Usage:

```
ceph osd crush tunables legacy|argonaut|bobtail|firefly|hammer|optimal|default
```

Subcommand unlink unlinks <name> from crush map (everywhere, or just at <ancestor>).

Usage:

```
ceph osd crush unlink <name> {<ancestor>}
```

Subcommand df shows OSD utilization

Usage:

```
ceph osd df {plain|tree}
```

Subcommand deep-scrub initiates deep scrub on specified osd.

Usage:

```
ceph osd deep-scrub <who>
```

Subcommand down sets osd(s) <id> [<id>...] down.

Usage:

```
ceph osd down <ids> [<ids>...]
```

Subcommand dump prints summary of OSD map.

Usage:

```
ceph osd dump {<int[0-]>}
```

Subcommand erasure-code-profile is used for managing the erasure code profiles. It uses some additional subcommands.

Subcommand get gets erasure code profile <name>.

Usage:

```
ceph osd erasure-code-profile get <name>
```

Subcommand ls lists all erasure code profiles.

Usage:

```
ceph osd erasure-code-profile ls
```

Subcommand rm removes erasure code profile <name>.

Usage:

```
ceph osd erasure-code-profile rm <name>
```

Subcommand set creates erasure code profile <name> with [<key[=value]> ...] pairs. Add a --force at the end to override an existing profile (IT IS RISKY).

Usage:

```
ceph osd erasure-code-profile set <name> {<profile> [<profile>...]}
```

Subcommand find find osd <id> in the CRUSH map and shows its location.

Usage:

```
ceph osd find <int[0-]>
```

Subcommand getcrushmap gets CRUSH map.

Usage:

```
ceph osd getcrushmap {<int[0-]>}
```

Subcommand getmap gets OSD map.

Usage:

```
ceph osd getmap {<int[0-]>}
```

Subcommand getmaxosd shows largest OSD id.

Usage:

```
ceph osd getmaxosd
```

Subcommand in sets osd(s) <id> [<id>...] in.

Usage:

```
ceph osd in <ids> [<ids>...]
```

Subcommand lost marks osd as permanently lost. THIS DESTROYS DATA IF NO MORE REPLICAS EXIST, BE CAREFUL.

Usage:

```
ceph osd lost <int[0-]> {--yes-i-really-mean-it}
```

Subcommand ls shows all OSD ids.

Usage:

```
ceph osd ls {<int[0-]>}
```

Subcommand lspools lists pools.

Usage:

```
ceph osd lspools {<int>}
```

Subcommand map finds pg for <object> in <pool>.

Usage:

```
ceph osd map <poolname> <objectname>
```

Subcommand metadata fetches metadata for osd <id>.

Usage:

```
ceph osd metadata {int[0-]} (default all)
```

Subcommand out sets osd(s) <id> [<id>...] out.

Usage:

```
ceph osd out <ids> [<ids>...]
```

Subcommand ok-to-stop checks whether the list of OSD(s) can be stopped without immediately making data unavailable. That is, all data should remain readable and writeable, although data redundancy may be reduced as some PGs may end up in a degraded (but active) state. It will return a success code if it is okay to stop the OSD(s), or an error code and informative message if it is not or if no conclusion can be drawn at the current time. When --max <num> is provided, up to <num> OSDs IDs will return (including the provided OSDs) that can all be stopped simultaneously. This allows larger sets of stoppable OSDs to be generated easily by providing a single starting OSD and a max. Additional OSDs are drawn from adjacent locations in the CRUSH hierarchy.

Usage:

```
ceph osd ok-to-stop <id> [<ids>...] [--max <num>]
```

Subcommand pause pauses osd.

Usage:

```
ceph osd pause
```

Subcommand perf prints dump of OSD perf summary stats.

Usage:

```
ceph osd perf
```

Subcommand pg-temp set pg_temp mapping pgid:[<id> [<id>...]] (developers only).

Usage:

```
ceph osd pg-temp <pgid> {<id> [<id>...]}
```

Subcommand force-create-pg forces creation of pg <pgid>.

Usage:

```
ceph osd force-create-pg <pgid>
```

Subcommand pool is used for managing data pools. It uses some additional subcommands.

Subcommand create creates pool.

Usage:

```
ceph osd pool create <poolname> {<int[0-]>} {<int[0-]>} {replicated|erasure}
{<erasure_code_profile>} {<rule>} {<int>} {--autoscale-mode=<on,off,warn>}
```

Subcommand delete deletes pool.

Usage:

```
ceph osd pool delete <poolname> {<poolname>} {--yes-i-really-really-mean-it}
```

Subcommand get gets pool parameter <var>.

Usage:

```
ceph osd pool get <poolname> size|min_size|pg_num|pgp_num|crush_rule|write_fadvise_dontneed
```

Only for tiered pools:

```
ceph osd pool get <poolname> hit_set_type|hit_set_period|hit_set_count|hit_set_fpp|
target_max_objects|target_max_bytes|cache_target_dirty_ratio|cache_target_dirty_high_ratio|
cache_target_full_ratio|cache_min_flush_age|cache_min_evict_age|
min_read_recency_for_promote|hit_set_grade_decay_rate|hit_set_search_last_n
```

Only for erasure coded pools:

```
ceph osd pool get <poolname> erasure_code_profile
```

Use all to get all pool parameters that apply to the pool's type:

```
ceph osd pool get <poolname> all
```

Subcommand get-quota obtains object or byte limits for pool.

Usage:

```
ceph osd pool get-quota <poolname>
```

Subcommand ls list pools

Usage:

```
ceph osd pool ls {detail}
```

Subcommand mksnap makes snapshot <snap> in <pool>.

Usage:

```
ceph osd pool mksnap <poolname> <snap>
```

Subcommand rename renames <srcpool> to <destpool>.

Usage:

```
ceph osd pool rename <poolname> <poolname>
```

Subcommand rmsnap removes snapshot <snap> from <pool>.

Usage:

```
ceph osd pool rmsnap <poolname> <snap>
```

Subcommand set sets pool parameter <var> to <val>.

Usage:

```
ceph osd pool set <poolname> size|min_size|pg_num|
pgp_num|crush_rule|hashpspool|nodelete|nopgchange|nosizechange|
hit_set_type|hit_set_period|hit_set_count|hit_set_fpp|debug_fake_ec_pool|
target_max_bytes|target_max_objects|cache_target_dirty_ratio|
cache_target_dirty_high_ratio|
cache_target_full_ratio|cache_min_flush_age|cache_min_evict_age|
min_read_recency_for_promote|write_fadvise_dontneed|hit_set_grade_decay_rate|
hit_set_search_last_n
<val> {--yes-i-really-mean-it}
```

Subcommand set-quota sets object or byte limit on pool.

Usage:

```
ceph osd pool set-quota <poolname> max_objects|max_bytes <val>
```

Subcommand stats obtain stats from all pools, or from specified pool.

Usage:

```
ceph osd pool stats {<name>}
```

Subcommand application is used for adding an annotation to the given pool. By default, the possible applications are object, block, and file storage (corresponding app-names are "rgw", "rbd", and "cephfs"). However, there might be other applications as well. Based on the application, there may or may not be some processing conducted.

Subcommand disable disables the given application on the given pool.

Usage:

```
ceph osd pool application disable <pool-name> <app> {--yes-i-really-mean-it}
```

Subcommand enable adds an annotation to the given pool for the mentioned application.

Usage:

```
ceph osd pool application enable <pool-name> <app> {--yes-i-really-mean-it}
```

Subcommand get displays the value for the given key that is associated with the given application of the given pool. Not passing the optional arguments would display all key-value pairs for all applications for all pools.

Usage:

```
ceph osd pool application get {<pool-name>} {<app>} {<key>}
```

Subcommand rm removes the key-value pair for the given key in the given application of the given pool.

Usage:

```
ceph osd pool application rm <pool-name> <app> <key>
```

Subcommand set associates or updates, if it already exists, a key-value pair with the given application for the given pool.

Usage:

```
ceph osd pool application set <pool-name> <app> <key> <value>
```

Subcommand primary-affinity adjust osd primary-affinity from 0.0 <=<weight> <= 1.0

Usage:

```
ceph osd primary-affinity <osdname (id|osd.id)> <float[0.0-1.0]>
```

Subcommand primary-temp sets primary_temp mapping pgid:<id>|-1 (developers only).

Usage:

```
ceph osd primary-temp <pgid> <id>
```

Subcommand repair initiates repair on a specified osd.

Usage:

```
ceph osd repair <who>
```

Subcommand reweight reweights osd to 0.0 < <weight> < 1.0.

Usage:

```
osd reweight <int[0-]> <float[0.0-1.0]>
```

Subcommand reweight-by-pg reweight OSDs by PG distribution [overload-percentage-for-consideration, default 120].

Usage:

```
ceph osd reweight-by-pg {<int[100-]>} {<poolname> [<poolname...]}
{--no-increasing}
```

Subcommand reweight-by-utilization reweights OSDs by utilization. It only reweights outlier OSDs whose utilization exceeds the average, eg. the default 120% limits reweight to those OSDs that are more than 20% over the average. [overload-threshold, default 120 [max_weight_change, default 0.05 [max_osds_to_adjust, default 4]]]

Usage:

```
ceph osd reweight-by-utilization {<int[100-]> {<float[0.0-]> {<int[0-]>}}}
{--no-increasing}
```

Subcommand rm removes osd(s) <id> [<id>...] from the OSD map.

Usage:

```
ceph osd rm <ids> [<ids>...]
```

Subcommand destroy marks OSD id as destroyed, removing its cephx entity's keys and all of its dm-crypt and daemon-private config key entries.

This command will not remove the OSD from crush, nor will it remove the OSD from the OSD map. Instead, once the command successfully completes, the OSD will show marked as destroyed.

In order to mark an OSD as destroyed, the OSD must first be marked as lost.

Usage:

```
ceph osd destroy <id> {--yes-i-really-mean-it}
```

Subcommand purge performs a combination of osd destroy, osd rm and osd crush remove.

Usage:

```
ceph osd purge <id> {--yes-i-really-mean-it}
```

Subcommand safe-to-destroy checks whether it is safe to remove or destroy an OSD without reducing overall data redundancy or durability. It will return a success code if it is definitely safe, or an error code and informative message if it is not or if no conclusion can be drawn at the current time.

Usage:

```
ceph osd safe-to-destroy <id> [<ids>...]
```

Subcommand scrub initiates scrub on specified osd.

Usage:

```
ceph osd scrub <who>
```

Subcommand set sets cluster-wide <flag> by updating OSD map. The full flag is not honored anymore since the Mimic release, and ceph osd set full is not supported in the Octopus release.

Usage:

```
ceph osd set pause|noup|nodown|noout|noin|nobackfill|
norebalance|norecover|noscrub|nodeep-scrub|notieragent
```

Subcommand setcrushmap sets crush map from input file.

Usage:

```
ceph osd setcrushmap
```

Subcommand setmaxosd sets new maximum osd value.

Usage:

```
ceph osd setmaxosd <int[0-]>
```

Subcommand set-require-min-compat-client enforces the cluster to be backward compatible with the specified client version. This subcommand prevents you from making any changes (e.g., crush tunables, or using new features) that would violate the current setting. Please note, This subcommand will fail if any connected daemon or client is not compatible with the features offered by the given <version>. To see the features and releases of all clients connected to cluster, please see ceph features.

Usage:

```
ceph osd set-require-min-compat-client <version>
```

Subcommand stat prints summary of OSD map.

Usage:

```
ceph osd stat
```

Subcommand tier is used for managing tiers. It uses some additional subcommands.

Subcommand add adds the tier <tierpool> (the second one) to base pool <pool> (the first one).

Usage:

```
ceph osd tier add <poolname> <poolname> {--force-nonempty}
```

Subcommand add-cache adds a cache <tierpool> (the second one) of size <size> to existing pool <pool> (the first one).

Usage:

```
ceph osd tier add-cache <poolname> <poolname> <int[0-]>
```

Subcommand cache-mode specifies the caching mode for cache tier <pool>.

Usage:

```
ceph osd tier cache-mode <poolname> writeback|readproxy|readonly|none
```

Subcommand remove removes the tier <tierpool> (the second one) from base pool <pool> (the first one).

Usage:

```
ceph osd tier remove <poolname> <poolname>
```

Subcommand remove-overlay removes the overlay pool for base pool <pool>.

Usage:

```
ceph osd tier remove-overlay <poolname>
```

Subcommand set-overlay set the overlay pool for base pool <pool> to be <overlaypool>.

Usage:

```
ceph osd tier set-overlay <poolname> <poolname>
```

Subcommand tree prints OSD tree.

Usage:

```
ceph osd tree {<int[0-]>}
```

Subcommand unpause unpauses osd.

Usage:

```
ceph osd unpause
```

Subcommand unset unsets cluster-wide <flag> by updating OSD map.

Usage:

```
ceph osd unset pause|noup|nodown|noout|noin|nobackfill|
norebalance|norecover|noscrub|nodeep-scrub|notieragent
```

### pg
It is used for managing the placement groups in OSDs. It uses some additional subcommands.

Subcommand debug shows debug info about pgs.

Usage:

```
ceph pg debug unfound_objects_exist|degraded_pgs_exist
```

Subcommand deep-scrub starts deep-scrub on <pgid>.

Usage:

```
ceph pg deep-scrub <pgid>
```

Subcommand dump shows human-readable versions of pg map (only 'all' valid with plain).

Usage:

```
ceph pg dump {all|summary|sum|delta|pools|osds|pgs|pgs_brief} [{all|summary|sum|delta|pools|osds|pgs|pgs_brief...]}
```

Subcommand dump_json shows human-readable version of pg map in json only.

Usage:

```
ceph pg dump_json {all|summary|sum|delta|pools|osds|pgs|pgs_brief} [{all|summary|sum|delta|pools|osds|pgs|pgs_brief...]}
```

Subcommand dump_pools_json shows pg pools info in json only.

Usage:

```
ceph pg dump_pools_json
```

Subcommand dump_stuck shows information about stuck pgs.

Usage:

```
ceph pg dump_stuck {inactive|unclean|stale|undersized|degraded [inactive|unclean|stale|undersized|degraded...]}
{<int>}
```

Subcommand getmap gets binary pg map to -o/stdout.

Usage:

```
ceph pg getmap
```

Subcommand ls lists pg with specific pool, osd, state

Usage:

```
ceph pg ls {<int>} {<pg-state> [<pg-state>...]}
```

Subcommand ls-by-osd lists pg on osd [osd]

Usage:

```
ceph pg ls-by-osd <osdname (id|osd.id)> {<int>}
{<pg-state> [<pg-state>...]}
```

Subcommand ls-by-pool lists pg with pool = [poolname]

Usage:

```
ceph pg ls-by-pool <poolstr> {<int>} {<pg-state> [<pg-state>...]}
```

Subcommand ls-by-primary lists pg with primary = [osd]

Usage:

```
ceph pg ls-by-primary <osdname (id|osd.id)> {<int>}
{<pg-state> [<pg-state>...]}
```

Subcommand map shows mapping of pg to osds.

Usage:

```
ceph pg map <pgid>
```

Subcommand repair starts repair on <pgid>.

Usage:

```
ceph pg repair <pgid>
```

Subcommand scrub starts scrub on <pgid>.

Usage:

```
ceph pg scrub <pgid>
```

Subcommand stat shows placement group status.

Usage:

```
ceph pg stat
```

### quorum
Cause a specific MON to enter or exit quorum.

Usage:

```
ceph tell mon.<id> quorum enter|exit
```

### quorum_status
Reports status of monitor quorum.

Usage:

```
ceph quorum_status
```

### report
Reports full status of cluster, optional title tag strings.

Usage:

```
ceph report {<tags> [<tags>...]}
```

### status
Shows cluster status.

Usage:

```
ceph status
```

### tell
Sends a command to a specific daemon.

Usage:

```
ceph tell <name (type.id)> <command> [options...]
```

List all available commands.

Usage:

```
ceph tell <name (type.id)> help
```

### version
Show mon daemon version

Usage:

```
ceph version
```

## OPTIONS
-i infile

will specify an input file to be passed along as a payload with the command to the monitor cluster. This is only used for specific monitor commands.

-o outfile

will write any payload returned by the monitor cluster with its reply to outfile. Only specific monitor commands (e.g. osd getmap) return a payload.

--setuser user

will apply the appropriate user ownership to the file specified by the option '-o'.

--setgroup group

will apply the appropriate group ownership to the file specified by the option '-o'.

-c ceph.conf, --conf=ceph.conf

Use ceph.conf configuration file instead of the default /etc/ceph/ceph.conf to determine monitor addresses during startup.

--id CLIENT_ID, --user CLIENT_ID

Client id for authentication.

--name CLIENT_NAME, -n CLIENT_NAME

Client name for authentication.

--cluster CLUSTER

Name of the Ceph cluster.

--admin-daemon ADMIN_SOCKET, daemon DAEMON_NAME

Submit admin-socket commands via admin sockets in /var/run/ceph.

--admin-socket ADMIN_SOCKET_NOPE

You probably mean --admin-daemon

-s, --status

Show cluster status.

-w, --watch

Watch live cluster changes on the default 'cluster' channel

-W, --watch-channel

Watch live cluster changes on any channel (cluster, audit, cephadm, or * for all)

--watch-debug

Watch debug events.

--watch-info

Watch info events.

--watch-sec

Watch security events.

--watch-warn

Watch warning events.

--watch-error

Watch error events.

--version, -v

Display version.

--verbose

Make verbose.

--concise

Make less verbose.

-f {json,json-pretty,xml,xml-pretty,plain,yaml}, --format

Format of output. Note: yaml is only valid for orch commands.

--connect-timeout CLUSTER_TIMEOUT

Set a timeout for connecting to the cluster.

--no-increasing

--no-increasing is off by default. So increasing the osd weight is allowed using the reweight-by-utilization or test-reweight-by-utilization commands. If this option is used with these commands, it will help not to increase osd weight even the osd is under utilized.

--block

block until completion (scrub and deep-scrub only)

## AVAILABILITY
ceph is part of Ceph, a massively scalable, open-source, distributed storage system. Please refer to the Ceph documentation at https://docs.ceph.com for more information.

## SEE ALSO
ceph-mon(8), ceph-osd(8), ceph-mds(8)

## COPYRIGHT
2010-2024, Inktank Storage, Inc. and contributors. Licensed under Creative Commons Attribution Share Alike 3.0 (CC-BY-SA-3.0)


ceph-authtool

title: ceph-authtool(8)

summary: ceph keyring manipulation tool

text:
# ceph-authtool(8)

## NAME
ceph-authtool - ceph keyring manipulation tool

## SYNOPSIS

```
ceph-authtool keyringfile
[ -l | --list ]
[ -p | --print-key ]
[ -C | --create-keyring ]
[ -g | --gen-key ]
[ --gen-print-key ]
[ --import-keyring otherkeyringfile ]
[ -n | --name entityname ]
[ -a | --add-key base64_key ]
[ --cap subsystem capability ]
[ --caps capfile ]
[ --mode mode ]
```

## DESCRIPTION
ceph-authtool is a utility to create, view, and modify a Ceph keyring file. A keyring file stores one or more Ceph authentication keys and possibly an associated capability specification. Each key is associated with an entity name, of the form {client,mon,mds,osd}.name.

WARNING Ceph provides authentication and protection against man-in-the-middle attacks once secret keys are in place. However, data over the wire is not encrypted, which may include the messages used to configure said keys. The system is primarily intended to be used in trusted environments.

## OPTIONS
-l, --list

will list all keys and capabilities present in the keyring

-p, --print-key

will print an encoded key for the specified entityname. This is suitable for the mount -o secret= argument

-C, --create-keyring

will create a new keyring, overwriting any existing keyringfile

-g, --gen-key

will generate a new secret key for the specified entityname

--gen-print-key

will generate a new secret key for the specified entityname, without altering the keyringfile, printing the secret to stdout

--import-keyring *secondkeyringfile*

will import the content of a given keyring to the keyringfile

-n, --name *name*

specify entityname to operate on

-a, --add-key *base64_key*

will add an encoded key to the keyring

--cap *subsystem* *capability*

will set the capability for given subsystem

--caps *capsfile*

will set all of capabilities associated with a given key, for all subsystems

--mode *mode*

will set the desired file mode to the keyring e.g: 0644, defaults to 0600

## CAPABILITIES
The subsystem is the name of a Ceph subsystem: mon, mds, or osd.

The capability is a string describing what the given user is allowed to do. This takes the form of a comma separated list of allow clauses with a permission specifier containing one or more of rwx for read, write, and execute permission. The allow * grants full superuser permissions for the given subsystem.

For example:

```
# can read, write, and execute objects
osd = "allow rwx"
# can access mds server
mds = "allow"
# can modify cluster state (i.e., is a server daemon)
mon = "allow rwx"
```

A librados user restricted to a single pool might look like:

```
mon = "allow r"
osd = "allow rw pool foo"
```

A client using rbd with read access to one pool and read/write access to another:

```
mon = "allow r"
osd = "allow class-read object_prefix rbd_children, allow pool templates r class-read, allow pool vms rwx"
```

A client mounting the file system with minimal permissions would need caps like:

```
mds = "allow"
osd = "allow rw pool data"
mon = "allow r"
```

## OSD CAPABILITIES
In general, an osd capability follows the grammar:

```
osdcap  := grant[,grant...]
grant   := allow (match capspec | capspec match)
match   := [ pool[=]<poolname> | object_prefix <prefix>

            | namespace[=]<rados-namespace>

            | tag <application-name> <key>=<value> ]
capspec := * | [r][w][x] [class-read] [class-write]
```

The capspec determines what kind of operations the entity can perform:

```
r           = read access to objects
w           = write access to objects
x           = can call any class method (same as class-read class-write)
class-read  = can call class methods that are reads
class-write = can call class methods that are writes
* or "all"  = equivalent to rwx, plus the ability to run osd admin commands,

              i.e. ceph osd tell ...
```

The match criteria restrict a grant based on the pool being accessed. Grants are additive if the client fulfills the match condition. For example, if a client has the osd capabilities: "allow r object_prefix prefix, allow w pool foo, allow x pool bar", then it has rw access to pool foo, rx access to pool bar, and r access to objects whose names begin with 'prefix' in any pool.

## CAPS FILE FORMAT
The caps file format consists of zero or more key/value pairs, one per line. The key and value are separated by an =, and the value must be quoted (with ' or ") if it contains any whitespace. The key is the name of the Ceph subsystem (osd, mds, mon), and the value is the capability string (see above).

## EXAMPLE
To create a new keyring containing a key for client.foo with a 0644 file mode:

```
ceph-authtool -C -n client.foo --gen-key keyring --mode 0644
```

To associate some capabilities with the key (namely, the ability to mount a Ceph file system):

```
ceph-authtool -n client.foo --cap mds 'allow' --cap osd 'allow rw pool=data' --cap mon 'allow r' keyring
```

To display the contents of the keyring:

```
ceph-authtool -l keyring
```

When mounting a Ceph file system, you can grab the appropriately encoded secret key with:

```
mount -t ceph serverhost:/ mountpoint -o name=foo,secret=`ceph-authtool -p -n client.foo keyring`
```

## AVAILABILITY
ceph-authtool is part of Ceph, a massively scalable, open-source, distributed storage system. Please refer to the Ceph documentation at https://docs.ceph.com for more information.

## SEE ALSO
ceph(8)

## COPYRIGHT
2010-2024, Inktank Storage, Inc. and contributors. Licensed under Creative Commons Attribution Share Alike 3.0 (CC-BY-SA-3.0)


ceph-conf

title: ceph-conf(8)

summary: ceph conf file tool

text:
# ceph-conf(8)

## NAME
ceph-conf - ceph conf file tool

## SYNOPSIS

```
ceph-conf -c conffile --list-all-sections
ceph-conf -c conffile -L
ceph-conf -c conffile -l prefix
ceph-conf key -s section1 ...
ceph-conf [-s section ] [-r] --lookup key
ceph-conf [-s section ] key
```

## DESCRIPTION
ceph-conf is a utility for getting information from a ceph configuration file. As with most Ceph programs, you can specify which Ceph configuration file to use with the -c flag.

Note that unlike other ceph tools, ceph-conf will only read from config files (or return compiled-in default values)--it will not fetch config values from the monitor cluster. For this reason it is recommended that ceph-conf only be used in legacy environments that are strictly config-file based. New deployments and tools should instead rely on either querying the monitor explicitly for configuration (e.g., ceph config get <daemon> <option>) or use daemons themselves to fetch effective config options (e.g., ceph-osd -i 123 --show-config-value osd_data). The latter option has the advantages of drawing from compiled-in defaults (which occasionally vary between daemons), config files, and the monitor's config database, providing the exact value that that daemon would be using if it were started.

## ACTIONS
ceph-conf performs one of the following actions:

-L, --list-all-sections

list all sections in the configuration file.

-l, --list-sections *prefix*

list the sections with the given prefix. For example, --list-sections mon would list all sections beginning with mon.

--lookup *key*

search and print the specified configuration setting. Note: --lookup is the default action. If no other actions are given on the command line, we will default to doing a lookup.

-h, --help

print a summary of usage.

## OPTIONS
-c *conffile*

the Ceph configuration file.

--filter-key *key*

filter section list to only include sections with given key defined.

--filter-key-value *key* ``=`` *value*

filter section list to only include sections with given key/value pair.

--name *type.id*

the Ceph name in which the sections are searched (default 'client.admin'). For example, if we specify --name osd.0, the following sections will be searched: [osd.0], [osd], [global]

--pid *pid*

override the $pid when expanding options. For example, if an option is configured like /var/log/$name.$pid.log, the $pid portion in its value will be substituted using the PID of ceph-conf instead of the PID of the process specified using the --name option.

-r, --resolve-search

search for the first file that exists and can be opened in the resulted comma delimited search list.

-s, --section

additional sections to search. These additional sections will be searched before the sections that would normally be searched. As always, the first matching entry we find will be returned.

## EXAMPLES
To find out what value osd 0 will use for the "osd data" option:

```
ceph-conf -c foo.conf  --name osd.0 --lookup "osd data"
```

To find out what value will mds a use for the "log file" option:

```
ceph-conf -c foo.conf  --name mds.a "log file"
```

To list all sections that begin with "osd":

```
ceph-conf -c foo.conf -l osd
```

To list all sections:

```
ceph-conf -c foo.conf -L
```

To print the path of the "keyring" used by "client.0":

```
ceph-conf --name client.0 -r -l keyring
```

## FILES
/etc/ceph/$cluster.conf, ~/.ceph/$cluster.conf, $cluster.conf

the Ceph configuration files to use if not specified.

## AVAILABILITY
ceph-conf is part of Ceph, a massively scalable, open-source, distributed storage system. Please refer to the Ceph documentation at https://docs.ceph.com for more information.

## SEE ALSO
ceph(8),

## COPYRIGHT
2010-2024, Inktank Storage, Inc. and contributors. Licensed under Creative Commons Attribution Share Alike 3.0 (CC-BY-SA-3.0)


ceph-dencoder

title: ceph-dencoder(8)

summary: ceph encoder/decoder utility

text:
# ceph-dencoder(8)

## NAME
ceph-dencoder - ceph encoder/decoder utility

## SYNOPSIS

```
ceph-dencoder [commands...]
```

## DESCRIPTION
ceph-dencoder is a utility to encode, decode, and dump ceph data structures. It is used for debugging and for testing inter-version compatibility.

ceph-dencoder takes a simple list of commands and performs them in order.

## COMMANDS
version

Print the version string for the ceph-dencoder binary.

import <file>

Read a binary blob of encoded data from the given file. It will be placed in an in-memory buffer.

export <file>

Write the contents of the current in-memory buffer to the given file.

list_types

List the data types known to this build of ceph-dencoder.

type <name>

Select the given type for future encode or decode operations.

skip <bytes>

Seek <bytes> into the imported file before reading data structure, use this with objects that have a preamble/header before the object of interest.

decode

Decode the contents of the in-memory buffer into an instance of the previously selected type. If there is an error, report it.

encode

Encode the contents of the in-memory instance of the previously selected type to the in-memory buffer.

dump_json

Print a JSON-formatted description of the in-memory object.

count_tests

Print the number of built-in test instances of the previously selected type that ceph-dencoder is able to generate.

select_test <n>

Select the given built-in test instance as the in-memory instance of the type.

get_features

Print the decimal value of the feature set supported by this version of ceph-dencoder. Each bit represents a feature. These correspond to CEPH_FEATURE_* defines in src/include/ceph_features.h.

set_features <f>

Set the feature bits provided to encode to f. This allows you to encode objects such that they can be understood by old versions of the software (for those types that support it).

## EXAMPLE
Say you want to examine an attribute on an object stored by ceph-osd. You can do this:

```
$ cd /mnt/osd.12/current/2.b_head
$ attr -l foo_bar_head_EFE6384B
Attribute "ceph.snapset" has a 31 byte value for foo_bar_head_EFE6384B
Attribute "ceph._" has a 195 byte value for foo_bar_head_EFE6384B
$ attr foo_bar_head_EFE6384B -g ceph._ -q > /tmp/a
$ ceph-dencoder type object_info_t import /tmp/a decode dump_json
{ "oid": { "oid": "foo",

      "key": "bar",

      "snapid": -2,

      "hash": 4024842315,

      "max": 0},

  "locator": { "pool": 2,

      "preferred": -1,

      "key": "bar"},

  "category": "",

  "version": "9'1",

  "prior_version": "0'0",

  "last_reqid": "client.4116.0:1",

  "size": 1681,

  "mtime": "2012-02-21 08:58:23.666639",

  "lost": 0,

  "wrlock_by": "unknown.0.0:0",

  "snaps": [],

  "truncate_seq": 0,

  "truncate_size": 0,

  "watchers": {}}
```

Alternatively, perhaps you wish to dump an internal CephFS metadata object, you might do that like this:

```
$ rados -p metadata get mds_snaptable mds_snaptable.bin
$ ceph-dencoder type SnapServer skip 8 import mds_snaptable.bin decode dump_json
{ "snapserver": { "last_snap": 1,

   "pending_noop": [],

   "snaps": [],

   "need_to_purge": {},

   "pending_create": [],

   "pending_destroy": []}}
```

## AVAILABILITY
ceph-dencoder is part of Ceph, a massively scalable, open-source, distributed storage system. Please refer to the Ceph documentation at https://docs.ceph.com for more information.

## SEE ALSO
ceph(8)

## COPYRIGHT
2010-2024, Inktank Storage, Inc. and contributors. Licensed under Creative Commons Attribution Share Alike 3.0 (CC-BY-SA-3.0)


ceph-post-file

title: ceph-post-file(8)

summary: post files for ceph developers

text:
# ceph-post-file(8)

## NAME
ceph-post-file - post files for ceph developers

## SYNOPSIS

```
ceph-post-file [-d description] [-u *user] file or dir ...
```

## DESCRIPTION
ceph-post-file will upload files or directories to ceph.com for later analysis by Ceph developers.

Each invocation uploads files or directories to a separate directory with a unique tag. That tag can be passed to a developer or referenced in a bug report (http://tracker.ceph.com/). Once the upload completes, the directory is marked non-readable and non-writeable to prevent access or modification by other users.

## WARNING
Basic measures are taken to make posted data be visible only to developers with access to ceph.com infrastructure. However, users should think twice and/or take appropriate precautions before posting potentially sensitive data (for example, logs or data directories that contain Ceph secrets).

## OPTIONS
-d *description*, --description *description*

Add a short description for the upload. This is a good opportunity to reference a bug number. There is no default value.

-u *user*

Set the user metadata for the upload. This defaults to whoami`@`hostname -f.

## EXAMPLES
To upload a single log:

```
ceph-post-file /var/log/ceph/ceph-mon.`hostname`.log
```

To upload several directories:

```
ceph-post-file -d 'mon data directories' /var/log/ceph/mon/*
```

## AVAILABILITY
ceph-post-file is part of Ceph, a massively scalable, open-source, distributed storage system. Please refer to the Ceph documentation at https://docs.ceph.com for more information.

## SEE ALSO
ceph(8), ceph-debugpack(8),

## COPYRIGHT
2010-2024, Inktank Storage, Inc. and contributors. Licensed under Creative Commons Attribution Share Alike 3.0 (CC-BY-SA-3.0)


ceph-rbdnamer

title: ceph-rbdnamer(8)

summary: udev helper to name RBD devices

text:
# ceph-rbdnamer(8)

## NAME
ceph-rbdnamer - udev helper to name RBD devices

## SYNOPSIS

```
ceph-rbdnamer num
```

## DESCRIPTION
ceph-rbdnamer prints the pool, namespace, image and snapshot names for a given RBD device to stdout. It is used by udev device manager to set up RBD device symlinks. The appropriate udev rules are provided in a file named 50-rbd.rules.

## AVAILABILITY
ceph-rbdnamer is part of Ceph, a massively scalable, open-source, distributed storage system. Please refer to the Ceph documentation at https://docs.ceph.com for more information.

## SEE ALSO
rbd(8), ceph(8)

## COPYRIGHT
2010-2024, Inktank Storage, Inc. and contributors. Licensed under Creative Commons Attribution Share Alike 3.0 (CC-BY-SA-3.0)


ceph-syn

title: ceph-syn(8)

summary: ceph synthetic workload generator

text:
# ceph-syn(8)

## NAME
ceph-syn - ceph synthetic workload generator

## SYNOPSIS

```
ceph-syn [ -m monaddr:port ] --syn command ...
```

## DESCRIPTION
ceph-syn is a simple synthetic workload generator for the Ceph distributed file system. It uses the userspace client library to generate simple workloads against a currently running file system. The file system need not be mounted via ceph-fuse(8) or the kernel client.

One or more --syn command arguments specify the particular workload, as documented below.

## OPTIONS
-d

Detach from console and daemonize after startup.

-c ceph.conf, --conf=ceph.conf

Use ceph.conf configuration file instead of the default /etc/ceph/ceph.conf to determine monitor addresses during startup.

-m monaddress[:port]

Connect to specified monitor (instead of looking through ceph.conf).

--num_client num

Run num different clients, each in a separate thread.

--syn workloadspec

Run the given workload. May be specified as many times as needed. Workloads will normally run sequentially.

## WORKLOADS
Each workload should be preceded by --syn on the command line. This is not a complete list.

mknap path snapname

Create a snapshot called snapname on path.

rmsnap path snapname

Delete snapshot called snapname on path.

rmfile path

Delete/unlink path.

writefile sizeinmb blocksize

Create a file, named after our client id, that is sizeinmb MB by writing blocksize chunks.

readfile sizeinmb blocksize

Read file, named after our client id, that is sizeinmb MB by writing blocksize chunks.

rw sizeinmb blocksize

Write file, then read it back, as above.

makedirs numsubdirs numfiles depth

Create a hierarchy of directories that is depth levels deep. Give each directory numsubdirs subdirectories and numfiles files.

walk

Recursively walk the file system (like find).

## AVAILABILITY
ceph-syn is part of Ceph, a massively scalable, open-source, distributed storage system. Please refer to the Ceph documentation at https://docs.ceph.com for more information.

## SEE ALSO
ceph(8), ceph-fuse(8)

## COPYRIGHT
2010-2024, Inktank Storage, Inc. and contributors. Licensed under Creative Commons Attribution Share Alike 3.0 (CC-BY-SA-3.0)


crushdiff

title: crushdiff(8)

summary: ceph crush map test tool

text:
# crushdiff(8)

## NAME
crushdiff - ceph crush map test tool

## SYNOPSIS

```
crushdiff [ --osdmap osdmap ] [ --pg-dump pg-dump ]
[ --compiled ] [ --verbose ] command crushmap
```

## DESCRIPTION
crushdiff is a utility that lets you test the effect of a crushmap change: number of pgs, objects, bytes moved. This is a wrapper around osdmaptool(8), relying on its --test-map-pgs-dump option to get the list of changed pgs. Additionally it uses pg stats to calculate the numbers of objects and bytes moved.

By default, crushdiff will use the cluster current osdmap and pg stats, which requires access to the cluster. Though one can use the --osdmap and --pg-dump options to test against previously obtained data.

## OPTIONS
--compiled

The input/output crushmap is compiled. If the options is not specified the expected/returned crushmap is in txt (decompiled) format.

--pg-dump <pg-dump>

JSON output of ceph pg dump. If not specified crushdiff will try to get data running the command itself.

--osdmap <osdmap>

The cluster osdmap, obtained with ceph osd getmap command. If not specified crushdiff will try to get data running the command itself.

--verbose

Produce diagnostic output.

## COMMANDS
compare crushmap

Compare the crushmap from crushmap file with the crushmap from the cluster osdmap. The output will show the expected number of pgs, objects, bytes moved when the new crushmap is installed.

export crushmap

Export crushmap to crushmap file from the cluster osdmap.

import crushmap

Import crushmap from crushmap file to the cluster osdmap.

## EXAMPLE
Get the current crushmap:

```
crushdiff export cm.txt
```

Edit the map:

```
$EDITOR cm.txt
```

Check the result:

```
crushdiff compare cm.txt
79/416 (18.99%) pgs affected
281/1392 (20.19%) objects affected
80/1248 (6.41%) pg shards to move
281/4176 (6.73%) pg object shards to move
730.52Mi/10.55Gi (6.76%) bytes to move
```

When running with --verbose option the output will also contain detailed information about the affected pgs, like below:

```
4.3     [0, 2, 1] -> [1, 4, 2]
4.b     [0, 1, 3] -> [2, 1, 3]
4.c     [4, 0, 1] -> [4, 1, 2]
```

i.e. a pg number, and its old and the new osd active sets.

If the result is satisfactory install the updated map:

```
crushdiff import cm.txt
```

## AVAILABILITY
crushdiff is part of Ceph, a massively scalable, open-source, distributed storage system. Please refer to the Ceph documentation at https://docs.ceph.com for more information.

## SEE ALSO
ceph(8), crushtool(8), osdmaptool(8),

## COPYRIGHT
2010-2024, Inktank Storage, Inc. and contributors. Licensed under Creative Commons Attribution Share Alike 3.0 (CC-BY-SA-3.0)