statd

title: statd(8)

summary: NSM service daemon

text:
# statd(8)

## NAME
rpc.statd - NSM service daemon

## SYNOPSIS
rpc.statd [-dh?FLNvV] [-H prog] [-n my-name] [-o outgoing-port] [-p listener-port] [-P path] [--nlm-port port] [--nlm-udp-port port]

## DESCRIPTION
File locks are not part of persistent file system state. Lock state is thus lost when a host reboots.

Network file systems must also detect when lock state is lost because a remote host has rebooted. After an NFS client reboots, an NFS server must release all file locks held by applications that were running on that client. After a server reboots, a client must remind the server of file locks held by applications running on that client.

For NFS version 2 [RFC1094] and NFS version 3 [RFC1813], the Network Status Monitor protocol (or NSM for short) is used to notify NFS peers of reboots. On Linux, two separate user-space components constitute the NSM service:

rpc.statd

A daemon that listens for reboot notifications from other hosts, and manages the list of hosts to be notified when the local system reboots

sm-notify

A helper program that notifies NFS peers after the local system reboots

The local NFS lock manager alerts its local rpc.statd of each remote peer that should be monitored. When the local system reboots, the sm-notify command notifies the NSM service on monitored peers of the reboot. When a remote reboots, that peer notifies the local rpc.statd, which in turn passes the reboot notification back to the local NFS lock manager.

## NSM OPERATION IN DETAIL
The first file locking interaction between an NFS client and server causes the NFS lock managers on both peers to contact their local NSM service to store information about the opposite peer. On Linux, the local lock manager contacts rpc.statd.

rpc.statd records information about each monitored NFS peer on persistent storage. This information describes how to contact a remote peer in case the local system reboots, how to recognize which monitored peer is reporting a reboot, and how to notify the local lock manager when a monitored peer indicates it has rebooted.

An NFS client sends a hostname, known as the client's caller_name, in each file lock request. An NFS server can use this hostname to send asynchronous GRANT calls to a client, or to notify the client it has rebooted.

The Linux NFS server can provide the client's caller_name or the client's network address to rpc.statd. For the purposes of the NSM protocol, this name or address is known as the monitored peer's mon_name. In addition, the local lock manager tells rpc.statd what it thinks its own hostname is. For the purposes of the NSM protocol, this hostname is known as my_name.

There is no equivalent interaction between an NFS server and a client to inform the client of the server's caller_name. Therefore NFS clients do not actually know what mon_name an NFS server might use in an SM_NOTIFY request. The Linux NFS client uses the server hostname from the mount command to identify rebooting NFS servers.

### Reboot notification
When the local system reboots, the sm-notify command reads the list of monitored peers from persistent storage and sends an SM_NOTIFY request to the NSM service on each listed remote peer. It uses the mon_name string as the destination. To identify which host has rebooted, the sm-notify command sends the my_name string recorded when that remote was monitored. The remote rpc.statd matches incoming SM_NOTIFY requests using this string, or the caller's network address, to one or more peers on its own monitor list.

If rpc.statd does not find a peer on its monitor list that matches an incoming SM_NOTIFY request, the notification is not forwarded to the local lock manager. In addition, each peer has its own NSM state number, a 32-bit integer that is bumped after each reboot by the sm-notify command. rpc.statd uses this number to distinguish between actual reboots and replayed notifications.

Part of NFS lock recovery is rediscovering which peers need to be monitored again. The sm-notify command clears the monitor list on persistent storage after each reboot.

## OPTIONS
-d, --no-syslog

Causes rpc.statd to write log messages on stderr instead of to the system log, if the -F option was also specified.

-F, --foreground

Keeps rpc.statd attached to its controlling terminal so that NSM operation can be monitored directly or run under a debugger. If this option is not specified, rpc.statd backgrounds itself soon after it starts.

-h, -?, --help

Causes rpc.statd to display usage information on stderr and then exit.

-H, --ha-callout prog

Specifies a high availability callout program. If this option is not specified, no callouts are performed. See the High-availability callouts section below for details.

-L, --no-notify

Prevents rpc.statd from running the sm-notify command when it starts up, preserving the existing NSM state number and monitor list.

Note: the sm-notify command contains a check to ensure it runs only once after each system reboot. This prevents spurious reboot notification if rpc.statd restarts without the -L option.

-n, --name ipaddr | hostname

This string is only used by the sm-notify command as the source address from which to send reboot notification requests.

The ipaddr form can be expressed as either an IPv4 or an IPv6 presentation address. If this option is not specified, rpc.statd uses a wildcard address as the transport bind address. See sm-notify(8) for details.

-N

Causes rpc.statd to run the sm-notify command, and then exit. Since the sm-notify command can also be run directly, this option is deprecated.

-o, --outgoing-port port

Specifies the source port number the sm-notify command should use when sending reboot notifications. See sm-notify(8) for details.

-p, --port port

Specifies the port number used for RPC listener sockets. If this option is not specified, rpc.statd will try to consult /etc/services, if gets port succeed, set the same port for all listener socket, otherwise chooses a random ephemeral port for each listener socket.

This option can be used to fix the port value of its listeners when SM_NOTIFY requests must traverse a firewall between clients and servers.

-T, --nlm-port port

Specifies the port number that lockd should listen on for NLM requests. This sets both the TCP and UDP ports unless the UDP port is set separately.

-U, --nlm-udp-port port

Specifies the UDP port number that lockd should listen on for NLM requests.

-P, --state-directory-path pathname

Specifies the pathname of the parent directory where NSM state information resides. If this option is not specified, rpc.statd uses /var/lib/nfs/statd by default.

After starting, rpc.statd attempts to set its effective UID and GID to the owner and group of the subdirectory sm of this directory. After changing the effective ids, rpc.statd only needs to access files in sm and sm.bak within the state-directory-path.

-v, -V, --version

Causes rpc.statd to display version information on stderr and then exit.

## CONFIGURATION FILE
Many of the options that can be set on the command line can also be controlled through values set in the [statd] or, in some cases, the [lockd] sections of the /etc/nfs.conf configuration file. Values recognized in the [statd] section include port, outgoing-port, name, state-directory-path, and ha-callout which each have the same effect as the option with the same name.

The values recognized in the [lockd] section include port and udp-port which have the same effect as the --nlm-port and --nlm-udp-port options, respectively.

## SECURITY
The rpc.statd daemon must be started as root to acquire privileges needed to create sockets with privileged source ports, and to access the state information database. Because rpc.statd maintains a long-running network service, however, it drops root privileges as soon as it starts up to reduce the risk of a privilege escalation attack.

During normal operation, the effective user ID it chooses is the owner of the state directory. This allows it to continue to access files in that directory after it has dropped its root privileges. To control which user ID rpc.statd chooses, simply use chown(1) to set the owner of the state directory.

You can also protect your rpc.statd listeners using the tcp_wrapper library or iptables(8). To use the tcp_wrapper library, add the hostnames of peers that should be allowed access to /etc/hosts.allow. Use the daemon name statd even if the rpc.statd binary has a different filename.

For further information see the tcpd(8) and hosts_access(5) man pages.

## ADDITIONAL NOTES
Lock recovery after a reboot is critical to maintaining data integrity and preventing unnecessary application hangs. To help rpc.statd match SM_NOTIFY requests to NLM requests, a number of best practices should be observed, including:

The UTS nodename of your systems should match the DNS names that NFS peers use to contact them

The UTS nodenames of your systems should always be fully qualified domain names

The forward and reverse DNS mapping of the UTS nodenames should be consistent

The hostname the client uses to mount the server should match the server's mon_name in SM_NOTIFY requests it sends

Unmounting an NFS file system does not necessarily stop either the NFS client or server from monitoring each other. Both may continue monitoring each other for a time in case subsequent NFS traffic between the two results in fresh mounts and additional file locking.

On Linux, if the lockd kernel module is unloaded during normal operation, all remote NFS peers are unmonitored. This can happen on an NFS client, for example, if an automounter removes all NFS mount points due to inactivity.

### High-availability callouts
rpc.statd can exec a special callout program during processing of successful SM_MON, SM_UNMON, and SM_UNMON_ALL requests, or when it receives SM_NOTIFY. Such a program may be used in High Availability NFS (HA-NFS) environments to track lock state that may need to be migrated after a system reboot.

The name of the callout program is specified with the -H option. The program is run with 3 arguments: The first is either add-client del-client or sm-notify depending on the reason for the callout. The second is the mon_name of the monitored peer. The third is the caller_name of the requesting lock manager for add-client or del-client , otherwise it is IP_address of the caller sending SM_NOTIFY. The forth is the state_value in the SM_NOTIFY request.

### IPv6 and TI-RPC support
TI-RPC is a pre-requisite for supporting NFS on IPv6. If TI-RPC support is built into rpc.statd, it attempts to start listeners on network transports marked 'visible' in /etc/netconfig. As long as at least one network transport listener starts successfully, rpc.statd will operate.

## ENVIRONMENT
RPC_STATD_NO_NOTIFY=

If set to a positive integer, has the same effect as --no-notify.

## FILES
/var/lib/nfs/statd/sm

directory containing monitor list

/var/lib/nfs/statd/sm.bak

directory containing notify list

/var/lib/nfs/statd/state

NSM state number for this host

/run/run.statd.pid

pid file

/etc/netconfig

network transport capability database

## SEE ALSO
sm-notify(8), nfs(5), rpc.nfsd(8), rpcbind(8), tcpd(8), hosts_access(5), iptables(8), netconfig(5)

RFC 1094 - "NFS: Network File System Protocol Specification"

RFC 1813 - "NFS Version 3 Protocol Specification"

OpenGroup Protocols for Interworking: XNFS, Version 3W - Chapter 11

## AUTHORS
Jeff Uphoff <juphoff@users.sourceforge.net>

Olaf Kirch <okir@monad.swb.de>

H.J. Lu <hjl@gnu.org>

Lon Hohberger <hohberger@missioncriticallinux.com>

Paul Clements <paul.clements@steeleye.com>

Chuck Lever <chuck.lever@oracle.com>


ganesha-cache-config

package: nfs-ganesha

title: ganesha-cache-config(8)

summary: NFS Ganesha Cache Configuration File

text:
# ganesha-cache-config(8)

## NAME
ganesha-cache-config - NFS Ganesha Cache Configuration File

## SYNOPSIS

```
/etc/ganesha/ganesha.conf
```

## DESCRIPTION
NFS-Ganesha reads the configuration data from: | /etc/ganesha/ganesha.conf

This file lists NFS-Ganesha Cache config options.

### MDCACHE {}
NParts (uint32, range 1 to 32633, default 7)

Partitions in the MDCACHE tree.

Cache_Size(uint32, range 1 to UINT32_MAX, default 32633)

Per-partition hash table size.

Use_Getattr_Directory_Invalidation(bool, default false)

Use getattr for directory invalidation.

Dir_Chunk(uint32, range 0 to UINT32_MAX, default 128)

Size of per-directory dirent cache chunks, 0 means directory chunking is not enabled. Dir_Chunk should always be enabled. Most FSAL modules especially FSAL_RGW/FSAL_GLUSTER need it to make readdir work well.

Detached_Mult(uint32, range 1 to UINT32_MAX, default 1)

Max number of detached directory entries expressed as a multiple of the chunk size.

Entries_HWMark(uint32, range 1 to UINT32_MAX, default 100000)

The point at which object cache entries will start being reused.

Entries_Release_Size(uint32, range 0 to UINT32_MAX, default 100)

The number of entries attempted to release each time when the handle cache has exceeded the entries high water mark.

Chunks_HWMark(uint32, range 1 to UINT32_MAX, default 1000)

The point at which dirent cache chunks will start being reused.

Chunks_LWMark(uint32, range 1 to UINT32_MAX, default 1000)

The target for reaping dirent cache chunks to drain the cache. Entries_HWMark, Dir_Chunk, Chunks_HWMark, Chunks_LWMark all play together. While a dirent chunk is cached, the object cache entries that are part of that dirent chunk will be retained in the cache indefinitely. Note that this means that possibly Chunks_LWMark * Dir_Chunk entries will be retained. If this is larger than Entries_HWMark, then the object cache may end up remaining above Entries_HWMark for a significant time. Consider the average size of the directories and the number of directories desired to be maintained in the cache and set Chunks_LWMark appropriately.

Note that the Chunks_LWMark defaults to the same value as Chunks_HWMark and these are set to 1/100 of Entries_HWMark suggesting an average directory size of 100. It may be desirable to set Chunks_LWMark less than Chunks_HWMark if it is desirable to allow large directory chunks to not immediately be re-used, but it is desirable to in the short term drain the dirent cache down to a smaller number.

LRU_Run_Interval(uint32, range 1 to 24 * 3600, default 90)

Base interval in seconds between runs of the LRU cleaner thread.

Cache_FDs(bool, true)

If "Cache_FDs" is set to false, the reaper thread aggressively closes FDs , significantly reducing the number of open FDs. This will help to maintain a minimal number of open FDs. If "Cache_FDs" is set to true (default), FDs are cached, and the LRU reaper thread closes FDs only when the current open FD count reaches or exceeds the "fds_lowat" threshold.

FD_Limit_Percent(uint32, range 0 to 100, default 99)

The percentage of the system-imposed maximum of file descriptors at which Ganesha will deny requests.

FD_HWMark_Percent(uint32, range 0 to 100, default 90)

The percentage of the system-imposed maximum of file descriptors above which Ganesha will make greater efforts at reaping.

FD_LWMark_Percent(uint32, range 0 to 100, default 50)

The percentage of the system-imposed maximum of file descriptors below which Ganesha will not reap file descriptors.

Reaper_Work(uint32, range 1 to 2000, default 0)

Roughly, the amount of work to do on each pass through the thread under normal conditions. (Ideally, a multiple of the number of lanes.) This setting is deprecated. Please use Reaper_Work_Per_Lane

Reaper_Work_Per_Lane(uint32, range 1 to UINT32_MAX, default 50)

This is the number of handles per lane to scan when performing LRU maintenance. This task is performed by the Reaper thread.

Biggest_Window(uint32, range 1 to 100, default 40)

The largest window (as a percentage of the system-imposed limit on FDs) of work that we will do in extremis.

Required_Progress(uint32, range 1 to 50, default 5)

Percentage of progress toward the high water mark required in in a pass through the thread when in extremis

Futility_Count(uint32, range 1 to 50, default 8)

Number of failures to approach the high watermark before we disable caching, when in extremis.

Dirmap_HWMark(uint32, range 1 to UINT32_MAX, default 10000)

The point at which dirmap entries are reused. This puts a practical limit on the number of simultaneous readdirs that may be in progress on an export for a whence-is-name FSAL (currently only FSAL_RGW)

## SEE ALSO
ganesha-config(8)


ganesha-config

title: ganesha-config(8)

summary: NFS Ganesha Configuration File

text:
# ganesha-config(8)

## NAME
ganesha-config - NFS Ganesha Configuration File

## SYNOPSIS

```
/etc/ganesha/ganesha.conf
```

## DESCRIPTION
NFS-Ganesha obtains configuration data from the configuration file:

/etc/ganesha/ganesha.conf

The configuration file consists of following parts:

### Comments
Empty lines and lines starting with '#' are comments.:

```
# This whole line is a comment
Protocol = TCP; # The rest of this line is a comment
```

### Blocks
Related options are grouped together into "blocks". A block is a name followed by parameters enclosed between "{" and "}". A block can contain other sub blocks as well.:

```
EXPORT
{

    Export_ID = 1;

    FSAL {

        Name = VFS:

    }
}
```

NOTE: FSAL is a sub block. Refer to BLOCKS section for list of blocks and options.

### Options
Configuration options can be of following types.

1. Numeric. Numeric options can be defined in octal, decimal, or hexadecimal. The format follows ANSI C syntax. eg.:

```
mode = 0755;  # This is octal 0755, 493 (decimal)
```

Numeric values can also be negated or logical NOT'd. eg.:

```
anonymousuid = -2; # this is a negative
mask = ~0xff; # Equivalent to 0xffffff00 (for 32 bit integers)
```

2. Boolean. Possible values are true, false, yes and no. 1 and 0 are not acceptable.

3. List. The option can contain a list of possible applicable values. Protocols = 3, 4, 9p;

4. String. String options such as export Path need not be enclosed in quotes but is recommended. If special characters are included, particularly non-ASCII characters, use of either single or double quotes may be required. Another set of strings that should be quoted are any strings that are a keyword or if spaces are part of the string.

### Including other config files
Additional files can be referenced in a configuration using '%include' or '%dir' or '%url' directives.:

```
%include <filename>
%dir <directory_name>
%url <url, e.g., rados://mypool/mynamespace/myobject>
```

The included file is inserted into the configuration text in place of the %include or %dir or %url line. Sub-inclusions may be to any depth. Filenames or directory or URLs may optionally use '"':

```
%include base.conf
%include "base.conf"
%dir /etc/ganesha/ganesha_config/
%dir "/etc/ganesha/ganesha_config/"
%url rados://mypool/mynamespace/myobject
%url "rados://mypool/mynamespace/myobject"
%url rados://mypool/myobject
%url "rados://mypool/myobject"
```

In the case of rados:// URLs, providing a two-component URL indicates that the default namespace should be used.

### Reloading Config
A config reload can be triggered by sending the ganesha.nfsd process a SIGHUP. Not all config options may be changed with reload, those that can will be documented in the individual sections.

In general, currently dynamic config is supported for EXPORT and LOG options.

## BLOCKS
NFS-Ganesha supports the following blocks:

### EXPORT {}
Along with its configuration options, the EXPORT block supports FSAL and CLIENT sub-blocks. See ganesha-export-config(8) for usage of this block and its sub-blocks.

### EXPORT_DEFAULTS {}
Refer to ganesha-export-config(8) for usage

### MDCACHE {}
Refer to ganesha-cache-config(8) for usage

### NFS_CORE_PARAM {}
Refer to ganesha-core-config(8) for usage

### NFS_IP_NAME {}
Refer to ganesha-core-config(8) for usage

### NFS_KRB5 {}
Refer to ganesha-core-config(8) for usage

### NFSv4 {}
Refer to ganesha-core-config(8) for usage

### _9P {}
Refer to ganesha-9p-config(8) for usage

### LOG {}
Refer to ganesha-log-config(8) for usage

1.**LOG { FACILITY {} }** 2.**LOG { FORMAT {} }**

### RADOS_KV {}
Refer to ganesha-core-config(8) for usage

### RADOS_URLS {}
Refer to ganesha-core-config(8) for usage

### FSAL_LIST {}
Refer to ganesha-core-config(8) for usage

### CEPH {}
Refer to ganesha-ceph-config(8) for usage

### GLUSTER {}
Refer to ganesha-gluster-config(8) for usage

### GPFS {}
Refer to ganesha-gpfs-config(8) for usage

### PROXY_V4 {}
Refer to ganesha-proxy-config(8) for usage

### PROXY_V3 {}
Refer to ganesha-proxy-v3-config(8) for usage

### RGW {}
Refer to ganesha-rgw-config(8) for usage

### VFS {}
Refer to ganesha-vfs-config(8) for usage

### XFS {}
Refer to ganesha-xfs-config(8) for usage

## EXAMPLE
Along with "ganesha.conf", for each installed FSAL, a sample config file is added at:

```
/etc/ganesha
```

## SEE ALSO
ganesha-log-config(8) ganesha-rgw-config(8) ganesha-vfs-config(8) ganesha-lustre-config(8) ganesha-xfs-config(8) ganesha-gpfs-config(8) ganesha-gluster-config(8) ganesha-9p-config(8) ganesha-proxy-config(8) ganesha-proxy-v3-config(8) ganesha-ceph-config(8) ganesha-core-config(8) ganesha-export-config(8)


ganesha-core-config

title: ganesha-core-config(8)

summary: NFS Ganesha Core Configuration File

text:
# ganesha-core-config(8)

## NAME
ganesha-core-config - NFS Ganesha Core Configuration File

## SYNOPSIS

```
/etc/ganesha/ganesha.conf
```

## DESCRIPTION
NFS-Ganesha reads the configuration data from: | /etc/ganesha/ganesha.conf

This file lists NFS related core config options.

### NFS_CORE_PARAM {}
Core parameters:

HAProxy_Hosts (host list, empty)

This is the list of hosts that can serve as HAProxy load balancers/proxies that will use the HAProxy protocol to indicate to Ganesha the actual end client IP address. This parameter may be repeated to extend the list. Host list entries can take on one of the following forms:

* Match any host @name Netgroup name x.x.x.x/y IPv4 network address, IPv6 addresses are also allowed but the format is too complex to show here

wildcarded If the string contains at least one ? or *

character (and is not simply "*"), the string is used to pattern match host names. Note that [] may also be used, but the pattern MUST have at least one ? or *

hostname Match a single host (match is by IP address, all

addresses returned by getaddrinfo will match, the getaddrinfo call is made at config parsing time)

IP address Match a single host

NFS_Port (uint16, range 0 to UINT16_MAX, default 2049)

Port number used by NFS Protocol.

MNT_Port (uint16, range 0 to UINT16_MAX, default 0)

Port number used by MNT Protocol.

NLM_Port (uint16, range 0 to UINT16_MAX, default 0)

Port number used by NLM Protocol.

Rquota_Port (uint16, range 0 to UINT16_MAX, default 875)

Port number used by Rquota Protocol.

Monitoring_Port (uint16, range 0 to UINT16_MAX, default 9587)

Port number used to export monitoring metrics.

Bind_addr(IPv4 or IPv6 addr, default 0.0.0.0)

The address to which to bind for our listening port.

NFS_Program(uint32, range 1 to INT32_MAX, default 100003)

RPC program number for NFS.

MNT_Program(uint32, range 1 to INT32_MAX, default 100005)

RPC program number for MNT.

NLM_Program(uint32, range 1 to INT32_MAX, default 100021)

RPC program number for NLM.

Drop_IO_Errors(bool, default false)

For NFSv3, whether to drop rather than reply to requests yielding I/O errors. It results in client retry.

Drop_Inval_Errors(bool, default false)

For NFSv3, whether to drop rather than reply to requests yielding invalid argument errors. False by default and settable with Drop_Inval_Errors.

Drop_Delay_Errors(bool, default false)

For NFSv3, whether to drop rather than reply to requests yielding delay errors. False by default and settable with Drop_Delay_Errors.

Plugins_Dir(path, default "/usr/lib64/ganesha")

Path to the directory containing server specific modules

Enable_NFS_Stats(bool, default true)

Whether to collect performance statistics. By default the performance counting is enabled. Enable_NFS_Stats can be enabled or disabled dynamically via ganesha_stats.

Enable_Fast_Stats(bool, default false)

Whether to use fast stats. If enabled this will skip statistics counters collection for per client and per export.

Enable_FSAL_Stats(bool, default false)

Whether to count and collect FSAL specific performance statistics. Enable_FSAL_Stats can be enabled or disabled dynamically via ganesha_stats

Enable_FULLV3_Stats(bool, default false)

Whether to count and collect "detailed statistics" for NFSv3. Enable_FULLV3_Stats can be enabled or disabled dynamically via ganesha_stats.

Enable_FULLV4_Stats(bool, default false)

Whether to count and collect "detailed statistics" for NFSv4. Enable_FULLV4_Stats can be enabled or disabled dynamically via ganesha_stats.

Enable_CLNT_AllOps_Stats(bool, default false)

Whether to count and collect statistics for all NFS operations requested by NFS clients. Enable_CLNT_AllOps_Stats can be enabled or disabled dynamically via ganesha_stats.

Short_File_Handle(bool, default false)

Whether to use short NFS file handle to accommodate VMware NFS client. Enable this if you have a VMware NFSv3 client. VMware NFSv3 client has a max limit of 56 byte file handles.

Manage_Gids_Expiration(int64, range 0 to 7*24*60*60, default 30*60)

How long the server will trust information it got by calling getgroups() when "Manage_Gids = TRUE" is used in a export entry.

heartbeat_freq(uint32, range 0 to 5000 default 1000)

Frequency of dbus health heartbeat in ms.

Enable_NLM(bool, default true)

Whether to support the Network Lock Manager protocol.

Disable_NLM_SHARE(bool, default false)

This option allows disabling support for the NLM4PROC_SHARE and NLM4PROC_UNSHARE RPC procedures that implement share reservations for NFSv3 via NLM. With this set to true, these procedures will fail.

Blocked_Lock_Poller_Interval(int64, range 0 to 180, default 10)

Polling interval for blocked lock polling thread

Protocols(enum list, default [3,4,9P])

Possible values:

3, 4, NFS3, NFS4, V3, V4, NFSv3, NFSv4, 9P

The protocols that Ganesha will listen for. This is a hard limit, as this list determines which sockets are opened. This list can be restricted per export, but cannot be expanded.

NSM_Use_Caller_Name(bool, default false)

Whether to use the supplied name rather than the IP address in NSM operations.

Clustered(bool, default true)

Whether this Ganesha is part of a cluster of Ganeshas. Its vendor specific option.

fsid_device(bool, default false)

Whether to use device major/minor for fsid.

resolve_fs_retries(uint32_t, range 1 to 1000, default 10)

How many times to attempt stat while resolving POSIX filesystems for exports.

resolve_fs_delay(uint32_t, range 1 to 1000, default 100)

How long to delay between stat attempts while resolving POSIX filesystems for exports.

mount_path_pseudo(bool, default false)

Whether to use Pseudo (true) or Path (false) for NFS v3 and 9P mounts. This option defaults to false for backward compatibility, however, for new setups, it's strongly recommended to be set true since it then means the same server path for the mount is used for both v3 and v4.x.

Note that as an export related option, it seems very desirable to be able to change this on config reload, unfortunately, at the moment it is NOT changeable on config reload. A restart is necessary to change this.

Dbus_Name_Prefix

DBus name prefix. Required if one wants to run multiple ganesha instances on single host. The prefix should be different for every ganesha instance. If this is set, the dbus name will be <prefix>.org.ganesha.nfsd

Enable_UDP(enum, values [False, True, Mount], default True)

Whether to create UDP listeners for Mount, NFS, NLM, RQUOTA, and register them with portmapper. Set to false, e.g., to run as non-root. Set to Mount to enable only Mount UDP listener.

Max_Uid_To_Group_Reqs(uint32, range 0 to INT32_MAX, default 0)

Maximum number of concurrent uid2grp requests that can be made by ganesha. In environments with a slow Directory Service Provider, where users are part of large number of groups, and Manage_Gids is set to True, uid2grp queries made by ganesha can fail if a large number of them are made in parallel. This option throttles the number of concurrent uid2grp queries that ganesha makes.

Enable_V3fh_Validation_For_V4(bool, default false)

Set true to enforce when v3 file handle used for v4

Readdir_Res_Size(uint32, range 4096 to 64*1024*1024, default 64*1024*1024)

Response size of readdir request. Suggested values are 4096,8192,16384 and 32768. Recommended 16384(16K) if readdir(ls command) operation performed on directory which has more files.

Readdir_Max_Count(uint32, range 32 to 1024*1024, default 1024*1024)

Maximum number of directory entries returned for a readdir request. Suggested values are 4096,8192,16384 and 32768. Recommended 16384(16K) if readdir(ls command) operation performed on directory which has more files.

Getattrs_In_Complete_Read(bool, default true)

Whether to call extra getattrs after read, in order to check file size and validate the EOF flag correctness. Needed for ESXi client compatibility when FSAL's don't set it correctly.

Enable_malloc_trim(bool, default false)

Set true to enable dynamic malloc_trim support.

Malloc_trim_MinThreshold(uint32, range 1 to INT32_MAX, default 15*1024)

Minimum threshold value to call malloc_trim. The malloc_trim will be called once memory allocation exceeds minimum value. Size in MB's. Note, this setting has no effect when Enable_malloc_trim is set to false.

enable_rpc_cred_fallback(bool, default false)

if Manage_Gids=True and group resolution fails, then use gid data from rpc request.

### Parameters controlling TCP DRC behavior:
DRC_Disabled(bool, default false)

Whether to disable the DRC entirely.

DRC_Recycle_Hiwat(uint32, range 1 to 1000000, default 1024)

High water mark for number of DRCs in recycle queue.

TCP_Npart(uint32, range 1 to 20, default 1)

Number of partitions in the tree for the TCP DRC.

DRC_TCP_Size(uint32, range 1 to 32767, default 1024)

Maximum number of requests in a transport's DRC.

DRC_TCP_Cachesz(uint32, range 1 to 255, default 127)

Number of entries in the O(1) front-end cache to a TCP Duplicate Request Cache.

DRC_TCP_Hiwat(uint32, range 1 to 256, default 64)

High water mark for a TCP connection's DRC at which to start retiring entries if we can.

DRC_TCP_Recycle_Npart(uint32, range 1 to 20, default 7)

Number of partitions in the recycle tree that holds per-connection DRCs so they can be used on reconnection (or recycled.)

DRC_TCP_Recycle_Expire_S(uint32, range 0 to 60*60, default 600)

How long to wait (in seconds) before freeing the DRC of a disconnected client.

DRC_TCP_Checksum(bool, default true)

Whether to use a checksum to match requests as well as the XID

### Parameters controlling UDP DRC behavior:
DRC_UDP_Npart(uint32, range 1 to 100, default 7)

Number of partitions in the tree for the UDP DRC.

DRC_UDP_Size(uint32, range 512, to 32768, default 32768)

Maximum number of requests in the UDP DRC.

DRC_UDP_Cachesz(uint32, range 1 to 2047, default 599)

Number of entries in the O(1) front-end cache to the UDP Duplicate Request Cache.

DRC_UDP_Hiwat(uint32, range 1 to 32768, default 16384)

High water mark for the UDP DRC at which to start retiring entries if we can

DRC_UDP_Checksum(bool, default true)

Whether to use a checksum to match requests as well as the XID.

### Parameters affecting the relation with TIRPC:
RPC_Max_Connections(uint32, range 1 to 1000000, default 1024)

Maximum number of connections for TIRPC.

RPC_Idle_Timeout_S(uint32, range 0 to 60*60, default 300)

Idle timeout (seconds). Default to 300 seconds.

MaxRPCSendBufferSize(uint32, range 1 to 1048576*9, default 1048576)

Size of RPC send buffer.

MaxRPCRecvBufferSize(uint32, range 1 to 1048576*9, default 1048576)

Size of RPC receive buffer.

RPC_Ioq_ThrdMax(uint32, range 1 to 1024*128 default 200)

TIRPC ioq max simultaneous io threads

RPC_GSS_Npart(uint32, range 1 to 1021, default 13)

Partitions in GSS ctx cache table

RPC_GSS_Max_Ctx(uint32, range 1 to 1048576, default 16384)

Max GSS contexts in cache. Default 16k

RPC_GSS_Max_Gc(uint32, range 1 to 1048576, default 200)

Max entries to expire in one idle check

### Parameters for TCP:
Enable_TCP_keepalive(bool, default true)

Whether tcp sockets should use SO_KEEPALIVE

TCP_KEEPCNT(UINT32, range 0 to 255, default 0 -> use system defaults)

Maximum number of TCP probes before dropping the connection

TCP_KEEPIDLE(UINT32, range 0 to 65535, default 0 -> use system defaults)

Idle time before TCP starts to send keepalive probes

TCP_KEEPINTVL(INT32, range 0 to 65535, default 0 -> use system defaults)

Time between each keepalive probe

### NFS_IP_NAME {}
Index_Size(uint32, range 1 to 51, default 17)

Configuration for hash table for NFS Name/IP map.

Expiration_Time(uint32, range 1 to 60*60*24, default 3600)

Expiration time for ip-name mappings.

### NFS_KRB5 {}
PrincipalName(string, default "nfs")

KeytabPath(path, default "")

Kerberos keytab.

CCacheDir(path, default "/var/run/ganesha")

The ganesha credential cache.

Active_krb5(bool, default false)

Whether to activate Kerberos 5. Defaults to true (if Kerberos support is compiled in)

### NFSv4 {}
Graceless(bool, default false)

Whether to disable the NFSv4 grace period.

Lease_Lifetime(uint32, range 1 to 120, default 60)

The NFSv4 lease lifetime.

Grace_Period(uint32, range 0 to 180, default 90)

The NFS grace period.

DomainName(string, default "localdomain")

Domain to use if we aren't using the nfsidmap.

IdmapConf(path, default "/etc/idmapd.conf")

Path to the idmap configuration file.

UseGetpwnam(bool, default false if using idmap, true otherwise)

Whether to use local password (PAM, on Linux) rather than nfsidmap.

Allow_Numeric_Owners(bool, default true)

Whether to allow bare numeric IDs in NFSv4 owner and group identifiers.

Only_Numeric_Owners(bool, default false)

Whether to ONLY use bare numeric IDs in NFSv4 owner and group identifiers.

Delegations(bool, default false)

Whether to allow delegations.

Deleg_Recall_Retry_Delay(uint32_t, range 0 to 10, default 1)

Delay after which server will retry a recall in case of failures

pnfs_mds(bool, default false)

Whether this a pNFS MDS server. For FSAL Gluster, if this is true, set pnfs_mds in gluster block as well.

pnfs_ds(bool, default false)

Whether this a pNFS DS server.

RecoveryBackend(enum, default "fs")

Use different backend for client info:

- fs : filesystem

- fs_ng: filesystem (better resiliency)

- rados_kv : rados key-value

- rados_ng : rados key-value (better resiliency)

- rados_cluster: clustered rados backend (active/active)

RecoveryRoot(path, default "/var/lib/nfs/ganesha")

Specify the root recovery directory for fs or fs_ng recovery backends.

RecoveryDir(path, default "v4recov")

Specify the recovery directory name for fs or fs_ng recovery backends.

RecoveryOldDir(path, "v4old")

Specify the recovery old directory name for fs recovery backend.

Minor_Versions(enum list, values [0, 1, 2], default [0, 1, 2])

List of supported NFSV4 minor version numbers.

Slot_Table_Size(uint32, range 1 to 1024, default 64)

Size of the NFSv4.1 slot table

Enforce_UTF8_Validation(bool, default false)

Set true to enforce valid UTF-8 for path components and compound tags

Max_Client_Ids(uint32, range 0 to UINT32_MAX, default 0)

Specify a max limit on number of NFS4 ClientIDs supported by the server. With filesystem recovery backend, each ClientID translates to one directory. With certain workloads, this could result in reaching inode limits of the filesystem that /var/lib/nfs/ganesha is part of. The above limit can be used as a guardrail to prevent getting into this situation.

Server_Scope(string, default "")

Specify the value which is common for all cluster nodes. For e.g., Name of the cluster or cluster-id.

Server_Owner(string, default "")

Connections to servers with the same server owner can be shared by the client. This is advertised to the client on EXCHANGE_ID.

### RADOS_KV {}
ceph_conf(string, no default)

Connection to ceph cluster, should be file path for ceph configuration.

userid(path, no default)

User ID to ceph cluster.

namespace(string, default NULL)

RADOS Namespace in which to store objects

pool(string, default "nfs-ganesha")

Pool for client info.

grace_oid(string, default "grace")

Name of the object containing the rados_cluster grace DB

nodeid(string, default result of gethostname())

Unique node identifier within rados_cluster

### RADOS_URLS {}
ceph_conf(string, no default)

Connection to ceph cluster, should be file path for ceph configuration.

userid(path, no default)

User ID to ceph cluster.

watch_url(url, no default)

rados:// URL to watch for notifications of config changes. When a notification is received, the server will issue a SIGHUP to itself.

### FSAL_LIST {}
name(string, no default)

This allows listing of the FSALs that will be used. This assures that the config blocks for those FSALs will not result in an error if no exports are configured using that FSAL. This parameter takes a list of FSAL names and the parameter may be listed multiple times.


ganesha-export-config

title: ganesha-export-config(8)

summary: NFS Ganesha Export Configuration File

text:
# ganesha-export-config(8)

## NAME
ganesha-export-config - NFS Ganesha Export Configuration File

## SYNOPSIS
/etc/ganesha/ganesha.conf

## DESCRIPTION
NFS-Ganesha obtains configuration data from the configuration file:

/etc/ganesha/ganesha.conf

This file lists NFS-Ganesha Export block config options.

### EXPORT PERMISSIONS
These options are all "export permissions" options, and are available in EXPORT_DEFAULTS {}, EXPORT {} and CLIENT {} blocks.

These options will all be dynamically updateable.

Access_Type(enum, default None)

Possible values:

None, RW, RO, MDONLY, MDONLY_RO

Protocols(enum list, default none)

Possible values:

3, 4, NFS3, NFS4, V3, V4, NFSv3, NFSv4, 9P

Transports(enum list, values [UDP, TCP, RDMA], default [UDP, TCP])

Anonymous_uid(anonid, range INT32_MIN to UINT32_MAX, default -2)

Anonymous_gid(anonid, range INT32_MIN to UINT32_MAX, default -2)

SecType(enum list, default [none, sys])

Possible values:

none, sys, krb5, krb5i, krb5p

PrivilegedPort(bool, default false)

Manage_Gids(bool, default false)

Squash(enum, default root_sqaush)

Possible values:

root, root_squash, rootsquash, rootid, root_id_squash, rootidsquash, all, all_squash, allsquash, all_anomnymous, allanonymous, no_root_squash, none, noidsquash

Each line of defaults above are synonyms

NFS_Commit(bool, default false)

Delegations(enum, default None)

Possible values:

None, read, write, readwrite, r, w, rw

### CLIENT {}
CLIENT blocks are used in EXPORT_DEFAULTS {}, EXPORT {}, and PSEUDOFS {} blocks.

Each CLIENT block provides a list of clients and the permissions granted those clients. These blocks form an ordered "access control list" for the export. If no client block matches for a particular client, then the permissions in the EXPORT {} block will be used (and if any permissions are not set between the CLIENT and the EXPORT, then the permissions set in EXPORT_DEFAULTS will be used and if still not set, then the code default will be used.

If an EXPORT does not have any CLIENT blocks then the list of CLIENT blocks in EXPORT_DEFAULTS (if any) will be used. No matter the source of the CLIENT blocks, if the client matches none, then the permissions in the EXPORT will be applied next.

To override a restrictive default client list with "everyone" a non-empty client list would have to be specified. Clients = "*" would do the trick.

Even when a CLIENT block matches a client, if a particular export permission is not explicit in that CLIENT block, the permission specified in the EXPORT block will be used, or if not specified there, from the EXPORT_DEFAULTS block, and if not specified there, the permission will default to the default code value noted in the permission option descriptions above.

Note that when the CLIENT blocks are processed on config reload, a new client access list is constructed and atomically swapped in. This allows adding, removing, and re-arranging clients as well as changing the access for any give client.

CLIENT blocks use all the EXPORT PERMISSIONS options (except PSEUDOFS CLIENT blocks which are limited to the same options the PSEUDOFS block is allowed) plus:

Clients(client list, empty)

Client list entries can take on one of the following forms. This parameter may be repeated to extend the list.

* Match any client @name Netgroup name x.x.x.x/y IPv4 network address, IPv6 addresses are also allowed but the format is too complex to show here

wildcarded If the string contains at least one ? or *

character (and is not simply "*"), the string is used to pattern match host names. Note that [] may also be used, but the pattern MUST have at least one ? or *

hostname Match a single client (match is by IP address, all

addresses returned by getaddrinfo will match, the getaddrinfo call is made at config parsing time)

IP address Match a single client

### EXPORT_DEFAULTS {}
All the EXPORT PERMISSIONS options plus:

Attr_Expiration_Time(int32, range -1 to INT32_MAX, default 60)

CLIENT (optional)

See the CLIENT {} block description. There may be any number of these.

### EXPORT_DEFAULTS { CLIENT {} }
See the CLIENT {} block description.

### EXPORT {}
All options below are dynamically changeable with config update unless specified below. This block may be repeated to define multiple exports.

All the EXPORT_PERMISSIONS plus:

Export_id (required):

An identifier for the export, must be unique and between 0 and 65535. If Export_Id 0 is specified, Pseudo must be the root path (/). Export_id is not dynamic per se, changing it essentially removes the old export and introduces a new export.

Path (required)

The directory in the exported file system this export is rooted on (may be ignored for some FSALs). It need not be unique if Pseudo and/or Tag are specified. Note that if it is not unique, and the core option mount_path_pseudo is not set true, a v3 mount using the path will ONLY be able to access the first export configured. To access other exports the Tag option would need to be used.

This option is NOT dynamically updateable since it fundamentally changes the export. To change the path exported, export_id should be changed also.

Pseudo (required v4)

This option specifies the position in the Pseudo Filesystem this export occupies if this is an NFS v4 export. It must be unique. By using different Pseudo options, the same Path may be exported multiple times. This option is used to place the export within the NFS v4 Pseudo Filesystem. This creates a single name space for NFS v4. Clients may mount the root of the Pseudo Filesystem and navigate to exports. Note that the Path and Tag options are not at all visible to NFS v4 clients.

Export id 0 is automatically created to provide the root and any directories necessary to navigate to exports if there is no other export specified with Pseudo = /;. Note that if an export is specified with Pseudo = /;, it need not be export id 0. Specifying such an export with FSAL { name = PSEUDO; } may be used to create a Pseudo Filesystem with specific options. Such an export may also use other FSALs (though directories to reach exports will ONLY be automatically created on FSAL PSEUDO exports).

This option is dynamically changeable and changing it will move the export within the pseudo filesystem. This may be disruptive to clients. Note that if the mount_path_pseudo NFS_CORE_PARAM option is true, the NFSv3 mount path will also change (that should not be disruptive to clients that have the export mounted).

Tag (no default)

This option allows an alternative access for NFS v3 mounts. The option MUST not have a leading /. Clients may not mount subdirectories (i.e. if Tag = foo, the client may not mount foo/baz). By using different Tag options, the same Path may be exported multiple times. This option is not dynamically updatable.

Filesystem_id(fsid, format is uint64.uint64, default unused)

This option allows overriding the filesystem ID provided by the underlying filesystem. Use of this option is discouraged, It will not work if VFS or GPFS is exporting a filesystem that has other filesystems mounted on sub-directories and exported with the same export. This option is not dynamically updateable.

MaxRead (64*1024*1024)

The maximum read size on this export

MaxWrite (64*1024*1024)

The maximum write size on this export

PrefRead (64*1024*1024)

The preferred read size on this export. Note that some older nfs client (e.g. libnfs 1.x) would not handle well for large preferred read size. If so, please try to decrease this size (usually less than 1M is suitable for older nfs client).

PrefWrite (64*1024*1024)

The preferred write size on this export. Note that some older nfs client (e.g. libnfs 1.x) would not handle well for large preferred write size. If so, please try to decrease this size (usually less than 1M is suitable for older nfs client).

PrefReaddir (16384)

The preferred readdir size on this export

MaxOffsetWrite (INT64_MAX)

Maximum file offset that may be written Range is 512 to UINT64_MAX

MaxOffsetRead (INT64_MAX)

Maximum file offset that may be read Range is 512 to UINT64_MAX

DisableReaddirPlus(bool, default false)

Trust_Readdir_Negative_Cache(bool, default false)

The following options may have limits on dynamic effect

UseCookieVerifier(bool, default true)

Updating UseCookieVerifier while a readdir is in progress may result in unexpected behavior.

Disable_ACL(bool, default false)

Disable_ACL is processed at create_export time currently which makes it effectively a static option.

Security_Label(bool, default false)

Attr_Expiration_Time(int32, range -1 to INT32_MAX, default 60)

Attr_Expiration_Time is evaluated when an MDCACHE entry is created, so the dynamic effect of this option may be constrained to new entries.

CLIENT (optional)

See the CLIENT {} block description. There may be any number of these.

FSAL (required)

See the EXPORT { FSAL {} } block. The FSAL for an export can not be changed dynamically. In order to change the FSAL, a new export must be created.

At this time, no FSAL actually supports any updatable options.

### EXPORT { CLIENT {} }
See the CLIENT {} block description.

### EXPORT { FSAL {} }
NFS-Ganesha supports the following FSALs: Ceph Gluster GPFS PROXY_V3 PROXY_V4 RGW VFS XFS LUSTRE LIzardFS KVSFS

Refer to individual FSAL config file for list of config options.

The FSAL blocks generally are less updatable

### PSEUDOFS {}
This block allows specifying some options for the pseudofs root export. It is very similar to an EXPORT block, except only the following options may be specified. CLIENT blocks may be used just like an EXPORT block, however, they are also limited to the same options.

This is basically a shortcut rather than having to fill out options in an EXPORT block. Note that Path, Pseudo, and Export_Id are not included as those values will be fixed ("/" for the paths, and 0 for Export_Id). Other options that don't make sense for the pseudofs root are also not allowed.

An empty PSEUDOFS {} block will produce the same default pseudofs root export as generated if no pseudofs root export is otherwise specified.

This block is most useful to override a restrictive CLIENT list in EXPORT_DEFAULTS {}.

These options will all be dynamically updateable.

Access_Type(enum, default MDONLY_RO)

Possible values:

None, MDONLY_RO

Transports(enum list, values [UDP, TCP, RDMA], default [TCP])

SecType(enum list, default [none, sys, krb5, krb5i, krb5p])

Possible values:

none, sys, krb5, krb5i, krb5p

PrivilegedPort(bool, default false)

Export_id(uint16, range 0 to UINT16_MAX, default 0)

An identifier for the export, must be unique and between 0 and 65535. Export_id is not dynamic per se, changing it essentially removes the old export and introduces a new export.

Filesystem_id(fsid, format is uint64.uint64, default 152.152)

Unlike standard exports, there is no underlying filesystem to get an ID from, so this option is important, however the default value may be used so it need not be specified. This option is not dynamically updateable.

DisableReaddirPlus(bool, default false)

Trust_Readdir_Negative_Cache(bool, default false)

The following options may have limits on dynamic effect

UseCookieVerifier(bool, default true)

Updating UseCookieVerifier while a readdir is in progress may result in unexpected behavior.

### PSEUDOFS { CLIENT {} }
See the CLIENT {} block description but note that beyond the Clients options that works as described, the other options available are as for the PSEUDOFS {} block.

If it is desired to override a restrictive CLIENT list in EXPORT_DEFAULTS, the following PSEUDOFS could be defined that will give all clients the default access to the pseudofs root that otherwise would have been granted if EXPORT_DEFAULTS was not used.

PSEUDOFS {

CLIENT {

Clients =

```
*
```

;

}

}

## DISCUSSION
The EXPORT blocks define the file namespaces that are served by NFS-Ganesha.

In best practice, each underlying filesystem has a single EXPORT defining how that filesystem is to be shared, however, in some cases, it is desirable to sub-divide a filesystem into multiple exports. The danger when this is done is that rogue clients may be able to spoof file handles and access portions of the filesystem not intended to be accessible to that client.

Some FSALs (currently FSAL_VFS, FSAL_GPFS, FSAL_XFS, and FSAL_LUSTRE) are built to support nested filesystems, for example:

/export/fs1 /export/fs1/some/path/fs2

In this case, it is possible to create a single export that exports both filesystems. There is a lot of complexity of what can be done there.

In discussions of filesystems, btrfs filesystems exported by FSAL_VFS may have subvolumes. Starting in NFS-Ganesha V4.0 FSAL_VFS treats these as separate filesystems that are integrated with all the richness of FSAL_VFS exports.

Another significant FSAL from an export point of view is FSAL_PSEUDO. This is used to build glue exports to build the unified NFSv4 name space. This name space may also be used by NFSv3 by setting the NFS_CORE_PARAM option:

mount_path_pseudo = TRUE;

If no FSAL_PSEUDO export is explicitly defined, and there is no EXPORT with:

Pseudo = "/";

NFS-Ganesha will build a FSAL_PSEUDO EXPORT with this Pseudo Path using Export_Id = 0. This automatic EXPORT may be replaced with an explicit EXPORT which need not have Export_Id = 0, it just must have Pseudo = "/" and Protocols = 4.

In building the Pseudo Filesystem, there is a subtle gotcha. Since NFSv4 clients actually moount the root of the Pseudo Filesystem and then use LOOKUP to traverse into the actual directory the sysadmin has mounted from the client, any EXPORTs from "/" to the desired EXPORT MUST have Protocols = 4 specified either in EXPORT_DEFAULTS {}, EXPORT {}, or EXPORT { CLIENT {} }. This is to assure that the client is allowed to traverse each EXPORT.

If Mount_Path_Pseudo = TRUE is being used and an export is desired to be NFSv3 only, Protocols = 3 MUST be specified in the EXPORT {} block. If Protocols is not specified in the EXPORT {} block and is only specified in an EXPORT { CLIENT {} } block, then that export will still be mounted in the Pseudo Filesystem but might not be traversable. Thus if the following two filesystems are exported:

/export/fs1 /export/fs1/some/path/fs2

And the EXPORTs look something like this:

EXPORT { Export_Id = 1; Path = /export/fs1; Peudo = /export/fs1; FSAL {

Name = VFS;

}

CLIENT {

Clients="*"; Protocols=3;

}

}

EXPORT {

Export_Id = 1; Path = /export/fs1/some/path/fs2; Peudo = /export/fs1/some/path/fs2; FSAL {

Name = VFS;

}

CLIENT {

Clients="*"; Protocols=3,4;

}

}

NFSv4 clients will not be able to access /export/fs1/some/path/fs2. The correct way to accomplish this is:

EXPORT { Export_Id = 1; Path = /export/fs1; Peudo = /export/fs1; Protocols=3;

FSAL

{

Name = VFS;

}

}

Note that an EXPORT { CLIENT {} } block is not necessary if the default export permissions are workable.

Note that in order for an EXPORT to be usable with NSFv4 it MUST either have Protocols = 4 specified in the EXPORT block, or the EXPORT block must not have the Protocols option at all such that it defaults to 3,4,9P. Note though that if it is not set and EXPORT_DEFAULTS just has Protocols = 3; then even though the export is mounted in the Pseudo Filesystem, it will not be accessible and the gotcha discussed above may be in play.

## CONFIGURATION RELOAD
In addition to the LOG {} configuration, EXPORT {} config is the main configuration that can be updated while NFS-Ganesha is running by issuing a SIGHUP.

This causes all EXPORT and EXPORT_DEFAULTS blocks to be reloaded. NFS-Ganesha V4.0 and later have some significant improvements to this since it was introduced in NFS-Ganesha V2.4.0. V2.8.0 introduced the ability to remove EXPORTs via SIGHUP configuration reload.

Significantly how things work now is:

On SIGHUP all the EXPORT and EXPORT_DEFAULTS blocks are re-read. There are three conditions that may occur:

An export may be added An export may be removed An export may be updated

A note on Export_Id and Path. These are the primary options that define an export. If Export_Id is changed, the change is treated as a remove of the old Export_Id and an addition of the new Export_Id. Path can not be changed without also changing Export_Id. The Tag and Pseudo options that also contribute to the uniqueness of an EXPORT may be changed.

Any removed exports are removed from the internal tables and if they are NFSv4 exports, unmounted from the Pseudo Filesystem, which will then be re-built as if those exports had not been present.

Any new exports are added to the internal tables, and if the export is an NFSv4 export, they are mounted into the Pseudo Filesystem.

Any updated exports will be modified with the least disruption possible. If the Pseduo option is changed, the export is unmounted from the Pseduo Filesystem in it's original location, and re-mounted in it's new location. Other options are updated atomically, though serially, so for a short period of time, the options may be mixed between old and new. In most cases this should not cause problems. Notably though, the CLIENT blocks are processed to form a new access control list and that list is atomically swapped with the old list. If the Protocols for an EXPORT are changed to include or remove NFSv4, the Pseduo Filesystem will also be updated.

Note that there is no pause in operations other than a lock being taken when the client list is being swapped out, however the export permissions are applied to an operation once. Notably for NFSv4, this is on a PUTFH or LOOKUP which changes the Current File Handle. As an example, if a write is in progress, having passed the permission check with the previous export permissions, the write will complete without interruption. If the write is part of an NFSv4 COMPOUND, the other operations in that COMPOUND that operate on the same file handle will also complete with the previous export permissions.

An update of EXPORT_DEFAULTS changes the export options atomically. These options are only used for those options not otherwise set in an EXPORT {} or CLIENT {} block and are applied when export permissions are evaluated when a new file handle is encountered.

The FSAL { Name } may not be changed and FSALs offer limited support for changing any options in the FSAL block. Some FSALs may validate and warn if any options in the FSAL block are changed when such a change is not supported.

## SEE ALSO
ganesha-config(8) ganesha-rgw-config(8) ganesha-vfs-config(8) ganesha-lustre-config(8) ganesha-xfs-config(8) ganesha-gpfs-config(8) ganesha-9p-config(8) ganesha-proxy-config(8) ganesha-ceph-config(8)


ganesha-log-config

title: ganesha-log-config(8)

summary: NFS Ganesha Log Configuration File

text:
# ganesha-log-config(8)

## NAME
ganesha-log-config - NFS Ganesha Log Configuration File

## SYNOPSIS

```
/etc/ganesha/ganesha.conf
```

## DESCRIPTION
NFS-Ganesha reads the configuration data from: | /etc/ganesha/ganesha.conf

This file lists NFS-Ganesha Log config options.

These options may be dynamically updated by issuing a SIGHUP to the ganesha.nfsd process.

### LOG {}
Default_log_level(token,default EVENT)

If this option is NOT set, the fall back log level will be that specified in the -N option on the command line if that is set, otherwise the fallback level is EVENT. If a SIGHUP is issued, any components not specified in LOG { COMPONENTS {} } will be reset to this value.

The log levels are:

NULL, FATAL, MAJ, CRIT, WARN, EVENT, INFO, DEBUG, MID_DEBUG, M_DBG, FULL_DEBUG, F_DBG

RPC_Debug_Flags(uint32, range 0 to UINT32_MAX, default 7)

Debug flags for TIRPC (default 7 matches log level default EVENT). These flags are only used if the TIRPC component is set to DEBUG

Display_UTC_Timestamp(bool, default false)

Flag to enable displaying UTC date/time in log messages instead of localtime.

### LOG { COMPONENTS {} }
Default_log_level(token,default EVENT)

These entries are of the form:

COMPONENT = LEVEL;

The components are:

ALL, LOG, MEMLEAKS, FSAL, NFSPROTO, NFS_V4, EXPORT, FILEHANDLE, DISPATCH, MDCACHE, MDCACHE_LRU, HASHTABLE, HASHTABLE_CACHE, DUPREQ, INIT, MAIN, IDMAPPER, NFS_READDIR, NFS_V4_LOCK, CONFIG, CLIENTID, SESSIONS, PNFS, RW_LOCK, NLM, RPC, TIRPC, NFS_CB, THREAD, NFS_V4_ACL, STATE, 9P, 9P_DISPATCH, FSAL_UP, DBUS, NFS_MSK

Some synonyms are:

FH = FILEHANDLE HT = HASHTABLE CACHE_INODE_LRU = MDCAHCE_LRU CACHE_INODE = MDCACHE INODE_LRU = MDCAHCE_LRU INODE = MDCACHE DISP = DISPATCH LEAKS = MEMLEAKS NFS3 = NFSPROTO NFS4 = NFS_V4 HT_CACHE = HASHTABLE_CACHE NFS_STARTUP = INIT NFS4_LOCK = NFS_V4_LOCK NFS4_ACL = NFS_V4_ACL 9P_DISP = 9P_DISPATCH

The log levels are:

NULL, FATAL, MAJ, CRIT, WARN, EVENT, INFO, DEBUG, MID_DEBUG, M_DBG, FULL_DEBUG, F_DBG default none

ALL is a special component that when set, sets all components to the specified value, overriding any that are explicitly set. Note that if ALL is then removed from the config and SIGHUP is issued, all components will revert to what is explicitly set, or Default_Log_Level if that is specified, or the original log level from the -N command line option if that was set, or the code default of EVENT.

TIRPC is a special component that also sets the active RPC_Debug_Flags. If the level for TIRPC is DEBUG or MID_DEBUG, the custom RPC_Debug_Flags set by that parameter will be used, otherwise flags will depend on the level the TIRPC component is set to:

NULL or FATAL: 0 CRIT or MAJ: TIRPC_DEBUG_FLAG_ERROR

WARN: TIRPC_DEBUG_FLAG_ERROR | TIRPC_DEBUG_FLAG_WARN

EVENT or INFO: TIRPC_DEBUG_FLAG_ERROR | TIRPC_DEBUG_FLAG_WARN | TIRPC_DEBUG_FLAG_EVENT

DEBUG or MID_DEBUG: RPC_Debug_Flags

FULL_DEBUG: 0xffffffff

### LOG { FACILITY {} }
This block may be repeated to configure multiple log facilities.

name(string, no default)

destination(string, no default, must be supplied)

max_level(token,default FULL_DEBUG)

The log levels are:

NULL, FATAL, MAJ, CRIT, WARN, EVENT, INFO, DEBUG, MID_DEBUG, M_DBG, FULL_DEBUG, F_DBG

headers(token, values [none, component, all], default all)

enable(token, values [idle, active, default], default idle)

### LOG { FORMAT {} }
date_format(enum,default ganesha)

Possible values:

ganesha, true, local, 8601, ISO-8601, ISO 8601, ISO, syslog, syslog_usec, false, none, user_defined

time_format(enum,default ganesha)

Possible values:

ganesha, true, local, 8601, ISO-8601, ISO 8601, ISO, syslog, syslog_usec, false, none, user_defined

user_date_format(string, no default)

user_time_format(string, no default)

EPOCH(bool, default true)

CLIENTIP(bool, default false)

HOSTNAME(bool, default true)

PROGNAME(bool, default true)

PID(bool, default true)

THREAD_NAME(bool, default true)

FILE_NAME(bool, default true)

LINE_NUM(bool, default true)

FUNCTION_NAME(bool, default true)

COMPONENT(bool, default true)

LEVEL(bool, default true)

OP_ID(bool, default false)

## SEE ALSO
ganesha-config(8)


ganesha-ceph-config

package: nfs-ganesha-ceph

title: ganesha-ceph-config(8)

summary: NFS Ganesha CEPH Configuration File

text:
# ganesha-ceph-config(8)

## NAME
ganesha-ceph-config - NFS Ganesha CEPH Configuration File

## SYNOPSIS

```
/etc/ganesha/ceph.conf
```

## DESCRIPTION
NFS-Ganesha install config example for CEPH FSAL: | /etc/ganesha/ceph.conf

This file lists CEPH specific config options.

### EXPORT { FSAL {} }
Name(string, "Ceph")

Name of FSAL should always be Ceph.

Filesystem(string, no default)

Ceph filesystem name string, for mounting an alternate filesystem within the cluster. The default is to mount the default filesystem in the cluster (usually, the first one created).

User_Id(string, no default)

cephx userid used to open the MDS session. This string is what gets appended to "client.". If not set, the ceph client libs will sort this out based on ceph configuration.

Secret_Access_Key(string, no default)

Key to use for the session (if any). If not set, then it uses the normal search path for cephx keyring files to find a key.

sec_label_xattr(char, default "security.selinux xattr of the file")

Enable NFSv4.2 security label attribute. Ganesha supports "Limited Server Mode" as detailed in RFC 7204. Note that not all FSALs support security labels.

cmount_path(string, no default)

If specified, the path within the ceph filesystem to mount this export on. It must be a subset of the EXPORT { Path } parameter. If this and the other EXPORT { FSAL {} } options are the same between multiple exports, those exports will share a single cephfs client. With the default, this effectively defaults to the same path as EXPORT { Path }.

### CEPH {}
Ceph_Conf(path, default "")

umask(mode, range 0 to 0777, default 0)

## SEE ALSO
ganesha-config(8) ganesha-log-config(8) ganesha-core-config(8) ganesha-export-config(8)


ganesha-gluster-config

package: nfs-ganesha-gluster

title: ganesha-gluster-config(8)

summary: NFS Ganesha Gluster Configuration File

text:
# ganesha-gluster-config(8)

## NAME
ganesha-gluster-config - NFS Ganesha Gluster Configuration File

## SYNOPSIS

```
/etc/ganesha/gluster.conf
```

## DESCRIPTION
NFS-Ganesha installs the following sample config file for Gluster FSAL: | /etc/ganesha/gluster.conf

This file lists Gluster specific config options.

### EXPORT { FSAL {} }
Name(string, "GLUSTER")

Name of FSAL should always be GLUSTER.

volume(string, no default, required)

hostname(string, no default, required)

volpath(path, default "/")

glfs_log(path, default "/var/log/ganesha/ganesha-gfapi.log")

up_poll_usec(uint64, range 1 to 60*1000*1000, default 10)

enable_upcall(bool, default true)

transport(enum, values [tcp, rdma], default tcp)

sec_label_xattr(char, default "security.selinux xattr of the file")

### GLUSTER {}
PNFS_MDS(bool, default FALSE)

Set this parameter to true to select this node as MDS

## SEE ALSO
ganesha-log-config(8) ganesha-core-config(8) ganesha-export-config(8)


title: ganesha-proxy-v3-config(8)

summary: NFSv3 Ganesha Proxy Configuration File

text:
# ganesha-proxy-v3-config(8)

## NAME
ganesha-proxy-v3-config - NFSv3 Ganesha Proxy Configuration File

## SYNOPSIS

```
/etc/ganesha/ganesha.conf
```

## DESCRIPTION
NFS-Ganesha install the following config file for Proxy FSAL: | /etc/ganesha/ganesha.conf

This file lists Proxy specific config options.

### EXPORT { FSAL {} }
Name(string, "proxy_v3")

Name of FSAL should always be proxy_v3.

Srv_Addr(ipv4_addr default "127.0.0.1")

FSAL_MAXIOSIZE(default 64 MB)

### PROXY_V3 {}
FSAL_MAXIOSIZE(default 64 MB)

maxread(uint64, default 1 MB)

range 1024 to FSAL_MAXIOSIZE Note that this value will get clamped to the backend's FSINFO response.

maxwrite(uint64, default 1 MB)

range 1024 to FSAL_MAXIOSIZE Note that this value will get clamped to the backend's FSINFO response.

num_sockets(uint32, default 32)

range 1 to 1000 Note that these sockets must be privileged ports, which may be limited on a given system. It is unlikely that 1000 privileged ports are available.

allow_lookup_optimization(bool, default true)

Set this to false to disable lookup optimization for current working directory and current working directory parent from the cache

## SEE ALSO
ganesha-log-config(8) ganesha-core-config(8) ganesha-export-config(8)


title: ganesha-proxy-v4-config(8)

summary: NFS Ganesha Proxy V4 Configuration File

text:
# ganesha-proxy-v4-config(8)

## NAME
ganesha-proxy-v4-config - NFS Ganesha Proxy V4 Configuration File

## SYNOPSIS

```
/etc/ganesha/ganesha.conf
```

## DESCRIPTION
NFS-Ganesha install the following config file for Proxy V4 FSAL: | /etc/ganesha/ganesha.conf

This file lists Proxy V4 specific config options.

### EXPORT { FSAL {} }
Name(string, "proxy_v4")

Name of FSAL should always be proxy_v4.

Retry_SleepTime(uint32, range 0 to 60, default 10)

Srv_Addr(ipv4_addr default "127.0.0.1")

NFS_Service(uint32, range 0 to UINT32_MAX, default 100003)

NFS_SendSize

must be greater than maxwrite+SEND_RECV_HEADER_SPACE

NFS_RecvSize

must be greater than maxread+SEND_RECV_HEADER_SPACE

MAX_READ_WRITE_SIZE(default 1 MB)

SEND_RECV_HEADER_SPACE(default 512 Bytes)

FSAL_MAXIOSIZE(default 64 MB)

NFS_SendSize(uint64, default MAX_READ_WRITE_SIZE + SEND_RECV_HEADER_SPACE)

range 512 + SEND_RECV_HEADER_SPACE to FSAL_MAXIOSIZE

NFS_RecvSize(uint64, default MAX_READ_WRITE_SIZE + SEND_RECV_HEADER_SPACE)

range 512 + SEND_RECV_HEADER_SPACE to FSAL_MAXIOSIZE

NFS_Port(uint16, range 0 to UINT16_MAX, default 2049)

Use_Privileged_Client_Port(bool, default true)

RPC_Client_Timeout(uint32, range 1 to 60*4, default 60)

Remote_PrincipalName(string, no default)

KeytabPath(string, default "/etc/krb5.keytab")

Credential_LifeTime(uint32, range 0 to 86400*2, default 86400)

Sec_Type(enum, values [krb5, krb5i, krb5p], default krb5)

Active_krb5(bool, default false)

Enable_Handle_Mapping(bool, default false)

HandleMap_DB_Dir(string, default "/var/ganesha/handlemap")

HandleMap_Tmp_Dir(string, default "/var/ganesha/tmp")

HandleMap_DB_Count(uint32, range 1 to 16, default 8)

HandleMap_HashTable_Size(uint32, range 1 to 127, default 103)

### PROXY_V4 {}
link_support(bool, default true)

symlink_support(bool, default true)

cansettime(bool, default true)

MAX_READ_WRITE_SIZE(default 1MB)

FSAL_MAXIOSIZE(default 64 MB)

SEND_RECV_HEADER_SPACE(default, 512 Bytes)

maxread(uint64, default MAX_READ_WRITE_SIZE)

range 512 to FSAL_MAXIOSIZE - SEND_RECV_HEADER_SPACE

maxwrite(uint64, default MAX_READ_WRITE_SIZE)

range 512 to FSAL_MAXIOSIZE - SEND_RECV_HEADER_SPACE

umask(mode, range 0 to 0777, default 0)

auth_xdev_export(bool, default false)

## SEE ALSO
ganesha-log-config(8) ganesha-core-config(8) ganesha-export-config(8)


ganesha-rados-cluster-design

package: nfs-ganesha-rados-grace

title: ganesha-rados-cluster-design(8)

summary: Clustered RADOS Recovery Backend Design

text:
# ganesha-rados-cluster-design(8)

## NAME
ganesha-rados-cluster-design - Clustered RADOS Recovery Backend Design

## OVERVIEW
This document aims to explain the theory and design behind the rados_cluster recovery backend, which coordinates grace period enforcement among multiple, independent NFS servers.

In order to understand the clustered recovery backend, it's first necessary to understand how recovery works with a single server:

## SINGLETON SERVER RECOVERY
NFSv4 is a lease-based protocol. Clients set up a relationship to the server and must periodically renew their lease in order to maintain their ephemeral state (open files, locks, delegations or layouts).

When a singleton NFS server is restarted, any ephemeral state is lost. When the server comes comes back online, NFS clients detect that the server has been restarted and will reclaim the ephemeral state that they held at the time of their last contact with the server.

## SINGLETON GRACE PERIOD
In order to ensure that we don't end up with conflicts, clients are barred from acquiring any new state while in the Recovery phase. Only reclaim operations are allowed.

This period of time is called the grace period. Most NFS servers have a grace period that lasts around two lease periods, however nfs-ganesha can and will lift the grace period early if it determines that no more clients will be allowed to recover.

Once the grace period ends, the server will move into its Normal operation state. During this period, no more recovery is allowed and new state can be acquired by NFS clients.

## REBOOT EPOCHS
The lifecycle of a singleton NFS server can be considered to be a series of transitions from the Recovery period to Normal operation and back. In the remainder of this document we'll consider such a period to be an epoch, and assign each a number beginning with 1.

Visually, we can represent it like this, such that each Normal -> Recovery transition is marked by a change in the epoch value:

```
+-------+-------+-------+---------------+-------+
| State | R | N | R | N | R | R | R | N | R | N |
+-------+-------+-------+---------------+-------+
| Epoch |   1   |   2   |       3       |   4   |
+-------+-------+-------+---------------+-------+
```

Note that it is possible to restart during the grace period (as shown above during epoch 3). That just serves to extend the recovery period and the epoch. A new epoch is only declared during a Recovery -> Normal transition.

## CLIENT RECOVERY DATABASE
There are some potential edge cases that can occur involving network partitions and multiple reboots. In order to prevent those, the server must maintain a list of clients that hold state on the server at any given time. This list must be maintained on stable storage. If a client sends a request to reclaim some state, then the server must check to make sure it's on that list before allowing the request.

Thus when the server allows reclaim requests it must always gate it against the recovery database from the previous epoch. As clients come in to reclaim, we establish records for them in a new database associated with the current epoch.

The transition from recovery to normal operation should perform an atomic switch of recovery databases. A recovery database only becomes legitimate on a recovery to normal transition. Until that point, the recovery database from the previous epoch is the canonical one.

## EXPORTING A CLUSTERED FILESYSTEM
Let's consider a set of independent NFS servers, all serving out the same content from a clustered backend filesystem of any flavor. Each NFS server in this case can itself be considered a clustered FS client. This means that the NFS server is really just a proxy for state on the clustered filesystem.

The filesystem must make some guarantees to the NFS server. First filesystem guarantee:

1.

The filesystem ensures that the NFS servers (aka the FS clients) cannot obtain state that conflicts with that of another NFS server.

This is somewhat obvious and is what we expect from any clustered filesystem outside of any requirements of NFS. If the clustered filesystem can provide this, then we know that conflicting state during normal operations cannot be granted.

The recovery period has a different set of rules. If an NFS server crashes and is restarted, then we have a window of time when that NFS server does not know what state was held by its clients.

If the state held by the crashed NFS server is immediately released after the crash, another NFS server could hand out conflicting state before the original NFS client has a chance to recover it.

This must be prevented. Second filesystem guarantee:

2.

The filesystem must not release state held by a server during the previous epoch until all servers in the cluster are enforcing the grace period.

In practical terms, we want the filesystem to provide a way for an NFS server to tell it when it's safe to release state held by a previous instance of itself. The server should do this once it knows that all of its siblings are enforcing the grace period.

Note that we do not require that all servers restart and allow reclaim at that point. It's sufficient for them to simply begin grace period enforcement as soon as possible once one server needs it.

## CLUSTERED GRACE PERIOD DATABASE
At this point the cluster siblings are no longer completely independent, and the grace period has become a cluster-wide property. This means that we must track the current epoch on some sort of shared storage that the servers can all access.

Additionally we must also keep track of whether a cluster-wide grace period is in effect. Any running nodes should all be informed when either of this info changes, so they can take appropriate steps when it occurs.

In the rados_cluster backend, we track these using two epoch values:

C: is the current epoch. This represents the current epoch value

of the cluster

R: is the recovery epoch. This represents the epoch from which

clients are allowed to recover. A non-zero value here means that a cluster-wide grace period is in effect. Setting this to 0 ends that grace period.

In order to decide when to make grace period transitions, each server must also advertise its state to the other nodes. Specifically, each server must be able to determine these two things about each of its siblings:

1.

Does this server have clients from the previous epoch that will require recovery? (NEED)

2.

Is this server enforcing the grace period by refusing non-reclaim locks? (ENFORCING)

We do this with a pair of flags per sibling (NEED and ENFORCING). Each server typically manages its own flags.

The rados_cluster backend stores all of this information in a single RADOS object that is modified using read/modify/write cycles. Typically we'll read the whole object, modify it, and then attempt to write it back. If something changes between the read and write, we redo the read and try it again.

## CLUSTERED CLIENT RECOVERY DATABASES
In rados_cluster the client recovery databases are stored as RADOS objects. Each NFS server has its own set of them and they are given names that have the current epoch (C) embedded in it. This ensures that recovery databases are specific to a particular epoch.

In general, it's safe to delete any recovery database that precedes R when R is non-zero, and safe to remove any recovery database except for the current one (the one with C in the name) when the grace period is not in effect (R==0).

## ESTABLISHING A NEW GRACE PERIOD
When a server restarts and wants to allow clients to reclaim their state, it must establish a new epoch by incrementing the current epoch to declare a new grace period (R=C; C=C+1).

The exception to this rule is when the cluster is already in a grace period. Servers can just join an in-progress grace period instead of establishing a new one if one is already active.

In either case, the server should also set its NEED and ENFORCING flags at the same time.

The other surviving cluster siblings should take steps to begin grace period enforcement as soon as possible. This entails "draining off" any in-progress state morphing operations and then blocking the acquisition of any new state (usually with a return of NFS4ERR_GRACE to clients that attempt it). Again, there is no need for the survivors from the previous epoch to allow recovery here.

The surviving servers must however establish a new client recovery database at this point to ensure that their clients can do recovery in the event of a crash afterward.

Once all of the siblings are enforcing the grace period, the recovering server can then request that the filesystem release the old state, and allow clients to begin reclaiming their state. In the rados_cluster backend driver, we do this by stalling server startup until all hosts in the cluster are enforcing the grace period.

## LIFTING THE GRACE PERIOD
Transitioning from recovery to normal operation really consists of two different steps:

1.

the server decides that it no longer requires a grace period, either due to it timing out or there not being any clients that would be allowed to reclaim.

2.

the server stops enforcing the grace period and transitions to normal operation

These concepts are often conflated in singleton servers, but in a cluster we must consider them independently.

When a server is finished with its own local recovery period, it should clear its NEED flag. That server should continue enforcing the grace period however until the grace period is fully lifted. The server must not permit reclaims after clearing its NEED flag, however.

If the servers' own NEED flag is the last one set, then it can lift the grace period (by setting R=0). At that point, all servers in the cluster can end grace period enforcement, and communicate that fact to the others by clearing their ENFORCING flags.


ganesha-rados-grace

title: ganesha-rados-grace(8)

text:
# ganesha-rados-grace(8)

## NAME
ganesha-rados-grace - manipulate the shared grace management database

## SYNOPSIS

```
ganesha-rados-grace [ --cephconf /path/to/ceph.conf ] [--ns namespace] [ --oid obj_id ] [ --pool pool_id ] [ --userid cephuser ] dump|add|start|join|lift|remove|enforce|noenforce|member [ nodeid ... ]
```

## DESCRIPTION
This tool allows the administrator to directly manipulate the database used by the rados_cluster recovery backend. Cluster nodes use that database to indicate their current state in order to coordinate a cluster-wide grace period.

The first argument should be a command to execute against the database. Any remaining arguments represent the nodeids of nodes in the cluster that should be acted upon.

Most commands will just fail if the grace database is not present. The exception to this rule is the add command which will create the pool, database and namespace if they do not already exist.

Note that this program does not consult ganesha.conf. If you use non-default values for ceph_conf, userid, grace_oid, namespace or pool in your RADOS_KV config block, then they will need to passed in via command-line options.

## OPTIONS
--cephconf

Specify the ceph.conf configuration that should be used (default is to use the normal search path to find one)

--ns

Set the RADOS namespace to use within the pool (default is NULL)

--oid

Set the object id of the grace database RADOS object (default is "grace")

--pool

Set the RADOS poolid in which the grace database object resides (default is "nfs-ganesha")

--userid

Set the cephx user ID to use when contacting the cluster (default is NULL)

## COMMANDS
dump

Dump the current status of the grace period database to stdout. This will show the current and recovery epoch serial numbers, as well as a list of hosts currently in the cluster and what flags they have set in their individual records.

add

Add the specified hosts to the cluster. This must be done before the given hosts can take part in the cluster. Attempts to modify the database by cluster hosts that have not yet been added will generally fail. New hosts are added with the enforcing flag set, as they are unable to hand out new state until their own grace period has been lifted.

start

Start a new grace period. This will begin a new grace period in the cluster if one is not already active and set the record for the listed cluster hosts as both needing a grace period and enforcing the grace period. If a grace period is already active, then this is equivalent to join.

join

Attempt to join an existing grace period. This works like start, but only if there is already an existing grace period in force.

lift

Attempt to lift the current grace period. This will clear the need grace flags for the listed hosts. If there are no more hosts in the cluster that require a grace period, then it will be fully lifted and the cluster will transition to normal operations.

remove

Remove one or more existing hosts from the cluster. This will remove the listed hosts from the grace database, possibly lifting the current grace period if there are no more hosts that need one.

enforce

Set the flag for the given hosts that indicates that they are currently enforcing the grace period; not allowing the acquisition of new state by clients.

noenforce

Clear the enforcing flag for the given hosts, meaning that those hosts are now allowing clients to acquire new state.

member

Test whether the given hosts are members of the cluster. Returns an error if any of the hosts are not present in the grace db omap.

## FLAGS
When the dump command is issued, ganesha-rados-grace will display a list of all of the nodes in the grace database, and any flags they have set. The flags are as follows:

E (Enforcing)

The node is currently enforcing the grace period by rejecting requests from clients to acquire new state.

N (Need Grace)

The node currently requires a grace period. Generally, this means that the node has clients that need to perform recovery.

## NODEID ASSIGNMENT
Each running ganesha daemon requires a nodeid string that is unique within the cluster. This can be any value as ganesha treats it as an opaque string. By default, the ganesha daemon will use the hostname of the node where it is running.

This may not be suitable when running under certain HA clustering infrastructure, so it's generally recommended to manually assign nodeid values to the hosts in the RADOS_KV config block of ganesha.conf.

## GANESHA CONFIGURATION
The ganesha daemon will need to be configured with the RecoveryBackend set to rados_cluster. If you use a non-default pool, namespace or oid, nodeid then those values will need to be set accordingly in the RADOS_KV config block as well.

## STARTING A NEW CLUSTER
First, add the given cluster nodes to the grace database. Assuming that the nodes in our cluster will have nodeids ganesha-1 through ganesha-3:

ganesha-rados-grace add ganesha-1 ganesha-2 ganesha-3

Once this is done, you can start the daemons on each host and they will coordinate to start and lift the grace periods as-needed.

## ADDING NODES TO A RUNNING CLUSTER
After this point, new nodes can then be added to the cluster as needed using the add command:

ganesha-rados-grace add ganesha-4

After the node has been added, ganesha.nfsd can then be started. It will then request a new grace period as-needed.

## REMOVING A NODE FROM THE CLUSTER
To remove a node from the cluster, first unmount any clients that have that node mounted (possibly moving them to other servers). Then execute the remove command with the nodeids to be removed from the cluster. For example:

ganesha-rados-grace remove ganesha-4

This will remove the ganesha-4's record from the database, and possibly lift the current grace period if one is active and it was the last one to need it.


ganesha-rgw-config

package: nfs-ganesha-rgw

title: ganesha-rgw-config(8)

summary: NFS Ganesha RGW Configuration File

text:
# ganesha-rgw-config(8)

## NAME
ganesha-rgw-config - NFS Ganesha RGW Configuration File

## SYNOPSIS

```
/etc/ganesha/rgw.conf
```

```
/etc/ganesha/rgw_bucket.conf
```

## DESCRIPTION
NFS-Ganesha install two config examples for RGW FSAL:

```
/etc/ganesha/rgw.conf
```

```
/etc/ganesha/rgw_bucket.conf
```

This file lists RGW specific config options.

### EXPORT { }
RGW supports exporting both the buckets and filesystem.

### EXPORT { FSAL {} }
Name(string, "RGW")

Name of FSAL should always be RGW.

User_Id(string, no default)

Access_Key(string, no default)

Secret_Access_Key(string, no default)

### RGW {}
The following configuration variables customize the startup of the FSAL's radosgw instance.

ceph_conf

optional full-path to the Ceph configuration file (equivalent to passing "-c /path/to/ceph.conf" to any Ceph binary

name

optional instance name (equivalent to passing "--name client.rgw.foohost" to the radosgw binary); the value provided here should be the same as the section name (sans brackets) of the radosgw facility in the Ceph configuration file (which must exist)

cluster

optional cluster name (equivalent to passing "--cluster foo" to any Ceph binary); use of a non-default value for cluster name is uncommon, but can be verified by examining the startup options of Ceph binaries

init_args

additional argument strings which will be passed verbatim to the radosgw instance startup process as if they had been given on the radosgw command line provided for customization in uncommon setups

## SEE ALSO
ganesha-log-config(8) ganesha-core-config(8) ganesha-export-config(8)


package: nfs-ganesha-utils

title: ganesha_conf(8)

summary: Ganesha configuration editor

text:
# ganesha_conf(8)

## NAME
ganesha_conf - Ganesha configuration editor

## SYNOPSIS
ganesha_conf set <block-description> [--key1 value1 [--key2 value2]...]

ganesha_conf del <block-description> [--key1 [--key2 ]...]

## DESCRIPTION
The set command adds or changes the given key value parameters in the block. It also creates the block if the block corresponding to the given block description (see section BLOCK DESCRIPTION below) is not present in the configuration.

The del command deletes given keys from the ganesha configuration block described by the block description. It will delete the block itself if no keys are provided.

## BLOCK DESCRIPTION:
Block description is a list of block names and possibly some key value parameters that uniquely identify a ganesha configuration block.

NFS Ganesha configuration file contains a list of blocks. Each block starts with a block name followed by a left brace, then a list of key = value; entries. The block may optionally have sub blocks (note the recursive definition!). Finally, the block ends with a right brace. (Note that key = value; entries can come after a sub block as well, but we don't allow this with ganesha_conf editor! All key value entries should come before any sub blocks.)

An example of a ganesha configuration block:

```
nfs_core_param {

    Nb_Worker = 256;

    Clustered = TRUE;

    NFS_Protocols = 3,4;
}
```

Since there should be only one nfs_core_param block, we just need the name of the block to uniquely identify it. So "nfs_core_param" would be its block description!

An example of a ganesha configuration block with a couple sub blocks:

```
log {

    default_log_level = EVENT;

    format {

        date_format = ISO-8601;

        time_format = ISO-8601;

        thread_name = TRUE;

    }

    components {

        all = EVENT;

    }
}
```

Ganesha configuration should have only one log block as well, so "log" would identify the log block. To identify format sub block inside the log block, "log format" would be the block description for the above format sub block. Similarly "log components" would be the block description for the above components sub block.

An export block is special in that there can be many export blocks, one for each export. A client block is also special. There can be many client blocks and they are always sub blocks inside export blocks. export_id key value uniquely identifies an export block. clients key value uniquely identifies a client block inside a given export block.

Here are couple export blocks with couple client blocks in them:

```
export {

    export_id = 1;

    path = /fs1/export1;

    pseudo = /fs1/export1;

    manage_gids = true;

    client {

        clients = 192.168.122.31;

        access_type = RW;

    }

    client {

        clients = *;

        access_type = RO;

    }
}
export {

    export_id = 2;

    path = /fs1/export2;

    pseudo = /fs1/export2;

    manage_gids = true;

    client {

        clients = 192.168.122.31;

        access_type = RW;

    }

    client {

        clients = 192.168.122.32;

        access_type = RO;

    }
}
```

To identify the correct export block, we need to supply its export_id. For example "export export_id 2" identifies the second export block above. export blocks can be uniquely identified by pseudo or path keys in some environments. One could specify "export path /fs1/export2" to identify the second export block as well. Similarly, a client block needs additional clients key value to identify the correct client block. For example, "export export_id 2 client clients 192.168.122.31" identifies the first client block in the second export block above!

## EXAMPLES:
1. To change number of ganesha worker threads:

ganesha_conf set nfs_core_param --nb_worker 256

2. To change the date and time format of ganesha log messages:

ganesha_conf set log format --date_format ISO-8601 --time_format ISO-8601

3. Create an export and allow client with IP address 192.168.122.31 to be able to do read write and all other clients to do read only:

ganesha_conf set export path /fs1/export2 --export_id 2 --pseudo /fs1/export2 --manage_gids true

ganesha_conf set export path /fs1/export2 client clients 192.168.122.31 --access_type RW

ganesha_conf set export path /fs1/export2 client clients 192.168.122.32 --access_type RO

## NOTES:
ganesha_conf by default uses /etc/ganesha/ganesha.conf file as the configuration file. If your environment uses a different file (or set of files), you can use CONFFILE environment variable to override the default configuration file. For example, "CONFFILE=/etc/ganesha/ganesha.main.conf ganesha_conf set nfs_core_param --nb_worker 256" will use /etc/ganesha/ganesha.main.conf file for changing the worker threads.

ganesha_conf can't handle comments within a block at this point.

Neither block descriptions nor key value parameters are verified to be valid ganesha configuration blocks or parameter values currently.


ganesha-vfs-config

package: nfs-ganesha-vfs

title: ganesha-vfs-config(8)

summary: NFS Ganesha VFS Configuration File

text:
# ganesha-vfs-config(8)

## NAME
ganesha-vfs-config - NFS Ganesha VFS Configuration File

## SYNOPSIS

```
/etc/ganesha/vfs.conf
```

## DESCRIPTION
NFS-Ganesha installs the config example for VFS FSAL: | /etc/ganesha/vfs.conf

This file lists VFS specific config options.

### EXPORT { FSAL {} }
Name(string, "vfs")

Name of FSAL should always be vfs.

pnfs(bool, default false)

fsid_type(enum)

Possible values: None, One64, Major64, Two64, uuid, Two32, Dev,Device

### VFS {}
link_support(bool, default true)

symlink_support(bool, default true)

cansettime(bool, default true)

maxread(uint64, range 512 to 64*1024*1024, default 64*1024*1024)

maxwrite(uint64, range 512 to 64*1024*1024, default 64*1024*1024)

umask(mode, range 0 to 0777, default 0)

auth_xdev_export(bool, default false)

only_one_user(bool, default false)

## SEE ALSO
ganesha-log-config(8) ganesha-core-config(8) ganesha-export-config(8)


exportd

package: nfs-server

title: exportd(8)

summary: NFSv4 Server Mount Daemon

text:
# exportd(8)

## NAME
nfsv4.exportd - NFSv4 Server Mount Daemon

## SYNOPSIS
/usr/sbin/nfsv4.exportd [options]

## DESCRIPTION
The nfsv4.exportd is used to manage NFSv4 exports. The NFS server (nfsd) maintains a cache of authentication and authorization information which is used to identify the source of each request, and then what access permissions that source has to any local filesystem. When required information is not found in the cache, the server sends a request to nfsv4.exportd to fill in the missing information. nfsv4.exportd uses a table of information stored in /var/lib/nfs/etab and maintained by exportfs(8), possibly based on the contents of exports(5), to respond to each request.

## OPTIONS
-d kind or --debug kind

Turn on debugging. Valid kinds are: all, auth, call, general and parse.

-l or --log-auth

Enable logging of responses to authentication and access requests from nfsd. Each response is then cached by the kernel for 30 minutes (or as set by --ttl below), and will be refreshed after 15 minutes (half the ttl time) if the relevant client remains active. Note that -l is equivalent to -d auth and so can be enabled in /etc/nfs.conf with "debug = auth" in the [exportd] section.

-i or --cache-use-ipaddr

Normally each client IP address is matched against each host identifier (name, wildcard, netgroup etc) found in /etc/exports and a combined identity is formed from all matching identifiers. Often many clients will map to the same combined identity so performing this mapping reduces the number of distinct access details that the kernel needs to store. Specifying the -i option suppresses this mapping so that access to each filesystem is requested and cached separately for each client IP address. Doing this can increase the burden of updating the cache slightly, but can make the log messages produced by the -l option easier to read.

-T or --ttl

Provide a time-to-live (TTL) for cached information given to the kernel. The kernel will normally request an update if the information is needed after half of this time has expired. Increasing the provided number, which is in seconds, reduces the rate of cache update requests, and this is particularly noticeable when these requests are logged with -l. However increasing also means that changes to hostname to address mappings can take longer to be noticed. The default TTL is 1800 (30 minutes).

-F or --foreground

Run in foreground (do not daemonize)

-h or --help

Display usage message.

-t N or --num-threads=N or --num-threads N

This option specifies the number of worker threads that nfsv4.exports spawns. The default is 1 thread, which is probably enough. More threads are usually only needed for NFS servers which need to handle mount storms of hundreds of NFS mounts in a few seconds, or when your DNS server is slow or unreliable.

-g or --manage-gids

Accept requests from the kernel to map user id numbers into lists of group id numbers for use in access control. An NFS request will normally (except when using Kerberos or other cryptographic authentication) contain a user-id and a list of group-ids. Due to a limitation in the NFS protocol, at most 16 groups ids can be listed. If you use the -g flag, then the list of group ids received from the client will be replaced by a list of group ids determined by an appropriate lookup on the server. Note that the 'primary' group id is not affected so a newgroup command on the client will still be effective. This function requires a Linux Kernel with version at least 2.6.21.

## CONFIGURATION FILE
Many of the options that can be set on the command line can also be controlled through values set in the [exportd] or, in some cases, the [nfsd] sections of the /etc/nfs.conf configuration file. Values recognized in the [exportd] section include cache-use-ipaddr , ttl, manage-gids, and debug which each have the same effect as the option with the same name.

## FILES
/etc/exports

input file for exportfs, listing exports, export options, and access control lists

## SEE ALSO
exportfs(8), exports(5), showmount(8), nfs.conf(5), firewall-cmd(1),

RFC 7530 - "Network File System (NFS) Version 4 Protocol"

RFC 8881 - "Network File System (NFS) Version 4 Minor Version 1 Protocol"