podman-troubleshooting

title: podman-troubleshooting(7)

text:
# podman-troubleshooting(7)

## A list of common issues and solutions for Podman

### 1) Variety of issues - Validate Version
A large number of issues reported against Podman are often found to already be fixed in more current versions of the project. Before reporting an issue, please verify the version you are running with podman version and compare it to the latest release documented on the top of Podman's ⟨README.md⟩.

If they differ, please update your version of PODMAN to the latest possible and retry your command before reporting the issue.

### 2) Can't use volume mount, get permission denied

```
$ podman run -v ~/mycontent:/content fedora touch /content/file
touch: cannot touch '/content/file': Permission denied
```

### Solution
This is sometimes caused by SELinux, and sometimes by user namespaces.

Labeling systems like SELinux require that proper labels are placed on volume content mounted into a container. Without a label, the security system might prevent the processes running inside the container from using the content. By default, Podman does not change the labels set by the OS.

To change a label in the container context, you can add either of two suffixes :z or :Z to the volume mount. These suffixes tell Podman to relabel file objects on the shared volumes. The z option tells Podman that two containers share the volume content. As a result, Podman labels the content with a shared content label. Shared volume labels allow all containers to read/write content. The Z option tells Podman to label the content with a private unshared label. Only the current container can use a private volume.

```
$ podman run -v ~/mycontent:/content:Z fedora touch /content/file
```

Make sure the content is private for the container. Do not relabel system directories and content. Relabeling system content might cause other confined services on your machine to fail. For these types of containers we recommend having SELinux separation disabled. The option --security-opt label=disable will disable SELinux separation for the container.

```
$ podman run --security-opt label=disable -v ~:/home/user fedora touch /home/user/file
```

In cases where the container image runs as a specific, non-root user, though, the solution is to fix the user namespace. This would include container images such as the Jupyter Notebook image (which runs as "jovyan") and the Postgres image (which runs as "postgres"). In either case, use the --userns switch to map user namespaces, most of the time by using the keep-id option.

```
$ podman run -v "$PWD":/home/jovyan/work --userns=keep-id jupyter/scipy-notebook
```

### 3) No such image or Bare keys cannot contain ':'
When doing a podman pull or podman build command and a "common" image cannot be pulled, it is likely that the /etc/containers/registries.conf file is either not installed or possibly misconfigured.

### Symptom

```
$ sudo podman build -f Dockerfile
STEP 1: FROM alpine
error building: error creating build container: no such image "alpine" in registry: image not known
```

or

```
$ sudo podman pull fedora
error pulling image "fedora": unable to pull fedora: error getting default registries to try: Near line 9 (last key parsed ''): Bare keys cannot contain ':'.
```

### Solution

- Verify that the /etc/containers/registries.conf file exists. If not, verify that the containers-common package is installed.

- Verify that the entries in the unqualified-search-registries list of the /etc/containers/registries.conf file are valid and reachable.

•

i.e. unqualified-search-registries = ["registry.fedoraproject.org", "quay.io", "registry.access.redhat.com"]

### 4) http: server gave HTTP response to HTTPS client
When doing a Podman command such as build, commit, pull, or push to a registry, TLS verification is turned on by default. If encryption is not used with those commands, this error can occur.

### Symptom

```
$ sudo podman push alpine docker://localhost:5000/myalpine:latest
Getting image source signatures
Get https://localhost:5000/v2/: http: server gave HTTP response to HTTPS client
```

### Solution
By default TLS verification is turned on when communicating to registries from Podman. If the registry does not require encryption the Podman commands such as build, commit, pull and push will fail unless TLS verification is turned off using the --tls-verify option. NOTE: It is not at all recommended to communicate with a registry and not use TLS verification.

- Turn off TLS verification by passing false to the tls-verify option.

- I.e. podman push --tls-verify=false alpine docker://localhost:5000/myalpine:latest

For a global workaround, users[1] can create the file /etc/containers/registries.conf.d/registry-NAME.conf (replacing NAME with the name of this registry) with the following content (replacing FULLY.QUALIFIED.NAME.OF.REGISTRY with the address of this registry):

```
[[registry]]
location = "FULLY.QUALIFIED.NAME.OF.REGISTRY"
insecure = true
```

[1] If you are using a Mac / Windows, you should execute podman machine ssh to login into podman machine before adding the insecure entry to the registry—conf file.

This is an insecure method and should be used cautiously.

### 5) rootless containers cannot ping hosts
When using the ping command from a non-root container, the command may fail because of a lack of privileges.

### Symptom

```
$ podman run --rm fedora ping -W10 -c1 redhat.com
PING redhat.com (209.132.183.105): 56 data bytes
--- redhat.com ping statistics ---
1 packets transmitted, 0 packets received, 100% packet loss
```

### Solution
It is most likely necessary to enable unprivileged pings on the host. Be sure the UID of the user is part of the range in the /proc/sys/net/ipv4/ping_group_range file.

To change its value you can use something like:

```
# sysctl -w "net.ipv4.ping_group_range=0 2000000"
```

To make the change persistent, you'll need to add a file in /etc/sysctl.d that contains net.ipv4.ping_group_range=0 $MAX_UID.

### 6) Build hangs when the Dockerfile contains the useradd command
When the Dockerfile contains a command like RUN useradd -u 99999000 -g users newuser the build can hang.

### Symptom
If you are using a useradd command within a Dockerfile with a large UID/GID, it will create a large sparse file /var/log/lastlog. This can cause the build to hang forever. Go language does not support sparse files correctly, which can lead to some huge files being created in your container image.

### Solution
If the entry in the Dockerfile looked like: RUN useradd -u 99999000 -g users newuser then add the --no-log-init parameter to change it to: RUN useradd --no-log-init -u 99999000 -g users newuser. This option tells useradd to stop creating the lastlog file.

### 7) Permission denied when running Podman commands
When rootless Podman attempts to execute a container on a non exec home directory a permission error will be raised.

### Symptom
If you are running Podman or Buildah on a home directory that is mounted noexec, then they will fail with a message like:

```
$ podman run centos:7
standard_init_linux.go:203: exec user process caused "permission denied"
```

### Solution
Since the administrator of the system set up your home directory to be noexec, you will not be allowed to execute containers from storage in your home directory. It is possible to work around this by manually specifying a container storage path that is not on a noexec mount. Simply copy the file /etc/containers/storage.conf to ~/.config/containers/ (creating the directory if necessary). Specify a graphroot directory which is not on a noexec mount point and to which you have read/write privileges. You will need to modify other fields to writable directories as well.

For example

```
$ cat ~/.config/containers/storage.conf
[storage]

  driver = "overlay"

  runroot = "/run/user/1000"

  graphroot = "/execdir/myuser/storage"

  [storage.options]

    mount_program = "/bin/fuse-overlayfs"
```

### 8) Permission denied when running systemd within a Podman container
When running systemd as PID 1 inside of a container on an SELinux separated machine, it needs to write to the cgroup file system.

### Symptom
Systemd gets permission denied when attempting to write to the cgroup file system, and AVC messages start to show up in the audit.log file or journal on the system.

### Solution
Newer versions of Podman (2.0 or greater) support running init based containers with a different SELinux labels, which allow the container process access to the cgroup file system. This feature requires container-selinux-2.132 or newer versions.

Prior to Podman 2.0, the SELinux boolean container_manage_cgroup allows container processes to write to the cgroup file system. Turn on this boolean, on SELinux separated systems, to allow systemd to run properly in the container. Only do this on systems running older versions of Podman.

```
# setsebool -P container_manage_cgroup true
```

### 9) Newuidmap missing when running rootless Podman commands
Rootless Podman requires the newuidmap and newgidmap programs to be installed.

### Symptom
If you are running Podman or Buildah as a rootless user, you get an error complaining about a missing newuidmap executable.

```
$ podman run -ti fedora sh
command required for rootless mode with multiple IDs: exec: "newuidmap": executable file not found in $PATH
```

### Solution
Install a version of shadow-utils that includes these executables. Note that for RHEL and CentOS 7, at least the 7.7 release must be installed for support to be available.

### 10) rootless setup user: invalid argument
Rootless Podman requires the user running it to have a range of UIDs listed in /etc/subuid and /etc/subgid.

### Symptom
A user, either via --user or through the default configured for the image, is not mapped inside the namespace.

```
$ podman run --rm -ti --user 1000000 alpine echo hi
Error: container create failed: container_linux.go:344: starting container process caused "setup user: invalid argument"
```

### Solution
Update the /etc/subuid and /etc/subgid with fields for users that look like:

```
$ cat /etc/subuid
johndoe:100000:65536
test:165536:65536
```

The format of this file is USERNAME:UID:RANGE

- username as listed in /etc/passwd or getpwent.

- The initial uid allocated for the user.

- The size of the range of UIDs allocated for the user.

This means johndoe is allocated UIDs 100000-165535 as well as his standard UID in the /etc/passwd file.

You should ensure that each user has a unique range of UIDs, because overlapping UIDs, would potentially allow one user to attack another user. In addition, make sure that the range of UIDs you allocate can cover all UIDs that the container requires. For example, if the container has a user with UID 10000, ensure you have at least 10000 subuids, and if the container needs to be run as a user with UID 1000000, ensure you have at least 1000000 subuids.

You could also use the usermod program to assign UIDs to a user.

If you update either the /etc/subuid or /etc/subgid file, you need to stop all running containers and kill the pause process. This is done automatically by the system migrate command, which can also be used to stop all the containers and kill the pause process.

```
# usermod --add-subuids 200000-201000 --add-subgids 200000-201000 johndoe
# grep johndoe /etc/subuid /etc/subgid
/etc/subuid:johndoe:200000:1001
/etc/subgid:johndoe:200000:1001
```

### 11) Changing the location of the Graphroot leads to permission denied
When I change the graphroot storage location in storage.conf, the next time I run Podman, I get an error like:

```
# podman run -p 5000:5000 -it centos bash
bash: error while loading shared libraries: /lib64/libc.so.6: cannot apply additional memory protection after relocation: Permission denied
```

For example, the admin sets up a spare disk to be mounted at /src/containers, and points storage.conf at this directory.

### Symptom
SELinux blocks containers from using arbitrary locations for overlay storage. These directories need to be labeled with the same labels as if the content was under /var/lib/containers/storage.

### Solution
Tell SELinux about the new containers storage by setting up an equivalence record. This tells SELinux to label content under the new path, as if it was stored under /var/lib/containers/storage.

```
# semanage fcontext -a -e /var/lib/containers /srv/containers
# restorecon -R -v /srv/containers
```

The semanage command above tells SELinux to set up the default labeling of /srv/containers to match /var/lib/containers. The restorecon command tells SELinux to apply the labels to the actual content.

Now all new content created in these directories will automatically be created with the correct label.

### 12) Anonymous image pull fails with 'invalid username/password'
Pulling an anonymous image that doesn't require authentication can result in an invalid username/password error.

### Symptom
If you pull an anonymous image, one that should not require credentials, you can receive an invalid username/password error if you have credentials established in the authentication file for the target container registry that are no longer valid.

```
$ podman run -it --rm docker://docker.io/library/alpine:latest ls
Trying to pull docker://docker.io/library/alpine:latest...ERRO[0000] Error pulling image ref //alpine:latest: Error determining manifest MIME type for docker://alpine:latest: unable to retrieve auth token: invalid username/password
Failed
Error: unable to pull docker://docker.io/library/alpine:latest: unable to pull image: Error determining manifest MIME type for docker://alpine:latest: unable to retrieve auth token: invalid username/password
```

This can happen if the authentication file is modified 'by hand' or if the credentials are established locally and then the password is updated later in the container registry.

### Solution
Depending upon which container tool was used to establish the credentials, use podman logout or docker logout to remove the credentials from the authentication file.

### 13) Running Podman inside a container causes container crashes and inconsistent states
Running Podman in a container and forwarding some, but not all, of the required host directories can cause inconsistent container behavior.

### Symptom
After creating a container with Podman's storage directories mounted in from the host and running Podman inside a container, all containers show their state as "configured" or "created", even if they were running or stopped.

### Solution
When running Podman inside a container, it is recommended to mount at a minimum /var/lib/containers/storage/ as a volume. Typically, you will not mount in the host version of the directory, but if you wish to share containers with the host, you can do so. If you do mount in the host's /var/lib/containers/storage, however, you must also mount in the host's /run/libpod and /run/containers/storage directories. Not doing this will cause Podman in the container to detect that temporary files have been cleared, leading it to assume a system restart has taken place. This can cause Podman to reset container states and lose track of running containers.

For running containers on the host from inside a container, we also recommend the Podman remote client ⟨docs/tutorials/remote_client.md⟩, which only requires a single socket to be mounted into the container.

### 14) Rootless 'podman build' fails EPERM on NFS:
NFS enforces file creation on different UIDs on the server side and does not understand user namespace, which rootless Podman requires. When a container root process like YUM attempts to create a file owned by a different UID, NFS Server denies the creation. NFS is also a problem for the file locks when the storage is on it. Other distributed file systems (for example: Lustre, Spectrum Scale, the General Parallel File System (GPFS)) are also not supported when running in rootless mode as these file systems do not understand user namespace.

### Symptom

```
$ podman build .
ERRO[0014] Error while applying layer: ApplyLayer exit status 1 stdout:  stderr: open /root/.bash_logout: permission denied
error creating build container: Error committing the finished image: error adding layer with blob "sha256:a02a4930cb5d36f3290eb84f4bfa30668ef2e9fe3a1fb73ec015fc58b9958b17": ApplyLayer exit status 1 stdout:  stderr: open /root/.bash_logout: permission denied
```

### Solution
Choose one of the following:

* Set up containers/storage in a different directory, not on an NFS share.

* Create a directory on a local file system.

* Edit ~/.config/containers/containers.conf and point the volume_path option to that local directory. (Copy /usr/share/containers/containers.conf if ~/.config/containers/containers.conf does not exist)

* Otherwise just run Podman as root, via sudo podman

### 15) Rootless 'podman build' fails when using OverlayFS:
The Overlay file system (OverlayFS) requires the ability to call the mknod command when creating whiteout files when extracting an image. However, a rootless user does not have the privileges to use mknod in this capacity.

### Symptom

```
$ podman build --storage-driver overlay .
STEP 1: FROM docker.io/ubuntu:xenial
Getting image source signatures
Copying blob edf72af6d627 done
Copying blob 3e4f86211d23 done
Copying blob 8d3eac894db4 done
Copying blob f7277927d38a done
Copying config 5e13f8dd4c done
Writing manifest to image destination
Storing signatures
Error: creating build container: Error committing the finished image: error adding layer with blob "sha256:8d3eac894db4dc4154377ad28643dfe6625ff0e54bcfa63e0d04921f1a8ef7f8": Error processing tar file(exit status 1): operation not permitted
$ podman build .
ERRO[0014] Error while applying layer: ApplyLayer exit status 1 stdout:  stderr: open /root/.bash_logout: permission denied
error creating build container: Error committing the finished image: error adding layer with blob "sha256:a02a4930cb5d36f3290eb84f4bfa30668ef2e9fe3a1fb73ec015fc58b9958b17": ApplyLayer exit status 1 stdout:  stderr: open /root/.bash_logout: permission denied
```

### Solution
Choose one of the following:

* Complete the build operation as a privileged user.

* Install and configure fuse-overlayfs.

* Install the fuse-overlayfs package for your Linux Distribution.

* Add mount_program = "/usr/bin/fuse-overlayfs" under [storage.options] in your ~/.config/containers/storage.conf file.

### 16) RHEL 7 and CentOS 7 based init images don't work with cgroup v2
The systemd version shipped in RHEL 7 and CentOS 7 doesn't have support for cgroup v2. Support for cgroup v2 requires version 230 of systemd or newer, which was never shipped or supported on RHEL 7 or CentOS 7.

### Symptom

```
# podman run --name test -d registry.access.redhat.com/rhel7-init:latest && sleep 10 && podman exec test systemctl status
c8567461948439bce72fad3076a91ececfb7b14d469bfa5fbc32c6403185beff
Failed to get D-Bus connection: Operation not permitted
Error: non zero exit code: 1: OCI runtime error
```

### Solution
You'll need to either:

- configure the host to use cgroup v1. On Fedora you can do:

```
# dnf install -y grubby
# grubby --update-kernel=ALL --args=”systemd.unified_cgroup_hierarchy=0"
# reboot

```

- update the image to use an updated version of systemd.

### 17) rootless containers exit once the user session exits
You need to set lingering mode through loginctl to prevent user processes to be killed once the user session completed.

### Symptom
Once the user logs out all the containers exit.

### Solution

```
# loginctl enable-linger $UID
```

### 18) podman run fails with bpf create: permission denied error
The Kernel Lockdown patches deny eBPF programs when Secure Boot is enabled in the BIOS. Matthew Garrett's post describes the relationship between Lockdown and Secure Boot and Jan-Philip Gehrcke's connects this with eBPF. RH bug 1768125 contains some additional details.

### Symptom
Attempts to run podman result in

````
#### Solution
One workaround is to disable Secure Boot in your BIOS.
### 19) error creating libpod runtime: there might not be enough IDs available in the namespace
Unable to pull images
#### Symptom
```console
$ podman unshare cat /proc/self/uid_map
	 0       1000          1
````

### Solution

```
$ podman system migrate
```

Original command now returns

```
$ podman unshare cat /proc/self/uid_map
	 0       1000          1
	 1     100000      65536
```

Reference subuid and subgid man pages for more detail.

### 20) Passed-in devices or files can't be accessed in rootless container
As a non-root user you have group access rights to a device or files that you want to pass into a rootless container with --device=... or --volume=...

### Symptom
Any access inside the container is rejected with "Permission denied".

### Solution
The runtime uses setgroups(2) hence the process loses all additional groups the non-root user has. Use the --group-add keep-groups flag to pass the user's supplementary group access into the container. Currently only available with the crun OCI runtime.

### 21) A rootless container running in detached mode is closed at logout
When running a container with a command like podman run --detach httpd as a rootless user, the container is closed upon logout and is not kept running.

### Symptom
When logging out of a rootless user session, all containers that were started in detached mode are stopped and are not kept running. As the root user, these same containers would survive the logout and continue running.

### Solution
When systemd notes that a session that started a Podman container has exited, it will also stop any containers that have been associated with it. To avoid this, use the following command before logging out: loginctl enable-linger. To later revert the linger functionality, use loginctl disable-linger.

LOGINCTL(1), SYSTEMD(1)

### 22) Containers default detach keys conflict with shell history navigation
Podman defaults to ctrl-p,ctrl-q to detach from a running containers. The bash and zsh shells default to ctrl-p for the displaying of the previous command. This causes issues when running a shell inside of a container.

### Symptom
With the default detach key combo ctrl-p,ctrl-q, shell history navigation (tested in bash and zsh) using ctrl-p to access the previous command will not display this previous command, or anything else. Conmon is waiting for an additional character to see if the user wants to detach from the container. Adding additional characters to the command will cause it to be displayed along with the additional character. If the user types ctrl-p a second time the shell display the 2nd to last command.

### Solution
The solution to this is to change the default detach_keys. For example in order to change the defaults to ctrl-q,ctrl-q use the --detach-keys option.

```
$ podman run -ti --detach-keys ctrl-q,ctrl-q fedora sh
```

To make this change the default for all containers, users can modify the containers.conf file. This can be done simply in your home directory, but adding the following lines to users containers.conf

```
$ cat >> ~/.config/containers/containers.conf << _eof
[engine]
detach_keys="ctrl-q,ctrl-q"
_eof
```

In order to effect root running containers and all users, modify the system wide defaults in /etc/containers/containers.conf.

### 23) Container with exposed ports won't run in a pod
A container with ports that have been published with the --publish or -p option can not be run within a pod.

### Symptom

```
$ podman pod create --name srcview -p 127.0.0.1:3434:3434 -p 127.0.0.1:7080:7080 -p 127.0.0.1:3370:3370                        4b2f4611fa2cbd60b3899b936368c2b3f4f0f68bc8e6593416e0ab8ecb0a3f1d
$ podman run --pod srcview --name src-expose -p 3434:3434 -v "${PWD}:/var/opt/localrepo":Z,ro sourcegraph/src-expose:latest serve /var/opt/localrepo
Error: cannot set port bindings on an existing container network namespace
```

### Solution
This is a known limitation. If a container will be run within a pod, it is not necessary to publish the port for the containers in the pod. The port must only be published by the pod itself. Pod network stacks act like the network stack on the host - you have a variety of containers in the pod, and programs in the container, all sharing a single interface and IP address, and associated ports. If one container binds to a port, no other container can use that port within the pod while it is in use. Containers in the pod can also communicate over localhost by having one container bind to localhost in the pod, and another connect to that port.

In the example from the symptom section, dropping the -p 3434:3434 would allow the podman run command to complete, and the container as part of the pod would still have access to that port. For example:

```
$ podman run --pod srcview --name src-expose -v "${PWD}:/var/opt/localrepo":Z,ro sourcegraph/src-expose:latest serve /var/opt/localrepo
```

### 24) Podman container images fail with fuse: device not found when run
Some container images require that the fuse kernel module is loaded in the kernel before they will run with the fuse filesystem in play.

### Symptom
When trying to run the container images found at quay.io/podman, quay.io/containers registry.access.redhat.com/ubi8 or other locations, an error will sometimes be returned:

```
ERRO error unmounting /var/lib/containers/storage/overlay/30c058cdadc888177361dd14a7ed7edab441c58525b341df321f07bc11440e68/merged: invalid argument
error mounting container "1ae176ca72b3da7c70af31db7434bcf6f94b07dbc0328bc7e4e8fc9579d0dc2e": error mounting build container "1ae176ca72b3da7c70af31db7434bcf6f94b07dbc0328bc7e4e8fc9579d0dc2e": error creating overlay mount to /var/lib/containers/storage/overlay/30c058cdadc888177361dd14a7ed7edab441c58525b341df321f07bc11440e68/merged: using mount program /usr/bin/fuse-overlayfs: fuse: device not found, try 'modprobe fuse' first
fuse-overlayfs: cannot mount: No such device
: exit status 1
ERRO exit status 1
```

### Solution
If you encounter a fuse: device not found error when running the container image, it is likely that the fuse kernel module has not been loaded on your host system. Use the command modprobe fuse to load the module and then run the container image afterwards. To enable this automatically at boot time, you can add a configuration file to /etc/modules.load.d. See man modules-load.d for more details.

### 25) podman run --rootfs link/to//read/only/dir does not work
An error such as "OCI runtime error" on a read-only filesystem or the error "{image} is not an absolute path or is a symlink" are often times indicators for this issue. For more details, review this issue ⟨https://github.com/containers/podman/issues/5895⟩.

### Symptom
Rootless Podman requires certain files to exist in a file system in order to run. Podman will create /etc/resolv.conf, /etc/hosts and other file descriptors on the rootfs in order to mount volumes on them.

### Solution
Run the container once in read/write mode, Podman will generate all of the FDs on the rootfs, and from that point forward you can run with a read-only rootfs.

```
$ podman run --rm --rootfs /path/to/rootfs true
```

The command above will create all the missing directories needed to run the container.

After that, it can be used in read-only mode, by multiple containers at the same time:

```
$ podman run --read-only --rootfs /path/to/rootfs ....
```

Another option is to use an Overlay Rootfs Mount:

```
$ podman run --rootfs /path/to/rootfs:O ....
```

Modifications to the mount point are destroyed when the container finishes executing, similar to a tmpfs mount point being unmounted.

### 26) Running containers with resource limits fails with a permissions error
On some systemd-based systems, non-root users do not have resource limit delegation permissions. This causes setting resource limits to fail.

### Symptom
Running a container with a resource limit options will fail with an error similar to the following:

--cpus, --cpu-period, --cpu-quota, --cpu-shares:

```
Error: OCI runtime error: crun: the requested cgroup controller `cpu` is not available
```

--cpuset-cpus, --cpuset-mems:

```
Error: OCI runtime error: crun: the requested cgroup controller `cpuset` is not available
```

This means that resource limit delegation is not enabled for the current user.

### Solution
You can verify whether resource limit delegation is enabled by running the following command:

```
$ cat "/sys/fs/cgroup/user.slice/user-$(id -u).slice/user@$(id -u).service/cgroup.controllers"
```

Example output might be:

```
memory pids
```

In the above example, cpu and cpuset are not listed, which means the current user does not have permission to set CPU or CPUSET limits.

If you want to enable CPU or CPUSET limit delegation for all users, you can create the file /etc/systemd/system/user@.service.d/delegate.conf with the contents:

```
[Service]
Delegate=memory pids cpu cpuset
```

After logging out and logging back in, you should have permission to set CPU and CPUSET limits.

### 27) exec container process '/bin/sh': Exec format error (or another binary than bin/sh)
This can happen when running a container from an image for another architecture than the one you are running on.

For example, if a remote repository only has, and thus send you, a linux/arm64 OS/ARCH but you run on linux/amd64 (as happened in https://github.com/openMF/community-app/issues/3323 due to https://github.com/timbru31/docker-ruby-node/issues/564).

### 28) Error: failed to create sshClient: Connection to bastion host (ssh://user@host:22/run/user/.../podman/podman.sock) failed.: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain
In some situations where the client is not on the same machine as where the podman daemon is running the client key could be using a cipher not supported by the host. This indicates an issue with one's SSH config. Until remedied using podman over ssh with a pre-shared key will be impossible.

### Symptom
The accepted ciphers per /etc/crypto-policies/back-ends/openssh.config are not one that was used to create the public/private key pair that was transferred over to the host for ssh authentication.

You can confirm this is the case by attempting to connect to the host via podman-remote info from the client and simultaneously on the host running journalctl -f and watching for the error userauth_pubkey: key type ssh-rsa not in PubkeyAcceptedAlgorithms [preauth].

### Solution
Create a new key using a supported algorithm e.g. ecdsa:

```
$ ssh-keygen -t ecdsa -f ~/.ssh/podman
```

Then copy the new id over:

```
$ ssh-copy-id -i ~/.ssh/podman.pub user@host
```

And then re-add the connection (removing the old one if necessary):

```
$ podman-remote system connection add myuser --identity ~/.ssh/podman ssh://user@host/run/user/1000/podman/podman.sock
```

And now this should work:

```
$ podman-remote info
```

### 29) Rootless CNI networking fails in RHEL with Podman v2.2.1 to v3.0.1.
A failure is encountered when trying to use networking on a rootless container in Podman v2.2.1 through v3.0.1 on RHEL. This error does not occur on other Linux distributions.

### Symptom
A rootless container is created using a CNI network, but the podman run command returns an error that an image must be built.

### Solution
In order to use a CNI network in a rootless container on RHEL, an Infra container image for CNI-in-slirp4netns must be created. The instructions for building the Infra container image can be found for v2.2.1 here, and for v3.0.1 here.

### 30) Container related firewall rules are lost after reloading firewalld
Container network can't be reached after firewall-cmd --reload and systemctl restart firewalld Running podman network reload will fix it but it has to be done manually.

### Symptom
The firewall rules created by podman are lost when the firewall is reloaded.

### Solution
@ranjithrajaram has created a systemd-hook to fix this issue

1) For "firewall-cmd --reload", create a systemd unit file with the following

```
[Unit]
Description=firewalld reload hook - run a hook script on firewalld reload
Wants=dbus.service
After=dbus.service
[Service]
Type=simple
ExecStart=/bin/bash -c '/bin/busctl monitor --system --match "interface=org.fedoraproject.FirewallD1,member=Reloaded" --match "interface=org.fedoraproject.FirewallD1,member=PropertiesChanged" | while read -r line ; do podman network reload --all ; done'
[Install]
WantedBy=default.target
```

2) For "systemctl restart firewalld", create a systemd unit file with the following

```
[Unit]
Description=podman network reload
Wants=firewalld.service
After=firewalld.service
PartOf=firewalld.service
[Service]
Type=simple
RemainAfterExit=yes
ExecStart=/usr/bin/podman network reload --all
[Install]
WantedBy=default.target
```

However, If you use busctl monitor then you can't get machine-readable output on RHEL 8. Since it doesn't have busctl -j as mentioned here by @yrro.

For RHEL 8, you can use the following one-liner bash script.

```
[Unit]
Description=Redo podman NAT rules after firewalld starts or reloads
Wants=dbus.service
After=dbus.service
Requires=firewalld.service
[Service]
Type=simple
ExecStart=/bin/bash -c "dbus-monitor --profile --system 'type=signal,sender=org.freedesktop.DBus,path=/org/freedesktop/DBus,interface=org.freedesktop.DBus,member=NameAcquired,arg0=org.fedoraproject.FirewallD1' 'type=signal,path=/org/fedoraproject/FirewallD1,interface=org.fedoraproject.FirewallD1,member=Reloaded' | sed -u '/^#/d' | while read -r type timestamp serial sender destination path interface member _junk; do if [[ $type = '#'* ]]; then continue; elif [[ $interface = org.freedesktop.DBus && $member = NameAcquired ]]; then echo 'firewalld started'; podman network reload --all; elif [[ $interface = org.fedoraproject.FirewallD1 && $member = Reloaded ]]; then echo 'firewalld reloaded'; podman network reload --all; fi; done"
Restart=Always
[Install]
WantedBy=default.target
```

busctl-monitor is almost usable in RHEL 8, except that it always outputs two bogus events when it starts up, one of which is (in its only machine-readable format) indistinguishable from the NameOwnerChanged that you get when firewalld starts up. This means you would get an extra podman network reload --all when this unit starts.

Apart from this, you can use the following systemd service with the python3 code.

```
[Unit]
Description=Redo podman NAT rules after firewalld starts or reloads
Wants=dbus.service
Requires=firewalld.service
After=dbus.service
[Service]
Type=simple
ExecStart=/usr/bin/python  /path/to/python/code/podman-redo-nat.py
Restart=always
[Install]
WantedBy=default.target
```

The code reloads podman network twice when you use systemctl restart firewalld.

```
import dbus
from gi.repository import GLib
from dbus.mainloop.glib import DBusGMainLoop
import subprocess
import sys
# I'm a bit confused on the return values in the code
# Not sure if they are needed.
def reload_podman_network():

    try:

        subprocess.run(["podman","network","reload","--all"],timeout=90)

        # I'm not sure about this part

        sys.stdout.write("podman network reload done\n")

        sys.stdout.flush()

    except subprocess.TimeoutExpired as t:

        sys.stderr.write(f"Podman reload failed due to Timeout {t}")

    except subprocess.CalledProcessError as e:

        sys.stderr.write(f"Podman reload failed due to {e}")

    except Exception as e:

        sys.stderr.write(f"Podman reload failed with an Unhandled Exception {e}")

    return False
def signal_handler(*args, **kwargs):

    if kwargs.get('member') == "Reloaded":

        reload_podman_network()

    elif kwargs.get('member') == "NameOwnerChanged":

        reload_podman_network()

    else:

        return None

    return None
def signal_listener():

    try:

        DBusGMainLoop(set_as_default=True)# Define the loop.

        loop = GLib.MainLoop()

        system_bus = dbus.SystemBus()

        # Listens to systemctl restart firewalld with a filter added, will cause podman network to be reloaded twice

        system_bus.add_signal_receiver(signal_handler,dbus_interface='org.freedesktop.DBus',arg0='org.fedoraproject.FirewallD1',member_keyword='member')

        # Listens to firewall-cmd --reload

        system_bus.add_signal_receiver(signal_handler,dbus_interface='org.fedoraproject.FirewallD1',signal_name='Reloaded',member_keyword='member')

        loop.run()

    except KeyboardInterrupt:

        loop.quit()

        sys.exit(0)

    except Exception as e:

        loop.quit()

        sys.stderr.write(f"Error occurred {e}")

        sys.exit(1)
if __name__ == "__main__":

    signal_listener()
```

### 31) Podman run fails with ERRO[0000] XDG_RUNTIME_DIR directory /run/user/0 is not owned by the current user or Error: creating tmpdir: mkdir /run/user/1000: permission denied.
A failure is encountered when performing podman run with a warning XDG_RUNTIME_DIR is pointing to a path which is not writable. Most likely podman will fail.

### Symptom
A rootless container is being invoked with cgroup configuration as cgroupv2 for user with missing or invalid systemd session.

Example cases

```
# su user1 -c 'podman images'
ERRO[0000] XDG_RUNTIME_DIR directory "/run/user/0" is not owned by the current user
```

```
# su - user1 -c 'podman images'
Error: creating tmpdir: mkdir /run/user/1000: permission denied
```

### Solution
Podman expects a valid login session for the rootless+cgroupv2 use-case. Podman execution is expected to fail if the login session is not present. In most cases, podman will figure out a solution on its own but if XDG_RUNTIME_DIR is pointing to a path that is not writable execution will most likely fail. Typical scenarios of such cases are seen when users are trying to use Podman with su - <user> -c '<podman-command>', or sudo -l and badly configured systemd session.

Alternatives:

- Execute Podman via systemd-run that will first start a systemd login session:

```
  $ sudo systemd-run --machine=username@ --quiet --user --collect --pipe --wait podman run --rm docker.io/library/alpine echo hello

```

- Start an interactive shell in a systemd login session with the command machinectl shell <username>@ and then run Podman

```
  $ sudo -i

  # machinectl shell username@

  Connected to the local host. Press ^] three times within 1s to exit session.

  $ podman run --rm docker.io/library/alpine echo hello

```

- Start a new systemd login session by logging in with ssh i.e. ssh <username>@localhost and then run Podman.

- Before invoking Podman command create a valid login session for your rootless user using loginctl enable-linger <username>

### 32) 127.0.0.1:7777 port already bound
After deleting a VM on macOS, the initialization of subsequent VMs fails.

### Symptom
After deleting a client VM on macOS via podman machine stop && podman machine rm, attempting to podman machine init a new client VM leads to an error with the 127.0.0.1:7777 port already bound.

### Solution
You will need to remove the hanging gv-proxy process bound to the port in question. For example, if the port mentioned in the error message is 127.0.0.1:7777, you can use the command kill -9 $(lsof -i:7777) in order to identify and remove the hanging process which prevents you from starting a new VM on that default port.

### Symptom
The sshd process running inside the container fails with the error "Error writing /proc/self/loginuid".

### Solution
If the /proc/self/loginuid file is already initialized then the CAP_AUDIT_CONTROL capability is required to override it.

This happens when running Podman from a user session since the /proc/self/loginuid file is already initialized. The solution is to run Podman from a system service, either using the Podman service, and then using podman -remote to start the container or simply by running something like systemd-run podman run .... In this case the container will only need CAP_AUDIT_WRITE.

### 34) Container creates a file that is not owned by the user's regular UID
After running a container with rootless Podman, the non-root user sees a numerical UID and GID instead of a username and groupname.

### Symptom
When listing file permissions with ls -l on the host in a directory that was passed as --volume /some/dir to podman run, the UID and GID are displayed rather than the corresponding username and groupname. The UID and GID numbers displayed are from the user's subordinate UID and GID ranges on the host system.

An example

```
$ mkdir dir1
$ chmod 777 dir1
$ podman run --rm -v ./dir1:/dir1:Z

             --user 2003:2003

             docker.io/library/ubuntu bash -c "touch /dir1/a; chmod 600 /dir1/a"
$ ls -l dir1/a
-rw-------. 1 102002 102002 0 Jan 19 19:35 dir1/a
$ less dir1/a
less: dir1/a: Permission denied
```

### Solution
If you want to read, chown, or remove such a file, enter a user namespace. Instead of running commands such as less dir1/a or rm dir1/a, you need to prepend the command-line with podman unshare, i.e., podman unshare less dir1/a or podman unshare rm dir1/a. To change the ownership of the file dir1/a to your regular user's UID and GID, run podman unshare chown 0:0 dir1/a. A file having the ownership 0:0 in the user namespace is owned by the regular user on the host. To use Bash features, such as variable expansion and globbing, you need to wrap the command with bash -c, e.g. podman unshare bash -c 'ls $HOME/dir1/a*'.

Would it have been possible to run Podman in another way so that your regular user would have become the owner of the file? Yes, you can use the option --userns keep-id:uid=$uid,gid=$gid to change how UIDs and GIDs are mapped between the container and the host. Let's try it out.

In the example above ls -l shows the UID 102002 and GID 102002. Set shell variables

```
$ uid_from_ls=102002
$ gid_from_ls=102002
```

Set shell variables to the lowest subordinate UID and GID

```
$ lowest_subuid=$(podman info --format "{{ (index .Host.IDMappings.UIDMap 1).HostID }}")
$ lowest_subgid=$(podman info --format "{{ (index .Host.IDMappings.GIDMap 1).HostID }}")
```

Compute the UID and GID inside the container that map to the owner of the created file on the host.

```
$ uid=$(( $uid_from_ls - $lowest_subuid + 1))
$ gid=$(( $gid_from_ls - $lowest_subgid + 1))
```

(In the computation it was assumed that there is only one subuid range and one subgid range)

```
$ echo $uid
2003
$ echo $gid
2003
```

The computation shows that the UID is 2003 and the GID is 2003 inside the container. This comes as no surprise as this is what was specified before with --user=2003:2003, but the same computation could be used whenever a username is specified or the --user option is not used.

Run the container again but now with UIDs and GIDs mapped

```
$ mkdir dir1
$ chmod 777 dir1
$ podman run --rm

  -v ./dir1:/dir1:Z

  --user $uid:$gid

  --userns keep-id:uid=$uid,gid=$gid

     docker.io/library/ubuntu bash -c "touch /dir1/a; chmod 600 /dir1/a"
$ id -u
tester
$ id -g
tester
$ ls -l dir1/a
-rw-------. 1 tester tester 0 Jan 19 20:31 dir1/a
$
```

In this example the --user option specified a rootless user in the container. As the rootless user could also have been specified in the container image, e.g.

```
$ podman image inspect --format "user: {{.User}}" IMAGE
user: hpc
```

the same problem could also occur even without specifying --user.

Another variant of the same problem could occur when using --user=root:root (the default), but where the root user creates non-root owned files in some way (e.g by creating them themselves, or switching the effective UID to a rootless user and then creates files).

See also the troubleshooting tip:

⟨#39-podman-run-fails-with-error-unrecognized-namespace-mode-keep-iduid1000gid1000-passed⟩

### 35) Passed-in devices or files can't be accessed in rootless container (UID/GID mapping problem)
As a non-root user you have access rights to devices, files and directories that you want to pass into a rootless container with --device=..., --volume=... or --mount=...

Podman by default maps a non-root user inside a container to one of the user's subordinate UIDs and subordinate GIDs on the host. When the container user tries to access a file, a "Permission denied" error could occur because the container user does not have the permissions of the regular user of the host.

### Symptom
•

Any access inside the container is rejected with "Permission denied" for files, directories or devices passed in to the container with --device=..,--volume=.. or --mount=.., e.g.

```
$ mkdir dir1
$ chmod 700 dir1
$ podman run --rm -v ./dir1:/dir1:Z

             --user 2003:2003

             docker.io/library/ubuntu ls /dir1
ls: cannot open directory '/dir1': Permission denied

```

### Solution
We follow essentially the same solution as in the previous troubleshooting tip:

⟨#34-container-creates-a-file-that-is-not-owned-by-the-users-regular-uid⟩

but for this problem the container UID and GID can't be as easily computed by mere addition and subtraction.

In other words, it might be more challenging to find out the UID and the GID inside the container that we want to map to the regular user on the host.

If the --user option is used together with a numerical UID and GID to specify a rootless user, we already know the answer.

If the --user option is used together with a username and groupname, we could look up the UID and GID in the file /etc/passwd of the container.

If the container user is not set via --user but instead from the container image, we could inspect the container image

```
$ podman image inspect --format "user: {{.User}}" IMAGE
user: hpc
```

and then look it up in /etc/passwd of the container.

If the problem occurs in a container that is started to run as root but later switches to an effictive UID of a rootless user, it might be less straightforward to find out the UID and the GID. Reading the Containerfile, Dockerfile or the /etc/passwd could give a clue.

To run the container with the rootless container UID and GID mapped to the user's regular UID and GID on the host follow these steps:

Set the uid and gid shell variables in a Bash shell to the UID and GID of the user that will be running inside the container, e.g.

```
$ uid=2003
$ gid=2003
```

and run

```
$ mkdir dir1
$ echo hello > dir1/file.txt
$ chmod 700 dir1/file.txt
$ podman run --rm

  -v ./dir1:/dir1:Z

  --user $uid:$gid

  --userns keep-id:uid=$uid,gid=$gid

  docker.io/library/alpine cat /dir1/file.txt
hello
```

See also the troubleshooting tip:

⟨#39-podman-run-fails-with-error-unrecognized-namespace-mode-keep-iduid1000gid1000-passed⟩

### 36) Images in the additional stores can be deleted even if there are containers using them
When an image in an additional store is used, it is not locked thus it can be deleted even if there are containers using it.

### Symptom
WARN[0000] Can't stat lower layer "/var/lib/containers/storage/overlay/l/7HS76F2P5N73FDUKUQAOJA3WI5" because it does not exist. Going through storage to recreate the missing symlinks.

### Solution
It is the user responsibility to make sure images in an additional store are not deleted while being used by containers in another store.

### 37) Syncing bugfixes for podman-remote or setups using Podman API
After upgrading Podman to a newer version an issue with the earlier version of Podman still presents itself while using podman-remote.

### Symptom
While running podman remote commands with the most updated Podman, issues that were fixed in a prior version of Podman can arise either on the Podman client side or the Podman server side.

### Solution
When upgrading Podman to a particular version for the required fixes, users often make the mistake of only upgrading the Podman client. However, suppose a setup uses podman-remote or uses a client that communicates with the Podman server on a remote machine via the REST API. In that case, it is required to upgrade both the Podman client and the Podman server running on the remote machine. Both the Podman client and server must be upgraded to the same version.

Example: If a particular bug was fixed in v4.1.0 then the Podman client must have version v4.1.0 as well the Podman server must have version v4.1.0.

### 38) Unexpected carriage returns are outputted on the terminal
When using the --tty (-t) flag, unexpected carriage returns are outputted on the terminal.

### Symptom
The container program prints a newline (\n) but the terminal outputs a carriage return and a newline (\r\n).

```
$ podman run --rm -t fedora echo abc | od -c
0000000   a   b   c  \r  \n
0000005
```

When run directly on the host, the result is as expected.

```
$ echo abc | od -c
0000000   a   b   c  \n
0000004
```

Extra carriage returns can also shift the prompt to the right.

```
$ podman run --rm -t fedora sh -c "echo 1; echo 2; echo 3" | cat -A
1^M$

    2^M$

        3^M$

            $
```

### Solution
Run Podman without the --tty (-t) flag.

```
$ podman run --rm fedora echo abc | od -c
0000000   a   b   c  \n
0000004
```

The --tty (-t) flag should only be used when the program requires user interaction in the termainal, for instance expecting the user to type an answer to a question.

Where does the extra carriage return \r come from?

The extra \r is not outputted by Podman but by the terminal. In fact, a reconfiguration of the terminal can make the extra \r go away.

```
$ podman run --rm -t fedora /bin/sh -c "stty -onlcr && echo abc" | od -c
0000000   a   b   c  \n
0000004
```

### 39) Podman run fails with Error: unrecognized namespace mode keep-id:uid=1000,gid=1000 passed
Podman 4.3.0 introduced the options uid and gid that can be given to --userns keep-id which are not recognized by older versions of Podman.

### Symptom
When using a Podman version older than 4.3.0, the options uid and gid are not recognized, and an "unrecognized namespace mode" error is raised.

```
$ uid=1000
$ gid=1000
$ podman run --rm

  --user $uid:$gid

  --userns keep-id:uid=$uid,gid=$gid

     docker.io/library/ubuntu /bin/cat /proc/self/uid_map
Error: unrecognized namespace mode keep-id:uid=1000,gid=1000 passed
$
```

### Solution
Use --uidmap and --gidmap options to describe the same UID and GID mapping.

Run

```
$ uid=1000
$ gid=1000
$ subuidSize=$(( $(podman info --format "{{ range

   .Host.IDMappings.UIDMap }}+{{.Size }}{{end }}" ) - 1 ))
$ subgidSize=$(( $(podman info --format "{{ range

   .Host.IDMappings.GIDMap }}+{{.Size }}{{end }}" ) - 1 ))
$ podman run --rm

  --user $uid:$gid

  --uidmap 0:1:$uid

  --uidmap $uid:0:1

  --uidmap $(($uid+1)):$(($uid+1)):$(($subuidSize-$uid))

  --gidmap 0:1:$gid

  --gidmap $gid:0:1

  --gidmap $(($gid+1)):$(($gid+1)):$(($subgidSize-$gid))

     docker.io/library/ubuntu /bin/cat /proc/self/uid_map

         0          1       1000

      1000          0          1

      1001       1001      64536
```

which uses the same UID and GID mapping as when specifying --userns keep-id:uid=$uid,gid=$gid with Podman 4.3.0 (or greater)

```
$ uid=1000
$ gid=1000
$ podman run --rm

  --user $uid:$gid

  --userns keep-id:uid=$uid,gid=$gid

     docker.io/library/ubuntu /bin/cat /proc/self/uid_map

         0          1       1000

      1000          0          1

      1001       1001      64536
```

Replace /bin/cat /proc/self/uid_map with /bin/cat /proc/self/gid_map to show the GID mapping.

### 40) Podman fails to find expected image with error locating pulled image , image not known
When trying to do a Podman command that pulls an image from local storage or a remote repository, an error is raised saying "image not known" or "error locating pulled image". Even though the image had been verified before the Podman command was invoked.

### Symptom
After verifying that an image is in place either locally or on a remote repository, a Podman command referencing that image will fail in a manner like:

```
# ls Containerfile
FROM registry.access.redhat.com/ubi8-minimal:latest
MAINTAINER Podman Community
USER root
# podman build .
STEP 1/2: FROM registry.access.redhat.com/ubi8-minimal
Trying to pull registry.access.redhat.com/ubi8-minimal:latest...
Getting image source signatures
Checking if image destination supports signatures
Copying blob a6577091999b done
Copying config abb1ba1bce done
Writing manifest to image destination
Storing signatures
Error: error creating build container: error locating pulled image "registry.access.redhat.com/ubi8-minimal:latest" name in containers storage: registry.access.redhat.com/ubi8-minimal:latest: image not known
```

### Solution
The general cause for this is a timing issue. To make Podman commands as efficient as possible, read and write locks are only established for critical sections within the code. When pulling an image from a repository, a copy of that image is first written to local storage using a write lock. This lock is released before the image is then acquired/read. If another process does a harmful command such as podman system prune --all or podman system reset or podman rmi --all, between the time the image is written and before the first process can acquire it, this type of image not known error can arise.

The maintainers of Podman have considered heavier-duty locks to close this timing window. However, the slowdown that all Podman commands would encounter was not considered worth the cost of completely closing this small timing window.

### 41) A podman build step with --mount=type=secret fails with operation not permitted
Executing a step in a Dockerfile/Containerfile which mounts secrets using --mount=type=secret fails with "operation not permitted" when running on a host filesystem mounted with nosuid and when using the runc runtime.

### Symptom
A RUN line in the Dockerfile/Containerfile contains a secret mount such as --mount=type=secret,id=MY_USER,target=/etc/dnf/vars/MY_USER. When running podman build the process fails with an error message like:

```
STEP 3/13: RUN --mount=type=secret,id=MY_USER,target=/etc/dnf/vars/MY_USER     --mount=type=secret,id=MY_USER,target=/etc/dnf/vars/MY_USER     ...: time="2023-06-13T18:04:59+02:00" level=error msg="runc create failed: unable to start container process: error during container init: error mounting \"/var/tmp/buildah2251989386/mnt/buildah-bind-target-11\" to rootfs at \"/etc/dnf/vars/MY_USER\": mount /var/tmp/buildah2251989386/mnt/buildah-bind-target-11:/etc/dnf/vars/MY_USER (via /proc/self/fd/7), flags: 0x1021: operation not permitted"
: exit status 1
ERRO[0002] did not get container create message from subprocess: EOF
```

### Solution

- Install crun, e.g. with dnf install crun.

- Use the crun runtime by passing --runtime /usr/bin/crun to podman build.

See also Buildah issue 4228 for a full discussion of the problem.

### 42) podman-in-podman builds that are file I/0 intensive are very slow
When using the overlay storage driver to do a nested podman build inside a running container, file I/O operations such as COPY of a large amount of data is very slow or can hang completely.

### Symptom
Using the default overlay storage driver, a COPY, ADD, or an I/O intensive RUN line in a Containerfile that is run inside another container is very slow or hangs completely when running a podman build inside the running parent container.

### Solution
This could be caused by the child container using fuse-overlayfs for writing to /var/lib/containers/storage. Writes can be slow with fuse-overlayfs. The solution is to use the native overlay filesystem by using a local directory on the host system as a volume to /var/lib/containers/storage like so: podman run --privileged --rm -it -v ./nested_storage:/var/lib/containers/storage parent:latest. Ensure that the base image of parent:latest in this example has no contents in /var/lib/containers/storage in the image itself for this to work. Once using the native volume, the nested container should not fall back to fuse-overlayfs to write files and the nested build will complete much faster.

If you don't have access to the parent run process, such as in a CI environment, then the second option is to change the storage driver to vfs in the parent image by changing changing this line in your storage.conf file: driver = "vfs". You may have to run podman system reset for this to take effect. You know it's changed when podman info |grep graphDriverName outputs graphDriverName: vfs. This method is slower performance than using the volume method above but is significantly faster than fuse-overlayfs

### 43) podman run --userns=auto fails with Error: creating container storage: not enough unused IDs in user namespace
Using --userns=auto when creating new containers does not work as long as any containers exist that were created with --userns=keep-id or --userns=nomap

### Symptom
1.

Run with --userns=auto $ podman run --rm -d --userns=auto alpine sleep 3600 The command succeeds.

2.

Run with --userns=auto $ podman run --rm -d --userns=auto alpine sleep 3600 The command succeeds.

3.

Run with --userns=keep-id $ podman run --rm -d --userns=keep-id alpine sleep 3600 The command succeeds.

4.

Run with --userns=auto $ podman run --rm -d --userns=auto alpine sleep 3600 The command fails with the error message Error: creating container storage: not enough unused IDs in user namespace

### Solution
Any existing containers that were created using --userns=keep-id or --userns=nomap must first be deleted before any new container can be created with --userns=auto

### 44) sudo podman run --userns=auto fails with Cannot find mappings for user containers
When rootful podman is invoked with --userns=auto, podman needs to pick subranges of subuids and subgids for the user namespace of the container. Rootful podman ensures that the subuid and subgid ranges for such containers do not overlap, but how can rootful podman prevent other tools from accidentally using these IDs?

It's not possible to block other tools that are running as root from using these IDs, but such tools would normally not use subuids and subgids that have already been assigned to a user in /etc/subuid and /etc/subgid.

The username containers on the host has a special function for rootful Podman. Rootful podman uses the subuids and subgids of the user containers when running --userns=auto containers. The user containers has no need for these subuids and subgids because no processes should be started as the user containers. In other words, the user containers is a special user that only exists on the system to reserve subuids and subgids for rootful podman.

containers is the default username but it can be changed by setting the option root-auto-userns-user in the file /etc/containers/storage.conf

### Symptom
Run rootful podman with --userns=auto

```
sudo podman run --rm --userns=auto alpine echo hello
```

The command fails with the error message:

```
ERRO[0000] Cannot find mappings for user "containers": no subuid ranges found for user "containers" in /etc/subuid
Error: creating container storage: not enough unused IDs in user namespace
```

The files /etc/subuid and /etc/subgid do not have any lines that start with containers:

### Solution
Add subuid and subgid ranges for the user containers in /etc/subuid and /etc/subgid or provide such ranges with /etc/nsswitch.conf. For details, see subid(5).

The following steps create the user containers and assigns big subuid and subgid ranges to it:

1.

Create the user containers sudo useradd --comment "Helper user to reserve subuids and subgids for Podman"

--no-create-home

--shell /usr/sbin/nologin

containers

2.

Check the subuid and subgid ranges of the user containers $ grep ^containers: /etc/subuid containers:720896:65536 $ grep ^containers: /etc/subgid containers:720896:65536 By default useradd assigns 65536 subuids and 65536 subgids to a new user. Typically you would like the reserved pool to be bigger than that. The bigger the size, the more containers could be started with sudo podman run --userns=auto ...

3.

Edit the line for the user containers in the files /etc/subuid and /etc/subgid to make the ranges bigger. Ensure that the subuid range of the user containers do not overlap with any other subuid ranges in the files /etc/subuid. Ensure that the subgid range of the user containers do not overlap with any other subgid ranges in the files /etc/subgid.

Test the echo command again

```
sudo podman run --rm --userns=auto alpine echo hello
```

The command succeeds and prints hello

### 45) Podman fails with lsetxattr(label=system_u:object_r:container_file_t:s0) /dir: operation not permitted
Strict file permissions prevent Podman from modifying SELinux context file labels in a bind-mounted directory. The error might also occur when using bind-mounted directories that reside on file systems that do not support XATTR (such as old versions of NFS).

### Symptom
1.

Create a directory owned by root sudo mkdir /var/test

2.

Give everyone read access to the directory sudo chmod 755 /var/test

3.

Run ls directly on the host ls /var/test The command succeeds.

4.

Run rootless podman with the directory bind-mounted. Set z as volume option. podman run --rm -v /var/test:/dir:z fedora ls /dir The command fails with the following error message: Error: lsetxattr(label=system_u:object_r:container_file_t:s0) /var/test: operation not permitted The error is expected because the unprivileged user does not have the privileges to modify the SELinux context of the directory /var/test

5.

Repeat the previous command but remove the volume option podman run --rm -v /var/test:/dir fedora ls /dir The command is blocked by SELinux and fails with the following error message: ls: cannot open directory '/dir': Permission denied

Note, access to the bind-mounted directory depends on SELinux rules. For example, bind-mounting the directory /usr/share succeeds even though the user does not have write permissions to the directory. The following command succeeds:

```
podman run --rm -v /usr/share:/dir fedora ls /dir
```

### Solution
Alternative 1

Use the O flag to mount the directory from the host as a temporary storage using the overlay file system.

```
podman run --rm -v /var/test:/dir:O fedora ls /dir
```

Modifications to the mount point are destroyed when the container finishes executing, similar to a tmpfs mount point being unmounted.

The following command succeeds:

```
podman run --rm -v /var/test:/dir:O fedora touch /dir/file
```

The command touch creates the file but the file is discarded when the container finishes.

Alternative 2

If the directory /var/test resides on a file system that supports XATTR, set a directory SELinux context that corresponds to the podman volume flag :z

```
sudo chcon -R system_u:object_r:container_file_t:s0 /var/test
```

The following command now succeeds

```
podman run --rm -v /var/test:/dir ls /dir
```

Alternative 3

Modify the default SELinux context configuration for the directory /var/test with the command

```
sudo semanage fcontext -a -s system_u -t container_file_t "/var/test(/.*)?"
```

Change the SElinux contexts recursively for files and directories under /var/test to match the new configuration

```
sudo restorecon -R -F -v /var/test
```

Alternative 4

Add the option --security-opt label=disable to disable SELinux for the Podman command.

```
podman run --rm --security-opt label=disable -v /var/test:/dir fedora ls /dir
```


ABORT

title: ABORT(7)

summary: abort the current transaction

text:
# ABORT(7)

## NAME
ABORT - abort the current transaction

## SYNOPSIS

```
ABORT [ WORK | TRANSACTION ] [ AND [ NO ] CHAIN ]
```

## DESCRIPTION
ABORT rolls back the current transaction and causes all the updates made by the transaction to be discarded. This command is identical in behavior to the standard SQL command ROLLBACK, and is present only for historical reasons.

## PARAMETERS
WORK

TRANSACTION

Optional key words. They have no effect.

AND CHAIN

If AND CHAIN is specified, a new transaction is immediately started with the same transaction characteristics (see SET TRANSACTION) as the just finished one. Otherwise, no new transaction is started.

## NOTES
Use COMMIT to successfully terminate a transaction.

Issuing ABORT outside of a transaction block emits a warning and otherwise has no effect.

## EXAMPLES
To abort all changes:

```
ABORT;
```

## COMPATIBILITY
This command is a PostgreSQL extension present for historical reasons. ROLLBACK is the equivalent standard SQL command.

## SEE ALSO
BEGIN(7), COMMIT(7), ROLLBACK(7)

branch: sisyphus


title: ALTER_AGGREGATE(7)

text:
# ALTER_AGGREGATE(7)

## NAME
ALTER_AGGREGATE - change the definition of an aggregate function

## SYNOPSIS

```
ALTER AGGREGATE name ( aggregate_signature ) RENAME TO new_name
ALTER AGGREGATE name ( aggregate_signature )

                OWNER TO { new_owner | CURRENT_ROLE | CURRENT_USER | SESSION_USER }
ALTER AGGREGATE name ( aggregate_signature ) SET SCHEMA new_schema
where aggregate_signature is:
* |
[ argmode ] [ argname ] argtype [ , ... ] |
[ [ argmode ] [ argname ] argtype [ , ... ] ] ORDER BY [ argmode ] [ argname ] argtype [ , ... ]
```

## DESCRIPTION
ALTER AGGREGATE changes the definition of an aggregate function.

You must own the aggregate function to use ALTER AGGREGATE. To change the schema of an aggregate function, you must also have CREATE privilege on the new schema. To alter the owner, you must also be a direct or indirect member of the new owning role, and that role must have CREATE privilege on the aggregate function's schema. (These restrictions enforce that altering the owner doesn't do anything you couldn't do by dropping and recreating the aggregate function. However, a superuser can alter ownership of any aggregate function anyway.)

## PARAMETERS
name

The name (optionally schema-qualified) of an existing aggregate function.

argmode

The mode of an argument: IN or VARIADIC. If omitted, the default is IN.

argname

The name of an argument. Note that ALTER AGGREGATE does not actually pay any attention to argument names, since only the argument data types are needed to determine the aggregate function's identity.

argtype

An input data type on which the aggregate function operates. To reference a zero-argument aggregate function, write * in place of the list of argument specifications. To reference an ordered-set aggregate function, write ORDER BY between the direct and aggregated argument specifications.

new_name

The new name of the aggregate function.

new_owner

The new owner of the aggregate function.

new_schema

The new schema for the aggregate function.

## NOTES
The recommended syntax for referencing an ordered-set aggregate is to write ORDER BY between the direct and aggregated argument specifications, in the same style as in CREATE AGGREGATE. However, it will also work to omit ORDER BY and just run the direct and aggregated argument specifications into a single list. In this abbreviated form, if VARIADIC "any" was used in both the direct and aggregated argument lists, write VARIADIC "any" only once.

## EXAMPLES
To rename the aggregate function myavg for type integer to my_average:

```
ALTER AGGREGATE myavg(integer) RENAME TO my_average;
```

To change the owner of the aggregate function myavg for type integer to joe:

```
ALTER AGGREGATE myavg(integer) OWNER TO joe;
```

To move the ordered-set aggregate mypercentile with direct argument of type float8 and aggregated argument of type integer into schema myschema:

```
ALTER AGGREGATE mypercentile(float8 ORDER BY integer) SET SCHEMA myschema;
```

This will work too:

```
ALTER AGGREGATE mypercentile(float8, integer) SET SCHEMA myschema;
```

## COMPATIBILITY
There is no ALTER AGGREGATE statement in the SQL standard.

## SEE ALSO
CREATE AGGREGATE (CREATE_AGGREGATE(7)), DROP AGGREGATE (DROP_AGGREGATE(7))


title: ALTER_COLLATION(7)

summary: change the definition of a collation

text:
# ALTER_COLLATION(7)

## NAME
ALTER_COLLATION - change the definition of a collation

## SYNOPSIS

```
ALTER COLLATION name REFRESH VERSION
ALTER COLLATION name RENAME TO new_name
ALTER COLLATION name OWNER TO { new_owner | CURRENT_ROLE | CURRENT_USER | SESSION_USER }
ALTER COLLATION name SET SCHEMA new_schema
```

## DESCRIPTION
ALTER COLLATION changes the definition of a collation.

You must own the collation to use ALTER COLLATION. To alter the owner, you must also be a direct or indirect member of the new owning role, and that role must have CREATE privilege on the collation's schema. (These restrictions enforce that altering the owner doesn't do anything you couldn't do by dropping and recreating the collation. However, a superuser can alter ownership of any collation anyway.)

## PARAMETERS
name

The name (optionally schema-qualified) of an existing collation.

new_name

The new name of the collation.

new_owner

The new owner of the collation.

new_schema

The new schema for the collation.

REFRESH VERSION

Update the collation's version. See Notes below.

## NOTES
When using collations provided by the ICU library, the ICU-specific version of the collator is recorded in the system catalog when the collation object is created. When the collation is used, the current version is checked against the recorded version, and a warning is issued when there is a mismatch, for example:

```
WARNING:  collation "xx-x-icu" has version mismatch
DETAIL:  The collation in the database was created using version 1.2.3.4, but the operating system provides version 2.3.4.5.
HINT:  Rebuild all objects affected by this collation and run ALTER COLLATION pg_catalog."xx-x-icu" REFRESH VERSION, or build PostgreSQL with the right library version.
```

A change in collation definitions can lead to corrupt indexes and other problems because the database system relies on stored objects having a certain sort order. Generally, this should be avoided, but it can happen in legitimate circumstances, such as when using pg_upgrade to upgrade to server binaries linked with a newer version of ICU. When this happens, all objects depending on the collation should be rebuilt, for example, using REINDEX. When that is done, the collation version can be refreshed using the command ALTER COLLATION ... REFRESH VERSION. This will update the system catalog to record the current collator version and will make the warning go away. Note that this does not actually check whether all affected objects have been rebuilt correctly.

When using collations provided by libc, version information is recorded on systems using the GNU C library (most Linux systems), FreeBSD and Windows.

Note

When using the GNU C library for collations, the C library's version is used as a proxy for the collation version. Many Linux distributions change collation definitions only when upgrading the C library, but this approach is imperfect as maintainers are free to back-port newer collation definitions to older C library releases.

When using Windows for collations, version information is only available for collations defined with BCP 47 language tags such as en-US.

Currently, there is no version tracking for the database default collation.

The following query can be used to identify all collations in the current database that need to be refreshed and the objects that depend on them:

```
SELECT pg_describe_object(refclassid, refobjid, refobjsubid) AS "Collation",

       pg_describe_object(classid, objid, objsubid) AS "Object"

  FROM pg_depend d JOIN pg_collation c

       ON refclassid = 'pg_collation'::regclass AND refobjid = c.oid

  WHERE c.collversion <> pg_collation_actual_version(c.oid)

  ORDER BY 1, 2;
```

## EXAMPLES
To rename the collation de_DE to german:

```
ALTER COLLATION "de_DE" RENAME TO german;
```

To change the owner of the collation en_US to joe:

```
ALTER COLLATION "en_US" OWNER TO joe;
```

## COMPATIBILITY
There is no ALTER COLLATION statement in the SQL standard.

## SEE ALSO
CREATE COLLATION (CREATE_COLLATION(7)), DROP COLLATION (DROP_COLLATION(7))


title: ALTER_CONVERSION(7)

summary: change the definition of a conversion

text:
# ALTER_CONVERSION(7)

## NAME
ALTER_CONVERSION - change the definition of a conversion

## SYNOPSIS

```
ALTER CONVERSION name RENAME TO new_name
ALTER CONVERSION name OWNER TO { new_owner | CURRENT_ROLE | CURRENT_USER | SESSION_USER }
ALTER CONVERSION name SET SCHEMA new_schema
```

## DESCRIPTION
ALTER CONVERSION changes the definition of a conversion.

You must own the conversion to use ALTER CONVERSION. To alter the owner, you must also be a direct or indirect member of the new owning role, and that role must have CREATE privilege on the conversion's schema. (These restrictions enforce that altering the owner doesn't do anything you couldn't do by dropping and recreating the conversion. However, a superuser can alter ownership of any conversion anyway.)

## PARAMETERS
name

The name (optionally schema-qualified) of an existing conversion.

new_name

The new name of the conversion.

new_owner

The new owner of the conversion.

new_schema

The new schema for the conversion.

## EXAMPLES
To rename the conversion iso_8859_1_to_utf8 to latin1_to_unicode:

```
ALTER CONVERSION iso_8859_1_to_utf8 RENAME TO latin1_to_unicode;
```

To change the owner of the conversion iso_8859_1_to_utf8 to joe:

```
ALTER CONVERSION iso_8859_1_to_utf8 OWNER TO joe;
```

## COMPATIBILITY
There is no ALTER CONVERSION statement in the SQL standard.

## SEE ALSO
CREATE CONVERSION (CREATE_CONVERSION(7)), DROP CONVERSION (DROP_CONVERSION(7))


title: ALTER_DATABASE(7)

summary: change a database

text:
# ALTER_DATABASE(7)

## NAME
ALTER_DATABASE - change a database

## SYNOPSIS

```
ALTER DATABASE name [ [ WITH ] option [ ... ] ]
where option can be:

    ALLOW_CONNECTIONS allowconn

    CONNECTION LIMIT connlimit

    IS_TEMPLATE istemplate
ALTER DATABASE name RENAME TO new_name
ALTER DATABASE name OWNER TO { new_owner | CURRENT_ROLE | CURRENT_USER | SESSION_USER }
ALTER DATABASE name SET TABLESPACE new_tablespace
ALTER DATABASE name SET configuration_parameter { TO | = } { value | DEFAULT }
ALTER DATABASE name SET configuration_parameter FROM CURRENT
ALTER DATABASE name RESET configuration_parameter
ALTER DATABASE name RESET ALL
```

## DESCRIPTION
ALTER DATABASE changes the attributes of a database.

The first form changes certain per-database settings. (See below for details.) Only the database owner or a superuser can change these settings.

The second form changes the name of the database. Only the database owner or a superuser can rename a database; non-superuser owners must also have the CREATEDB privilege. The current database cannot be renamed. (Connect to a different database if you need to do that.)

The third form changes the owner of the database. To alter the owner, you must own the database and also be a direct or indirect member of the new owning role, and you must have the CREATEDB privilege. (Note that superusers have all these privileges automatically.)

The fourth form changes the default tablespace of the database. Only the database owner or a superuser can do this; you must also have create privilege for the new tablespace. This command physically moves any tables or indexes in the database's old default tablespace to the new tablespace. The new default tablespace must be empty for this database, and no one can be connected to the database. Tables and indexes in non-default tablespaces are unaffected.

The remaining forms change the session default for a run-time configuration variable for a PostgreSQL database. Whenever a new session is subsequently started in that database, the specified value becomes the session default value. The database-specific default overrides whatever setting is present in postgresql.conf or has been received from the postgres command line. Only the database owner or a superuser can change the session defaults for a database. Certain variables cannot be set this way, or can only be set by a superuser.

## PARAMETERS
name

The name of the database whose attributes are to be altered.

allowconn

If false then no one can connect to this database.

connlimit

How many concurrent connections can be made to this database. -1 means no limit.

istemplate

If true, then this database can be cloned by any user with CREATEDB privileges; if false, then only superusers or the owner of the database can clone it.

new_name

The new name of the database.

new_owner

The new owner of the database.

new_tablespace

The new default tablespace of the database. This form of the command cannot be executed inside a transaction block.

configuration_parameter

value

Set this database's session default for the specified configuration parameter to the given value. If value is DEFAULT or, equivalently, RESET is used, the database-specific setting is removed, so the system-wide default setting will be inherited in new sessions. Use RESET ALL to clear all database-specific settings. SET FROM CURRENT saves the session's current value of the parameter as the database-specific value. See SET(7) and Chapter 20 for more information about allowed parameter names and values.

## NOTES
It is also possible to tie a session default to a specific role rather than to a database; see ALTER ROLE (ALTER_ROLE(7)). Role-specific settings override database-specific ones if there is a conflict.

## EXAMPLES
To disable index scans by default in the database test:

```
ALTER DATABASE test SET enable_indexscan TO off;
```

## COMPATIBILITY
The ALTER DATABASE statement is a PostgreSQL extension.

## SEE ALSO
CREATE DATABASE (CREATE_DATABASE(7)), DROP DATABASE (DROP_DATABASE(7)), SET(7), CREATE TABLESPACE (CREATE_TABLESPACE(7))


title: ALTER_DEFAULT_PRIVILEGES(7)

summary: define default access privileges

text:
# ALTER_DEFAULT_PRIVILEGES(7)

## NAME
ALTER_DEFAULT_PRIVILEGES - define default access privileges

## SYNOPSIS

```
ALTER DEFAULT PRIVILEGES

    [ FOR { ROLE | USER } target_role [, ...] ]

    [ IN SCHEMA schema_name [, ...] ]

    abbreviated_grant_or_revoke
where abbreviated_grant_or_revoke is one of:
GRANT { { SELECT | INSERT | UPDATE | DELETE | TRUNCATE | REFERENCES | TRIGGER }

    [, ...] | ALL [ PRIVILEGES ] }

    ON TABLES

    TO { [ GROUP ] role_name | PUBLIC } [, ...] [ WITH GRANT OPTION ]
GRANT { { USAGE | SELECT | UPDATE }

    [, ...] | ALL [ PRIVILEGES ] }

    ON SEQUENCES

    TO { [ GROUP ] role_name | PUBLIC } [, ...] [ WITH GRANT OPTION ]
GRANT { EXECUTE | ALL [ PRIVILEGES ] }

    ON { FUNCTIONS | ROUTINES }

    TO { [ GROUP ] role_name | PUBLIC } [, ...] [ WITH GRANT OPTION ]
GRANT { USAGE | ALL [ PRIVILEGES ] }

    ON TYPES

    TO { [ GROUP ] role_name | PUBLIC } [, ...] [ WITH GRANT OPTION ]
GRANT { { USAGE | CREATE }

    [, ...] | ALL [ PRIVILEGES ] }

    ON SCHEMAS

    TO { [ GROUP ] role_name | PUBLIC } [, ...] [ WITH GRANT OPTION ]
REVOKE [ GRANT OPTION FOR ]

    { { SELECT | INSERT | UPDATE | DELETE | TRUNCATE | REFERENCES | TRIGGER }

    [, ...] | ALL [ PRIVILEGES ] }

    ON TABLES

    FROM { [ GROUP ] role_name | PUBLIC } [, ...]

    [ CASCADE | RESTRICT ]
REVOKE [ GRANT OPTION FOR ]

    { { USAGE | SELECT | UPDATE }

    [, ...] | ALL [ PRIVILEGES ] }

    ON SEQUENCES

    FROM { [ GROUP ] role_name | PUBLIC } [, ...]

    [ CASCADE | RESTRICT ]
REVOKE [ GRANT OPTION FOR ]

    { EXECUTE | ALL [ PRIVILEGES ] }

    ON { FUNCTIONS | ROUTINES }

    FROM { [ GROUP ] role_name | PUBLIC } [, ...]

    [ CASCADE | RESTRICT ]
REVOKE [ GRANT OPTION FOR ]

    { USAGE | ALL [ PRIVILEGES ] }

    ON TYPES

    FROM { [ GROUP ] role_name | PUBLIC } [, ...]

    [ CASCADE | RESTRICT ]
REVOKE [ GRANT OPTION FOR ]

    { { USAGE | CREATE }

    [, ...] | ALL [ PRIVILEGES ] }

    ON SCHEMAS

    FROM { [ GROUP ] role_name | PUBLIC } [, ...]

    [ CASCADE | RESTRICT ]
```

## DESCRIPTION
ALTER DEFAULT PRIVILEGES allows you to set the privileges that will be applied to objects created in the future. (It does not affect privileges assigned to already-existing objects.) Currently, only the privileges for schemas, tables (including views and foreign tables), sequences, functions, and types (including domains) can be altered. For this command, functions include aggregates and procedures. The words FUNCTIONS and ROUTINES are equivalent in this command. (ROUTINES is preferred going forward as the standard term for functions and procedures taken together. In earlier PostgreSQL releases, only the word FUNCTIONS was allowed. It is not possible to set default privileges for functions and procedures separately.)

You can change default privileges only for objects that will be created by yourself or by roles that you are a member of. The privileges can be set globally (i.e., for all objects created in the current database), or just for objects created in specified schemas.

As explained in Section 5.7, the default privileges for any object type normally grant all grantable permissions to the object owner, and may grant some privileges to PUBLIC as well. However, this behavior can be changed by altering the global default privileges with ALTER DEFAULT PRIVILEGES.

Default privileges that are specified per-schema are added to whatever the global default privileges are for the particular object type. This means you cannot revoke privileges per-schema if they are granted globally (either by default, or according to a previous ALTER DEFAULT PRIVILEGES command that did not specify a schema). Per-schema REVOKE is only useful to reverse the effects of a previous per-schema GRANT.

### Parameters
target_role

The name of an existing role of which the current role is a member. Default access privileges are not inherited, so member roles must use SET ROLE to access these privileges, or ALTER DEFAULT PRIVILEGES must be run for each member role. If FOR ROLE is omitted, the current role is assumed.

schema_name

The name of an existing schema. If specified, the default privileges are altered for objects later created in that schema. If IN SCHEMA is omitted, the global default privileges are altered. IN SCHEMA is not allowed when setting privileges for schemas, since schemas can't be nested.

role_name

The name of an existing role to grant or revoke privileges for. This parameter, and all the other parameters in abbreviated_grant_or_revoke, act as described under GRANT(7) or REVOKE(7), except that one is setting permissions for a whole class of objects rather than specific named objects.

## NOTES
Use psql(1)'s \ddp command to obtain information about existing assignments of default privileges. The meaning of the privilege display is the same as explained for \dp in Section 5.7.

If you wish to drop a role for which the default privileges have been altered, it is necessary to reverse the changes in its default privileges or use DROP OWNED BY to get rid of the default privileges entry for the role.

## EXAMPLES
Grant SELECT privilege to everyone for all tables (and views) you subsequently create in schema myschema, and allow role webuser to INSERT into them too:

```
ALTER DEFAULT PRIVILEGES IN SCHEMA myschema GRANT SELECT ON TABLES TO PUBLIC;
ALTER DEFAULT PRIVILEGES IN SCHEMA myschema GRANT INSERT ON TABLES TO webuser;
```

Undo the above, so that subsequently-created tables won't have any more permissions than normal:

```
ALTER DEFAULT PRIVILEGES IN SCHEMA myschema REVOKE SELECT ON TABLES FROM PUBLIC;
ALTER DEFAULT PRIVILEGES IN SCHEMA myschema REVOKE INSERT ON TABLES FROM webuser;
```

Remove the public EXECUTE permission that is normally granted on functions, for all functions subsequently created by role admin:

```
ALTER DEFAULT PRIVILEGES FOR ROLE admin REVOKE EXECUTE ON FUNCTIONS FROM PUBLIC;
```

Note however that you cannot accomplish that effect with a command limited to a single schema. This command has no effect, unless it is undoing a matching GRANT:

```
ALTER DEFAULT PRIVILEGES IN SCHEMA public REVOKE EXECUTE ON FUNCTIONS FROM PUBLIC;
```

That's because per-schema default privileges can only add privileges to the global setting, not remove privileges granted by it.

## COMPATIBILITY
There is no ALTER DEFAULT PRIVILEGES statement in the SQL standard.

## SEE ALSO
GRANT(7), REVOKE(7)


title: ALTER_DOMAIN(7)

summary: change the definition of a domain

text:
# ALTER_DOMAIN(7)

## NAME
ALTER_DOMAIN - change the definition of a domain

## SYNOPSIS

```
ALTER DOMAIN name

    { SET DEFAULT expression | DROP DEFAULT }
ALTER DOMAIN name

    { SET | DROP } NOT NULL
ALTER DOMAIN name

    ADD domain_constraint [ NOT VALID ]
ALTER DOMAIN name

    DROP CONSTRAINT [ IF EXISTS ] constraint_name [ RESTRICT | CASCADE ]
ALTER DOMAIN name

     RENAME CONSTRAINT constraint_name TO new_constraint_name
ALTER DOMAIN name

    VALIDATE CONSTRAINT constraint_name
ALTER DOMAIN name

    OWNER TO { new_owner | CURRENT_ROLE | CURRENT_USER | SESSION_USER }
ALTER DOMAIN name

    RENAME TO new_name
ALTER DOMAIN name

    SET SCHEMA new_schema
where domain_constraint is:
[ CONSTRAINT constraint_name ]
{ NOT NULL | CHECK (expression) }
```

## DESCRIPTION
ALTER DOMAIN changes the definition of an existing domain. There are several sub-forms:

SET/DROP DEFAULT

These forms set or remove the default value for a domain. Note that defaults only apply to subsequent INSERT commands; they do not affect rows already in a table using the domain.

SET/DROP NOT NULL

These forms change whether a domain is marked to allow NULL values or to reject NULL values. You can only SET NOT NULL when the columns using the domain contain no null values.

ADD domain_constraint [ NOT VALID ]

This form adds a new constraint to a domain. When a new constraint is added to a domain, all columns using that domain will be checked against the newly added constraint. These checks can be suppressed by adding the new constraint using the NOT VALID option; the constraint can later be made valid using ALTER DOMAIN ... VALIDATE CONSTRAINT. Newly inserted or updated rows are always checked against all constraints, even those marked NOT VALID. NOT VALID is only accepted for CHECK constraints.

DROP CONSTRAINT [ IF EXISTS ]

This form drops constraints on a domain. If IF EXISTS is specified and the constraint does not exist, no error is thrown. In this case a notice is issued instead.

RENAME CONSTRAINT

This form changes the name of a constraint on a domain.

VALIDATE CONSTRAINT

This form validates a constraint previously added as NOT VALID, that is, it verifies that all values in table columns of the domain type satisfy the specified constraint.

OWNER

This form changes the owner of the domain to the specified user.

RENAME

This form changes the name of the domain.

SET SCHEMA

This form changes the schema of the domain. Any constraints associated with the domain are moved into the new schema as well.

You must own the domain to use ALTER DOMAIN. To change the schema of a domain, you must also have CREATE privilege on the new schema. To alter the owner, you must also be a direct or indirect member of the new owning role, and that role must have CREATE privilege on the domain's schema. (These restrictions enforce that altering the owner doesn't do anything you couldn't do by dropping and recreating the domain. However, a superuser can alter ownership of any domain anyway.)

## PARAMETERS
name

The name (possibly schema-qualified) of an existing domain to alter.

domain_constraint

New domain constraint for the domain.

constraint_name

Name of an existing constraint to drop or rename.

NOT VALID

Do not verify existing stored data for constraint validity.

CASCADE

Automatically drop objects that depend on the constraint, and in turn all objects that depend on those objects (see Section 5.14).

RESTRICT

Refuse to drop the constraint if there are any dependent objects. This is the default behavior.

new_name

The new name for the domain.

new_constraint_name

The new name for the constraint.

new_owner

The user name of the new owner of the domain.

new_schema

The new schema for the domain.

## NOTES
Although ALTER DOMAIN ADD CONSTRAINT attempts to verify that existing stored data satisfies the new constraint, this check is not bulletproof, because the command cannot “see” table rows that are newly inserted or updated and not yet committed. If there is a hazard that concurrent operations might insert bad data, the way to proceed is to add the constraint using the NOT VALID option, commit that command, wait until all transactions started before that commit have finished, and then issue ALTER DOMAIN VALIDATE CONSTRAINT to search for data violating the constraint. This method is reliable because once the constraint is committed, all new transactions are guaranteed to enforce it against new values of the domain type.

Currently, ALTER DOMAIN ADD CONSTRAINT, ALTER DOMAIN VALIDATE CONSTRAINT, and ALTER DOMAIN SET NOT NULL will fail if the named domain or any derived domain is used within a container-type column (a composite, array, or range column) in any table in the database. They should eventually be improved to be able to verify the new constraint for such nested values.

## EXAMPLES
To add a NOT NULL constraint to a domain:

```
ALTER DOMAIN zipcode SET NOT NULL;
```

To remove a NOT NULL constraint from a domain:

```
ALTER DOMAIN zipcode DROP NOT NULL;
```

To add a check constraint to a domain:

```
ALTER DOMAIN zipcode ADD CONSTRAINT zipchk CHECK (char_length(VALUE) = 5);
```

To remove a check constraint from a domain:

```
ALTER DOMAIN zipcode DROP CONSTRAINT zipchk;
```

To rename a check constraint on a domain:

```
ALTER DOMAIN zipcode RENAME CONSTRAINT zipchk TO zip_check;
```

To move the domain into a different schema:

```
ALTER DOMAIN zipcode SET SCHEMA customers;
```

## COMPATIBILITY
ALTER DOMAIN conforms to the SQL standard, except for the OWNER, RENAME, SET SCHEMA, and VALIDATE CONSTRAINT variants, which are PostgreSQL extensions. The NOT VALID clause of the ADD CONSTRAINT variant is also a PostgreSQL extension.

## SEE ALSO
CREATE DOMAIN (CREATE_DOMAIN(7)), DROP DOMAIN (DROP_DOMAIN(7))


title: ALTER_EVENT_TRIGGER(7)

text:
# ALTER_EVENT_TRIGGER(7)

## NAME
ALTER_EVENT_TRIGGER - change the definition of an event trigger

## SYNOPSIS

```
ALTER EVENT TRIGGER name DISABLE
ALTER EVENT TRIGGER name ENABLE [ REPLICA | ALWAYS ]
ALTER EVENT TRIGGER name OWNER TO { new_owner | CURRENT_ROLE | CURRENT_USER | SESSION_USER }
ALTER EVENT TRIGGER name RENAME TO new_name
```

## DESCRIPTION
ALTER EVENT TRIGGER changes properties of an existing event trigger.

You must be superuser to alter an event trigger.

## PARAMETERS
name

The name of an existing trigger to alter.

new_owner

The user name of the new owner of the event trigger.

new_name

The new name of the event trigger.

DISABLE/ENABLE [ REPLICA | ALWAYS ]

These forms configure the firing of event triggers. A disabled trigger is still known to the system, but is not executed when its triggering event occurs. See also session_replication_role.

## COMPATIBILITY
There is no ALTER EVENT TRIGGER statement in the SQL standard.

## SEE ALSO
CREATE EVENT TRIGGER (CREATE_EVENT_TRIGGER(7)), DROP EVENT TRIGGER (DROP_EVENT_TRIGGER(7))


title: ALTER_EXTENSION(7)

summary: change the definition of an extension

text:
# ALTER_EXTENSION(7)

## NAME
ALTER_EXTENSION - change the definition of an extension

## SYNOPSIS

```
ALTER EXTENSION name UPDATE [ TO new_version ]
ALTER EXTENSION name SET SCHEMA new_schema
ALTER EXTENSION name ADD member_object
ALTER EXTENSION name DROP member_object
where member_object is:

  ACCESS METHOD object_name |

  AGGREGATE aggregate_name ( aggregate_signature ) |

  CAST (source_type AS target_type) |

  COLLATION object_name |

  CONVERSION object_name |

  DOMAIN object_name |

  EVENT TRIGGER object_name |

  FOREIGN DATA WRAPPER object_name |

  FOREIGN TABLE object_name |

  FUNCTION function_name [ ( [ [ argmode ] [ argname ] argtype [, ...] ] ) ] |

  MATERIALIZED VIEW object_name |

  OPERATOR operator_name (left_type, right_type) |

  OPERATOR CLASS object_name USING index_method |

  OPERATOR FAMILY object_name USING index_method |

  [ PROCEDURAL ] LANGUAGE object_name |

  PROCEDURE procedure_name [ ( [ [ argmode ] [ argname ] argtype [, ...] ] ) ] |

  ROUTINE routine_name [ ( [ [ argmode ] [ argname ] argtype [, ...] ] ) ] |

  SCHEMA object_name |

  SEQUENCE object_name |

  SERVER object_name |

  TABLE object_name |

  TEXT SEARCH CONFIGURATION object_name |

  TEXT SEARCH DICTIONARY object_name |

  TEXT SEARCH PARSER object_name |

  TEXT SEARCH TEMPLATE object_name |

  TRANSFORM FOR type_name LANGUAGE lang_name |

  TYPE object_name |

  VIEW object_name
and aggregate_signature is:
* |
[ argmode ] [ argname ] argtype [ , ... ] |
[ [ argmode ] [ argname ] argtype [ , ... ] ] ORDER BY [ argmode ] [ argname ] argtype [ , ... ]
```

## DESCRIPTION
ALTER EXTENSION changes the definition of an installed extension. There are several subforms:

UPDATE

This form updates the extension to a newer version. The extension must supply a suitable update script (or series of scripts) that can modify the currently-installed version into the requested version.

SET SCHEMA

This form moves the extension's objects into another schema. The extension has to be relocatable for this command to succeed.

ADD member_object

This form adds an existing object to the extension. This is mainly useful in extension update scripts. The object will subsequently be treated as a member of the extension; notably, it can only be dropped by dropping the extension.

DROP member_object

This form removes a member object from the extension. This is mainly useful in extension update scripts. The object is not dropped, only disassociated from the extension.

See Section 38.17 for more information about these operations. You must own the extension to use ALTER EXTENSION. The ADD/DROP forms require ownership of the added/dropped object as well.

## PARAMETERS
name

The name of an installed extension.

new_version

The desired new version of the extension. This can be written as either an identifier or a string literal. If not specified, ALTER EXTENSION UPDATE attempts to update to whatever is shown as the default version in the extension's control file.

new_schema

The new schema for the extension.

object_name

aggregate_name

function_name

operator_name

procedure_name

routine_name

The name of an object to be added to or removed from the extension. Names of tables, aggregates, domains, foreign tables, functions, operators, operator classes, operator families, procedures, routines, sequences, text search objects, types, and views can be schema-qualified.

source_type

The name of the source data type of the cast.

target_type

The name of the target data type of the cast.

argmode

The mode of a function, procedure, or aggregate argument: IN, OUT, INOUT, or VARIADIC. If omitted, the default is IN. Note that ALTER EXTENSION does not actually pay any attention to OUT arguments, since only the input arguments are needed to determine the function's identity. So it is sufficient to list the IN, INOUT, and VARIADIC arguments.

argname

The name of a function, procedure, or aggregate argument. Note that ALTER EXTENSION does not actually pay any attention to argument names, since only the argument data types are needed to determine the function's identity.

argtype

The data type of a function, procedure, or aggregate argument.

left_type

right_type

The data type(s) of the operator's arguments (optionally schema-qualified). Write NONE for the missing argument of a prefix operator.

PROCEDURAL

This is a noise word.

type_name

The name of the data type of the transform.

lang_name

The name of the language of the transform.

## EXAMPLES
To update the hstore extension to version 2.0:

```
ALTER EXTENSION hstore UPDATE TO '2.0';
```

To change the schema of the hstore extension to utils:

```
ALTER EXTENSION hstore SET SCHEMA utils;
```

To add an existing function to the hstore extension:

```
ALTER EXTENSION hstore ADD FUNCTION populate_record(anyelement, hstore);
```

## COMPATIBILITY
ALTER EXTENSION is a PostgreSQL extension.

## SEE ALSO
CREATE EXTENSION (CREATE_EXTENSION(7)), DROP EXTENSION (DROP_EXTENSION(7))


title: ALTER_FOREIGN_DATA_WRAPPER(7)

summary: change the definition of a foreign\-data wrapper

text:
# ALTER_FOREIGN_DATA_WRAPPER(7)

## NAME
ALTER_FOREIGN_DATA_WRAPPER - change the definition of a foreign-data wrapper

## SYNOPSIS

```
ALTER FOREIGN DATA WRAPPER name

    [ HANDLER handler_function | NO HANDLER ]

    [ VALIDATOR validator_function | NO VALIDATOR ]

    [ OPTIONS ( [ ADD | SET | DROP ] option ['value'] [, ... ]) ]
ALTER FOREIGN DATA WRAPPER name OWNER TO { new_owner | CURRENT_ROLE | CURRENT_USER | SESSION_USER }
ALTER FOREIGN DATA WRAPPER name RENAME TO new_name
```

## DESCRIPTION
ALTER FOREIGN DATA WRAPPER changes the definition of a foreign-data wrapper. The first form of the command changes the support functions or the generic options of the foreign-data wrapper (at least one clause is required). The second form changes the owner of the foreign-data wrapper.

Only superusers can alter foreign-data wrappers. Additionally, only superusers can own foreign-data wrappers.

## PARAMETERS
name

The name of an existing foreign-data wrapper.

HANDLER handler_function

Specifies a new handler function for the foreign-data wrapper.

NO HANDLER

This is used to specify that the foreign-data wrapper should no longer have a handler function. Note that foreign tables that use a foreign-data wrapper with no handler cannot be accessed.

VALIDATOR validator_function

Specifies a new validator function for the foreign-data wrapper. Note that it is possible that pre-existing options of the foreign-data wrapper, or of dependent servers, user mappings, or foreign tables, are invalid according to the new validator. PostgreSQL does not check for this. It is up to the user to make sure that these options are correct before using the modified foreign-data wrapper. However, any options specified in this ALTER FOREIGN DATA WRAPPER command will be checked using the new validator.

NO VALIDATOR

This is used to specify that the foreign-data wrapper should no longer have a validator function.

OPTIONS ( [ ADD | SET | DROP ] option ['value'] [, ... ] )

Change options for the foreign-data wrapper. ADD, SET, and DROP specify the action to be performed. ADD is assumed if no operation is explicitly specified. Option names must be unique; names and values are also validated using the foreign data wrapper's validator function, if any.

new_owner

The user name of the new owner of the foreign-data wrapper.

new_name

The new name for the foreign-data wrapper.

## EXAMPLES
Change a foreign-data wrapper dbi, add option foo, drop bar:

```
ALTER FOREIGN DATA WRAPPER dbi OPTIONS (ADD foo '1', DROP bar);
```

Change the foreign-data wrapper dbi validator to bob.myvalidator:

```
ALTER FOREIGN DATA WRAPPER dbi VALIDATOR bob.myvalidator;
```

## COMPATIBILITY
ALTER FOREIGN DATA WRAPPER conforms to ISO/IEC 9075-9 (SQL/MED), except that the HANDLER, VALIDATOR, OWNER TO, and RENAME clauses are extensions.

## SEE ALSO
CREATE FOREIGN DATA WRAPPER (CREATE_FOREIGN_DATA_WRAPPER(7)), DROP FOREIGN DATA WRAPPER (DROP_FOREIGN_DATA_WRAPPER(7))