NAME

pve-firewall - PVE Firewall Daemon

SYNOPSIS

pve-firewall <COMMAND> [ARGS] [OPTIONS]

pve-firewall compile

Compile and print firewall rules. This is useful for testing.

pve-firewall help [OPTIONS]

Get help about specified command.

--extra-args <array>

--verbose <boolean>

pve-firewall localnet

Print information about local network.

pve-firewall restart

Restart the Proxmox VE firewall service.

pve-firewall simulate [OPTIONS]

Simulate firewall rules. This does not simulates the kernel routing table, but simply assumes that routing from source zone to destination zone is possible.

--dest <string>

--dport <integer>

--from (host|outside|vm\d+|ct\d+|vmbr\d+/\S+) (default = outside)

--protocol (tcp|udp) (default = tcp)

--source <string>

--sport <integer>

--to (host|outside|vm\d+|ct\d+|vmbr\d+/\S+) (default = host)

--verbose <boolean> (default = 0)

pve-firewall start [OPTIONS]

Start the Proxmox VE firewall service.

--debug <boolean> (default = 0)

pve-firewall status

Get firewall status.

pve-firewall stop

Stop the Proxmox VE firewall service. Note, stopping actively removes all Proxmox VE related iptable rules rendering the host potentially unprotected.

DESCRIPTION

Proxmox VE Firewall provides an easy way to protect your IT infrastructure. You can setup firewall rules for all hosts inside a cluster, or define rules for virtual machines and containers. Features like firewall macros, security groups, IP sets and aliases help to make that task easier.

While all configuration is stored on the cluster file system, the iptables-based firewall service runs on each cluster node, and thus provides full isolation between virtual machines. The distributed nature of this system also provides much higher bandwidth than a central firewall solution.

The firewall has full support for IPv4 and IPv6. IPv6 support is fully transparent, and we filter traffic for both protocols by default. So there is no need to maintain a different set of rules for IPv6.

ZONES

The Proxmox VE firewall groups the network into the following logical zones:

Host

VM

For each zone, you can define firewall rules for incoming and/or outgoing traffic.

CONFIGURATION FILES

All firewall related configuration is stored on the proxmox cluster file system. So those files are automatically distributed to all cluster nodes, and the pve-firewall service updates the underlying iptables rules automatically on changes.

You can configure anything using the GUI (i.e. Datacenter → Firewall, or on a Node → Firewall), or you can edit the configuration files directly using your preferred editor.

Firewall configuration files contain sections of key-value pairs. Lines beginning with a # and blank lines are considered comments. Sections start with a header line containing the section name enclosed in [ and ].

/etc/pve/firewall/cluster.fw

[OPTIONS]
# enable firewall (cluster-wide setting, default is disabled)
enable: 1

/etc/pve/nodes/<nodename>/host.fw

/etc/pve/firewall/<VMID>.fw

FIREWALL RULES

Firewall rules consists of a direction (IN or OUT) and an action (ACCEPT, DENY, REJECT). You can also specify a macro name. Macros contain predefined sets of rules and options. Rules can be disabled by prefixing them with |.

Firewall rules syntax.

[RULES]
DIRECTION ACTION [OPTIONS]
|DIRECTION ACTION [OPTIONS] # disabled rule
DIRECTION MACRO(ACTION) [OPTIONS] # use predefined macro

The following options can be used to refine rule matches.

--dest <string>

--dport <string>

--icmp-type <string>

--iface <string>

--log <alert | crit | debug | emerg | err | info | nolog | notice | warning>

--proto <string>

--source <string>

--sport <string>

Here are some examples:

[RULES]
IN SSH(ACCEPT) -i net0
IN SSH(ACCEPT) -i net0 # a comment
IN SSH(ACCEPT) -i net0 -source 192.168.2.192 # only allow SSH from 192.168.2.192
IN SSH(ACCEPT) -i net0 -source 10.0.0.1-10.0.0.10 # accept SSH for IP range
IN SSH(ACCEPT) -i net0 -source 10.0.0.1,10.0.0.2,10.0.0.3 #accept ssh for IP list
IN SSH(ACCEPT) -i net0 -source +mynetgroup # accept ssh for ipset mynetgroup
IN SSH(ACCEPT) -i net0 -source myserveralias #accept ssh for alias myserveralias
|IN SSH(ACCEPT) -i net0 # disabled rule
IN  DROP # drop all incoming packages
OUT ACCEPT # accept all outgoing packages

SECURITY GROUPS

A security group is a collection of rules, defined at cluster level, which can be used in all VMs' rules. For example you can define a group named “webserver” with rules to open the http and https ports.

# /etc/pve/firewall/cluster.fw
[group webserver]
IN  ACCEPT -p tcp -dport 80
IN  ACCEPT -p tcp -dport 443

Then, you can add this group to a VM’s firewall

# /etc/pve/firewall/<VMID>.fw
[RULES]
GROUP webserver

IP ALIASES

IP Aliases allow you to associate IP addresses of networks with a name. You can then refer to those names:

# pve-firewall localnet
local hostname: example
local IP address: 192.168.2.100
network auto detect: 192.168.0.0/20
using detected local_network: 192.168.0.0/20

#  /etc/pve/firewall/cluster.fw
[ALIASES]
local_network 1.2.3.4 # use the single IP address

IP SETS

IP sets can be used to define groups of networks and hosts. You can refer to them with ‘+name` in the firewall rules’ source and dest properties.

The following example allows HTTP traffic from the management IP set.

IN HTTP(ACCEPT) -source +management

# /etc/pve/firewall/cluster.fw
[IPSET management]
192.168.2.10
192.168.2.10/24

# /etc/pve/firewall/cluster.fw
[IPSET blacklist]
77.240.159.182
213.87.123.0/24

/etc/pve/firewall/<VMID>.fw
[IPSET ipfilter-net0] # only allow specified IPs on net0
192.168.2.10

SERVICES AND COMMANDS

The firewall runs two service daemons on each node:

There is also a CLI command named pve-firewall, which can be used to start and stop the firewall service:

# pve-firewall start
# pve-firewall stop

To get the status use:

# pve-firewall status

The above command reads and compiles all firewall rules, so you will see warnings if your firewall configuration contains any errors.

If you want to see the generated iptables rules you can use:

# iptables-save

DEFAULT FIREWALL RULES

The following traffic is filtered by the default firewall configuration:

 # iptables-save

LOGGING OF FIREWALL RULES

By default, all logging of traffic filtered by the firewall rules is disabled. To enable logging, the loglevel for incoming and/or outgoing traffic has to be set in Firewall → Options. This can be done for the host as well as for the VM/CT firewall individually. By this, logging of Proxmox VE’s standard firewall rules is enabled and the output can be observed in Firewall → Log. Further, only some dropped or rejected packets are logged for the standard rules (see default firewall rules (man pve-firewall(8))).

loglevel does not affect how much of the filtered traffic is logged. It changes a LOGID appended as prefix to the log output for easier filtering and post-processing.

loglevel is one of the following flags:

A typical firewall log output looks like this:

VMID LOGID CHAIN TIMESTAMP POLICY: PACKET_DETAILS

In case of the host firewall, VMID is equal to 0.

IN REJECT -p icmp -log nolog
IN REJECT -p icmp

IN REJECT -p icmp -log debug

TIPS AND TRICKS

modprobe ip_conntrack_ftp

# apt-get install suricata
# modprobe nfnetlink_queue

# /etc/pve/firewall/<VMID>.fw
[OPTIONS]
ips: 1
ips_queues: 0

# /etc/default/suricata
NFQUEUE=0

NOTES ON IPV6

The firewall contains a few IPv6 specific options. One thing to note is that IPv6 does not use the ARP protocol anymore, and instead uses NDP (Neighbor Discovery Protocol) which works on IP level and thus needs IP addresses to succeed. For this purpose link-local addresses derived from the interface’s MAC address are used. By default the NDP option is enabled on both host and VM level to allow neighbor discovery (NDP) packets to be sent and received.

Beside neighbor discovery NDP is also used for a couple of other things, like auto-configuration and advertising routers.

By default VMs are allowed to send out router solicitation messages (to query for a router), and to receive router advertisement packets. This allows them to use stateless auto configuration. On the other hand VMs cannot advertise themselves as routers unless the “Allow Router Advertisement” (radv: 1) option is set.

As for the link local addresses required for NDP, there’s also an “IP Filter” (ipfilter: 1) option which can be enabled which has the same effect as adding an ipfilter-net* ipset for each of the VM’s network interfaces containing the corresponding link local addresses. (See the Standard IP set ipfilter-net* (man pve-firewall(8)) section for details.)

PORTS USED BY PROXMOX VE

MACRO DEFINITIONS

COPYRIGHT AND DISCLAIMER

Copyright © 2007-2022 Proxmox Server Solutions GmbH

This program is free software: you can redistribute it and/or modify it under the terms of the GNU Affero General Public License as published by the Free Software Foundation, either version 3 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more details.

You should have received a copy of the GNU Affero General Public License along with this program. If not, see https://www.gnu.org/licenses/

AUTHOR

Proxmox Server Solutions Gmbh
www.proxmox.com

NOTES

1.

Suricata IPS