innfeed.conf

package: inn

title: innfeed.conf(5)

summary: Configuration file for innfeed

text:
# innfeed.conf(5)

## NAME
innfeed.conf - Configuration file for innfeed

## IN A NUTSHELL
The innfeed.conf file configures to which remote peers innfeed sends NNTP feeds.

A common entry to parameterize "news.server.com" as an outgoing feed is:

```
    peer news.server.com {
        ip-name: "news.server.com"
    }
```

If standard NNTP port 119 is not used, you may specify an alternate port as follows:

```
    peer news.server.com {
        ip-name: "news.server.com"
        port-number: 433
    }
```

After any changes, run "inncheck" to perform basic syntax checks, and reload this configuration file with the following command which makes innd respawn a new instance of innfeed (assuming "innfeed!" is the name of the corresponding channel feed in newsfeeds):

```
    ctlinnd flush innfeed!
```

## DESCRIPTION
The configuration file innfeed.conf in pathetc is used to control the innfeed(8) program. It is a fairly free-format file that consists of three types of entries: key:value, peer and group. Comments are from the hash character "#" to the end of the line.

key:value entries are a keyword and a value separated by a colon (which can itself be surrounded by whitespace). For example:

```
    max-connections: 10
```

A legal key starts with a letter and contains only letters, digits, and the "_" and "-" characters. There are 5 different types of values: integers, floating-point numbers, characters, booleans, and strings.

Integer and floating-point numbers are as to be expected, except that exponents in floating-point numbers are not supported. A boolean value is either "true", "yes", "on", "false", "no" or "off" (case is not significant). A character value is a single-quoted character as defined by the C-language. A string value is any other sequence of characters. If the string needs to contain whitespace, then it must be quoted with double quotes, and uses the same format for embedding non-printing characters as normal C-language string.

Peer entries look like:

```
    peer <name> {
        # body ...
    }
```

The word "peer" is required. The <name> is the same as the site name in INN's newsfeeds configuration file. The body of a peer entry contains some number (possibly zero) of key:value entries.

Group entries look like:

```
    group <name> {
        # body ...
    }
```

The word "group" is required. The <name> is any string valid as a key. The body of a group entry contains any number of the three types of entries. So key:value pairs can be defined inside a group, and peers can be nested inside a group, and other groups can be nested inside a group.

key:value entries that are defined outside of all peer and group entries are said to be at "global scope". There are global key:value entries that apply to the process as a whole (for example the location of the backlog file directory), and there are global key:value entries that act as defaults for peers. When innfeed looks for a specific value in a peer entry (for example, the maximum number of connections to set up), if the value is not defined in the peer entry, then the enclosing groups are examined for the entry (starting at the closest enclosing group). If there are no enclosing groups, or the enclosing groups do not define the key:value, then the value at global scope is used.

A small example could be:

```
    # Global value applied to all peers that have
    # no value of their own.
    max-connections: 5
    # A peer definition.  "uunet" is the name used by innd
    # in the newsfeeds configuration file.
    peer uunet {
        ip-name: usenet1.uu.net
    }
    peer vixie {
        ip-name: gw.home.vix.com
        max-connections: 10       # Override global value.
    }
    # A group of two peers which can handle more connections
    # than normal.
    group fast-sites {
        max-connections: 15
        # Another peer.  The "max-connections" value from the
        # "fast-sites" group scope is used.  The "ip-name" value
        # defaults to the peer's name.
        peer data.ramona.vix.com {
        }
        peer bb.home.vix.com {
            max-connections: 20   # He can really cook.
        }
    }
```

Given the above configuration file, the defined peers would have the following values for the max-connections key:

```
    uunet                  5
    vixie                 10
    data.ramona.vix.com   15
    bb.home.vix.com       20
```

innfeed ignores key:value pairs it is not interested in. Some configuration file values can be set via a command-line option, in which case that setting overrides the settings in the file.

Configuration files can be included in other configuration files via the syntax:

```
    $INCLUDE filename
```

There is a maximum nesting depth of 10.

For a fuller example configuration file, see the supplied innfeed.conf.

## GLOBAL VALUES
The following listing show all the keys that apply to the process as whole. These are not required (compiled-in defaults are used where needed).

news-spool

This key requires a pathname value and defaults to patharticles in inn.conf. It specifies where the top of the article spool is. This corresponds to the -a command-line option.

input-file

This key requires a pathname value. It specifies the pathname (relative to the backlog-directory value) that should be read in funnel-file mode. This corresponds to giving a filename as an argument on the command-line (i.e. its presence also implies that funnel-file mode should be used). The default is unset; innfeed then runs in channel or batch mode.

pid-file

This key requires a pathname value and defaults to innfeed.pid. It specifies the pathname (relative to pathrun in inn.conf) where the pid of the innfeed process should be stored. This corresponds to the -p command-line option.

debug-level

This key defines the debug level for the process. Default is 0. A non-zero number generates a lot of messages to stderr, or to the config-defined log-file. This corresponds to the -d command-line option. If a file named innfeed.debug exists in the pathlog directory (as set in inn.conf), then debug-level is automatically set to 1. This is a cheap way of avoiding continual reloading of the newsfeeds file when debugging. Note that debug messages still go to log-file.

debug-shrinking

This key requires a boolean value and defaults to false (the debug file is allowed to grow without bound). If set to true, this file is truncated when its size reaches a certain limit. See backlog-limit for more details.

initial-sleep

This key requires a positive integer. The default value is 2. It defines the number of seconds to wait when innfeed (or a fork) starts, before beginning to open connections to remote hosts.

fast-exit

This key requires a boolean value and defaults to false. If set to true, when innfeed receives a SIGTERM or SIGQUIT signal, it will close its listeners as soon as it can, even if it means dropping articles.

use-mmap

This key requires a boolean value and defaults to true. When innfeed is given file names to send (a fairly rare use case) instead of storage API tokens, it specifies whether mmaping should be used if innfeed has been built with mmap(2) support. If article data on disk is not in NNTP-ready format (CR/LF at the end of each line), then after mmaping, the article is read into memory and fixed up, so mmaping has no positive effect (and possibly some negative effect depending on your system), and so in such a case this value should be "false", which corresponds to the -M command-line option.

log-file

This key requires a pathname value and defaults to innfeed.log. It specifies where any logging messages that could not be sent via syslog(3) should go (such as those generated when a positive value for debug-level is used). This corresponds to the -l command-line option. This pathname is relative to pathlog in inn.conf.

log-time-format

This key requires a format string suitable for strftime(3). It is used for messages sent via syslog(3) and to the status-file. Default value is "%a %b %d %H:%M:%S %Y".

backlog-directory

This key requires a pathname value and defaults to innfeed. It specifies where the current innfeed process should store backlog files. This corresponds to the -b command-line option. This pathname is relative to pathspool in inn.conf.

backlog-highwater

This key requires a positive integer value and defaults to 5. It specifies how many articles should be kept on the backlog file queue before starting to write new entries to disk.

backlog-ckpt-period

This key requires a positive integer value and defaults to 30. It specifies how many seconds elapse between checkpoints and rewrites of the input backlog file. Too small a number will mean frequent disk accesses; too large a number will mean after a crash, innfeed will re-offer more already-processed articles than necessary.

backlog-newfile-period

This key requires a positive integer value and defaults to 600. It specifies how many seconds elapse before each check for externally generated backlog files that are to be picked up and processed.

backlog-rotate-period

This key requires a positive integer value and defaults to 60. It specifies how many seconds elapse before innfeed moves an externally generated backlog file to the input backlog file (if backlog-newfile-period seconds have elapsed) or in the absence of such a file, moves the output backlog file to the input backlog file. No moves occur if the input backlog file is not empty.

dns-retry

This key requires a positive integer value and defaults to 900. It defines the number of seconds between attempts to re-lookup host information that previously failed to be resolved.

dns-expire

This key requires a positive integer value and defaults to 86400. It defines the number of seconds between refreshes of name to address DNS translation. This is so long-running processes do not get stuck with stale data, should peer IP addresses change.

gen-html

This key requires a boolean value and defaults to false. It specifies whether the status-file should be HTML-ified.

status-file

This key requires a pathname value and defaults to innfeed.status. An absolute pathname can be used. It specifies the pathname (relative to pathhttp when gen-html is true; otherwise, pathlog as set in inn.conf) where the periodic status of the innfeed process should be stored. This corresponds to the -S command-line option.

connection-stats

This key requires a boolean value and defaults to false. If the value is true, then whenever the transmission statistics for a peer are logged, each active connection logs its own statistics. This corresponds to the -z command-line option.

host-queue-highwater

This key requires a positive integer value and defaults to 10. It defines how many articles will be held internally for a peer before new arrivals cause article information to be spooled to the backlog file.

stats-period

This key requires a positive integer value and defaults to 600. It defines how many seconds innfeed waits between generating statistics on transfer rates.

stats-reset

This key requires a positive integer value and defaults to 43200. It defines how many seconds innfeed waits before resetting all internal transfer counters back to zero (after logging one final time). This is so a innfeed process running more than a day will generate "final" stats that will be picked up by logfile processing scripts.

initial-reconnect-time

This key requires a positive integer value and defaults to 30. It defines how many seconds to first wait before retrying to reconnect after a connection failure. If the next attempt fails too, then the reconnect time is approximately doubled until the connection succeeds, or max-reconnect-time is reached.

max-reconnect-time

This key requires an integer value and defaults to 3600. It defines the maximum number of seconds to wait between attempt to reconnect to a peer. The initial value for reconnection attempts is defined by initial-reconnect-time, and it is doubled after each failure, up to this value.

stdio-fdmax

This key requires a non-negative integer value and defaults to 0. If the value is greater than zero, then whenever a network socket file descriptor is created and it has a value less than this, the file descriptor will be dup'ed to bring the value up greater than this. This is to leave lower numbered file descriptors free for stdio. Certain systems, Sun's in particular, require this. SunOS 4.1.x usually requires a value of 128; 32-bit Solaris versions or 32-bit applications running on 64-bit Solaris, as well as 64-bit Solaris versions prior to 11.0 require a value of 256. See the Solaris documentation about file descriptors <https://support.oracle.com/knowledge/Sun%20Microsystems/1005979_1.html> for more details. The default if this is not specified, is 0.

### Special keys for imapfeed
The following keys are used with imapfeed to authenticate to a remote host. Several parameters may be included at global scope:

deliver-authname

The authname is who you want to authenticate as.

deliver-password

This is the appropriate password for authname.

deliver-username

The username is who you want to "act" as, that is, who is actually going to be using the server.

deliver-realm

In this case, the "realm" is the realm in which the specified authname is valid. Currently this is only needed by the DIGEST-MD5 SASL mechanism.

deliver-rcpt-to

A printf(3)-style format string for creating the envelope recipient address. The pattern MUST include a single string specifier which will be replaced with the newgroup (e.g. "bb+%s"). The default is "+%s".

deliver-to-header

An optional printf(3)-style format string for creating a To header field to be prepended to the article. The pattern MUST include a single string specifier which will be replaced with the newgroup (e.g. "post+%s@domain"). If not specified, the To header field will not be prepended.

## GLOBAL PEER DEFAULTS
All the key:value pairs mentioned in this section can be specified at global scope. They may also be specified inside a group or peer definition. Note that when peers are added dynamically (i.e. when innfeed receives an article for an unspecified peer), it will add the peer site using the parameters specified at global scope.

### Required keys
No keys are currently required. They all have a default value, if not present in the configuration file.

### Optional keys
The following keys are optional:

article-timeout

This key requires a non-negative integer value. The default value is 600. If no articles need to be sent to the peer for this many seconds, then the peer is considered idle and all its active connections are torn down.

response-timeout

This key requires a non-negative integer value. The default value is 300. It defines the maximum amount of time to wait for a response from the peer after issuing a command.

initial-connections

This key requires a non-negative integer value. The default value is 1. It defines the number of connections to be opened immediately when setting up a peer binding. A value of 0 means no connections will be created until an article needs to be sent.

max-connections

This key requires a positive integer value. The default value is 2 but may be increased if needed or for large feeds. It defines the maximum number of connections to run in parallel to the peer. A value of 0 specifies an unlimited number of maximum connections. In general, use of an unlimited number of maximum connections is not recommended. Do not ever set max-connections to zero with dynamic-method 0 set, as this will saturate peer hosts with connections.

close-period

This key requires a positive integer value and defaults to 86400. It is the maximum number of seconds a connection should be kept open. Some NNTP servers do not deal well with connections being held open for long periods.

dynamic-method

This key requires an integer value between 0 and 3. The default value is 3. It controls how connections are opened, up to the maximum specified by max-connections. In general (and specifically, with dynamic-method 0), a new connection is opened when the current number of connections is below max-connections, and an article is to be sent while no current connections are idle. Without further restraint (i.e. using dynamic-method 0), in practice this means that max-connections connections are established while articles are being sent. Use of other dynamic-method settings imposes a further limit on the amount of connections opened below that specified by max-connections. This limit is calculated in different ways, depending of the value of dynamic-method. Users should note that adding additional connections is not always productive -- just because opening twice as many connections results in a small percentage increase of articles accepted by the remote peer, this may be at considerable resource cost both locally and at the remote site, whereas the remote site might well have received the extra articles sent from another peer a fraction of a second later. Opening large numbers of connections is considered antisocial.

The meanings of the various settings are:

0 (no method)

Increase of connections up to max-connections is unrestrained.

1 (maximize articles per second)

Connections are increased (up to max-connections) and decreased so as to maximize the number of articles per second sent, while using the fewest connections to do this.

2 (set target queue length)

Connections are increased (up to max-connections) and decreased so as to keep the queue of articles to be sent within the bounds set by dynamic-backlog-low and dynamic-backlog-high, while using the minimum resources possible. As the queue will tend to fill if the site is not keeping up, this method ensures that the maximum number of articles are offered to the peer while using the minimum number of connections to achieve this.

3 (combination)

This method uses a combination of methods 1 and 2 above. For sites accepting a large percentage of articles, method 2 will be used to ensure these sites are offered as complete a feed as possible. For sites accepting a small percentage of articles, method 1 is used, to minimize remote resource usage. For intermediate sites, an appropriate combination is used.

dynamic-backlog-low

This key requires a floating-point value between 0 and 100. It represents (as a percentage) the low water mark for the host queue. If the host queue falls below this level while using dynamic-method 2 or 3, and if 2 or more connections are open, innfeed will attempt to drop connections to the host. An Infinite Impulse Response (IIR) filter is applied to the value to prevent connection flap (see dynamic-filter). The default value is 20.0. This value must be smaller than dynamic-backlog-high.

dynamic-backlog-high

This key requires a floating-point value between 0 and 100. It represents (as a percentage) the high water mark for the host queue. If the host queue rises above this level while using dynamic-method 2 or 3, and if less than max-connections are open to the host, innfeed will attempt to open further connections to the host. An Infinite Impulse Response (IIR) filter is applied to the value to prevent connection flap (see dynamic-filter). The default value is 50.0. This value must be larger than dynamic-backlog-low.

dynamic-backlog-filter

This key requires a floating-point value between 0 and 1. It represents the filter coefficient used by the Infinite Impulse Response (IIR) filter used to implement dynamic-method 2 and 3. The default value of this filter is 0.7, giving a time constant of 1/(1-0.7) articles. Higher values will result in slower response to queue fullness changes; lower values in faster response.

max-queue-size

This key requires a positive integer value. The default value is 20. It defines the maximum number of articles to process at one time when using streaming to transmit to a peer. Larger numbers mean more memory consumed as articles usually get pulled into memory (see the description of use-mmap).

streaming

This key requires a boolean value. Its default value is true. It defines whether streaming commands are used to transmit articles to the peers.

no-check-high

This key requires a floating-point number which must be in the range [0.0, 100.0]. When running transmitting with the streaming commands, innfeed attempts an optimization called "no-CHECK mode". This involves not asking the peer if it wants the article, but just sending it. This optimization occurs when the percentage of the articles the peer has accepted gets larger than this number. If this value is set to 100.0, then this effectively turns off no-CHECK mode, as the percentage can never get above 100.0. If this value is too small, then the number of articles the peer rejects will get bigger (and your bandwidth will be wasted). The default value of 95.0 usually works pretty well.

no-check-low

This key requires a floating-point number which must be in the range [0.0, 100.0], and it must be smaller that the value for no-check-high. When running in no-CHECK mode, as described above, if the percentage of articles the remote server accepts drops below this number, then the no-CHECK optimization is turned off until the percentage gets above the no-check-high value again. If there is small difference between this and the no-check-high value (less than about 5.0), then innfeed may frequently go in and out of no-CHECK mode. If the difference is too big, then it will make it harder to get out of no-CHECK mode when necessary (wasting bandwidth). Keeping this to between 5.0 and 10.0 less than no-check-high usually works pretty well. The default value is 90.0.

no-check-filter

This is a floating-point value representing the time constant, in articles, over which the CHECK/no-CHECK calculations are done. The default value is 50.0, which will implement an Infinite Impulse Response (IIR) filter of time constant 50. This roughly equates to making a decision about the mode over the previous 50 articles. A higher number will result in a slower response to changing percentages of articles accepted; a lower number will result in a faster response.

port-number

This key requires a positive integer value. It defines the TCP/IP port number to use when connecting to the remote. Usually, port number 119 is used, which is the default value.

force-ipv4

This key requires a boolean value. By default, it is set to false. Setting it to true is the same as setting bindaddress6 to "none" and removing bindaddress from "none" if it was set.

drop-deferred

This key requires a boolean value. By default, it is set to false. When set to true, and a peer replies with code 431 or 436 (try again later), innfeed just drops the article and does not try to re-send it. This is useful for some peers that keep on deferring articles for a long time to prevent innfeed from trying to offer the same article over and over again.

min-queue-connection

This key requires a boolean value. By default, it is set to false. When set to true, innfeed will attempt to use a connection with the least queue size (or the first empty connection). If this key is set to true, it is recommended that dynamic-method be set to 0. This allows for article propagation with the least delay.

no-backlog

This key requires a boolean value. It specifies whether spooling should be enabled (false, the default) or disabled (true). Note that when no-backlog is set, articles reported as spooled are actually silently discarded.

backlog-limit

This key requires a non-negative integer value. If the number is 0 (the default), then backlog files are allowed to grow without bound when the peer is unable to keep up with the article flow. If this number is greater than 0, then it specifies the size (in bytes) the backlog file should get truncated to when the backlog file reaches a certain limit. The limit depends on whether backlog-factor or backlog-limit-highwater is used. This parameter also applies to the debug file when debug-shrinking is set to true, and has the same effect on this file as the one has on backlog files.

backlog-factor

This key requires a floating-point value, which must be larger than 1.0. It is used in conjunction with the peer key backlog-limit. If backlog-limit has a value greater than zero, then when the backlog file gets larger than the value backlog-limit * backlog-factor, then the backlog file will be truncated to the size backlog-limit. For example, if backlog-limit has a value of 1000000, and backlog-factor has a value of 2.0, then when the backlog file gets to be larger than 2000000 bytes in size, it will be truncated to 1000000 bytes. The front portion of the file is removed, and the trimming happens on line boundaries, so the final size may be a bit less than this number. If backlog-limit-highwater is defined too, then backlog-factor takes precedence. The default value of backlog-factor is 1.1.

This parameter also applies to the debug file when debug-shrinking is set to true, and has the same effect on this file as the one has on backlog files.

backlog-limit-highwater

This key requires a positive integer value that must be larger than the value for backlog-limit. The default value is 0. If the size of the backlog file gets larger than this value (in bytes), then the backlog file will be shrunk down to the size of backlog-limit. If both backlog-factor and backlog-limit-highwater are defined, then the value of backlog-factor is used.

This parameter also applies to the debug file when debug-shrinking is set to true, and has the same effect on this file as the one has on backlog files.

backlog-feed-first

This key requires a boolean value. By default it is set to false. When set to true, the backlog is fed before new files. This is intended to enforce in-order delivery, so setting this to true when initial-connections or max-connections is more than 1 is inconsistent.

bindaddress

This key requires a string value. It specifies which outgoing IPv4 address innfeed should bind the local end of its connection to. It must be an IPv4 address in dotted-quad format (nnn.nnn.nnn.nnn), "any", or "none". If not set or set to "any", innfeed defaults to letting the kernel choose this address. If set to "none", innfeed will not use IPv4 for outgoing connections to peers in this scope (i.e. it forces IPv6). If not set in innfeed.conf, innfeed defaults to the value of sourceaddress from inn.conf (which by default is unset).

bindaddress6

This key requires a string value. It behaves like bindaddress except for outgoing IPv6 connections. It must be in numeric IPv6 format (note that a value containing colons must be enclosed in double quotes), "any", or "none". If set to "none", innfeed will not use IPv6 for outgoing connections to peers in this scope. If not set in innfeed.conf, innfeed defaults to the value of sourceaddress6 from inn.conf (which by default is unset).

username

This key requires a string value. If the value is defined, then innfeed tries to authenticate by AUTHINFO USER and this value used for user name. password must also be defined, if this key is defined.

password

This key requires a string value. The value is the password used for AUTHINFO PASS. username must also be defined, if this key is defined.

## PEER VALUES
As previously explained, the peer definitions can contain redefinitions of any of the key:value pairs described in the section about global peer defaults above. There is one key:value pair that is specific to a peer definition.

ip-name

This key requires a word value. The word is either one of the host's FQDNs, or the dotted-quad IP address of the peer for IPv4, or the colon-separated IP address of the peer for IPv6. If this value is not specified, then the name of the peer in the enclosing peer block is taken to also be its ip-name.

## RELOADING
If innfeed gets a SIGHUP signal, then it will reread the configuration file. All values at global scope except for backlog-directory can be changed (although note that bindaddress and bindaddress6 changes will only affect new connections).

Any new peers are added and any missing peers have their connections closed.

The log file is also reopened.

## EXAMPLE
For a comprehensive example, see the sample innfeed.conf distributed with INN and installed as a starting point.

Here are examples of how to format values:

```
    eg-string:         "New\tconfig\tfile\n"
    eg-long-string:    "A long string that goes
                       over multiple lines.  The
                       newline is kept in the
                       string except when quoted
                       with a backslash \
                       as here."
    eg-simple-string:  A-no-quote-string
    eg-integer:        10
    eg-boolean:        true
    eg-char:           'a'
    eg-ctrl-g:         '\007'
```

## HISTORY
Written by James Brister <brister@vix.com> for InterNetNews. Converted to POD by Julien Elie.

Earlier versions of innfeed (up to 0.10.1) were shipped separately; innfeed is now part of INN and shares the same version number. Please note that the innfeed.conf format has changed dramatically since version 0.9.3.

## SEE ALSO
inn.conf(5), innfeed(8), newsfeeds(5).

branch: sisyphus


innreport.conf

title: innreport.conf(5)

summary: Configuration file for innreport

text:
# innreport.conf(5)

## NAME
innreport.conf - Configuration file for innreport

## DESCRIPTION
The file pathetc/innreport.conf permits configuring the behaviour of innreport. It consists of a series of lines; blank lines and what follows a number sign ("#") in a line are ignored. The structure of this file is:

```
    section default {
        libpath             "/path/to/libraries";
        logpath             "/path/to/logs";
        module              "innreport_inn";
        display_conf_file   "innreport-display.conf";
        unwanted_log        "unwanted.log";
        text                true;
        html                true;  # Enable HTML reports.
        # Other options to set.
    };
```

Only this default section needs being configured. It begins with "section default {" and ends with "};". Each line in the section consists of an option name followed with one or more spaces or tabulations, its value and a semi-colon ";".

The display_conf_file option defines the name of the display configuration file to use. This file, located in libpath, has the same syntax as innreport.conf. If you want to use a custom display configuration, just rename the default innreport-display.conf file to another name, and set it in display_conf_file (otherwise, innreport-display.conf will be overwritten each time you update INN).

Sections defined in the display configuration file are used to parameterize the display, how and what to report, but should not normally be changed (unless you precisely know what you are doing). These sections are written in a domain-specific language not intended to be modified (you may break the reports, especially if changing data, double, format, primary, sort, total or value parameters). The easiest parameters to change, if needed, would be:

- title (the titles of sections in innreport),

- name (the names of the columns),

- color (the color of curves in the graphs),

- numbering (a boolean value to configure whether each row of HTML reports is prefixed with a rank number),

- text, html, and skip (boolean values to configure whether a given section should be present in respectively text, HTML or both reports),

- top (the maximum number of values to display); you can specify different values for text and HTML reports with top_text and top_html.

## OPTIONS
The following options can be set in the default section of the general configuration file innreport.conf:

archive

Sets whether HTML pages and graphs should be archived if HTML reports are enabled. When this option is set to "true", which is the default, a date will be added in the file name of each report and each graph. Otherwise, only the latest HTML report is kept. Use of the -[no]archive flag with innreport(8) takes precedence over the configuration file.

casesensitive

When this option is set to "true", which is the default, case sensitive searches are done in news log files for predetermined patterns. Use of the -[no]casesensitive flag with innreport(8) takes precedence over the configuration file.

cycle

Sets how many report files should be kept if HTML reports are enabled and archived. The default is 0, meaning all archives are kept. The value "none" also means 0. Use of the -cycle flag with innreport(8) takes precedence over the configuration file.

display_conf_file

Defines the display configuration file to use, located in the directory set with the libpath option. The default is "innreport-display.conf".

encoding

Specifies the character set to use in XML and Content-Type declarations of HTML reports. The default is "ISO-8859-1".

footer

Specifies a text to append to the end of the latest "<div>" tag of each HTML report, where the version of innreport is displayed. This option is unset by default.

graph

If the Perl graphic library GD.pm is installed and HTML reports are enabled, graphs are also generated in PNG if supported by "GD", which is the case in modern versions (GIF will otherwise be generated). This option is set to "true" by default, meaning to enable graph generation. Use of the -[no]graph flag with innreport(8) takes precedence over the configuration file.

graph_bg

Specifies the background colour of graphs in HTML reports as an HTML hexadecimal colour value. The default is "#FFFFFF" (white).

graph_fg

Specifies the foreground colour of graphs in HTML reports as an HTML hexadecimal colour value. The default is "#000000" (black).

graph_width

Sets the width in pixels of graphs in HTML reports. The default is 550.

html

This is a mandatory option with no default value. It serves two purposes: generating HTML reports besides the plain text report printed to standard output, and configuring in the sections other than default (in the display configuration file) whether a section of the report should be present in the HTML output. If set to "true", and the -nohtml flag not given when calling innreport, HTML reports will be generated. These reports will not contain sections individually having html set to "false" in the display configuration file (by default, everything is present in the HTML reports). Otherwise, if the -nohtml flag is given, no HTML reports will be generated.

If set to "false", and the -html flag not given when calling innreport, no HTML reports will be generated. Otherwise, if the -html flag is given, HTML reports will be generated but only the sections individually having html set to "true" in the display configuration file will be present in the reports.

html_body

Specifies additional elements to append to the "<body>" tag of HTML reports. This option is unset by default. Note that using this option leads to invalid XHTML. You should provide a custom CSS file with the html_css_url option instead.

html_css_url

Specifies the name of a CSS file to use to apply a custom style to HTML reports. The file is located in the directory set with the html_dir option. It may also be a URL, in which case the file may be in another location. If unset, which is the default, innreport creates a default innreport.css custom style sheet in html_dir and uses it for HTML reports.

In case you wish to supply your own style, take a look at what is defined in the default CSS and the HTML reports, and adapt it for your needs. Across several innreport releases that may change classes, make sure to keep old class definitions for backward compatibility with previously generated HTML reports still referring to that custom file.

html_dir

Sets the directory for web pages if HTML reports are enabled. The default is the value of pathhttp as set in inn.conf. Use of the -dir flag with innreport(8) takes precedence over the configuration file.

html_file_extension

Defines the file extension of generated HTML reports, if enabled. A dot (".") should begin the extension. The default is ".html". You may want to use ".xhtml" instead to let the web server serve correct MIME type, as innreport generates XHTML 1.1 pages.

html_footer_file

Specifies the name of a file in html_dir whose contents will be appended just before the "</body>" tag of each HTML report. This option is unset by default.

html_header_file

Specifies the name of a file in html_dir whose contents will be appended just after the "<body>" tag of each HTML report. This option is unset by default.

html-unknown

Whether to report unknown entries from news log files in HTML reports, when set to "true". This option takes by default the same value as the one for text plain reports (as set with the unknown option). Use of the -[no]html-unknown flag with innreport(8) takes precedence over the configuration file.

html_xsl_url

Specifies the name of an XSL file to use to apply styling operations to HTML reports. The file is located in the directory set with the html_dir option. It may also be a URL, in which case the file may be in another location. This option is unset by default. When set, a corresponding XML style sheet declaration is added to HTML reports. It may be useful for XHTML compliance with older browsers (see <https://www.w3.org/MarkUp/2004/xhtml-faq#ie>).

More generally, if you are interested in XSL transformation, you may want to have a look at innreport-filter.xslt provided in the contrib directory.

img_dir

Sets the web path for pictures in HTML pages if HTML reports are enabled, and graphs are wanted. The path is relative to the directory set with the html_dir option. The default is "pics". Use of the -webpath flag with innreport(8) takes precedence over the configuration file.

index

Sets the name of the index web page if HTML reports are enabled. The default is "index.html". You may want to use "index.xhtml" instead to let the web server serve correct MIME type, as innreport generates XHTML 1.1 pages.

Use of the -index flag with innreport(8) takes precedence over the configuration file.

libpath

Defines the directory in which the Perl module set with the module option is located. This option is mandatory, and usually set to the same value as pathlib in inn.conf.

logpath

Defines the directory in which the log file for newsgroups not locally carried by the news server, as set with the unwanted_log option, is located. This option is mandatory, and usually set to the same value as pathlog in inn.conf.

max_unknown

How many unrecognized entries from news log files to report, when the unknown or html-unknown options are set. The default is 50. Using "-1" means there is no limit. Use of the -maxunrec flag with innreport(8) takes precedence over the configuration file.

module

Defines the Perl module used by innreport to parse and summarize log files. This option is mandatory, and usually set to "innreport_inn". This value should not normally be changed unless you know exactly what you are doing. The Perl module that will be loaded is module.pm located in the directory set with the libpath option.

separator

Sets the separator between hours, minutes and seconds in file names if HTML reports are enabled and archived. The default is "." as ":" may not be properly read by all web browsers. Be careful not to use "/" as a separator, and to use only a valid file name character. Use of the -separator flag with innreport(8) takes precedence over the configuration file.

text

This is a mandatory option with no default value. If set to "true", innreport will print its text report to standard output. This report will not contain sections individually having text set to "false" in the display configuration file (by default, everything is present in the text report).

If set to "false", innreport will only print to standard output the sections individually having text set to "true" in the display configuration file (by default, the text report will be empty).

title

Specifies the text to use as first words of text and HTML reports as well as in the title of HTML reports. By default, "Daily Usenet report" is used.

transparent

If set to "true", the background of generated graphs in HTML reports will be transparent. The default is "false".

unknown

Whether to report unknown entries from news log files in plain text reports, when set to "true". This option is enabled by default. Use of the -[no]unknown flag with innreport(8) takes precedence over the configuration file.

unwanted_log

Defines the name of the file used by innreport to maintain a count of the number of articles that were rejected because they were posted to newsgroups that do not exist in the active file of the news server. This file is updated by innreport while processing the news.notice log file and it is maintained in reverse numeric order (the most popular rejected group first). This option is mandatory, and usually set to "unwanted.log". This value should not normally be changed unless you know exactly what you are doing. The file is located in the directory set with the logpath option.

Note that logtrash has to be set to true in inn.conf for this file to be generated.

## HISTORY
Written by Julien Elie for InterNetNews.

## SEE ALSO
innreport(8).


innwatch.ctl

title: innwatch.ctl(5)

text:
# innwatch.ctl(5)

## NAME
innwatch.ctl - List of supervisory actions taken by innwatch

## DESCRIPTION
The file pathetc/innwatch.ctl is used to determine what actions are taken during the periodic supervisions by innwatch.

The file consists of a series of lines; blank lines and lines beginning with a number sign ("#") are ignored. All other lines consist of seven fields, each preceded by a delimiting character, for example:

```
    !state!when!condition!test!limit!command!reason
```

or:

```
    @state@when@condition@test@limit@command@reason
```

The delimiter can be any one of several non-alphanumeric characters that does not appear elsewhere in the line; there is no way to quote it to include it in any of the fields. Any of "!", ",", ":", "@", ";", or "?" is a good choice. Each line can have a different delimiter; the first character on each line is the delimiter for that line. White space surrounding delimiters, except before the first, is ignored, and does not form part of the fields; white space within fields is permitted. All delimiters must be present.

The first field is the state to enter if the condition for this control line is true. It is used as an internal state indicator and in ctlinnd messages to control the server. If this field is empty, the line number is used.

The second field specifies when this control line should be used. It consists of a list of states and special indicators, separated by whitespace. If the current state matches against any of the states in this field, this line will be used as described below. The values that may be used are:

"-"

This line matches if the current state is the same as the label on this line, or if the current state is "run", the initial state. This is also the default state if this field is empty.

"+"

This line matches if the current state is "run".

"*"

This line always matches.

"label"

This line matches if the current state is the specified "label".

"-label"

This line matches if the current state is not the specified "label".

The third field specifies a shell command that is invoked if this line matches. Do not use any shell filename expansion characters such as "*", "?", or "[" (even quoted, they're not likely to work as intended). The command is executed with its current directory set to the news spool articles directory (patharticles).

If the command succeeds, as indicated by its exit status, it is expected to have printed a single integer to standard output. This gives the value of this control line, to be used below. If the command fails, the line is ignored.

The fourth field specifies the operator to use to test the value returned above. It should be one of the two letter numeric test operators defined in test(1) such as "eq", "lt" and the like. The leading dash ("-") should not be included.

The fifth field specifies a constant with which to compare the value using the operator just defined. This is done by invoking the command:

```
    test value -operator constant
```

The line is said to succeed if it returns true.

The sixth field specifies what should be done if the line succeeds, and in some cases if it fails. Any of the following words may be used:

"throttle"

Causes innwatch to throttle the server if this line succeeds. It also sets the state to the value of the line's label. If the line fails, and the state was previously equal to the label on this line (that is, this line had previously succeeded), then a "ctlinnd go" command will be sent to the server, and innwatch will return to the "run" state. The "ctlinnd throttle" command is only performed if the current state is "run" or a state other than the label of this line, regardless of whether the command succeeds.

"pause"

Is identical to "throttle" except that the server is paused via the "ctlinnd pause" command.

"shutdown"

Sends a "ctlinnd shutdown" command to the server. It is for emergency use only.

"flush"

Sends a "ctlinnd flush" command to the server.

"go"

Causes innwatch to send a "ctlinnd go" command to the server and to set the state to "run".

"exit"

Causes innwatch to exit.

"skip"

The remainder of the control file is skipped for the current pass.

The last field specifies the reason that is used in those "ctlinnd" commands that require one. More strictly, it is part of the reason, as innwatch appends some information to it.

In order to enable other sites to recognize the state of the local innd server, this field should usually be set to one of several standard values. Use for instance "No space" if the server is rejecting articles because of a lack of filesystem resources, or "loadav" if the server is rejecting articles because of a lack of CPU resources.

Once innwatch has taken some action as a consequence of its control line, it skips the rest of the control file for this pass. If the action was to restart the server (that is, issue a "ctlinnd go" command), then the next pass will commence almost immediately, so that innwatch can discover any other condition that may mean that the server should be suspended again.

## EXAMPLES

```
    !!! inndf . ! lt ! 10000 ! throttle ! No space (spool)
    !!! inndf -i . ! lt ! 1000 ! throttle ! No space (inodes)
```

The first line causes the server to be throttled if the free space drops below 10000 units (using whatever units inndf uses), and restarted again when free space increases above the threshold.

The second line does the same for inodes.

The next three lines act as a group and should appear in the following order. It is easier to explain them, however, if they are described from the last up.

```
    ! load ! load hiload ! loadavg ! lt ! 5 ! go ! loadav
    : hiload : + load : loadavg : gt : 8 : throttle : loadav
    / load / + / loadavg / gt / 6 / pause / loadav
```

The final line causes the server to be paused if innwatch is in the "run" state and the load average rises to, or above, six. The state is set to "load" when this happens. The previous line causes the server to be throttled when innwatch is in the "run" or "load" state, and the load average rises above eight. The state is set to "hiload" when this happens. Note that innwatch can switch the server from "pause" to "throttle" if the load average rises from below six to between six and seven, and then to above eight. The first line causes the server to be sent a "ctlinnd go" command if innwatch is in the "load" or "hiload" state, and the load average drops below five.

Note that all three lines assume a mythical command "loadavg" that is assumed to print the current load average as an integer. In more practical circumstances, a pipe of uptime into AWK is more likely to be useful.

## BUGS
The "run" state is not actually identified by the label with that three letter name, and using it will not work as expected ("go" is the wanted state in that case).

Using an unusual character for the delimiter such as "(", "*", "&", "`", "'", and the like, is likely to lead to obscure and hard to locate bugs.

## HISTORY
Written by <kre@munnari.oz.au> for InterNetNews. Rewritten into POD by Julien Elie.

## SEE ALSO
ctlinnd(8), inndf(8), news.daily(8), rc.news(8).


moderators

title: moderators(5)

text:
# moderators(5)

## NAME
moderators - Submission addresses for moderated groups

## DESCRIPTION
When an unapproved article is posted locally to a moderated newsgroup, it is not passed off to innd for normal handling; instead, it is sent via e-mail to the submission address for that newsgroup. The submission address is determined using this configuration file.

The file pathetc/moderators is a list of associations between uwildmat patterns matching newsgroups and the submission address for those newsgroups. Blank lines and lines starting with a number sign ("#") are ignored. All other lines should consist of two fields separated by a colon:

```
    <pattern>:<address>
```

The first field is a uwildmat pattern matching the group or groups to which this line applies. The first matching line is used, so more specific patterns should be listed before general patterns.

The second field, the submission address, should be a simple e-mail address with one exception: at most one %s may occur anywhere in the address. If present, it will be replaced by the name of the newsgroup, with all periods in the name changed to dashes ("-"). If there is a literal "%" in the submission address, it must be written as "%%", even if not followed by an "s".

With the %s syntax, periods are converted to dashes for historical reasons, from back in the days when periods in the local part of addresses were not always handled correctly. It's probably no longer necessary, but so much now depends on it that it can't be easily changed.

It's intended that the sample moderators file included in the INN distribution always be sufficient for all world-wide newsgroups. The hosts behind moderators.isc.org have graciously volunteered to handle forwarding tasks for all world-wide newsgroups so that individual sites don't have to keep track of the submission addresses for moderated groups. The forwarding database used by moderators.isc.org is coordinated by <moderators-request@isc.org>; if you know of a world-wide newsgroup hierarchy that is not correctly handled by moderators.isc.org, please send the details to that address.

Given that, the only thing you should have to add to the sample file under normal circumstances are the forwarding addresses for local or limited-distribution moderated groups.

If this file doesn't exist, or if a post is made to a moderated group that has no matching entry in this file, nnrpd falls back on the value of moderatormailer set in inn.conf and, failing that, rejects the post.

## EXAMPLES
Here is a sample file:

```
    example.important:announce@example.com
    example.*:%s@smtp.example.com
    *:%s@moderators.isc.org
```

Using the above file, postings to the moderated newsgroup in the left column below will be sent to the address shown in the right column below:

```
    example.important   announce@example.com
    example.x-announce  example-x-announce@smtp.example.com
    alt.dev.null        alt-dev-null@moderators.isc.org
```

Note that periods are changed to dashes and dashes are left alone with the %s syntax, so two moderated newsgroups whose names differ only by changing a period to a dash would go to the same address. Such newsgroup pairs should therefore not be created.

## HISTORY
Written by Rich $alz <rsalz@uunet.uu.net> for InterNetNews. Rewritten in POD by Russ Allbery <eagle@eyrie.org>.

## SEE ALSO
inn.conf(5), libinn_uwildmat(3), nnrpd(8).


motd.news

title: motd.news(5)

text:
# motd.news(5)

## NAME
motd.news - Message of the day information for feeders or readers

## DESCRIPTION
Two files, found in pathetc, contain local information for news feeders or news readers in a free-form format. The entire files are returned verbatim to any client that issues the LIST MOTD command. This might be used for new information, notification of upcoming downtime, or similar purposes.

These files are named motd.innd (for news feeders) and motd.nnrpd (for news readers); they are used by innd and nnrpd, respectively.

Make sure that these files are encoded in UTF-8. They should also be "dot-stuffed", that is to say that a line beginning with a dot should have that dot doubled.

Be aware that use of the LIST MOTD command is not widespread (though documented in RFC 6048) and most news clients will never ask for this file.

If one or both of these files are missing, it is not an error. The server will just send the client the appropriate response for an unmaintained motd.innd or motd.nnrpd file.

On a fresh INN install, samples for these files are installed in pathetc; you will therefore find in motd.innd.sample and motd.nnrpd.sample a few examples of messages that can be communicated to news clients.

## HISTORY
Rewritten in POD by Russ Allbery <eagle@eyrie.org> for InterNetNews.

## SEE ALSO
inn.conf(5).


newsfeeds

title: newsfeeds(5)

text:
# newsfeeds(5)

## NAME
newsfeeds - Determine where Usenet articles are sent

## IN A NUTSHELL
The newsfeeds file configures what innd does with incoming accepted articles.

A minimalist newsfeeds file which parameters an outgoing NNTP feed to "news.server.com", using path identity "server" (as it appears in Path header fields), of all the articles you receive is:

```
    # Mandatory line, with no exclusion patterns defined.
    ME:::
    # A real-time feed through innfeed of all your articles.
    news.server.com/server\
        :*\
        :Tm:innfeed!
    # The channel feed entry for innfeed.
    # Change "/usr/lib/news/bin" to the directory path
    # of the innfeed program.
    innfeed!:!*:Tc,Wnm*:/usr/lib/news/bin/innfeed
```

In order to find an external feed, you can ask in the news.admin.peering newsgroup. Several news administrators will certainly respond and gracefully provide you with a news feed.

Other entries may be added in newsfeeds depending on your needs. See the examples below and also in the archive(8), controlchan(8), news2mail(8), ninpaths(8) and perl-nocem(8) man pages for the most widely used.

If you exclude local hierarchies or newsgroups from the feed, make sure to add "!control,!control.*" so that cancel articles or any kind of control articles related to excluded hierarchies or newsgroups are not propagated to the remote peer. Its newsfeeds entry then looks like:

```
    news.server.com/server\
        :*,!local.*,@*bina*,!control,!control.*\
        :Tm:innfeed!
```

After any changes, run "inncheck" to perform basic syntax checks, and reload this configuration file with the following command:

```
    ctlinnd reload newsfeeds 'new peer'
```

## DESCRIPTION
The file pathetc/newsfeeds specifies how incoming articles should be distributed to other programs and files on the server. It is parsed by the InterNetNews server innd(8) when it starts up, or when directed to by ctlinnd(8). innd doesn't send articles to remote sites itself, so newsfeeds doesn't directly determine which remote news servers articles are sent to. Instead, it specifies what batch files should be created or which programs should be run (and what information should be sent to them), and then this information is used by programs like innxmit(8) and innfeed(8) to feed articles to remote sites.

Used in conjunction with delayer(1) in the configuration of the related funnel feed, innfeed can even send a delayed feed to remote peers or to another local instance of innd, notably to permit the rejection of spam before it actually arrives.

The newsfeeds file isn't used solely to set up feeding accepted articles to remote sites but also to pass them (or bits of information about them) to any local programs or files that want that data. For example, controlchan(8), a daemon that processes incoming control messages, runs out of newsfeeds, as could a news to mail gateway.

The file is interpreted as a set of lines, parsed according to the following rules: if a line ends with a backslash, the backslash, the newline, and any whitespace at the start of the next line is deleted. This is repeated until the entire "logical" line is collected. If the logical line is blank or starts with a number sign ("#"), it is ignored.

All other lines are interpreted as feed entries. An entry should consist of four colon-separated fields; two of the fields may have optional sub-fields, marked off by a slash. Fields or sub-fields that take multiple parameters should be separated by a comma. Extra whitespace can cause problems and should be avoided. Except for the site names, case is significant. The format of an entry is:

```
    sitename[/exclude,exclude,...]\
        :pattern,pattern...[/distribution,distribution...]\
        :flag,flag...\
        :parameter
```

Each field is described below.

The sitename is the name of the site to which a news article can be sent. It is used for writing log entries and for determining if an article should be forwarded to a site. (A "site" is the generic term for some destination of newsfeed data; it often corresponds to a remote news peer, but doesn't have to. For example, a local archiving program run from newsfeeds is also a "site".) If sitename already appears in the article's Path header field body, then the article will not be sent to the site. The name is usually whatever the remote site uses to identify itself in the Path header field body, but can be almost any word.

Be careful, though, to avoid having the sitename accidentally match a path identity in the Path header field unintentionally. For this reason, special local entries (such as archivers or gateways) should probably end with an exclamation point to make sure that they do not have the same name as any real site. For example, "gateway" is an obvious name for the local entry that forwards articles out to a mailing list. If a site with the name "gateway" posts an article, when the local site receives the article it will see the name in the Path header field and not send the article to its own "gateway" entry. Since "gateway!" can't appear as an individual path identity in the Path header field (as "!" is the delimiter), that would be a better thing to use for sitename.

(Another way to avoid this problem is with the "Ap" flag; see the description below.)

If an entry has an exclusion sub-field, the article will not be sent to that site if any of exclude appear in the Path header field body. (It's sometimes convenient to have the sitename be an abbreviated form of the name of the remote site, since all the sitenames to which an article is sent are written to the log and using shorter sitenames can therefore improve performance for large servers. In this case, the path identities of those sites should be given as exclude entries and the "Ap" flag used so that the abbreviated sitename doesn't accidentally match some other path identities in Path header fields.)

The same sitename can be used more than once and the appropriate action will be taken for each entry that should receive the article, but this is recommended only for program feeds to avoid confusion. Case is not significant in site names.

The comma-separated pattern specifies which groups to send to the site; it is interpreted to build a "subscription list" for the site. The default subscription is to get all groups carried by the server. It is a uwildmat pattern supporting poison ("@") wildcards; see the libinn_uwildmat man page for full details on the pattern matching language. pattern will be matched against every newsgroup carried by the server and all newsgroups that match will be added to the subscription list for the site.

Normally, a given article (or information about it) is sent to a site if any of the newsgroups to which the article was posted are in that site's subscription list. If a newsgroup matches an "@" pattern in pattern, then not only is it not added to the subscription list, but any articles crossposted to that newsgroup also will not be sent to that site even if other newsgroups to which it was crossposted are in that site's subscription list. This is called a poison pattern (because matching groups are "poisoned").

For example, to receive all comp.* groups, but only comp.sources.unix within the sources newsgroups, the following pattern can be used:

```
    comp.*,!comp.sources.*,comp.sources.unix
```

Note that the trailing ".*" is required; the pattern has to match the whole newsgroup name. "comp.sources.*" could be written "comp.sources*" and would exclude the newsgroup comp.sources (if it exists) as well as the groups in the comp.sources.* hierarchy, but note that this would also exclude a newsgroup named comp.sources-only (whereas the above pattern would add that group to the site subscription list since it matches "comp.*" and none of the other patterns).

For another example, to feed alt.* and misc.* to a given site but not any articles posted to alt.binaries.warez (even if they're also crossposted to other alt.* or misc.* groups), the following pattern can be used:

```
    alt.*,@alt.binaries.warez,misc.*
```

Note, however, that if you reversed the "alt.*" and "@alt.binaries.warez" entries, this pattern would be equivalent to "alt.*,misc.*", since the last matching pattern determines whether a given newsgroup matches and the poison logic only applies if the poison entry is the last matching entry.

Control messages follow slightly different propagation rules than normal articles; see innd(8) for the details. Note that most subscriptions should have "!junk,!control,!control.*" in their pattern list due to those propagation rules (and since "junk" is a special internal newsgroup; see wanttrash in inn.conf(5) for more details on what it's used for) and that the best way to keep control messages local to a site is with a distribution.

A subscription can be further modified by specifying distributions that the site should or should not receive. The default is to send all articles to all sites that subscribe to any of the groups where it has been posted, but if an article has a Distribution header field and any distributions are specified, then they are checked according to the following rules:

1.

If the Distribution header field matches any of the values in the sub-field, the article is sent.

2.

If a distribution starts with an exclamation point, and it matches the Distribution header field, the article is not sent.

3.

If the Distribution header field does not match any distribution in the site's entry and no negations were used, the article is not sent.

4.

If the Distribution header field does not match any distribution in the site's entry and any distribution started with an exclamation point, the article is sent.

If an article has more than one distribution specified, then each one is handled according according to the above rules. If any of the specified distributions indicate that the article should be sent, it is; if none do, it is not sent. In other words, the rules are used as a logical or.

It is almost definitely a mistake to have a single feed that specifies distributions that start with an exclamation point along with some that don't.

Distributions are text words, not patterns; entries like "*" or "all" have no special meaning.

The flag field is described in "FLAG VALUES". The interpretation of the parameter field depends on the type of feed and is explained in more detail in "FEED TYPES". It can be omitted for some types of feeds.

The site named "ME" is special. There must be exactly one such entry, and it should be the first entry in the file. If the "ME" entry has an exclusion sub-field, incoming articles are rejected completely if any of the names specified in that exclusion sub-field appear in their Path header field bodies. If the "ME" entry has a subscription list, that list is prepended to the subscription list of all other entries. For example, "*,!control,!control.*,!junk,!foo.*" could be used to set the default subscription list for all other feeds so that local postings are not propagated unless "foo.*" explicitly appears in the site's subscription list. This feature tends to be somewhat confusing since the default subscription is prepended and can be overridden by other patterns.

If the "ME" entry has a distribution sub-field, only articles that match that distribution list are accepted and all other articles with a distribution are rejected. A common use for this is to put something like "/!local" in the "ME" entry to reject local postings from other misconfigured sites. The distribution sub-field of "ME" has no effect on the acceptance or rejection of articles that do not have a Distribution header field.

An empty "ME" entry is possible, in which case no exclusion patterns will be defined.

Finally, it is also possible to set variables in newsfeeds and use them later in the file. A line starting with "$" sets a variable. For example:

```
    $LOCALGROUPS=local.*,example.*
    $CONTROLGROUPS=control,control.*
```

This sets the variable "LOCALGROUPS" to "local.*,example.*" and the variable "CONTROLGROUPS" to "control,control.*". They can later be used elsewhere in the file, such as in a site entry like:

```
    news.example.com:$LOCALGROUPS:Tf,Wnm:
```

which is then completely equivalent to:

```
    news.example.com:local.*,example.*:Tf,Wnm:
```

Variables aren't solely simple substitution. If either "!" or "@" immediately precedes the variable and the value of the variable contains commas, that character will be duplicated before each comma. This somewhat odd-sounding behavior is designed to make it easier to use variables to construct feed patterns. The utility becomes more obvious when you observe that the line:

```
    news.example.net:*,@$LOCALGROUPS,!$CONTROLGROUPS:Tf,Wnm:
```

is therefore equivalent to:

```
    news.example.net:*,@local.*,@example.*,!control,!control.*:Tf,Wnm:
```

which (as explained below) excludes all of the groups in $LOCALGROUPS and unwanted control articles from the feed to that site.

## FLAG VALUES
The flags parameter specifies miscellaneous parameters, including the type of feed, what information should be sent to it, and various limitations on what articles should be sent to a site. They may be specified in any order and should be separated by commas. Flags that take values should have the value immediately after the flag letter with no whitespace. The valid flags are:

< size

An article will only be sent to this site if it is less than size bytes long. The default is no limit.

> size

An article will only be sent to this site if it is greater than size bytes long. The default is no limit.

A checks

An article will only be sent to this site if it meets the requirements specified in checks, which should be chosen from the following set. checks can be multiple letters if appropriate. Note that this flag is not effective on funnel targets; it has to be used on every funnel entry (for instance, Af is not effective on the innfeed! funnel target and therefore has to be specified on every funnelled news site).

c

Exclude all kinds of control messages.

C

Only send control messages, not regular articles.

d

Only send articles with a Distribution header field. Combined with a particular distribution value in the distribution part of the site entry, this can be used to limit articles sent to a site to just those with a particular distribution.

e

Only send articles where every newsgroup listed in the Newsgroups header field body exists in the active file.

f

Don't send articles rejected by filters. This is only useful when dontrejectfiltered is set to true in inn.conf. With that variable set, this lets one accept all articles but not propagate filtered ones to some sites.

j

Propagate articles according to their Newsgroups header field. This is only useful when wanttrash is set to true in inn.conf. With that variable set, articles accepted and filed in "junk" (due to wanttrash) are fed to peers based on their subscription pattern applied to the Newsgroups header field as though they were accepted and all those groups were locally carried. Otherwise, they are propagated to sites that receive the "junk" newsgroup. This variable is useful if you want to run INN with a minimal active file and propagate all posts.

o

Only send articles for which overview data was stored.

O

Send articles to this site that don't have an Injection-Info or X-Trace header field, even if the "O" flag is also given.

p

Only check the exclusions against the Path header field of articles; don't check the site name. This is useful if your site names aren't the same as the path identities in Path header fields added by those remote sites, or for program feeds where the site name is arbitrary and unrelated to the Path header field body.

If both "c" and "C" are given, the last specified one takes precedence.

B high/low

If a site is being fed by a file, channel, or exploder (see below), the server will normally start trying to write the information as soon as possible. Providing a buffer may give better system performance and help smooth out overall load if a large batch of news comes in. The value of this flag should be two numbers separated by a slash. high specifies the point at which the server can start draining the feed's I/O buffer, and low specifies when to stop writing and begin buffering again; the units are bytes. The default is to do no buffering, sending output as soon as it is possible to do so.

C count

If this flag is specified, an article will only be sent to this site if the number of groups it is posted to, plus the square of the number of groups followups would appear in, is no more than count. 30 is a good value for this flag, allowing for instance:

- crossposts to only 5 groups when followups aren't set;

- crossposts to up to 26 groups when followups are set to two groups;

- crossposts to up to 29 groups when followups are set to a single group;

- crossposts to up to 30 groups when followups are set to poster.

Note that if an article does not contain a Followup-To header field, the number of groups followups would appear in is the number of groups it is posted to.

F name

Specifies the name of the file that should be used if it's necessary to begin spooling for the site (see below). If name is not an absolute path, it is taken to be relative to pathoutgoing in inn.conf. If name is a directory, the file togo in that directory will be used as the file name.

G count

If this flag is specified, an article will only be sent to this site if it is posted to no more than count newsgroups. This has the problem of filtering out many FAQs, as well as newsgroup creation postings and similar administrative announcements. Either the C flag or the U flag is a better solution.

H count

If this flag is specified, an article will only be sent to this site if it has count or fewer sites in its Path header field body. This flag should only be used as a rough guide because of the loose interpretation of the Path header field; some sites put the poster's name in the header field body, and some sites that might logically be considered to be one hop become two because they put the posting workstation's name in the header field body. The default value for count if not specified is one. (Also see the O flag, which is sometimes more appropriate for some uses of this flag.)

I size

The flag specifies the size of the internal buffer for a file feed. If there are more file feeds than allowed by the system, they will be buffered internally in least-recently-used order. If the internal buffer grows bigger than size bytes, however, the data will be written out to the appropriate file. The default value is 16 KB.

N status

Restricts the articles sent to this site to those in newsgroups with the moderation status given by status. If status is "m", only articles in moderated groups are sent; if status is "u", only articles in unmoderated groups are sent.

O originator

If this flag is specified, an article will only be sent to this site if it contains an Injection-Info header field (or an X-Trace header field if no Injection-Info header field is found) and the first field of this header field matches originator. originator is a uwildmat expression without commas or a list of such expressions, separated by "/". The article is never sent if the first character of the pattern begins with "@" and the rest of the pattern matches. One use of this flag is to restrict the feed to locally generated posts by using an originator pattern that matches the Injection-Info header field added by the local server.

P priority

The nice priority that this channel or program feed should receive. This should be a positive number between 0 and 20 and is the priority that the new process will run with. This flag can be used to raise the priority to normal if you're using the nicekids parameter in inn.conf.

Q hashfeed

Specifies the hashfeed match expression for this site. It must be in the form "value/mod" or "start-end/mod". The Message-ID of the article is hashed using MD5, which results in a 128-bit hash. The lowest 32 bits are then taken by default as the hashfeed value (which is an integer). If the hashfeed value modulus "mod" plus one equals "value" or is between "start" and "end", the article will be fed to this site. All these numbers must be integers. It is a deterministic way to control the flow of articles and to split a feed. For instance:

```
    Q1/2      Feeds about 50% of all articles to this site.
    Q2/2      Feeds the other 50% of all articles.

```

Another example with three sites:

```
    Q1-3/10   Feeds about 30% of all articles.
    Q4-5/10   Feeds about 20% of all articles.
    Q6-10/10  Feeds about 50% of all articles.

```

If this flag is specified multiple times, the contents will be logically "OR"ed together (just one match is needed).

You can use an extended syntax of the form "value/mod_offset" or "start-end/mod_offset". As MD5 generates a 128-bit return value, it is possible to specify from which byte-offset the 32-bit integer used by hashfeed starts. The default value for "offset" is "_0" and thirteen overlapping values from "_0" to "_12" can be used. Only up to four totally independent values exist: "_0", "_4", "_8" and "_12".

Therefore, it allows generating a second level of deterministic distribution. Indeed, if a news server is fed "Q1/2", it can go on splitting thanks to "Q1-3/9_4" for instance. Up to four levels of deterministic distribution can be used.

The algorithm is compatible with the one used by Diablo 5.1 and up. If you want to use the legacy quickhashing method used by Diablo before 5.1, you can put an "@" sign just after the Q flag (for instance "Q@1-3/10", but the distribution of the messages is not perfect with this legacy method whose use is discouraged and for which offsets cannot be used).

S size

If the amount of data queued for the site gets to be larger than size bytes, the server will switch to spooling, appending to a file specified by the F flag, or pathoutgoing/sitename if F is not specified. Spooling usually happens only for channel or exploder feeds, when the spawned program isn't keeping up with its input.

T type

This flag specifies the type of feed for this site. type should be a letter chosen from the following set:

```
    c        Channel
    f        File
    l        Log entry only
    m        Funnel (multiple entries feed into one)
    p        Program
    x        Exploder

```

Each feed is described below in "FEED TYPES". The default is Tf, for a file feed.

U count

If this flag is specified, an article will only be sent to this site if followups to this article would be posted to no more than count newsgroups. (Also see C for a more complex way of handling this.)

W items

For a file, channel, or exploder feed, this flag controls what information will be sent to this site. For a program feed, only the asterisk ("*") has any effect. items should be chosen from the following set:

b

Size of the article (in wire format, meaning with CRLF at the end of each line, periods doubled at the beginning of lines, and ending in a line with a single period) in bytes.

e

The time the article will expire as seconds since epoch if it has an Expires header field, 0 otherwise.

f

The storage API token of the article (the same as "n"). The article can be retrieved given the storage API token by using sm(1).

g

The newsgroup the article is in; if cross-posted, then the first of the groups to which the article was posted that this site gets. (The difference from "G" is that this sends the newsgroup to which the article was posted even if it is a control message.)

h

The history hash key of the article (derived from the message ID).

m

The message ID of the article.

n

The storage API token of the article. The article can be retrieved given the storage API token by using sm(1).

p

The time the article was posted a seconds since epoch.

s

The site that fed the article to the server. This is taken from either the Path header field body or the IP address of the sending site depending on the value of logipaddr in inn.conf. If logipaddr is true and the IP address is 0.0.0.0 (meaning that the article was fed from localhost by a program like rnews(1)), the Path header field body will be sent instead.

t

The time the article was received as seconds since epoch.

*

The names of the appropriate funnel entries, or all sites that get the article (see below for more details).

D

The body of the Distribution header field of the article, or "?" if there is no such header field in the article.

G

Where the article is stored. If the newsgroup is crossposted, this is generally the first of the groups to which it was posted that this site receives; however, control messages are filed in control or control.* (which is the difference between this item and "g").

H

All of the headers, followed by a blank line. The Xref header field will already be present, and a Bytes header field containing the article's size in bytes as in the "b" item will be added to the headers. If used, this should be the only item in the list.

N

The body of the Newsgroups header field.

P

The body of the Path header field.

O

Overview data for the article.

R

Information needed for replication (the Xref header field without the site name).

More than one letter can be given. If multiple items are specified, they will be written in the order specified separated by spaces. ("H" should be the only item if given, but if it's not a newline will be sent before the beginning of the headers.) The default is Wn.

The "H" and "O" items are intended for use by programs that create news overview databases or require similar information. WnteO is the flag to generate input needed by the overchan(8) program.

The asterisk ("*") has special meaning. Normally it expands to a space-separated list of all sites that received the current article. If, however, this site is a target of a funnel feed (in other words, if it is named by other sites which have the Tm flag), then the asterisk expands to the names of the funnel feeds that received the article. Similarly, if the site is a program feed, an asterisk in the parameter field will be expanded into the list of funnel feeds that received the article. A program feed cannot get the site list unless it is the target of other Tm feeds.

## FEED TYPES
innd provides four basic types of feeds: log, file, program, and channel. An exploder is a special type of channel. In addition, several entries can feed into the same feed; these are funnel feeds, which refer to an entry that is one of the other types. Funnel feeds are partially described above with the description of the W* flag. A funnel feed gets every article that would be sent to any of the feeds that funnel into it and normally include the W* flag in their flags so that the program processing that feed knows which sites received which articles. The most common funnel feed is innfeed(8).

Note that the term "feed" is technically a misnomer, since the server doesn't transfer articles itself and only writes data to a file, program, or log telling another program to transfer the articles.

The simplest feed is a log feed (Tl). Other than a mention in the news log file, pathlog/news, no data is written out. This is equivalent to a Tf entry writing to /dev/null, except that no file is ever opened. Flushing a log feed does nothing.

A file feed (Tf) is the next simplest type of feed. When the site should receive an article, the specified data is written out to the file named by the parameter field. If parameter is not an absolute path, it is taken to be relative to pathoutgoing in inn.conf. If parameter is not given, it defaults to pathoutgoing/sitename. The file name should be unique (two file feeds should not ever point to the same file).

File feeds are designed for use by external programs that periodically process the written data. To cooperate with innd properly, such external programs should first rename the batch file and then send a flush command for that site to innd using ctlinnd(8). innd will then write out any buffered data, close the file, and reopen it (under the original name), and the program can process the data in the renamed file at its leisure. File feeds are most frequently used in combination with nntpsend(8).

A program feed (Tp) spawns a given program for every article that the site receives. The parameter field must be the command line to execute, and should contain one instance of %s, which will be replaced by the storage API token of the article (the actual article can be retrieved by the program using sm(1)). The program will not receive anything on standard input (unlike earlier versions of INN, where the article is sent to the program on stdin), and standard output and error from the program will be set to the error log (pathlog/errlog). innd will try to avoid spawning a shell if the command has no shell meta-characters; this feature can be defeated if necessary for some reason by appending a semi-colon to the end of the command. The full path name of the program to be run must be specified unless the command will be run by the shell (and it is strongly recommended that the full path name always be specified regardless).

If a program feed is the target of a funnel, and if W* appears in the flags of the site, a single asterisk may be present in the parameter and will be replaced by a space-separated list of names of the sites feeding into the funnel which received the relevant article. If the site is not the target of a funnel, or if the W* flag is not used, the asterisk has no special meaning.

Flushing a program feed does nothing.

For a channel (Tc) or exploder (Tx) feed, the parameter field again names the process to start. As with program feeds, the full path to the program must be specified. However, rather than spawning the program for every article, it is spawned once and then whenever the site receives an article, the data specified by the site flags is written to the standard input of the spawned program. Standard output and error are set as with program feeds. If the process exits, it will be restarted automatically. If the process cannot be started, the server will spool input to a file named pathoutgoing/sitename and will try to start the process again later.

When a channel or exploder feed is flushed, the server closes its end of the pipe to the program's standard input. Any pending data that has not been written will be spooled; see the description of the S flag above. The server will then spawn a new instance of the program. No signal is sent to the program; it is up to the program handling a channel or exploder feed to notice end of file on its standard input and exit appropriately.

Exploders are a special type of channel feed. In addition to the channel feed behavior described above, exploders can also be sent command lines. These lines start with an exclamation point and their interpretation is up to the exploder. The following commands are generated automatically by the server:

```
    !newgroup group
    !rmgroup group
    !flush
    !flush site
```

These commands are sent whenever the ctlinnd(8) command of the same name is received by the server. In addition, the ctlinnd(8) "send" command can be used to send an arbitrary command line to an exploder. The primary exploder is buffchan(8).

Finally, Tm feeds are the input to a funnel. The parameter field of the site should name the site handling articles for all of the funnel inputs.

## EXAMPLES
The syntax of the newsfeeds file is so complex because you can specify a staggering variety of feeds. INN is capable of interacting with a wide variety of programs that do various things with news articles. Far and away the most common two entries in newsfeeds, however, are file feeds for nntpsend(8) and funnel feeds for innfeed(8).

The former look like this:

```
    feed.example.com:*,!control,!control.*,!junk:Tf,Wnm:
```

which generates a file named pathoutgoing/feed.example.com containing one line per article consisting of the storage API token, a space, and the message ID.

The latter look like this:

```
    feed.example.com:*,!control,!control.*,!junk:Tm:innfeed!
```

Very similar, except that this is the input to a funnel feed named "innfeed!". One could also write this as:

```
    example/feed.example.com:*,!control,!control.*,!junk:Ap,Tm:innfeed!
```

(note the Ap so that articles that contain just "example" in the Path header field body will still be sent), which is completely equivalent except that this will be logged in pathlog/news as going to the site "example" rather than "feed.example.com".

The typical feed entry for innfeed(8) is a good example of a channel feed that's the target of various funnel feeds (make sure the path to innfeed is properly set):

```
    innfeed!:!*:Tc,Wnm*:<pathbin in inn.conf>/innfeed -y
```

Note that the pattern for this feed is just "!*" so that it won't receive any articles directly. The feed should only receive those articles that would go to one of the funnel feeds that are feeding into it. innfeed(8) will receive one line per article on its standard input containing the storage API token, the message ID, and a space-separated list of sites that should receive that article.

Here's a more esoteric example of a channel feed:

```
    watcher!:*:Tc,Wbnm\
        :exec awk '$1 > 1000000 { print "BIG", $2, $3 }' \
         >> <pathlog>/bigart
```

This receives the byte size of each article along with the storage API token and Message-ID, and appends to a bigart log file a line for every article that's over a million bytes. This is actually rather a strange way to write this since INN can do the size check itself; the following is equivalent:

```
    watcher!:*:Tc,>1000000,Wbnm\
        :exec awk '{ print "BIG", $2, $3}' >> <pathlog>/bigart
```

Here's a cute, really simple news to mail gateway that also serves as an example of a fairly fancy program feed:

```
    mailer!:!*:W*,Tp\
        :sm %s | innmail -s "News article" *
```

Remember that %s is replaced by the storage API token, so this retrieves the article and pipes it into innmail (which is safer than programs like Mail(1) because it doesn't parse the body for tilde commands) with a given subject line. Note the use of "*" in the command line and W* in the flags; this entry is designed to be used as the target of funnel feeds such as:

```
    peter@example.com:news.software.nntp:Tm:mailer!
    sue@example.com:news.admin.misc:Tm:mailer!
```

Suppose that the server receives an article crossposted between news.admin.misc and news.software.nntp. The server will notice that the article should be sent to the site "peter@example.com" and the site "bob@example.com", both of which funnel into "mailer!", so it will look at the "mailer!" site and end up executing the command line:

```
    sm @...@ | innmail -s "News article" \
        peter@example.com sue@example.com
```

which will mail the article to both Peter and Sue.

Finally, another very useful example of a channel feed: the standard entry for controlchan(8). Make sure its path is properly set.

```
    controlchan!\
        :!*,control,control.*,!control.cancel\
        :AC,Tc,Wnsm:<pathbin in inn.conf>/controlchan
```

This program only wants information about articles posted to a control newsgroup other than control.cancel, which due to the sorting of control messages described in innd(8) will send it all control messages except for cancel messages. controlchan gets the storage API token, the name of the sending site (for processing old-style ihave and sendme control messages, be sure to read about logipaddr in controlchan(8)), and the message ID for each article.

For many other examples, including examples of the special "ME" site entry, see the example newsfeeds file distributed with INN. Also see the install documentation that comes with INN for information about setting up the standard newsfeeds entries used by most sites.

## HISTORY
Written by Rich $alz <rsalz@uunet.uu.net> for InterNetNews. Reformatted and rewritten in POD by Russ Allbery <eagle@eyrie.org>.

## SEE ALSO
active(5), buffchan(8), controlchan(8), ctlinnd(8), delayer(1), inn.conf(5), innd(8), innfeed(8), innxmit(8), libinn_uwildmat(3), nntpsend(8).


newsgroups

title: newsgroups(5)

text:
# newsgroups(5)

## NAME
newsgroups - List of newsgroups and their short descriptions

## DESCRIPTION
The file pathdb/newsgroups contains a list of newsgroups for which a short description is available. This file is generally updated by controlchan(8) whenever a control message is received; it is used by nnrpd in response to LIST NEWSGROUPS and is only meant to provide information to users. News readers often show the list of carried newsgroups along with these descriptions.

It is not necessary that all the groups carried by the news server (that is to say all the groups listed in the active file) be listed in the newsgroups file. And it is also not necessary that all the groups listed in the newsgroups file be carried by the news server. Nonetheless, it is of course better if the active and newsgroups files have exactly the same newsgroups.

If you use "ctlinnd newgroup" to manually create a group, only the active file is updated. You should then edit the newsgroups file in order to add a short description for the created group. The same goes for manually removing or changing the status of a newsgroup.

Each line of the newsgroups file consists of two fields separated by at least one tabulation:

```
    <name>\t<description>
```

The first field is the name of the newsgroup. The second field is its description.

You can get the newsgroups file of another NNTP server with getlist(1).

## PREFERRED FORMAT FOR A ONE-LINE NEWSGROUP DESCRIPTION
As far as the format of the newsgroups file is concerned, there is a preferred format for each line. Since news administrators do not generally have the time to fix up the lines that are being automatically included from newgroup or checkgroups messages, this information is provided so that control message senders can craft better control messages. It will also be useful for news administrators to know how to format the description of their local newsgroups.

There should be at least one hard tab (8 column tab stops) between the group name and the description. If the group name is at least 16 characters, it should be followed with one tab. If the group name is at least 8 characters, it should be followed with two tabs. And in the unlikely event the group name is less than 8 characters, it should be followed with three tabs. For instance:

```
    misc.transport.rail.europe  Railroads & railways in all of Europe.
    news.admin.nocem    NoCeM protocol policy issues and information.
    news.groups         Discussions and lists of newsgroups.
```

The total line length should be at most 79 columns. The description should start with a capital and not be more than 55 characters (79 - 24) long. If the group name is longer than 24 characters, the description should be correspondingly shorter. If the group is moderated, it should have " (Moderated)" (note the space before the opening parenthesis) at the very end of the description, not counted as part of the length of the description. This text must be exactly that, with no variations, as it is used by news software to find moderated groups.

Here is an example of moderated newsgroup:

```
    news.lists.misc     News-related statistics and lists. (Moderated)
```

Traditionally, all newsgroup descriptions ended with a period, but this is not necessary and steals away one character that is occasionally useful for forming a better description.

Some over-long descriptions could be made to easily fit the length by dropping useless wordings like "Discussion of" which do not meaningfully contribute to the description. Others are usually pretty easy to get to no more than column eighty, except when the group names start getting really long. Hopefully then the group name itself contains quite a bit of description.

In some cases, a longer description really will be necessary; they can of course be used within the newsgroups file. However, they will probably be less readable and less useful for some Usenet users.

Descriptions must not contain any control characters (octets between 0x00 and 0x1F).

## ENCODING OF THE DESCRIPTIONS
There is, at present, no good mechanism for managing the character set of the newsgroup descriptions. Many non-English hierarchies include newsgroup descriptions in their native languages, since this is more useful for their users, and those are included verbatim in the newsgroups file. This unfortunately means that different lines of the file will require different character set settings to read properly, and those character sets are not documented in the file. Hopefully some future standard will provide a way to address this; in the meantime, using UTF-8 for non-ASCII characters is recommended.

## MINIMAL NEWSGROUPS FILE
The minimal newsgroups file shipped with INN is:

```
    control             Various control messages (no posting).
    control.cancel      Cancel messages (no posting).
    control.checkgroups Hierarchy check control messages (no posting).
    control.newgroup    Newsgroup creation control messages (no posting).
    control.rmgroup     Newsgroup removal control messages (no posting).
    junk                Unfiled articles (no posting).
```

These lines must be tab-delimited, so please be careful in case you copy and paste them from above.

## HISTORY
Written by Julien Elie for InterNetNews. The preferred format for a one-line newsgroup description is based on the policies by which the newsgroups file in <https://ftp.isc.org/pub/usenet/CONFIG/> is maintained; they were originally written by David Lawrence <tale@isc.org> and updated by Russ Allbery <eagle@eyrie.org>.

## SEE ALSO
active(5), controlchan(8), ctlinnd(8), getlist(1), nnrpd(8).