cdcc

package: DCC

title: cdcc(8)

text:
# cdcc(8)

## NAME
cdcc — Control Distributed Checksum Clearinghouse

## SYNOPSIS
cdcc

[-VBdq] [-h homedir] [-c ids] [op1 op2 ... [-]]

## DESCRIPTION
Cdcc is used to clear, control, and query the control file used by Distributed Checksum Clearinghouse clients such as dccm(8). The host names, UDP port numbers, IDs, and passwords local clients use to talk to servers as well as IP addresses, round trip times, and other information are contained in the map file. While cdcc is set-UID, it uses the real UID only when accessing the map file. It refuses to display sensitive information such as passwords unless the real UID is the same as the effective UID. Note that cdcc needs to be set to a UID that can read and write the map file, but that UID need not be 0.

Cdcc is also used to send commands to DCC servers to tell them to stop, reload their lists of DCC IDs, turn on tracing, and so forth.

Many commands sent to DCC servers require a numeric DCC ID and a password recognized by the server. A DCC password is a 1-32 character string that does not contain blank, tab, newline or carriage return characters. The ID is specified with the id operation. If cdcc is run with a real UID that can read the ids file and a password is not specified (see the password operation), then the current password for the specified ID in the ids file will be used. If no ids file is available and a password and DCC ID are not specified, cdcc uses the anonymous DCC client-ID. DCC servers do not expect a password from clients using the anonymous client-ID, but they also won't honor control requests.

Operations that modify the map file can only be performed when the real UID is sufficient to modify the file directly. Trying to perform an operation that requires a password without specifying a server-ID or without using a UID that can access the ids file produces an error message complaining about a "privileged operation."

Commands and operations are read from the command line or from stdin. A series of op1 op2 ... operations followed a - (a dash) causes operations to be read from stdin after the command line operations are processed. Semi-colons or newlines separate commands in UNIX command-line "words," as well as when commands are read from stdin. Since each command line operation must be a shell "word," quotes are often required as in

% cdcc "load map.txt"

or % cdcc "host localhost;info" stats

### OPTIONS
The following options are available:

-V

displays the version of the DCC controller. Two or more -V options show the options with which it was built.

-B

sends error messages from the DCC server to both stderr and stdout instead of only stderr.

-d

enables debugging output from the DCC client software. Additional -d options increase the number of messages. See the debug command.

-q

quiets initial complaints about the map file and some messages about successful commands. See the quiet command.

-h homedir

overrides the default DCC home directory, /var/lib/dcc. See the homedir operation.

-c ids

specifies the file containing DCC IDs and passwords known by local DCC servers. An ids file that can be read by others cannot be used. The format of the ids file is described in dccd(8).

op1 op2 ...

are operations or commands such as "id 100; stop". Commands or operations specified on the command line are performed before the first interactive request. The last command can be - to specify that additional commands should be read from stdin.

### OPERATIONS
Local operations include the following:

help [command]

lists information about one or all available commands and operations.

exit

stops cdcc

grey [on | off]

switches between DCC and greylist servers.

homedir [path]

displays or specifies the DCC home directory, /var/lib/dcc.

file [map]

displays or specifies the name or path of the map file. The string "-" specifies the default file /var/lib/dcc/map.

new map [map]

creates a new, empty file for DCC server host names, port numbers, passwords, and so forth. There must not already be a file of the same name. The default is map in the DCC home directory.

delete host[,port]

deletes the entry in the map file for host and UDP port. If greylist mode has been set with the grey on command, the entry for the grelist server at host is deleted.

add host[,port] [RTT+adj|RTT-adj] [Greylist] [client-ID [password]]

adds an entry to the map file. The port can be "-" to specify the default DCC server port number. An adjustment to the round trip time is between -2000 and +2000 and follows the string RTT. The adjustment is added to the average measured round trip time when the DCC client software picks the "nearest" DCC server, or the server with the smallest RTT. If an IP address is mentioned more than once in the list of servers, for example because it is among the addresses for more than one server name, conflicts among RTT adjustments are resolved by picking the adjustment with the largest absolute value.

Greylist marks an entry for a greylist servers. Greylist is assumed if greylist mode has been set with the grey on command, See dccd(8).

If both the client-ID and the password are absent, the anonymous client-ID, 1, is used. The string anon is equivalent to the anonymous client-ID. A null password string is assumed if the password is missing and the client-ID is 1 or also missing.

load info-file

loads the current parameter file with the host names, port numbers, IDs, and passwords in info-file. Standard input is understood if info-file is "-". A suitable file can be created with the info operation. It consists of ignored blank or comment lines starting with '#' and other lines in the same format as the arguments to the add operation. Note that output of the info command will lack passwords unless it is run by a privileged user.

host [hostname]

specifies the host name of the DCC server to which commands should be sent. If hostname is "-", the current default DCC server is chosen.

port [port]

specifies the UDP port number of the DCC server to which commands should be sent. The default is 6277 or 6276 depending on the setting of the greylist mode controlled with the grey command.

password secret

specifies the password with which to sign commands sent to the DCC server specified with the server and port operations.

id [ID]

specifies or displays the numeric DCC ID for commands sent to the DCC server specified with the server and port operations. If no password is specified with the password command, the password is sought in the local ids.

info [-N]

displays information about the connections to DCC servers. It starts with the current date and name of the current map file or says that cdcc is using the implicit file created with the server and port operations. It then says when host names will next be resolved into IP addresses, the smallest round trip time to the IP addresses of known DCC servers. The host name, UDP port number (or dash if it is the default), DCC client-ID, and password (if cdcc is used by a privileged user) are shown in one line per configured DCC server. The currently preferred IP address is indicated by an asterisk. The "brand" of the server, its DCC ID, and its IP address are displayed in one line per IP address. The performance of the server at each IP address in the most recent 32 operations is displayed in a second line. The second line ends with the measured delay imposed by the server on requests with this client's ID.

-N displays the reverse DNS name of each server.

RTT [-N]

measures the round trip time to the DCC servers. It does this by discarding accumulated information and forcing a probe of all listed server IP addresses. Beware that when run with sufficient privilege, the RTT operation is like the info and load operations and displays cleartext passwords.

-N displays the reverse DNS name of each server.

debug Op Ar on | off | TTL=x

increases or decreases debugging information from the DCC client software or sets the IP TTL on queries to the server. See -d. Some operating systems do not include the functions required to change the IP TTL. Others include the required functions but have no apparent effect.

quiet [on | off]

makes commands more quiet or more verbose.

IPv6 [on | off | only]

clients to try to use IPv6 and IPv4, IPv4 only, or IPv6 only.

SOCKS [on | off]

tell DCC to use the SOCKS5 protocol if they have been built with a SOCKS library. The socks library linked with the DCC client must be configured appropriately, often including knowing which DCC servers must be connected via the SOCKS proxy and which can be reached directly. DCC clients use SOCKS functions such as Rsendto() with all or no servers depending on the setting of this switch.

src [- |] IPaddress[,IPv6address]

displays or configures the source address of DCC client requests. - removes the explicit configuration of the source, while IPaddress or IPaddress,IPv6address sets it. This makes sense only on multi-homed hosts. It can be useful for passing firewalls.

### DCC SERVER COMMANDS
Commands that can be sent to a DCC server include the following. Most of the commands must be used with the server's ID specified with the id command. The specified ID is included in the commands sent to the server The command itself is digitally signed with the first password associated with the ID in the ids file. The server requires that the signature match one of the passwords associated with the ID in its ids file.

delck type hex1 hex2 hex3 hex4

asks the server to delete the type checksum with value hex1 hex2 hex3 hex4. The type and checksum values can be found in dccproc(8) and dccm(8) log files or computed with dccproc -QC. There are very few situations where it makes sense to bother to delete checksums. For example, mail that was accidentally reported with a target count of "MANY" is either private and so will not be seen by other people and so will not be affected, or it is bulk and its source so must have already been whitelisted by recipients.

stats [all | clear]

displays current status and statistics from the current DCC server or for all known DCC servers. The server's counters will be cleared after they are displayed when the server's ID has been specified with the id ID operation.

clients [-nsiaVAK] [-I id] [max [thold]] [addr[/prefix]]

display some of the clients recently seen by the server. -n

display only the IP addresses and not the names of clients.

-s

sort the clients by the number of requests they have made.

-i

count clients with the same client-ID as single entities.

-I id

display information only about clients using client-ID id.

-a

produce 24 hour average numbers of requests.

-A

display only anonymous clients.

-K

display only clients using client-IDs other than the anonymous ID of 1.

-V

include the DCC protocol versions used by clients.

max

display only the max most recent clients.

max thold

display the most recent max clients that have made at least thold requests.

addr[/prefix]

restricts the results to the DCC client with that IP address or clients with addresses in that block of addresses. Individual clients in a blacklisted block of addresses in the server's /var/lib/dcc/blacklist file are not displayed unless explicitly requested by address or address block.

Clients with IDs marked with an asterisk (*) used the wrong password for that client-ID.

The mechanism that implements this command involves asking the DCC server for the first approximately 100 clients, then the second about 100, and so on, If entries change position in the complete list maintained by the server between requests, the displayed list will have duplicate or missing entries. Only clients since the last use of stats clear are displayed.

stop

tells the DCC server to exit.

system stop

tells the DCC server to exit so that the operating system can be shut down. This tells the DCC server on some systems to delete the dcc_db.hash file to speed system shut down. The file will be rebuilt automatically by dbclean when the DCC server is restarted.

clean stop

tells the DCC server to exit after applying fsync() to the database.

reload IDs

tells the local DCC server to reload its DCC ids file immediately. This command is not strictly needed. Every several minutes, the DCC server notices if the file has been changed and automatically reads it.

flood check

tells the DCC server to check for changes in the flod file and try to restart any of the streams to peers that are broken.

flood shutdown

tells the DCC server to cleanly stop flooding checksums to and from peers. The server will wait for sending and receiving peers to agree to stop. Each flood shutdown or flood halt request increases a count of reasons why the server should not flood checksums.

flood halt

tells the DCC server to abruptly stop flooding checksums to and from peers.

flood rewind server-ID

tells the DCC server to ask its peer with server-ID to rewind and resend its stream of checksums.

flood ffwd in server-ID

tells the DCC server to ask its peer to "fast forward" or skip to the end of the incoming flood.

flood ffwd out server-ID

tells the DCC server to "fast forward" or skip to the current end of the flood to its peer.

flood resume

tells the DCC server to reduce the number of reasons to not flood checksums increased by flood shutdown and flood halt. When the number of reasons reaches zero, the server tries to resume flooding.

flood list

displays the list of current incoming and outgoing floods. Each line contains the server-ID of the peer, the IP address and port used for the outgoing flood, the address for the incoming flood if different, and the host name. Only the server-IDs of flooding peers are disclosed with the server's ID.

flood stats [clear] { server-ID | all }

displays counts of checksum reports sent and received by the current flooding connections to and from server-ID or all flooding connections and then optionally clears the counts.

DB clean

is used by dbclean to tell the server that the database expiration has begun.

DB new

is used by dbclean to tell the server that the database cleaning is complete.

flush cache

tells the server to flush its cache and to keep it clean.

cache ok

tells the server to resume normal operations after flush cache.

clock check

asks the DCC server to say how much its clock differs from the local clock.

clock kludge +/-seconds

adjusts the timestamps in server commands to make it possible to control servers with inaccurate clocks.

trace default

turns on ANON and CLNT tracing and turns off all others.

trace mode {on|off}

turns the server's tracing mode on or off. Mode must be one of: ADMN

administrative requests from cdcc

ANON

errors by anonymous clients

CLNT

errors by authenticated clients

RLIM

rate-limited messages

QUERY

all queries and reports

RIDC

messages concerning the report-ID cache that is used to detect duplicate reports from clients

FLOOD1

messages about inter-server flooding connections

FLOOD2

messages about flooded reports

IDS

unknown server-IDs in flooded reports

BL

blacklisted clients

DB

odd database events

WLIST

reports of whitelisted checksums from authenticated, not anonymous DCC clients

cdcc exits with 0 on success, and >0 if an error occurs in operations specified on the command line.

## FILES
/var/lib/dcc

default DCC home directory unless changed by the homedir operation.

map

memory mapped file in the home DCC home directory of server host names, port numbers, passwords, measured round trip times (RTT), and so forth.

ids

list of IDs and passwords, as described in dccd(8). It is only required by systems running the DCC server, but is used by cdcc if available.

## SEE ALSO
dbclean(8), dcc(8), dccd(8), dblist(8), dccifd(8), dccm(8), dccproc(8), dccsight(8).

## HISTORY
Implementation of cdcc was started at Rhyolite Software in 2000. This document describes version 2.3.167.


dbclean

title: dbclean(8)

text:
# dbclean(8)

## NAME
dbclean — Clean Distributed Checksum Clearinghouse Database

## SYNOPSIS
dbclean

[-dfFNPSVqu] [-i id] [-a [server-addr][,port]] [-h homedir]

[-H hash-file-dir] [-D db-file-dir] [-G on] [-R mode]

[-s hash-size] [-e seconds] [-E spamsecs]

[-L ltype,facility.level]

## DESCRIPTION
Dbclean creates empty, rebuilds corrupted, and deletes or expires old reports of checksums from DCC databases. It should be installed where it will be found with the path given the DCC server daemon when the daemon needs to expand the hash table. See dccd(8). It should also be run by the daily cron(8) job, /usr/lib/dcc/cron-dccd.

The whitelist in /var/lib/dcc/whitelist or /var/lib/dcc/grey_whitelist are built into the DCC server's database. Changes to the whitelist are not effective until dbclean is run. White or blacklists can also be used by DCC clients and work better.

### OPTIONS
The following options are available. Most of them should set by changing DBCLEAN_LOGDAYS and DBCLEAN_ARGS in the /var/lib/dcc/dcc_conf control file.

-d

enables debugging output. Additional -d options increase the number of messages.

-F

uses write() instead of mmap() and msync() in some cases to modify the DCC database. This works better on some versions of Solaris provided the entire DCC database fits in RAM and provided the file system has not been tuned for the large, random accesses of a DCC database. It is the default on Solaris except when the database is in a memory mapped file system or the entire database fits in RAM. Do not use -F with -f, -H, or -D.

-f

uses mmap() and msync() to modify the DCC database. Do not use -f with -F, -H, or -D.

-N

creates a new, empty database. There must not be an existing database and the DCC server, dccd(8), must not be running.

-P

expires old checksums from a database using the -e -and -E values from the preceding use of dbclean. -P cannot be used with -e or -E. Using -P differs from not using -e or -E, because in the absence of all three, their default values are used.

-S

says that the DCC server, dccd(8), is not running and so dbclean should run stand-alone and not try to tell the DCC server about changes to the database. -i is not needed with -S.

-V

displays the version of the DCC database cleaner. Two or more -V options show the options with which it was built.

-q

quiets the announcement to stderr of the final results and debugging messages turned on with -d. Results are still sent to the system log.

-u

should be used when the file system containing the dcc_db and dcc_db.hash files is "ultra-fast," such as a solid-state drive. This setting postpones writing some data during normal operation until the system is shutdown. This reduces wear on the SSD at the cost of writing GBytes of data when the system is shutdown. Writing GBytes to a rotating disk at system shutdown unacceptable, but not noticeable to a non-rotating disk that operates 10 or 100 times faster.

-i id

specifies the DCC ID recognized by the local DCC server as its own. This ID allows the DCC server to recognize commands from dbclean to stop using the database while it is being cleaned.

-a [server-addr][,port]

is commonly used to specify a UDP port or IP address of the local server other than the default.

-h homedir

overrides the default DCC home directory, /var/lib/dcc.

-H hash-file-dir

puts the dcc_db.hash hash table file in the hash-file-dir directory with a symbolic link. Using -H to put the hash table in a memory, "tmpfs", or "swap" file system such as /dev/shm or /tmp significantly speeds up the DCC server, dccd(8), on operating systems such as Linux and Solaris that lack the MAP_NOSYNC flag for the mmap(8) system call. The memory file system must have space for two copies of the dcc_db.hash file. -H is undesirable on FreeBSD and other systems with MAP_NOSYNC except when -D is used. The DCC database file, dcc_db, should usually be in file system that is not cleared by operating system rebooting and not in a memory file system.

Use or stop using -H by adding it to or removing it from DBCLEAN_ARGS in /var/lib/dcc/dcc_conf and manually running or waiting for the nightly run of the /usr/lib/dcc/cron-dccd cron job.

Do not use -H with -F or -f.

-D db-file-dir

puts the /var/lib/dcc/dcc_db database and the /var/lib/dcc/dcc_db.hash hash table files in the db-file-dir directory with symbolic links. Using -D to put the files in a memory, "tmpfs", or "swap" file system such as /dev/shm or /tmp significantly speeds up the DCC server, dccd(8), but causes several GBytes of database transfers before DCC is effective whenever the operating system is restarted. -D should not be used except at sites with more than one DCC server installation and where at least one of the other local servers is not using -D. -D should only be used on systems that are rarely rebooted, because up to one month is required for the database reach equilibrium. The memory file system must have space for two copies of the dcc_db file.

Use or stop using -D by adding it to or removing it from DBCLEAN_ARGS in /var/lib/dcc/dcc_conf and manually running or waiting for the nightly run of the /usr/lib/dcc/cron-dccd cron job.

Do not use -D with -F or -f.

-G on

cleans a greylist database in /var/lib/dcc/grey_db and /var/lib/dcc/grey_db.hash instead of of a DCC server database.

-R mode

repairs a database or does a quick cleaning. Mode must be one of the following: bad

to repair a broken database.

quick

for a quick, superficial cleaning during the day.

hash

to rebuild a hash not sent to disk before the system was rebooted.

failsafe

to work around missing nightly cleaning by the cron(8) job, /usr/lib/dcc/cron-dccd

del

to finish processing a delete command received by dccd(8).

-s hash-size

specifies a size for the hash table in /var/lib/dcc/dcc_db.hash. By default the hash table is rebuilt to be approximately 80% full based on an estimate of the number of distinct checksums in the database file.

-e seconds

specifies that reports older than seconds and with totals below 10 targets should be deleted. Reports older than seconds of checksums that have been reported more recently are summarized in the database. The default value is 1DAY or the value of -E, whichever is smaller. The 1 day default is reduced if the system does not appear to have enough RAM to hold the database. The minimum is 1 hour. Seconds can also be NEVER or a number of hours, days, or weeks followed by HOURS, H, DAYS, D, WEEKS or W. DCC servers that are not very busy and are isolated or do not receive "floods" of checksums from busy servers should use longer values to increase their chances of recognizing bulk mail.

-E spamsecs

changes the expiration of checksums with more than 10 targets from the default of 30DAYS or the explicit value of -e, whichever is larger. The default is reduced if the system does not have enough RAM to hold the database. Spamsecs can also be NEVER or a number of hours, days, or weeks followed by HOURS, H, DAYS, D, WEEKS or W.

-L ltype,facility.level

specifies how messages should be logged. Ltype must be error, info, or off to indicate which of the two types of messages are being controlled or to turn off all syslog(3) messages from dbclean. Level must be a syslog(3) level among EMERG, ALERT, CRIT, ERR, WARNING, NOTICE, INFO, and DEBUG. Facility must be among AUTH, AUTHPRIV, CRON, DAEMON, FTP, KERN, LPR, MAIL, NEWS, USER, UUCP, and LOCAL0 through LOCAL7. The default is equivalent to -L info,MAIL.NOTICE -L error,MAIL.ERR

dbclean exits 0 on success, and > 0 if an error occurs.

## FILES
/var/lib/dcc

is the DCC home directory containing data and control files.

dcc_conf

is the DCC control file.

dcc_db

is the main file containing mail checksums.

dcc_db.hash

mail checksum database hash table.

grey_db

is the database of greylist checksums.

grey_db.hash

is the greylist database hash table.

dcc_db-new, dcc_db-new.hash, grey_db-new, grey_db-new.hash

new database and hash files until they are renamed.

dcc_db-old, grey_db-old

previous database files.

ids

list of IDs and passwords, as described in dccd(8).

whitelist

contains the DCC server whitelist in the format described in dcc(8).

grey_whitelist

contains the greylist server whitelist.

## SEE ALSO
cdcc(8), cron(8), dcc(8), dccd(8), dblist(8), dccifd(8), dccm(8), dccproc(8).

## HISTORY
Implementation of dbclean was started at Rhyolite Software, in 2000. This document describes version 2.3.167.


dblist

title: dblist(8)

text:
# dblist(8)

## NAME
dblist — Database List Distributed Checksum Clearinghouse

## SYNOPSIS
dblist

[-vVHD] [-G on | off] [-h homedir]

[-C 'type h1 h2 h3 h4'] [-I server-ID] [-A dbaddr] [-L pathlen]

[-P pages] [-T timestamp] [file1 file2 ...]

## DESCRIPTION
Dblist lists the contents of a DCC database as it does some consistency checking.

-v

lists more of the database. Additional information is produced with additional -v arguments.

-V

displays the version of the DCC database lister. Two or more -V options show the options with which it was built.

-H

turns off the listing of the hash table as well as the analysis of the hash table. Determining the worst case and average lengths of chains in the hash table can take a long time for a large database on a small computer.

-D

turns off the listing of the data or checksum records.

-G on

lists a greylist database.

-h homedir

overrides the default DCC home directory, /var/lib/dcc.

-C 'type h1 h2 h3 h4'

limits the listing to records containing that checksum or one of the other checksums specified with -C. If the four hexadecimal values h1 h2 h3 h4 are absent, records with the matching type will be listed. If type is absent, any checksum with the four hexadecimal values will be listed. If the hexadecimal values h1 h2 h3 h4 are absent, all checksums of the specified type will be listed. The alternate form 'server-ID {simple|commercial|ignore|rogue} at ID' can be used to select server-ID records of some types.

As many as 16 checksums can be specified.

-I server-ID

limits the listing to records with that server-ID or one of the other server-IDs specified with -I. As many as 16 server-IDs can be specified.

-A dbaddr

excludes database records before dbaddr.

-L pathlen

excludes records with path lengths shorter than pathlen.

-P pages

ignores all but the last pages of the database.

-T timetamp

excludes records with other timestamps. A timestamp with a missing microsecond value matches any record with that second. As many as 16 timestamps can be specified.

file1 file2 ...

are names of databases to be listed. The default is /var/lib/dcc/dcc_db and its companion, /var/lib/dcc/dcc_db.hash. When -G on is used, the default files are /var/lib/dcc/grey_db and /var/lib/dcc/grey_db.hash.

By default, the sizes of the main file and the hash table as well as how much they contain and values related to the performance of the hash are displayed.

With a single -v, most of the mail database file and the contents of memory mapped server flooding positions in the /var/lib/dcc/flod.map file are listed. The listing starts with the serial number of the database file which is when old entries were last removed from it by dbclean(8) That is followed by similar lines showing the oldest timestamp of checksums not expired by dbclean and of mail that is not "spam."

The flooding positions from the /var/lib/dcc/flod.map file are record offsets or addresses in the main database file.

A typical record in the main database file looks like:

```
02/07/02 20:25:12.497032    5	      1601		2fe5b94
     path: 103<-101<-1601
  Body	    6	    e2d3f96a c65aea01 3fece361 edff9ecf  2f21364 772d2
  Fuz1	    many    6ff56fe8 ffc312d7 a5fe8f13 12a537ae  2f21364 200a9
  Fuz2	    many    fac882b8 03eea34f bd792c40 2fe6fd54  2f21364 72816
```

That example was received by a DCC server with server-ID 1601 at about 8:25 UTC on the evening of February 7, 2000. The report was about a mail message set to 5 addressees. The report was then sent or ‘flooded’ to the server with server-ID 101 which in turn sent it to a server with server-ID 103. That server sent it to the local DCC server. The record is at the address 0x2fe5b94 in the database. The record contains 3 checksums. The simple checksum of the body of the message was 0xe2d3f96a 0xc65aea01 0x3fece361 0xedff9ecf The total number of recipients of messages with this body checksum known in the database is 6, which implies this checksum had been previously reported with a target count of 1. The previous report in the database of a message with this body checksum is at 0x2f21364. The hash table chain for this body checksum is computed to start at 0x772d2. This report included two fuzzy checksums. Both have been previously reported as having been sent to many targets.

An asterisk (*) before the type of the checksum indicates that this checksum is redundant. A report of many addressees makes all preceding reports redundant.

The flooding of some database records is delayed, as shown by the string delayed after the server-ID.

The string trimmed after the server-ID marks older reports that have had uninteresting checksums removed. The string compressed after the server-ID would indicate that this older report has been trimmed and compressed with older reports.

With two -v arguments, records added to the database by dbclean(8) from the server whitelist are also displayed.

Three -v arguments cause the hash table to be displayed. Three typical hash table entries look like:

```
      19b8:   19ee   19b7
      19b9:   19c0      0    90120 Fuz1
      19ba:      0      0  1b72300 Fuz1
```

The entry in slot number 0x19b8 is unused or free. Slot number 0x19b9 is the start of a chain of collisions or entries with the same hash value of 0x19b9. The next slot in this chain is at 0x19c0. The corresponding Fuz1 checksum is at 0x9012 in the database. The third slot at 0x19ba is also that of a Fuz1 checksum, but it is not part of a hash chain and its database record is at 0x1b72300.

## FILES
/var/lib/dcc

is the DCC home directory containing data and control files.

dcc_db

main file of DCC checksums.

grey_dcc_db

main file of checksums used for greylisting.

dcc_db.hash

database hash table for /var/lib/dcc/dcc_db.

grey_dcc_db.hash

database hash table for /var/lib/dcc/grey_dcc.

flod.map

grey_flod.map

memory mapped flooding positions.

## SEE ALSO
cdcc(8), dcc(8), dbclean(8), dccd(8), dccifd(8), dccm(8), dccproc(8).

## HISTORY
Implementation of dblist was started at Rhyolite Software, in 2000. This document describes version 2.3.167.


dcc

title: dcc(8)

summary: Distributed Checksum Clearinghouse

text:
# dcc(8)

## NAME
DCC — Distributed Checksum Clearinghouse

## DESCRIPTION
The Distributed Checksum Clearinghouse or DCC is a cooperative, distributed system intended to detect "bulk" mail or mail sent to many people. It allows individuals receiving a single mail message to determine that many other people have received essentially identical copies of the message and so reject or discard the message.

Source is available at https://www.dcc-servers.net/dcc/ free for organizations that do not sell spam or virus filtering services.

### How the DCC Is Used
The DCC can be viewed as a tool for end users to enforce their right to "opt-in" to streams of bulk mail by refusing bulk mail except from sources in a "whitelist." Whitelists are the responsibility of DCC clients, since only they know which bulk mail they solicited.

False positives or mail marked as bulk by a DCC server that is not bulk occur only when a recipient of a message reports it to a DCC server as having been received many times or when the "fuzzy" checksums of differing messages are the same. The fuzzy checksums ignore aspects of messages in order to compute identical checksums for substantially identical messages. The fuzzy checksums are designed to ignore only differences that do not affect meanings. So in practice, you do not need to worry about DCC false positive indications of "bulk," but not all bulk mail is unsolicited bulk mail or spam. You must either use whitelists to distinguish solicited from unsolicited bulk mail or only use DCC indications of "bulk" as part of a scoring system such as SpamAssassin. Besides unsolicited bulk email or spam, bulk messages include legitimate mail such as order confirmations from merchants, legitimate mailing lists, and empty or test messages.

A DCC server estimates the number copies of a message by counting checksums reported by DCC clients. Each client must decide which bulk messages are unsolicited and what degree of "bulkiness" is objectionable. Client DCC software marks, rejects, or discards mail that is bulk according to local thresholds on target addresses from DCC servers and unsolicited according to local whitelists.

DCC servers are usually configured to receive reports from as many targets as possible, including sources that cannot be trusted to not exaggerate the number of copies of a message they see. A user of a DCC client angry about receiving a message could report it with 1,000,000 separate DCC reports or with a single report claiming 1,000,000 targets. An unprincipled user could subscribe a "spam trap" to mailing lists such as those of the IETF or CERT. Such abuses of the system area not problems, because much legitimate mail is "bulk." You cannot reject bulk mail unless you have a whitelist of sources of legitimate bulk mail.

DCC can also be used by an Internet service provider to detect bulk mail coming from its own customers. In such circumstances, the DCC client might be configured to only log bulk mail from unexpected (not whitelisted) customers.

### What the DCC Is
A DCC server accumulates counts of cryptographic checksums of messages but not the messages themselves. It exchanges reports of frequently seen checksums with other servers. DCC clients send reports of checksums related to incoming mail to a nearby DCC server running dccd(8). Each report from a client includes the number of recipients for the message. A DCC server accumulates the reports and responds to clients the the current total number of recipients for each checksum. The client adds an SMTP header to incoming mail containing the total counts. It then discards or rejects mail that is not whitelisted and has counts that exceed local thresholds.

A special value of the number of addressees is "MANY" and means this message was certainly bulk and might be unsolicited, perhaps because it came from a locally blacklisted source or was addressed to an invalid address or "spam trap." The special value "MANY" is merely the largest value that fits in the database field containing the count of addressees. That "infinite" accumulated total can be reached with millions of independent reports as well as with one or two.

DCC servers flood or send reports of checksums of bulk mail to neighboring servers.

To keep a server's database of checksums from growing without bound, checksums are forgotten when they become old. Checksums of bulk mail are kept longer. See dbclean(8).

DCC clients pick the nearest working DCC server using a small shared or memory mapped file, /var/lib/dcc/map. It contains server names, port numbers, passwords, recent performance measures, and so forth. This file allows clients to use quick retransmission timeouts and to waste little time on servers that have temporarily stopped working or become unreachable. The utility program cdcc(8) is used to maintain this file as well as to check the health of servers.

### X-DCC Headers
The DCC software includes several programs used by clients. Dccm(8) uses the sendmail "milter" interface to query a DCC server, add header lines to incoming mail, and reject mail whose total checksum counts are high. Dccm is intended to be run with SMTP servers using sendmail.

Dccproc(8) adds header lines to mail presented by file name or stdin, but relies on other programs such as procmail to deal with mail with large counts. Dccsight(8) is similar but deals with previously computed checksums.

Dccifd(8) is similar to dccproc but is not run separately for each mail message and is more efficient. It receives mail messages via a socket somewhat like dccm, but with a simpler protocol that can be used by Perl scripts or other programs. Dccifd can also be used as a postfix "Before-Queue Content Filter."

DCC SMTP header lines are of one of the forms:

```
X-DCC-brand-Metrics: client server-ID; bulk cknm1=count cknm2=count ...
X-DCC-brand-Metrics: client; whitelist
```

where whitelist

appears if the global /var/lib/dcc/whiteclnt or per-user file marks the message as good.

brand

is the "brand name" of the DCC server, such as "RHYOLITE".

client

is the name or IP address of the DCC client that added the header line to the SMTP message.

server-ID

is the numeric ID of the DCC server that the DCC client contacted.

bulk

is present if one or more checksum counts exceeded the DCC client's thresholds to make the message "bulky."

bulk rep

is present if the DCC reputation of the IP address of the sender is bad.

cknm1,cknm2,...

are types of checksums: IP

address of SMTP client

env_From

SMTP envelope value

From

SMTP header line

Message-ID

SMTP header line

Received

last Received: header line in the SMTP message

substitute

SMTP header line chosen by the DCC client, prefixed with the name of the header

Body

SMTP body ignoring white-space

Fuz1

filtered or "fuzzy" body checksum

Fuz2

another filtered or "fuzzy" body checksum

rep

DCC reputation of the mail sender or the estimated probability that the message is bulk.

Counts for IP, env_From, From, Message-Id, Received, and substitute checksums are omitted by the DCC client if the server says it has no information. Counts for Fuz1 and Fuz2 are omitted if the message body is empty or contains too little of the right kind of information for the checksum to be computed.

count

is the total number of recipients of messages with that checksum reported directly or indirectly to the DCC server. The special count "MANY" means that DCC client have claimed that the message is directed at millions of recipients. "MANY" imples the message is definitely bulk, but not necessarily unsolicited. The special counts "OK" and "OK2" mean the checksum has been marked "good" or "half-good" by DCC servers.

### Mailing lists
Legitimate mailing list traffic differs from spam only in being solicited by recipients. Each client should have a private whitelist.

DCC whitelists can also mark mail as unsolicited bulk using blacklist entries for commonly forged values such as "From: user@public.com".

### White and Blacklists
DCC server and client whitelist files share a common format. Server files are always named whitelist and one is required to be in the DCC home directory with the other server files. Client whitelist files are named /var/lib/dcc/whiteclnt in the DCC home directory or a per-user subdirectory of the directory specified with the -U option for dccm(8) or dccifd(8). They specify mail that should not be reported to a DCC server or that is always unsolicited and almost certainly bulk.

A DCC whitelist file contains blank lines, comments starting with "#", and lines of the following forms:

include file

Copies the contents of file into the whitelist. It cannot occur in an included file. The file name is relative to the DCC home directory if not absolute.

count value

lines specify checksums that should be white- or blacklisted. count env_From 821-path

count env_To dest-mailbox

count From 822-mailbox

count Message-ID <string>

count Received string

count Substitute header string

count Hex ctype cksum

count IP hosts

MANY value

indicates that millions of targets have received messages with the header, IP address, or checksum value.

OK value

OK2 value

say that messages with the header, IP address, or checksum value are OK and should not reported to DCC servers or be greylisted. OK2 says that the message is "half OK." Two OK2 checksums associated with a message are equivalent to one OK.

A DCC server never shares or floods reports containing checksums marked in its whitelist with OK or OK2 to other servers. A DCC client does not report or ask its server about messages with a checksum marked OK or OK2 in the client whitelist. This is intended to allow a DCC client to keep private mail so private that even its checksums are not disclosed.

MX IP hosts

MXDCC IP hosts

mark an IP address or block of addresses of trusted mail relays including MX servers, smart hosts, and bastion or DMZ relays. The DCC clients dccm(8), dccifd(8), and dccproc(8) parse and skip initial Received: headers added by listed MX servers to determine the external sources of mail messages. Unsolicited bulk mail that has been forwarded through listed addresses is discarded by dccm(8) and dccifd(8) as if with -a DISCARD instead of rejected. MXDCC marks addresses that are MX servers that run DCC clients. The checksums for a mail message that has been forwarded through an address listed as MXDCC are queried instead of reported by a DCC client.

dccd(8) treats MXDCC and MX lines in the /var/lib/dcc/whitelist file as if they were OK lines.

SUBMIT IP hosts

marks an IP address or block of addresses of SMTP submission clients such as web browsers that cannot tolerate 4yz temporary rejections but that cannot be trusted to not send spam. Since they are local addresses, DCC Reputations are not computed for them.

dccd(8) ignores SUBMIT lines in the /var/lib/dcc/whitelist file.

value in count value lines can be

dest-mailbox

is an RFC 821 address or a local user name.

821-path

is an RFC 821 address.

822-mailbox

is an RFC 822 address with optional name.

Substitute header

is the name of an SMTP header such as "Sender" or the name of one of two SMTP envlope values, "HELO," or "Mail_Host" for the resolved host name from the 821-path in the message.

Hex ctype cksum

starts with the string Hex followed a checksum type, and a string of four hexadecimal numbers obtained from a DCC log file or the dccproc(8) command using -CQ. The checksum type is body, Fuz1, or Fuz2 or one of the preceding checksum types such as env_From.

hosts

is a host name, an IPv4 or IPv6 address, a block of IP addresses specified as starting and ending addresses separated by a dash (-), or a block in the standard xxx/mm form. A host name is converted to IP addresses with DNS, the /etc/hosts file, or other mechanisms.

The /var/lib/dcc/whitelist file used by the DCC server. dccd(8), treats all host names, IP addresses, and address blocks the same. Each IP address must be added to the DCC database as its checksum. DCC servers only hear about checksums and so could not use a list of IP addresses. To prevent accidentally adding billions of records to the database (contemplate a line like "OK IP fe80::0/120), server whitelist entries cannot specify blocks larger than 65,536 or /16.

The DCC clients, dccifd(8), dccm(8) or dccproc(8), know about IP addresses and so their whitelists can contain IP addresses. The global /var/lib/dcc/whiteclnt file or a per-user whiteclnt file can contain up to 64 ranges of 256 or more IP addresses. Smaller ranges are added as individual entries.

option setting

can only be in a DCC client whiteclnt file used by dccifd(8), dccm(8) or dccproc(8). Settings in per-user whiteclnt files override settings in the global /var/lib/dcc/whiteclnt file. Setting can be any of the following: option log-all

to log all mail messages.

option log-normal

to log only messages that meet the logging thresholds.

option log-subdirectory-day

option log-subdirectory-hour

option log-subdirectory-minute

puts log files for mail messages in subdirectories of the userdirs/addr/log directory specified with -U userdirs for dccm(8) or dccifd(8). The subsdirectories are of the form JJJ, JJJ/HH, or JJJ/HH/MM where JJJ is the current julian day, HH is the current hour, and MM is the current minute. See also -l logdir for dccm(8), dccifd(8), and dccproc(8).

option DCC-on

option DCC-off

to control DCC filtering.

option greylist-on

option greylist-off

to control greylisting if enabled in dccm(8) or dccifd(8) with -G. Greylisting for other recipients in the same SMTP transaction can still cause greylist temporary rejections.

option greylist-ignore-spam-on

option greylist-ignore-spam-off

causes greylisting to ignore the results of other filters. If off, spam is rejected regardless of greylist embargoes and future embargoes for the sending IP address are restored or reset. If this option is on, greylist delays or embargoes are required before spam is rejected and future embargoes on spam sending IP addresses are not reset.

option greylist-log-on

option greylist-log-off

to control per-user logging of greylisted mail messages. Logging of greylisted messages in the main log directory is not affected.

option DCC-rep-off

option DCC-rep-on

to honor or ignore DCC Reputations computed by the DCC server.

option DNSBL1-off

option DNSBL1-on

option DNSBL2-off

option DNSBL2-on

option DNSBL3-off

option DNSBL3-on

option DNSBL4-off

option DNSBL4-on

honor or ignore results of DNS blacklist checks configured with -B for dccm(8), dccifd(8), and dccproc(8).

option MTA-first

option MTA-last

consider MTA determinations of spam or not-spam first so they can be overridden by whiteclnt files, or last so that they can override whiteclnt files.

option forced-discard-ok

option no-forced-discard

control whether dccm(8) and dccifd(8) are allowed to discard a message for one mailbox for which it is spam when it is not spam and must be delivered to another mailbox. This can happen if a mail message is addressed to two or more mailboxes with differing whitelists. Discarding can be undesirable because false positives are not communicated to mail senders. To avoid discarding, dccm(8) and dccifd(8) running in proxy mode temporarily reject SMTP envelope Rcpt To values that involve differing whiteclnt files.

option threshold type,rej-thold

has the same effects as -c type,rej-thold for dccproc(8) or -t type,rej-thold for dccm(8) and dccifd(8). It is useful only in per-user whiteclnt files to override the global DCC checksum thresholds.

option spam-trap-discard

option spam-trap-reject

say that mail should be reported to the DCC server as extremely bulk or with target counts of MANY. Greylisting, DNS blacklist (DNSBL), and other checks are turned off. Spam-trap-discard tells the MTA to accept the message while spam-trap-reject tells the MTA to reject the message. Use Spam-trap-discard for spam traps that should not be disclosed. Spam-trap-reject can be used on catch-all mailboxes that might receive legitimate mail by typographical errors and that senders should be told about.

option not-spam-trap

turns off spam-trap-discard and spam-trap-reject.

In the absence of explicit settings, the default in the main whiteclnt file is equivalent to

option log-normal

option DCC-on

option greylist-on

option greylist-ignore-spam-off

option greylist-log-on

option DCC-rep-off

option DNSBL1-off

option DNSBL2-off

option DNSBL3-off

option DNSBL4-off

option MTA-last

option no-forced-discard

The defaults for individual recipient whiteclnt files are the same except as change by explicit settings in the main file.

Checksums of the IP address of the SMTP client sending a mail message are practically unforgeable, because it is impractical for an SMTP client to "spoof" its address or pretend to use some other IP address. That would make the IP address of the sender useful for whitelisting, except that the IP address of the SMTP client is often not available to users of dccproc(8). In addition, legitimate mail relays make whitelist entries for IP addresses of little use. For example, the IP address from which a message arrived might be that of a local relay instead of the home address of a whitelisted mailing list.

Envelope and header From values can be forged, so whitelist entries for their checksums are not entirely reliable.

Checksums of env_To values are never sent to DCC servers. They are valid in only whiteclnt files and used only by dccm(8), dccifd(8), and dccproc(8) when the envelope Rcpt To value is known.

### Greylists
The DCC server, dccd(8), can be used to maintain a greylist database for some DCC clients including dccm(8) and dccifd(8). Greylisting involves temporarily refusing mail from unfamiliar SMTP clients and is unrelated to filtering with a Distributed Checksum Clearinghouse.

See http://projects.puremagic.com/greylisting/

### Privacy
Because sending mail is a less private act than receiving it, and because sending bulk mail is usually not private at all and cannot be very private, the DCC tries first to protect the privacy of mail recipients, and second the privacy of senders of mail that is not bulk.

DCC clients necessarily disclose some information about mail they have received. The DCC database contains checksums of mail bodies, header lines, and source addresses. While it contains significantly less information than is available by "snooping" on Internet links, it is important that the DCC database be treated as containing sensitive information and to not put the most private information in the DCC database. Given the contents of a message, one might determine whether that message has been received by a system that subscribes to the DCC. Guesses about the sender and addressee of a message can also be validated if the checksums of the message have been sent to a DCC server.

Because the DCC is distributed, organizations can operate their own DCC servers, and configure them to share or "flood" only the checksums of bulk mail that is not in local whitelists.

DCC clients should not report the checksums of messages known to be private to a DCC server. For example, checksums of messages local to a system or that are otherwise known a priori to not be unsolicited bulk should not be sent to a remote DCC server. This can accomplished by adding entries for the sender to the client's local whitelist file. Client whitelist files can also include entries for email recipients whose mail should not be reported to a DCC server.

### Security
Whenever considering security, one must first consider the risks. The worst DCC security problems are unauthorized commands to a DCC service, denial of the DCC service, and corruption of DCC data. The worst that can be done with remote commands to a DCC server is to turn it off or otherwise cause it to stop responding. The DCC is designed to fail gracefully, so that a denial of service attack would at worst allow delivery of mail that would otherwise be rejected. Corruption of DCC data might at worst cause mail that is already somewhat "bulk" by virtue of being received by two or more people to appear have higher recipient numbers. Since DCC users must whitelist all sources of legitimate bulk mail, this is also not a concern. Such security risks should be addressed, but only with defenses that don't cost more than the possible damage from an attack.

The DCC must contend with senders of unsolicited bulk mail who resort to unlawful actions to express their displeasure at having their advertising blocked. Because the DCC protocol is based on UDP, an unhappy advertiser could try to flood a DCC server with packets supposedly from subscribers or non-subscribers. DCC servers defend against that attack by rate-limiting requests from anonymous users.

Also because of the use of UDP, clients must be protected against forged answers to their queries. Otherwise an unsolicited bulk mail advertiser could send a stream of "not spam" answers to an SMTP client while simultaneously sending mail that would otherwise be rejected. This is not a problem for authenticated clients of the DCC because they share a secret with the DCC. Unauthenticated, anonymous DCC clients do not share any secrets with the DCC, except for unique and unpredictable bits in each query or report sent to the DCC. Therefore, DCC servers cryptographically sign answers to unauthenticated clients with bits from the corresponding queries. This protects against attackers that do not have access to the stream of packets from the DCC client.

The passwords or shared secrets used in the DCC client and server programs are "cleartext" for several reasons. In any shared secret authentication system, at least one party must know the secret or keep the secret in cleartext. You could encrypt the secrets in a file, but because they are used by programs, you would need a cleartext copy of the key to decrypt the file somewhere in the system, making such a scheme more expensive but no more secure than a file of cleartext passwords. Asymmetric systems such as that used in UNIX allow one party to not know the secrets, but they must be and are designed to be computationally expensive when used in applications like the DCC that involve thousands or more authentication checks per second. Moreover, because of "dictionary attacks," asymmetric systems are now little more secure than keeping passwords in cleartext. An adversary can compare the hash values of combinations of common words with /etc/passwd hash values to look for bad passwords. Worse, by the nature of a client/server protocol like that used in the DCC, clients must have the cleartext password. Since it is among the more numerous and much less secure clients that adversaries would seek files of DCC passwords, it would be a waste to complicate the DCC server with an asymmetric system.

The DCC protocol is vulnerable to dictionary attacks to recover passwords. An adversary could capture some DCC packets, and then check to see if any of the 100,000 to 1,000,000 passwords in so called "cracker dictionaries" applied to a packet generated the same signature. This is a concern only if DCC passwords are poorly chosen, such as any combination of words in an English dictionary. There are ways to prevent this vulnerability regardless of how badly passwords are chosen, but they are computationally expensive and require additional network round trips. Since DCC passwords are created and typed into files once and do not need to be remembered by people, it is cheaper and quite easy to simply choose good passwords that are not in dictionaries.

### Reliability
It is better to fail to filter unsolicited bulk mail than to fail to deliver legitimate mail, so DCC clients fail in the direction of assuming that mail is legitimate or even whitelisted.

A DCC client sends a report or other request and waits for an answer. If no answer arrives within a reasonable time, the client retransmits. There are many things that might result in the client not receiving an answer, but the most important is packet loss. If the client's request does not reach the server, it is easy and harmless for the client to retransmit. If the client's request reached the server but the server's response was lost, a retransmission to the same server would be misunderstood as a new report of another copy of the same message unless it is detected as a retransmission by the server. The DCC protocol includes transactions identifiers for this purpose. If the client retransmitted to a second server, the retransmission would be misunderstood by the second server as a new report of the same message.

Each request from a client includes a timestamp to aid the client in measuring the round trip time to the server and to let the client pick the closest server. Clients monitor the speed of all of the servers they know including those they are not currently using, and use the quickest.

### Client and Server-IDs
Servers and clients use numbers or IDs to identify themselves. ID 1 is reserved for anonymous, unauthenticated clients. All other IDs are associated with a pair of passwords in the ids file, the current and next or previous and current passwords. Clients included their client IDs in their messages. When they are not using the anonymous ID, they sign their messages to servers with the first password associated with their client-ID. Servers treat messages with signatures that match neither of the passwords for the client-ID in their own ids file as if the client had used the anonymous ID.

Each server has a unique server-ID less than 32768. Servers use their IDs to identify checksums that they flood to other servers. Each server expects local clients sending administrative commands to use the server's ID and sign administrative commands with the associated password.

Server-IDs must be unique among all systems that share reports by "flooding." All servers must be told of the IDs all other servers whose reports can be received in the local /var/lib/dcc/flod file described in dccd(8). However, server-IDs can be mapped during flooding between independent DCC organizations.

Passwd-IDs are server-IDs that should not be assigned to servers. They appear in the often publicly readable /var/lib/dcc/flod and specify passwords in the private /var/lib/dcc/ids file for the inter-server flooding protocol

The client identified by a client-ID might be a single computer with a single IP address, a single but multi-homed computer, or many computers. Client-IDs are not used to identify checksum reports, but the organization operating the client. A client-ID need only be unique among clients using a single server. A single client can use different client-IDs for different servers, each client-ID authenticated with a separate password.

An obscure but important part of all of this is that the inter-server flooding algorithm depends on server-IDs and timestamps attached to reports of checksums. The inter-server flooding mechanism requires cooperating DCC servers to maintain reasonable clocks ticking in UTC. Clients include timestamps in their requests, but as long as their timestamps are unlikely to be repeated, they need not be very accurate.

### Installation Considerations
DCC clients on a computer share information about which servers are currently working and their speeds in a shared memory segment. This segment also contains server host names, IP addresses, and the passwords needed to authenticate known clients to servers. That generally requires that dccm(8), dccproc(8), dccifd(8), and cdcc(8) execute with an UID that can write to the DCC home directory and its files. The sendmail interface, dccm, is a daemon that can be started by an "rc" or other script already running with the correct UID. The other two, dccproc and cdcc need to be set-UID because they are used by end users. They relinquish set-UID privileges when not needed.

Files that contain cleartext passwords including the shared file used by clients must be readable only by "owner."

The data files required by a DCC can be in a single "home" directory, /var/lib/dcc. Distinct DCC servers can run on a single computer, provided they use distinct UDP port numbers and home directories. It is possible and convenient for the DCC clients using a server on the same computer to use the same home directory as the server.

The DCC source distribution includes sample control files. They should be modified appropriately and then copied to the DCC home directory. Files that contain cleartext passwords must not be publicly readable.

The DCC source includes "feature" m4 files to configure sendmail to use dccm(8) to check a DCC server about incoming mail.

See also the INSTALL.html file.

### Client Installation
Installing a DCC client starts with obtaining or compiling program binaries for the client server data control tool, cdcc(8). Installing the sendmail DCC interface, dccm(8), or dccproc(8), the general or procmail(1) interface is the main part of the client installation. Connecting the DCC to sendmail with dccm is most powerful, but requires administrative control of the system running sendmail.

As noted above, cdcc and dccproc should be set-UID to a suitable UID. Root or 0 is thought to be safe for both, because they are careful to release privileges except when they need them to read or write files in the DCC home directory. A DCC home directory, /var/lib/dcc should be created. It must be owned and writable by the UID to which cdcc is set.

After the DCC client programs have been obtained, contact the operator(s) of the chosen DCC server(s) to obtain each server's host name, port number, and a client-ID and corresponding password. No client-IDs or passwords are needed touse DCC servers that allow anonymous clients. Use the load or add commands of cdcc to create a map file in the DCC home directory. It is usually necessary to create a client whitelist file of the format described above. To accommodate users sharing a computer but not ideas about what is solicited bulk mail, the client whitelist file can be any valid path name and need not be in the DCC home directory.

If dccm is chosen, arrange to start it with suitable arguments before sendmail is started. See the homedir/dcc_conf file and the misc/rcDCC script in the DCC source. The procmail DCCM interface, dccproc(8), can be run manually or by a procmailrc(5) rule.

### Server Installation
The DCC server, dccd(8), also requires that the DCC home directory exist. It does not use the client shared or memory mapped file of server addresses, but it requires other files. One is the /var/lib/dcc/ids file of client-IDs, server-IDs, and corresponding passwords. Another is a flod file of peers that send and receive floods of reports of checksums with large counts. Both files are described in dccd(8).

The server daemon should be started when the system is rebooted, probably before sendmail. See the misc/rcDCC and misc/start-dccd files in the DCC source.

The database should be cleaned regularly with dbclean(8) such as by running the crontab job that is in the misc directory.

## SEE ALSO
cdcc(8), dbclean(8), dcc(8), dccd(8), dccifd(8), dccm(8), dccproc(8), dblist(8), dccsight(8), sendmail(8).

## HISTORY
Distributed Checksum Clearinghouses are based on an idea of Paul Vixie with code designed and written at Rhyolite Software starting in 2000. This document describes version 2.3.167.


dccd

title: dccd(8)

text:
# dccd(8)

## NAME
dccd — Distributed Checksum Clearinghouse Daemon

## SYNOPSIS
dccd

[-dVbfFQ] -i server-ID [-n brand] [-h homedir] -I [host-ID][,user]

[-a [server-addr][,server-port]] [-q qsize]

[-G [on,][weak-body,][weak-IP,][embargo][,window][,white]]

[-W [rate][,chg][,dbsize]] [-K [no-]type] [-T [no-]tracemode] [-u anon-delay[,inflate]] [-C dbclean] [-L ltype,facility.level]

[-R [RL_SUB],[RL_ANON],[RL_ALL_ANON],[RL_BUGS]]

## DESCRIPTION
Dccd receives reports of checksums related to mail received by DCC clients and queries about the total number of reports of particular checksums. A DCC server never receives mail, address, headers, or other information from clients, but only cryptographically secure checksums of such information. A DCC server cannot determine the text or other information that corresponds to the checksums it receives. It only acts as a clearinghouse of total counts of checksums computed by clients.

Each DCC server is identified by a numeric server-ID. Each DCC client is identified by a client-ID, either explicitly listed in the /var/lib/dcc/ids file or the special anonymous client-ID. Many computers are expected to share a single client-ID. A server-ID is between 100 and 32768 while a client-ID is between 32768 and 16777215. DCC server-IDs need be known only to DCC servers and the people running them. The passwords associated with DCC server-IDs should be protected, because DCC servers listen to commands authenticated with server-IDs and their associated passwords. Each client that does not use the anonymous ID must know the client-ID and password used by each of its servers. A single client computer can use different passwords with different server computers. See the /var/lib/dcc/ids file.

A /var/lib/dcc/whitelist of known good (or bad) sources of email prevents legitimate mailing lists from being seen as unsolicited bulk email by DCC clients. The whitelist used by a DCC server is built into the database when old entries are removed by dbclean(8). Each DCC client has its own, local whitelist, and in general, whitelists work better in DCC clients than servers.

A dccd /var/lib/dcc/whitelist file containing IP addresses that should be in client whiteclnt files is useful. When -T WLIST tracing is enabled (as it is by default), dccd complains to the system log when an authenticated client reports mail from IP addresses listed as OK, MX, or MXDCC. It is often useful to have a /var/lib/dcc/whitecommon file containing whitelisted IP addresses.

The effectiveness of a Distributed Checksum Clearinghouse increases as the number of subscribers increases. Flooding reports of checksums among DCC servers increases the effective number of subscribers to each server. Each dccd daemon tries to maintain TCP/IP connections to the other servers listed in the /var/lib/dcc/flod file, and send them reports containing checksums with total counts exceeding thresholds. Changes in the flod file are noticed automatically within minutes.

Controls on report flooding are specified in the flod file. Each line specifies a hostname and port number to which reports should be flooded, a server-ID to identify and authenticate the output stream, a server-ID to identify and authenticate an input stream from the same server, and flags with each ID. The ability to delete reports of checksums is handy, but could be abused. If del is not present among the in-opts options for the incoming ID, incoming delete requests are logged and then ignored. Floods from DCC "brands" that count only mail to spam traps and whose servers use the -Q option to count extremely bulk mail should be marked with traps. They can be seen as counting millions of targets, so the traps flag on their /var/lib/dcc/flod file entry changes their incoming flooded reports counts to many.

Dccd automatically checks its /var/lib/dcc/flod and /var/lib/dcc/ids files periodically. Cdcc(8) has the commands new ids and flood check to tell dccd to check those two files immediately. Both files are also checked for changes after the SIGHUP signal.

### OPTIONS
The following options are available. Most of them should set by changing the /var/lib/dcc/dcc_conf control file.

-d

enables debugging output. Additional -d options increase the number of messages.

-V

displays the version of the DCC server daemon. Two or more -V options show the options with which it was built.

-b

causes the server to not detach itself from the controlling tty or put itself into the background.

-F

uses write() instead of mmap() and msync() in some cases to modify the DCC database. It is the default on Solaris except when the database is in a memory mapped file system. See -f.

-f

uses mmap() and msync() to modify the DCC database. See -F.

-Q

causes the server to treat reports of checksums as queries except from DCC clients marked trusted in the /var/lib/dcc/ids file with rpt-ok. See -u to turn off access by anonymous or unauthenticated clients.

-i server-ID

specifies the ID of this DCC server. Each server identifies itself as responsible for checksums that it forwards to other servers.

-n brand

is an arbitrary string of letters and numbers that identifies the organization running the DCC server. The brand is required, and appears in the SMTP X-DCC headers generated by the DCC.

-h homedir

overrides the default DCC home directory, /var/lib/dcc.

-I [host-ID][,user]

sets the UID and GID of the process or the server's name for assertions of its -i server-ID flooded to peers. The default name is the first 16 characters of the host name. If present, user must be a valid user name.

-a [server-addr][,server-port]

adds an hostname or IP address to the list of local IP addresses that the server answers. Multiple -a options can be used to specify a subset of the available network interfaces or to use more than one port number. The default without any -a options is to listen on all local IP addresses. It can be useful to list some of the IP addresses of multi-homed hosts to deal with firewalls. By default server-port is 6277 for DCC servers and 6276 for greylist servers. It is the UDP port at which DCC requests are received and the TCP port for incoming floods of reports. If server-addr is absent and if the getifaddrs(8) function is supported, separate UDP sockets are bound to each configured network interface so that each DCC clients receives replies from the IP addresses to which corresponding request are sent. If dccd is started before all network interfaces are turned on or there are interfaces that are turned on and off or change their addresses such as PPP interfaces, then the special string @ should be used to tell dccd to bind to an INADDR_ANY UDP socket.

Outgoing TCP connections to flood checksum reports to other DCC servers used the IP address of a single -a option, but only if there is single option that is not localhost. See also the /var/lib/dcc/flod file.

-q qsize

specifies the maximum size of the queue of requests from anonymous or unauthenticated clients. The default value is the maximum DCC RTT in seconds times 200 or 1000.

-G [on,][weak-body,][weak-IP,][embargo][,window][,white]

changes dccd to a greylist server for dccm(8) or dccifd(8). Greylisting consists of temporarily rejecting or embargoing mail from unfamiliar combinations of SMTP client IP address, SMTP envelope sender, and SMTP envelope recipient. If the SMTP client persists for embargo seconds and so is probably not an open proxy, worm-infected personal computer, or other transient source of spam, the triple of (IP address,sender,recipient) is added to a database similar to the usual DCC database. If the SMTP client does not try again after embargo seconds and before window seconds after the first attempt, the triple is forgotten. If the SMTP client persists past the embargo, the triple is added to the database and becomes familiar and the message is accepted. Familiar triples are remembered for white seconds after the last accepted mail message. The triple is forgotten if it is ever associated with unsolicited bulk email. All three durations can be a number of minutes, hours, days, or weeks followed by MINUTES, M, HOURS, H, DAYS, D, WEEKS or W. The default is -G 270seconds,7days,63days. The first duration or the embargo should be longer than open proxies can linger retransmitting. The second window time should be as long as legitimate mail servers persist in retransmitting to recognize embargoed messages whose retransmissions were not received because of network or other problems. The white time should be long enough to recognize and not embargo messages from regular senders.

Usually the DCC greylist system requires that an almost identical copy of the message be retransmitted during the embargo. If weak-body is present, any message with the same triple of sender IP address, sender mail address, and target mail address ends the embargo, even if the body of the message differs.

If weak-IP is present, all mail from an SMTP client at an IP address is accept after any message from the same IP address has been accepted.

Unlike DCC checksums, the contents of greylist databases are private and do not benefit from broad sharing. However, large installations can use more two or more greylist servers flooding triples among themselves. Flooding among greylist servers is controlled by the /var/lib/dcc/grey_flod file.

All greylist cooperating or flooding greylist servers must use the same -G values.

Clients of greylist servers cannot be anonymous and must have client-IDs and passwords assigned in the /var/lib/dcc/ids file. This implies that cdcc commands directed to greylist servers must specify the server-ID.

White- and blacklists are honored by the DCC clients. whitelisted messages are embargoed or checked with a greylist server. The greylist triples of blacklisted messages, messages whose DCC counts make them spam, and other messages known to be spam are sent to a greylist server to be removed from the greylist database and cause an embargo on the next messages with those triples.

Messages whose checksums match greylist server whitelists are not embargoed and the checksums of their triples are not added to the greylist database.

The target counts of embargoed messages are reported to the DCC network to improve the detection of bulk mail.

-W [rate][,chg][,dbsize]

controls quick database cleaning. If the database is larger than dbsize in MBytes, the database has not recently been cleand and is not about to be cleaned, and dccd is receiving fewer than rate requests per second, or if telling DCC clients that the database is about to be cleaned reduces the requests/second by chg, then dccd starts dbclean(8) for a quick database cleaning. The cleaning is abandoned if it takes too long. The defaults are equivalent to -W 1.0,40.0,RSS where RSS is the maximum dccd resident set size displayed in the system log when the database is opened. A rate of -W 0.0 disables quick cleanings.

-K [no-]type

marks checksums of type (not) be kept or counted in the database (unless they appear in the /var/lib/dcc/whitelist file). Explicit settings add to or remove from the initial contents of the list, which is equivalent to -K Body -K Fuz1 -K Fuz2.

-T [no-]tracemode

causes the server to trace or record some operations. tracemode must be one of the following: ADMN

administrative requests from the control program, cdcc(8)

ANON

errors by anonymous clients

CLNT

errors by authenticated clients

RLIM

rate-limited messages

QUERY

all queries and reports

RIDC

some messages concerning the report-ID cache that is used to detect duplicate reports from clients

FLOOD1

messages about inter-server flooding connections

FLOOD2

messages about flooded reports

BL

requests from clients in the /var/lib/dcc/blacklist file.

DB

odd database events including long chains of duplicate checksums

WLIST

reports of whitelisted checksums from authenticated, not anonymous DCC clients

The default is ANON CLNT WLIST except for a greylist server which uses ANON CLNT WLIST IDS.

-u anon-delay[,inflate]

changes the number of milliseconds anonymous or unauthenticated clients must wait for answers to their queries and reports. The purpose of this delay is to discourage large anonymous clients. The anon-delay is multiplied by 1 plus the number of recent anonymous requests from IPv4 addresses in a /24 block or IPv6 addresses a /56 block divided by the inflate value. The string FOREVER turns off all anonymous or unauthenticated access not only for checksum queries and reports but also cdcc(8) stats requests. A missing value for inflate turns off inflation.

The default value is 50, except when -G is used in which case FOREVER is assumed and required.

-C dbclean

changes the default name or path of the program used to rebuild the hash table when it becomes too full. The default value is /usr/lib/dcc/dbclean. The value can include arguments as in -C '/usr/lib/dcc/dbclean -F'. Dbclean should not be run by dccd except in emergencies such as database corruption or hash table overflow. Dbclean(8) should be run daily with the /usr/lib/dcc/cron-dccd cron script

-L ltype,facility.level

specifies how messages should be logged. Ltype must be error, info, or off to indicate which of the two types of messages are being controlled or to turn off all syslog(3) messages from dccd. Level must be a syslog(3) level among EMERG, ALERT, CRIT, ERR, WARNING, NOTICE, INFO, and DEBUG. Facility must be among AUTH, AUTHPRIV, CRON, DAEMON, FTP, KERN, LPR, MAIL, NEWS, USER, UUCP, and LOCAL0 through LOCAL7. The default is equivalent to -L info,MAIL.NOTICE -L error,MAIL.ERR

-R [RL_SUB],[RL_ANON],[RL_ALL_ANON],[RL_BUGS]

sets one or more of the four rate-limits. RL_SUB limits the number of DCC transactions per second from subscribers or DCC clients with known client-IDs and passwords. This limit applies to each IP address independently. RL_ANON limits the number of DCC transactions per second from anonymous DCC clients. This limit applies to each IP address independently. It is better to use -u than to change this value to exclude anonymous clients.

RL_ALL_ANON limits the number of DCC transactions per second from all anonymous DCC clients. This limit applies to all anonymous clients as a group, regardless of their IP addresses.

RL_BUGS limits the number of complaints or error messages per second for all anonymous DCC clients as a group as well as for each DCC client by IP address.

The default is equivalent to -R 400,50,2000,0.1

## FILES
/var/lib/dcc

is the DCC home directory containing data and control files.

dcc_conf

is the DCC control file.

dcc_db

is the database of mail checksums.

dcc_db.hash

is the mail checksum database hash table.

grey_db

is the database of greylist checksums.

grey_db.hash

is the greylist database hash table.

flod

contains lines controlling DCC flooding of the form:

host[,rport][;src[,lport]] rem-ID [passwd-ID [o-opt [i-opt]]]

where absent optional values are signaled with "-" and host

is the IP address or name of a DCC server and rport is the name or number of the TCP port used by the remote server.

src

and lport are the source IP address or host name and TCP port from which the outgoing flooding connection should come. The string * specifies any source IP address. Incoming flooding connections must arrive at an address and port specified with -a.

rem-id

is the server-ID of the remote DCC server.

passwd-ID

is a server-ID that is not assigned to a server, but whose first password is used to sign checksum reports sent to the remote system. Either of its passwords are required with incoming reports. If it is absent or "-", outgoing floods are signed with the first password of the local server in the ids file and incoming floods must be signed with either password of the remote server-ID.

i-opt and o-opt

are comma separated lists of off

turns off flooding to the remote or local system.

no-del

says checksum delete requests are refused by the remote or local server and so turns off sending or accepting delete requests, respectively. By default, delete requests are sent to remote servers and accepted in incoming floods if and only if the peers are exchanging DCC reputations.

del

says delete requests are accepted by the remote or local server.

no-log-del

turns off logging of incoming requests to delete checksums.

passive

is used to tell a server outside a firewall to expect a peer inside to create both of the pair of input and output TCP connections used for flooding. The peer inside the firewall should use SOCKS or NAT on its flod file entry for this system.

SOCKS

is used to tell a server inside a firewall that it should create both of the TCP connections used for flooding and that SOCKS protocol should be used. The peer outside the firewall should use passive on its flod file entry for this system.

NAT

differs from SOCKS only by not using the SOCKS protocol.

IDS->result

converts server-IDs in flooded reports. IDS may be the string ‘self’ to specify the server's own ID. IDS can instead be the string ‘all’ to specify all server-IDs or a pair of server-IDs separated by a dash to specify an inclusive range. result can be the string ‘self’ to translate to the server's own ID. ‘ok’ sends or receives reports without translation. The string ‘reject’ to not send outgoing or refuse incoming reports. Only the first matching conversion is applied. For example, when ‘self->ok,all->reject’ is applied to a locally generated report, the first conversion is made and the second is ignored.

leaf=path-len

does not send reports with paths longer than path-len server-IDs. A path-len of 0 blocks reports from this server.

IPv4

requires only IPv4 addresses to connect to this flooding peer.

IPv6

requires only IPv6 addresses to connect to this flooding peer.

vers

specifies the version of the DCC flooding protocol used by the remote DCC server with a string such as ‘version2’.

trace1

sends information about a single peer like the cdcc(8) command trace FLOOD1 on does for all peers.

trace2

sends information about individual flooded reports like the cdcc(8) command trace FLOOD2 on does for all peers.

grey_flod

is the equivalent of the /var/lib/dcc/flod file used by dccd when it is a greylist server.

flod.map

is an automatically generated file in which dccd records its progress sending or flooding reports to DCC peers.

grey_flod.map

is the equivalent of the /var/lib/dcc/flod.map file used by dccd when it is a greylist server.

ids

contains the IDs and passwords known by the DCC server. An ids file that can be read by others cannot be used. It contains blank lines, comments starting with "#" and lines of the form: id[,rpt-ok][,no-reps][,trace][,delay=ms[*inflate]] pass1 [pass2]

where id

is a DCC client-ID or server-ID.

trace

logs activity from clients and flooding peers using the ID.

rpt-ok

overrides -Q by saying that this client is trusted to report only checksums for unsolicited bulk mail.

no-reps

disables DCC Reputation reports from and responses to clients using this ID.

delay=ms[*inflate]

delays answers to systems using the client id. The delay in milliseconds is multiplied by 1 plus the number of recent requests from an IP address using id divided by the inflate value. See -u.

pass1

is the password currently used by clients with identifier id. It is a 1 to 32 character string that does not contain blank, tab, newline or carriage return characters.

pass2

is the optional next password that those clients will use. A DCC server accepts either password if both are present in the file.

Both passwords can be absent if the entry not used except to tell dccd that server-IDs in the flooded reports are valid. The string unknown is equivalent to the null string.

whitelist

contains the DCC server whitelist. It is not used directly but is loaded into the database when dbclean(8) is run.

grey_whitelist

contains the greylist server whitelist. It is not used directly but is loaded into the database when dbclean(8) is run with -G.

blacklist

if present, contains a list of IP addresses and blocks of IP addresses of DCC clients and flooding peers that are ignored. Each line in the file should be blank, a comment starting with '#', or an IP address or block of IP addresses in the form [trace,][ok,][bad,][no-anon][flood-ok] address

Addresses are single IPv4 or IPv6 addresses, CIDR blocks in the usual form, or a pair of addresses separated by a hyphen (-) specifying an inclusive range. The last line in the file that cover an address applies. Changes to the file are automatically noticed within a few minutes. An IP address is assigned all of the flags from all of lines that apply to it. Addresses or blocks of addresses can be preceded with ok to "punch holes" in blacklisted blocks or specify tracing without blacklisting. Trace logs activity. No-anon blacklists clients only when they use the anonymous client-ID. flood-ok says that DCC flood peering is allowed. Bad is assumed when only an address is present. This mechanism is intended for no more than a few dozen blocks of addresses.

dccd_clients

contains client IP addresses and activity counts.

grey_clients

contains greylist client IP addresses and activity counts.

## EXAMPLES
dccd is usually started with other system daemons with something like the script /usr/lib/dcc/rcDCC. That scripts uses values in /var/lib/dcc/dcc_conf to start the server. With the argument stop, /usr/lib/dcc/rcDCC can be used to stop the daemon.

The database grows too large unless old reports are removed. dbclean(8) should be run daily with the /usr/lib/dcc/cron-dccd cron(8) script

## SEE ALSO
cdcc(8), dcc(8), dbclean(8), dblist(8), dccifd(8), dccm(8), dccproc(8). dccsight(8),

## HISTORY
dccd is based on an idea from Paul Vixie. It was designed and written at Rhyolite Software, starting in 2000. This document describes version 2.3.167.