NAME
innfeed.conf - configuration file for innfeed
DESCRIPTION
This man page describes the configuration file for version
0.9.4 (beta) of innfeed. This format has changed dramati-
cally since version 0.9.3.
The file innfeed.conf is used to control the innfeed(1) pro-
gram. It is a fairly free-format file that consists of three
types of entries: key/value, peer and group. Comments are
taken from the hash character ``#'' to the end of the line.
Key/value entries are a keyword and a value separated by a
colon (which can itself be surrounded by whitespace). For
example:
max-connections: 10
A legal key starts with a letter and contains only letters,
numbers and ``_'', ``-''.
There are 5 different type of values: integers, floating-
point numbers, characters, booleans, and strings. Integer
and floating point numbers are as to be expected except that
exponents in floating point numbers are not supported. A
boolean value is either ``true'' or ``false'' (case is not
significant). A character value is a single-quoted character
as defined by the C-language. A string value is any other
sequence of characters. If the string needs to contain whi-
tespace, then it must be quoted with double quotes, and uses
the same format for embedding non-printing characters as
normal C-language string.
Peer entries look like:
peer <name> {
# body ...
}
The word ``peer'' is required. The ``<name>'' is the same as
the site name in INN's newsfeeds file. The body of a peer
entry contains some number (possibly zero) of key/value
entries.
Group entries look like:
group <name> {
# body
}
The word ``group'' is required. The ``<name>'' is any string
valid as a key. The body of a group entry contains any
number of the three types of entries. So key/value pairs can
be defined inside a group, and peers can be nested inside a
group, and other groups can be nested inside a group.
Key/value entries that are defined outside of all peer and
group entries are said to be at ``global scope''. There are
global key/value entries that apply to the process as a
whole (for example the location of the backlog file direc-
tory), and there are global key/value entries that act as
defaults for peers. When innfeed looks for a specific value
in a peer entry (for example, the maximum number of connec-
tions to set up), if the value is not defined in the peer
entry, then the enclosing groups are examined for the entry
(starting at the closest enclosing group). If there are no
enclosing groups, or the enclosing groups don't define the
key/value, then the value at global scope is used.
A small example could be:
# Global value for the process
backlog-directory: /var/news/spool/innfeed
# Global value applied to all peers that have
# no value of their own.
max-connections: 5
# A peer definition. ``uunet'' is the name used by innd in
# the newsfeeds file.
peer uunet {
ip-name: usenet1.uu.net
}
peer vixie {
ip-name: gw.home.vix.com
max-connections: 10 # override global value.
}
# A group of two peers who can handle more connections
# than normal
group fast-sites {
max-connections: 15
# Another peer. The ``max-connections'' value from the
# ``fast-sites'' group scope is used. The ``ip-name'' value
# defaults to the peer's name.
peer data.ramona.vix.com {
}
peer bb.home.vix.com {
max-connections: 20 # he can really cook.
}
}
Given the above configuration file, the defined peers would
have the following values for the ``max-connections'' key.
uunet 5
vixie 10
data.ramona.vix.com 15
bb.home.vix.com 20
Innfeed ignores key/value pairs it is not interested in. Any
config file value that can be set via a command line option,
is not used if the command-line option is given.
GLOBAL VALUES
The following listing show all the keys that apply to the
process as whole. These are not required, but innfeed will
issue a warning if they are not defined.
news-spool
This key requires a pathname value. It specifies where
the top of the article spool is. This corresponds to
the ``-a'' command-line option.
pid-file
This key requires a pathname value. It specifies the
pathname (relative to the backlog-directory) where the
pid of the innfeed process should be stored. This
corresponds to the ``-p'' command-line option.
debug-level
This key defines the debug level for the process. A
non-zero number generates a lot of messages to stderr,
or to the config-defined ``log-file''. This
corresponsds to the ``-d'' command-line option.
use-mmap
This key requires a boolean value. It specifies whether
mmap'ing should be used if innfeed has been built with
mmap support. If article data on disk is not in NNTP-
ready format (CR/LF at the end of each line), then
after mmap'ing the article is read into memory and
fixed up, so mmap'ing has no positive effect (and pos-
sibly some negative effect depending on your system),
and so in such a case this value should be false. This
corresponds to the ``-M'' command-line option.
log-file
This key requires a pathname value. It specifies where
any logging messages that couldn't be sent via sys-
log(3) should go (such as those generated when a
positive value for ``debug-value'', is used). This
corresponds to the ``-l'' command-line option. A rela-
tive pathname is relative to the ``backlog-directory''
value.
backlog-directory
This key requires a pathname value. It specifies where
the current innfeed process should store backlog files.
This corresponds to the ``-b'' command-line option.
backlog-highwater
This key requires a positive integer value. It speci-
fies how many articles should be kept on the backlog
file queue before starting to write new entries to
disk.
backlog-ckpt-period
This key requires a positive integer value. It speci-
fies how many seconds between checkpoints of the input
backlog file. To small a number will mean frequent disk
accesses, to large a number will mean after a crash
innfeed will re-offer more already-processed articles
than necessary.
backlog-newfile-period
This key requires a positive integer value. It speci-
fies how many seconds between checks for externally
generated backlog files that are to be picked up and
processed.
dns-retry
The key requires a positive integer value. It defines
the number of seconds between attempts to re-lookup
host information that previous failed to be resolved.
dns-expire
The key requires a positive integer value. It defines
the number of seconds between refreshes of name to
address DNS translation. This is so long running
processes don't get stuck with stale data, should peer
ip addresses change..
close-period
The key requires a positive integer value. It is the
maximum number of seconds a connection should be kept
open. Some NNTP servers don't deal well with connec-
tions being held open for long periods.
gen-html
This key requires a boolean value. It specifies whether
the status-file should be HTML-ified.
status-file
This key requires a pathname value. It specifies the
pathname (relative to the backlog-directory) where the
periodic status of the innfeed process should be
stored. This corresponds to the ``-S'' command-line
option.
connection-stats
This key requires a boolean value. If the value is
true, then whenever the transmission statistics for a
peer are logged, then each active connection logs its
own statistics. This corresponds to the `` - z''
command-line option.
host-queue-highwater
This key requires a postive integer value. It defines
how many articles will be held internally for a peer
before new arrivals cause article information to be
spooled to the backlog file.
stats-period
This key requires a positive integer value. It defines
how many seconds innfeed waits between generating
statistics on transfer rates.
stats-reset
This key requires a positive integer value. It defines
how many seconds innfeed waits before resetting all
internal transfer counters back to zero (after logging
one final time). This is so a innfeed-process running
more than a day will generate ``final'' stats that will
be picked up by logfile processing scripts.
initial-reconnect-time
This key requires a postive integer value. It defines
how many seconds to first wait before retrying to
reconnect after a connection failure. If the next
attempt fails too, then the reconnect time is approxi-
mately doubled until the connection succeeds, or max-
reconnection-time is reached.
max-reconnect-time
This key requires an integer value. It defines the max-
imum number of seconds to wait between attempt to
reconnect to a peer. The initial value for reconnection
attempts is defined by initial-reconnect-time, and it
is doubled after each failure, up to this value.
GLOBAL PEER DEFAULTS
All the key/value pairs mentioned in this section must be
specified at global scope. They may also be specified inside
a group or peer definition.
article-timeout
This key requires a non-negative integer value. If no
articles need to be sent to the peer for this many
seconds, then the peer is considered idle and all its
active connections are torn down.
response-timeout
This key requires a non-negative integer value. It
defines the maximum amount of time to wait for a
response from the peer after issuing a command.
initial-connections
This key requires a non-negative integer value. It
defines the number of connections to be opened immedi-
ately when setting up a peer binding. A value of 0
means no connections will be created until an article
needs to be sent.
max-connections
This key requires positive integer value. It defines
the maximum number of connections to run in parallel to
the peer.
max-queue-size
This key requires a positive integer value. It defines
the maximum number of articles to process at one time
when using streaming to transmit to a peer. Larger
numbers mean more memory consumed as articles usually
get pulled into memory (see the description of use-
mmap).
streaming
This key requires a boolean value. It defines whether
streaming commands are used to transmit articles to the
peers.
no-check-high
This key requires a floating-point number which must be
in the range [0.0, 100.0]. When running transmitting
with the streaming commands, innfeed attempts an optim-
ization called ``no-CHECK'' mode. This involves not
asking the peer if it wants the article, but just send-
ing it. This optimization occurs when the percentage of
the articles the peer has accepted gets larger than
this number. If this value is set to 100.0, then this
effectivly turns off no-CHECK mode, as the percentage
can never get above 100.0. If this value is too small,
then the number of articles the peer rejects will get
bigger (and your bandwidth will be wasted). A value of
95.0 is usually pretty good. NOTE: In innfeed 0.9.3 and
earlier this value was in the range [0.0, 9.0].
no-check-low:
This key requires a floating-point number which must be
in the range [0.0, 100.0), and it must be smaller that
the value for no-check-high. When running in no-CHECK
mode, as described above, if the percentage of articles
the remote accepts drops below this number, then the
no-CHECK optimization is turned off until the percen-
tage gets above the no-check-high value again. If there
is small difference between this and the no-check-high
value (less than about 5.0), then innfeed may fre-
quently go in and out of no-CHECK mode. If the differ-
ence is too big, then it will make it harder to get out
of no-CHECK mode when necessary (wasting bandwidth).
Keeping this to between 5.0 and 10.0 less than no-
check-high is usually pretty good.
port-number
This key requires a postive integer value. It defines
the tcp/ip port number to use when connecting to the
remote.
backlog-limit
This key requires a non-negative integer value. If the
number is 0 then backlog files are allowed to grown
without bound when the peer is unable to keep up with
the article flow. If this number is greater than 0 then
it specifies the size (in bytes) the backlog file
should get truncated to when the backlog file reaches a
certain limit. The limit depends on whether backlog-
factor or backlog-limit-high is used.
backlog-factor
This key requires a floating point value, which must be
larger than 1.0. It is used in conjunction with the
peer key backlog-limit. If backlog-limit has a value
greater than zero, then when the backlog file gets
larger than the value backlog-limit * backlog-factor,
then the backlog file will be truncated to the size
backlog-limit. For example if backlog-limit has a value
of 1000000, and backlog-factor has a value of 2.0, then
when the backlogfile gets to be larger than 2000000
bytes in size, it will be truncated to 1000000 bytes.
The front portion of the file is removed, and the trim-
ing happens on line boundaries, so the final size may
be a bit less than this number. If backlog-limit-
highwater is defined too, then backlog-factor takes
precedence.
backlog-limit-highwater
This key requires a positive integer value that must be
larger than the value for backlog-limit. If the size of
the backlog file gets larger than this value (in
bytes), then the backlog file will be dhrunk down to
the size of backlog-limit. If both backlog-factor and
backlog-limit-highwater are defined, then the value of
backlog-factor is used.
PEER VALUES
As previously explained, the peer definitions can contain
redefinitions of any of the key/value pairs described in the
GLOBAL PEER DEFAULTS section above. There is one key/value
pair that is specific to a peer definition.
ip-name
This key requires a word value. The word is the host's
FQDN, or the dotted quad ip-address. If this value is
not specified then the name of the peer is taken to
also be its ip-name. See the entry for
data.ramona.vix.com in the example below.
RELOADING
If innfeed gets a SIGHUP signal, then it will reread the
config file. All values at global scope except for
``backlog-directory'' can be changed. Any new peers are
added and any missing peers have their connections closed.
EXAMPLE
Below is the sample innfeed.conf file.
#
# innfeed.conf file. See the comment block at the
# end for a fuller description.
#
##
## Global values. Not specific to any peer. These
## are optional, but if used will override the
## compiled in values. Command-line options used
## will override these values.
##
backlog-directory: /var/news/spool/innfeed
log-file: innfeed.log
debug-level: 0
status-file: innfeed.status
use-mmap: false
pid-file: innfeed.pid
connection-stats: false
news-spool: /var/news/spool/articles
gen-html: false
backlog-factor: 1.10
max-reconnect-time: 3600
##
## Global scope. Defaults for all peers. These
## must all exist. Any of them can be redefined
## inside a peer definition.
##
article-timeout: 600
response-timeout: 300
initial-connections: 1
max-connections: 5
max-queue-size: 25
streaming: true
no-check-high: 95.0
no-check-low: 90.0
port-number: 119
backlog-limit: 0
##
## Peers.
##
peer decwrl {
ip-name: news1.pa.dec.com
}
peer uunet {
ip-name: news.uunet.uu.net
max-connections: 10
}
peer data.ramona.vix.com {
# ip-name defaults to data.ramona.vix.com
streaming: false
}
peer bb.home.vix.com {
ip-name: 192.5.5.33
}
# Blank lines are ignored. Exerything after a '#'
# is ignored too.
#
# Format is:
# key : value
#
# See innfeed.conf(5) for a description of
# necessary & useful keys. Unknown keys and their
# values are ignored.
#
# Values may be a integer, floating-point, c-style
# single-quoted characters, boolean, and strings.
#
# If a string value contains whitespace, or
# embedded quotes, or the comment character
# (``#''), then the whole string must be quoted
# with double quotes. Inside the quotes, you may
# use the standard c-escape sequence
# (,07,
#
# Examples:
# eg-string: "NewConfigfile0
# eg-long-string: "A long string that goes
# over multiple lines. The
# newline is kept in the
# string except when quoted
# with a backslash # as here."
# eg-simple-string: A-no-quote-string
# eg-integer: 10
# eg-boolean: true
# eg-char: 'a'
# eg-ctrl-g: ' 07'
HISTORY
Written by James Brister <brister@vix.com> for InterNetNews.
This is revision 1.4, dated 1997/02/14.
SEE ALSO
innfeed(1), newsfeeds(5)
Man(1) output converted with
man2html