A Guide to INN ( Internet Net News )

This guide includes overview of INN server, simple description of configuration files format used by INN, way to add new INN feeders, description of GUP ( Group Update ), contents log files of INN and Unofficial INN CONFIGURATION FAQ.

Overwiev of INND daemon

Description

Innd, the InterNetNews daemon, handles all incoming NNTP feeds. It reads the active(5), newsfeeds(5), and hosts.nntp(5) files into memory. It then opens the NNTP port to receive articles from remote sites, a Unix-domain stream socket to receive articles from local processes such as nnrpd(8) and rnews(1), and a Unix-domain datagram socket for use by ctlinnd(8) to direct the server to perform cer- tain actions. It also opens the history(5) database and two log files to replace its standard output and standard error. If the ``-p'' flag is used, then the NNTP port is assumed to be open on the specified descriptor. (If this flag is used, then innd assumes it is running with the proper permissions and it will not call chown(2) on any files or directories it creates.)

Once the files and sockets have been opened, innd waits for connections and data to be ready on its ports by using select(2) and non-blocking I/O. If no data is available, then it will flush its in-core data structures. The default number of seconds to timeout before flushing is 300 seconds. This timeout may be changed by using the ``-t'' flag.

To limit the number of incoming NNTP connections, use the ``-i'' flag. A value of zero will suppress this check. The default is 50.

To limit the number of files that will be kept open for out- going file feeds, use the ``-o'' flag. The default is the number of available descriptors minus some reserved for internal use.

To limit the size of an article, use the ``-l'' flag. If this flag is used, then any article bigger than size bytes will be rejected. The default is no checking, which can also be obtained by using a value of zero.

Innd rejects articles that are too old. While this behavior can be controlled by the history database, occasionally a site dumps a batch of very old news back onto the network. Use the ``-c'' flag to specify a cutoff. For example ``-c21'' will reject any articles that were posted more than 21 days ago. A value of zero will suppress this check. The default is 14 days.

Innd normally puts itself into the background, sets its standard output and error to log files, and disassociates itself from the terminal. Using the ``-d'' flag instructs the server to not do this, while using the ``-f'' flag just leaves the server running the foreground. The logs are nor- mally buffered; use the ``-u'' flag to have them unbuffered.

To start the server in a paused or throttled state (see ctlinnd(8)) use the ``-m'' flag to set the initial running mode. The argument should start with a single letter g, p, or t, to emulate the ``go,'' ``pause,'' or ``throttle'' com- mands, respectively.

If the ``-r'' flag is used, the server will renumber the active file as if a ``renumber'' command were sent.

If the ``-s'' flag is used, then innd will not do any work but will instead just check the syntax of the newsfeeds file. It will exit with an error status if there are any errors; the actual errors will be reported in syslog(3).

If innd gets an NOSPC error (see intro(2)) while trying to write the active file, an article file, or the history data- base, it will send itself a ``throttle'' command. This will also happen if it gets too many I/O errors while writing to any files.

The ``-n'' flag specifies whether or not pausing or throt- tling the server should also disable future newsreading processes. A value of ``y'' will make newreaders act as the server, a value of ``n'' will allow newsreading even when the server is not running. The default is to allow reading.

If the ``-S'' flag is used, then innd will run in ``slave'' mode. When running as a slave, the server will only accept articles from the specified host, which must use the ``xreplic'' protocol extension described below. Note that the host must either appear in the hosts.nntp file, or the server must be started with the ``-a'' flag.

By default, if a host if not mentioned in the hosts.nntp file, then the connection is handed off to nnrpd. If the ``-a'' flag is used, then any host can connect and transfer articles.

If the ``-x'' flag is used, then an Xref header will be added to all articles even if they are not cross-posted.

Inndstart is a small front-end program that opens the NNTP port, sets its userid and groupid to the news maintainer, and then execs innd with the ``-p'' flag and a minimal secure, environment. This is a small, easily-understood front-end program that can be used if a site does not want to run innd with root privileges.

Control Messages

Arriving articles that have a Control header or have a Sub- ject header that starts with the five characters ``cmsg '' are called control messages. Except for the cancel message, these messages are implemented by external programs in the /usr/lib/news/bin/control directory. (Cancel messages update the history database, so they must be handled inter- nally; the cost of syncing, locking, then unlocking would be too high given the number of cancel messages that are received.)

When a control message arrives, the first word of the text is converted to lowercase and used as the name of the pro- gram to execute; if the named program does not exist, then a program named default is executed. All control programs are invoked with four parameters. The first is the address of the person who posted the message; this is taken from the Sender header. If that header is empty, then it is taken from the From header. The second parameter is the address to send replies to; this is taken from the Reply-To header. If that header is empty then the poster's address is used. The third parameter will be a name under which the article is filed, relative to the news spool directory. The fourth parameter is the host that sent the article, as specified on the Path line.

The distribution of control message is also different from those of standard articles. Control messages are normally filed in the newsgroup named control. They can be filed in subgroups, however, based on the control message command. For example, a newgroup mes- sage will be filed in control.newgroup if that group exists, otherwise it will be filed in control.

Sites may explicitly have the ``control'' newsgroup in their subscription list, although it is usually best to exclude it. If a control message is posted to a group whose name ends with the four characters ``.ctl'' then the suffix is stripped off and what is left is used as the group name. For example, a cancel message posted to ``news.admin.ctl'' will be sent to all sites that subscribe to ``control'' or ``news.admin.'' Newgroup and rmgroup messages receive addi- tional special treatment. If the message is approved and posted to the name of the group being created or removed, then the message will be sent to all sites whose subscrip- tion patterns would cause them to receive articles posted in that group.

Protocol Differences

Innd implements the NNTP commands defined in RFC 977, with the following differences:

  1. The ``list'' maybe followed by an optional ``active'', ``active.times'', or ``newsgroups'' argument. This common extension is not fully supported; see nnrpd(8).
  2. The ``authinfo user'' and ``authinfo pass'' commands are implemented. These are based on the reference Unix implementation; no other documentation is available.
  3. A new command, ``mode reader'', is provided. This com- mand will cause the server to pass the connection on to nnrpd. The command ``mode query'' is intended for future use, and is currently treated the same way.
  4. A new command, ``xreplic news.group:art[,news.group:art]'', is provided. This is similar to the ``ihave'' command (the same reply codes are used) except for the data that follows the command word. The data consists of entries separated by a single comma. Each entry consists of a newsgroup name, a colon, and an article number. Once processed, the article will be filed in the newsgroup and article numbers specified in the command.
  5. A new command, ``xpath messageid'', is provided. The server responds with a 223 response and a space- separated list of filenames where the article was filed.
  6. The only other commands implemented are ``head'', ``help'', ``ihave'', ``quit'', and ``stat''.

Header Modification

Innd modifiers as few article headers as possible, although it could be better in this area.

The following headers, if present, are removed: Date-Received Posted Posting-Version Received Relay-Version Empty headers and headers that consist of nothing but whi- tespace are also dropped.

The local site's name and an exclamation point are prepended to the Path header.

The Xref header is removed. If the article is cross-posted a new header is generated.

The Lines header will be added if it is missing.

Innd does not rewrite incorrect headers. For example, it will not replace an incorrect Lines header, but will reject the article.

Logging

Innd reports all incoming articles in its log file. This is a text file with a variable number of space-separated fields in one of the following formats:

mon dd hh:mm:ss.mmm + feed site...

mon dd hh:mm:ss.mmm j feed site...

mon dd hh:mm:ss.mmm c feed site...

mon dd hh:mm:ss.mmm - feed reason...

The first three fields are the date and time to millisecond resolution. The fifth field is the site that sent the arti- cle (based on the Path header) and the sixth field is the article's Message-ID; they will be a question mark if the information is not available.

The fourth field indicates whether the article was accepted or not. If it is a plus sign, then the article was accepted. If it is the letter ``j'' then the article was accepted, but all of newsgroups have an ``j'' in their active field, so the article was filed into the ``junk'' newsgroup. If the fourth field is the letter ``c'', then a cancel message was accepted before the original article arrived. In all three cases, the article has been accepted and the ``site..'' field contains the space-separated list of sites to which the article is being sent.

If the fourth field is a minus sign, then the article was rejected. The reasons for rejection include: "%s" header too long "%s" wants to cancel <%s> by "%s" Article exceeds local limit of %s bytes Article posted in the future -- "%s" Bad "%s" header Can't write history Duplicate Duplicate "%s" header EOF in headers Linecount %s != %s +- %s Missing %s header No body No colon-space in "%s" header No space Space before colon in "%s" header Too old -- "%s" Unapproved for "%s" Unwanted newsgroup "%s" Unwanted distribution "%s" Whitespace in "Newsgroups" header -- "%s" Where ``%s'', above, is replaced by more specific informa- tion.

Innd also makes extensive reports through syslog. The first word of the log message will be the name of the site if the entry is site-specific (such as a ``connected'' message). The first word will be ``ME'' if the message relates to the server itself, such as when a read error occurs.

If the second word is the four letters ``cant'' then an error is being reported. In this case, the next two words generally name the system call or library routine that failed, and the object upon which the action was being per- formed. The rest of the line may contain other information.

In other cases, the second word attempts to summarize what change has been made, while the rest of the line gives more specific information. The word ``internal'' generally indi- cates an internal logic error.

History

Written by Rich $alz for InterNetNews. This is revision 1.28, dated 1993/03/18.

Description of configuration files format

There are some files contains configuration data for INN.

inn.conf
The file /usr/lib/news/conf/inn.conf is used to determine various parameters. Blank lines and lines starting with a number sign (``#'') are ignored. All other lines specify parameters that may be read, and should be of the following form: name : [optional whitespace] value Everything after the whitespace and up to the end of the line is taken as the value; multi-word values should not be put in quotes. The case of names is significant - server is not the same as Server or SERVER. Some parameters specified in the file may be overridden by environment variables, and some file parameters may be used to mask real data, such as when hiding a cluster of hosts behind a single electronic mail hostname. The current set of parameters is as follows:
fromhost
This is the name of the host to use when building the From header line. The default is the fully-qualified domain name of the local host. The value of the FROMHOST environment variable, if it exists, overrides this.
moderatormailer
This names the default machine that contains forwarding aliases for all moderated groups. It is only used if the moderators(5) file doesn't exist, or if the group is not matched by that file. The value is interpreted as a pattern match; see moderators(5).
organization
This specifies what to put in the Organization header if it is blank. The value of the ORGANIZATION environ- ment variable, if it exists, overrides this.
pathhost
This specifies how to name the local site when building the Path header line. The default is the fully- qualified domain name of the local host.
server
This specifies the name of the NNTP server to which an article should be posted. The value of the NNTPSERVER environment variable, if it exists, overrides this.
domain
This should be the domain name of the local host. It should not have a leading period, and it should not be a full host address. It is used only if the GetFQDN routine in libinn(3) cannot get the fully-qualified domain name by using either the gethostname(2) or gethostbyname(3) calls. The check is very simple; if either routine returns a name with a period in it, then it is assumed to have the full domain name.
Three parameters are used only by nnrpd when accepting post- ings from clients:
mime-version
If this parameter is present, then nnrpd will add the necessary MIME (Multipurpose Internet Mail Extensions) headers to all any articles that do not have a Mime- Version header. This parameter specifies the MIME ver- sion, and should normally be ``1.0''.
mime-contenttype
If MIME headers are being added, this parameter speci- fies the value of the Content-Type header. The default value is ``text/plain; charset=US-ASCII.''
mime-encoding
If MIME headers are being added, this parameter speci- fies the value of the Content-Transfer-Encoding header. The default value is ``7bit.''

EXAMPLE

  • fromhost: foo.com
  • moderatormailer: %s@uunet.uu.net
  • organization: Foo, Incorporated
  • #pathhost -- use FQDN.
  • server: news.foo.com
  • domain: foo.com
hosts.nntp
The file /usr/lib/news/conf/hosts.nntp is read by innd(8) to get the list of hosts that feed the local site Usenet news using the NNTP protocol. The server reads this file at start-up or when directed to by ctlinnd(8). When a hosts connects to the NNTP port of the system on which innd is running, the server will do a check to see if their Internet address is the same as one of the hosts named in this file. If the host is not mentioned, then innd will spawn an nnrpd(8) to process the connection, with the accepted con- nection on standard input and standard output. Comments begin with a number sign (``#'') and continue through the end of the line. Blank lines and comments also ignored. All other lines should consist of two or three fields separated by a colon.
The first field
should be either an Internet address in dotted-quad format or an address that can be parsed by gethostbyname(3). If a host's entry has multiple addresses, all of them will be added to the access list.
Second field,
may be blank, is the minimal disk free space then innd will be rejecting remote connection from this host till disk space less then defined value.

Note: Origin inn1.4 server don't support this field.

The third field
field, which may be blank, is the password the foreign host is required to use when first connecting.
The fourth field,
which may be omitted, is a list of newsgroups to which the host may post articles. This list is parsed as a newsfeeds(5) subscription list; groups not in the list are ignored.
Since innd is usually started at system boot time, the local nameserver may not be fully operational when innd parses this file. As a work-around, a ctlinnd ``reload'' command can be performed after a delay of an hour or so. It is also possible to provide both a host's name and its dotted-quad address in the file.

EXAMPLE

  • ## FOO has a password, UUNET doesn't.
  • ## UUNET cannot post to local group.s
  • ## These are comment lines.
  • news.foo.com::magic
  • uunet.uu.net:::!foo.*
  • news.relcom.net:20000:
nnrp.access
The file /usr/lib/news/conf/nnrp.access specifies the access control for those NNTP sites that are not handled by the main InterNetNews daemon innd(8). The nnrpd(8) server reads it when first spawned by innd. Comments begin with a number sign (``#'') and continue through the end of the line. Blank lines and comments are ignored. All other lines should consist of five fields separated by colons:

hosts:perms:username:password:patterns

The first field
is a wildmat(3)-style pattern specifying the names or Internet address of a set of hosts. Before a match is checked, the client's hostname (or its Internet address if gethostbyaddr(3) fails) is converted to lowercase. Each line is matched in turn, and the last successful match is taken as the correct one.
The second field
is a set of letters specifying the permis- sions granted to the client. The perms should be chosen from the following set: R The client can retrieve articles P The client can post articles
The third and fourth fields
specify the username and pass- word that the client must use to authenticate themselves before the server will accept any articles. Note that no authentication (other then a matching entry in this file) is required for newsreading. If they are empty, then no pass- word is required. Whitespace in these fields will result in the client being unable to properly authenticate themselves and may be used to disable access.
The fifth field
is a set of patterns identifying the news- groups that the client is allowed to access. The patterns are interpreted in the same manner as the newsfeeds(5) file. The default, however, denies access to all groups.
The access file is normally used to provide host-level access control for reading and posting articles. There are times, however, when this is not sufficient and user-level access control is needed. Whenever an NNTP ``authinfo'' command is used, the nnrpd server re-reads this file and looks for a matching username and password. If the local newsreaders are modified to send the ``authinfo'' command, then all host entries can have no access and specific users can be granted the appropriate read and post access.

EXAMPLE

  • ## host:perm:user:pass:groups
  • ## Default is no access.
  • *:: -no- : -no- :!*
  • ## FOO hosts have no password, can read anything.
  • *.foo.com:Read Post:::*
  • ## A related workstation can't access FOO newsgroups.
  • lenox.foo.net:RP:martha:hiatt:*,!foo.*
expire.ctl
The file /usr/lib/news/conf/expire.ctl is the default con- trol file for the expire(8) program, which reads it at start-up. Blank lines and lines beginning with a number sign (``#'') are ignored. All other lines should be in one of two formats. The first format specifies how long to keep a record of fully-expired articles. This is useful when a newsfeed intermittently offers older news that is not kept around very long. (The case of very old news is handled by the ``-c'' flag of innd(8).) There should only be one line in this format, which looks like this: /remember/:days Where days is a floating-point number that specifies the upper limit to remember a Message-ID, even if the article has already expired. (It does not affect article expira- tions.) Most of the lines in the file will consist of five colon- separated fields, as follows: pattern:modflag:keep:default:purge The pattern field is comma-separated set of single wildmat(3)-style patterns that specify the newsgroups to which the rest of the line applies. Since the file is interpreted in order, the most general patterns should be specified first, and the most specific patterns should be specified last. The modflag field can be used to further limit newsgroups to which the line applies, and should be chosen from the fol- lowing set: M Only moderated groups U Only unmoderated groups A All groups The next three fields are used to determine how long an article should be kept. Each field should be either a number of days (fractions like ``8.5'' are allowed) or the word ``never.'' The most common use is to specify the default value for how long an article should be kept. The first and third fields - keep and purge - specify the boun- daries within which an Expires header will be honored. They are ignored if an article has no Expires header. The fields are specified in the file as ``lower-bound default upper- bound,'' and they are explained in this order. Since most articles do not have explicit expiration dates, however, the second field tends to be the most important one. The keep field specifies how many days an article should be kept before it will be removed. No article in the newsgroup will be removed if it has been filed for less then keep days, regardless of any expiration date. If this field is the word ``never'' then an article cannot have been kept for enough days so it will never be expired. The default field specifies how long to keep an article if no Expires header is present. If this field is the word ``never'' then articles without explicit expiration dates will never be expired. The purge field specifies the upper bound on how long an article can be kept. No article will be kept longer then the number of days specified by this field. All articles will be removed after then have been kept for purge days. If purge is the word ``never'' then the article will never be deleted. It is often useful to honor the expiration headers in arti- cles, especially those in moderated groups. To do this, set keep to zero, default to whatever value you wish, and purge to never. To ignore any Expires header, set all three fields to the same value. There must be exactly one line with a pattern of ``*'' and a modflags of ``A'' - this matches all groups and is used to set the expiration default. It should be the first expira- tion line.

EXAMPLE

Way to add new INN feeders

Description of GUP ( Group Update )

The automated news update program accepts the following commands: site, include, exclude, delete, list, newsgroups, help and quit. The 'site' command *must* come prior to any of the other commands After that any order of commands is acceptable. Note that commands are processed in order and that the 'list' command displays the current state as affected by any preceeding commands.

Syntax:

Where:

sitename
must be a valid site registered in the gup config file
password
Must match the registered password
pattern
Is a single 'wildmat' pattern of a newsgroup.
In the case of 'Include' and 'Exclude' it is matched against the active file. With the 'delete' command. is matched against the current group list. The 'include' command adds all groups that match the pattern into your current group list. The 'exclude' command is typically used in conjunction with a wildcard 'include' command, eg: include comp.sys.* exclude comp.sys.weirdosystems In the above example, all of the groups in the comp.sys hierarchy will be included, except for comp.sys.weirdosystems. In other words it's a merely a convenient way of refining a large Include list. The 'delete' command removes all matching 'include' and 'exclude' patterns from the current group list. newsgroups lists out all the valid newsgroups that match the pattern. quit is used to stop gup from parsing the rest of the mail, that typically has signatures and such baggage on the end of it
Contents log files of INN

Directory are containing the log files is /usr/lib/news/log. There are some log files such as:

news.error
Reflects all fatal erros.
news.crit
Reflects all crit errors.
news.notice
There is information sended by news programs. In patiqular connect a sytes to innd daemon, information from other news programs ( innxmit, rnews e.t.c ).
news.state
Reflects current state of innd daemon. You can see this information by click Operational Status of the INN Server on the first page of WWW server.
INNXMIT
INNXMIT dirrectory are containing a log files of innxmit programs. All files are named according name of site is feeded by innxmit.
expire.log
Contents of the file is log information from expire program.