On 2012-06-12, at 04:14, Jamie Baddeley wrote:

ISTR a tool that some dodgy geezer called jabley wrote years ago called "aggregate" or "aggregate-ios" that might help. Rumour has it he lurks on this list so he might be able to expound it's virtues.

I dare say Joe might insist using things like awk or sed to help with parsing the input file or something.

aggregate-ios was in fact once an awk script. But then Michael Shields at AboveNet re-wrote it in perl. aggregate is written in C, though, which you'd think would make it fast and efficient. You'd think. ftp://ftp.isc.org/isc/aggregate/aggregate-1.6.tar.gz Joe AGGREGATE(1) AGGREGATE(1) NAME aggregate - optimise a list of route prefixes to help make nice short filters SYNOPSIS aggregate [-m max-length] [-o max-opt-length] [-p default-length] [-q] [-t] [-v] DESCRIPTION Takes a list of prefixes in conventional format on stdin, and performs two optimisations to attempt to reduce the length of the prefix list. The first optimisation is to remove any supplied prefixes which are superfluous because they are already included in another supplied pre- fix. For example, 203.97.2.0/24 would be removed if 203.97.0.0/17 was also supplied. The second optimisation identifies adjacent prefixes that can be com- bined under a single, shorter-length prefix. For example, 203.97.2.0/24 and 203.97.3.0/24 can be combined into the single prefix 203.97.2.0/23. OPTIONS -m max-length Sets the maximum prefix length for entries read from stdin max_length bits. The default is 32. Prefixes with longer lengths will be discarded prior to processing. -o max-opt-length Sets the maximum prefix length for optimisation to max-opt- length bits. The default is 32. Prefixes with longer lengths will not be subject to optimisation. -p default-length Sets the default prefix length. There is no default; without this option a prefix without a mask length is treated as invalid. Use -p 32 -m 32 -o 32 to aggregate a list of host routes specified as bare addresses, for example. -q Sets quiet mode -- instructs aggregate never to generate warning messages or other output on stderr. -t Silently truncate prefixes that seem to have an inconsistent prefix: e.g. an input prefix 203.97.2.226/24 would be truncated to 203.97.2.0/24. Without this option an input prefix 203.97.2.226/24 would not be accepted, and a warning about the inconsistent mask would be generated. -v Sets verbose mode. This changes the output format to display the source line number that the prefix was obtained from, together with a preceding "-" to indicate a route that can be suppressed, or a "+" to indicate a shorter-prefix aggregate that was added by aggregate as an adjacency optimisation. Note that verbose output continues even if -q is selected. DIAGNOSTICS Aggregate exits 0 on success, and >0 if an error occurs. EXAMPLES The following list of prefixes: 193.58.204.0/22 193.58.208.0/22 193.193.160.0/22 193.193.168.0/22 193.243.164.0/22 194.126.128.0/22 194.126.132.0/22 194.126.134.0/23 194.151.128.0/19 195.42.240.0/21 195.240.0.0/16 195.241.0.0/16 is optimised as followed by aggregate (output shown using the -v flag): aggregate: maximum prefix length permitted will be 24 [ 0] + 193.58.204.0/21 [ 1] - 193.58.204.0/22 [ 2] - 193.58.208.0/22 [ 3] 193.193.160.0/22 [ 4] 193.193.168.0/22 [ 5] 193.243.164.0/22 [ 0] + 194.126.128.0/21 [ 6] - 194.126.128.0/22 [ 7] - 194.126.132.0/22 [ 8] - 194.126.134.0/23 [ 9] 194.151.128.0/19 [ 10] 195.42.240.0/21 [ 0] + 195.240.0.0/15 [ 11] - 195.240.0.0/16 [ 12] - 195.241.0.0/16 Note that 193.58.204.0/22 and 193.58.208.0/22 were combined under the single prefix 193.58.204.0/21, and 194.126.134.0/23 was suppressed because it was included in 194.126.132.0/22. The number in square brackets at the beginning of each line indicates the original line num- ber, or zero for new prefixes that were introduced by aggregate. The output without the -v flag is as follows: 193.58.204.0/21 193.193.160.0/22 193.193.168.0/22 193.243.164.0/22 194.126.128.0/21 194.151.128.0/19 195.42.240.0/21 195.240.0.0/15 SEE ALSO aggregate-ios(1) HISTORY Aggregate was written by Joe Abley <jabley(a)mfnx.net>, and has been rea- sonably well tested. It is suitable for reducing customer prefix fil- ters for production use without extensive hand-proving of results. Autoconf bits were donated by Michael Shields <michael.shields(a)mfn.com>. The -t option was suggested by Robin John- son <robbat2(a)fermi.orbis-terrarum.net>, and the treatment of leading zeros on octet parsing was changed following comments from Arnold Nip- per <arnold(a)nipper.de>. An early version of aggregate would attempt to combine adjacent pre- fixes regardless of whether the first prefix lay on an appropriate bit boundary or not (pointed out with great restraint by Robert Noland <rnoland(a)2hip.net>). BUGS Common unix parsing of IPv4 addresses understands the representation of individual octets in octal or hexadecimal, following a "0" or "0x" pre- fix, respectively. That convention has been deliberately disabled here, since resources such as the IRR do not follow the convention, and con- fusion can result. For extremely sensitive applications, judicious use of the -v option together with a pencil and paper is probably advisable. Joe Abley 2001 November 2 AGGREGATE(1) [monster:~]%