[H-GEN] IP Traffic Monitoring

[Tony Nugent]

[ Humbug *General* list - semi-serious discussions about Humbug and     ]
[ Unix-related topics. Posts from non-subscribed addresses will vanish. ]

On Wed Sep 10 2003 at 09:34, Russell Stuart wrote:

> I need some pointers for debugging an ISDN link.  The link is currently
> overloaded, and I want to know why.

While I manage a site using a 128k onramp2 isdn connection[1], I
wouldn't call myself a guru but (hopefully) I can point you in the
right directions...

  [1] a linux box with an idsn card on their internet gateway,
  relatively easy to setup with the right configuation magic, and
  the only hassle has been that one of the lines occasionally hangs
  up (I think when someone [accidently?] dials it's phone number).

> The General Interface Statistics screen of iptraf says that approx 50%

(iptraf?  you really trust what iptraf says?  It may be prudent to
confirm its conclusions before going off on a wild goose-chase :)

> of the traffic falls under the "BadIp" heading, which the manual says
> are IP packets with checksum errors.  An "ip -statistics link" says
> there are no errors, but I suspect that mean link layer errors, not
> layer 3/4 errors.  Is there some other way to confirm there are IP
> checksum errors?

Have you tried doing a tcpdump on the idsn interface? eg:
	tcpdump -ni ippp0 icmp

> Given that there are, is there some way to see the contents of those
> packets?  Not unsurprisingly, they don't seem to get to tcpdump.

oh, you have :)  ok, these packets must be getting rejected/dropped
at the kernel level[2] - but perhaps you are not looking at the
right things?  For example, icmp or arp (or other) packets being
might be useful as symptoms to diagnose the problem (indeed, an
over-generation of such traffic may *be* the problem).

SNMP might also be useful for gathering statistics (snmpwalk can
give you an amazing amount of information - but please make sure
that the daemon is firewalled/tcp-wrapped for localhost access
only).

Also, you should be able to gather "real" (crude raw) statistics
from the relevant /proc/ files (eg "watch cat /proc/net/dev" and
other similar spells).  You might also need to look at doing some
tweaking to the kernel settings in /proc/sys/net/ipv4/*[2].

  [2] the files in the /proc/sys/net/ipv4/ heirachy can be used to
  control the generation of, and response to, these sorts of
  packets.  For example, it might be possible to make the kernel
  accept these packets even if they would normally be
  rejected/dropped (eg, such as the conf/default/rp_filter setting
  which controls the dropping of packets arriving from an IP on an
  interface that is not the route back to the source host).

> I would like to know where they are coming from.  They don't seem to be
> coming from the sources I expect, as quick check shows the links from
> those sources have bad latency, as you would expect from an overloaded
> link, but no dropped packets.

Are both 64k channels up (assuming 128k onramp2)?  Check with:
"isdnctrl status ippp0 ; isdnctrl status ippp1".  Perhaps you might
find other features of the isdnctrl utility useful for debugging and
tweaking your isdn settings (it has a man page and descriptive help
output).

Is the routing table correct? (for you to check eg: "ip r sh ta all")

Also, there is an iptables option to identify "invalid" packets
(--state INVALID), you might try to see if you can log/dump these
packets (if they are being accepted by the kernel).

  You could then drop these packets with netfilter, or (another
  possible alternative) fwmark them in the mangle table then use
  "/sbin/ip rule" to re-route them to another routing table where
  you can send them elsewhere - perhaps with a "default blackhole"
  to drop them?

Finally, who is your idsn provider?  Do they have technical staff
who can help, or web management pages that allow you to check on
things like your network statistics?  (Telstra does, on both
counts... once you finally reach them, I have generally found their
tech guys very helpful).

Hope this helps (at least for ideas).  Good luck.

Cheers
Tony

--
* This is list (humbug) general handled by majordomo at lists.humbug.org.au .
* Postings to this list are only accepted from subscribed addresses of
* lists 'general' or 'general-post'.  See http://www.humbug.org.au/