So you have a load balanced internet setup using multiple WAN Links that provide internet to your network. This is great except that it can often hide issues with WAN links by spreading the load across working links. Here is a simple solution to monitoring load balanced WAN links using Smokeping and a little bit of routing.

Setup

You have multiple WAN links such as ADSL, Wireless, Fibre, Ethernet over copper that you want to load balance for any number of reasons. Because the load balancing connections will automatically load balance around a failure you may want to be able to monitor each WAN link to identify issues on that link. For this example I will use the following information.

Network Setup.

  • 3 x WAN interfaces that are terminated onto a Sonicwall
  • A Sonicwall with a LAN IP of 192.168.50.250
  • A Linux (I use Ubuntu) server on the network with an IP address of 192.168.50.5. This doesn’t need to be powerful, most monitoring machines that I build are VMs with 512MB Memory and a 10GB Hard drive so any old machine will be suitable.
  • Sendmail and smokeping are already setup and configured correctly to be working.

You will also need one reliable and pingable IP on the internet for each WAN interface. In this example I have selected the following three.

  • 8.8.8.8
  • 8.8.4.4
  • 4.2.2.2

Smokeping Setup

Now that we have all the information we setup Smokeping to monitor those IP Addresses. Add the following code to your Targets configuration file to create a ‘Control’ Menu with the three graphs, one for each WAN Interface.

Notice that there are a few other entries in this Targets other then the targets that we discussed earlier. This is to assist in identifying if an issue is on a connection or a WAN interface. For example if there is packet loss on the Gateway connection then the issue is more likely on the LAN. if the LAN is solid but the WAN is reporting loss then the issue is most likely on the WAN.

Now add the following to the Alerts configuration file

Once setup and working you will start to see information appear in your smokeping setup for the monitored interfaces that should look something similar to this.

X1X2Monitor

A configured and working X1 and X2 ping test

Now that setup alone will not correctly monitor the individual WAN ports because the traffic will be automatically balanced to a working WAN interface in the event that an outage or issue does occur. The next steps allows the Sonicwall (or any load balancing device) to send the traffic down the appropriate WAN interface.

Sonicwall Setup

X1Setup

Example X1 route policy for Sonicwall (Click for larger view)

Once Smokeping is setup and working and you are receiving information for the probes the next step is to setup the routing that will allow the probes to monitor the correct interface.

  1. Once in your Sonicwall interface select Routing under Network (I am going to perform this setup in a Sonicwall but it could be done with any load balancing device.)
  2. Add a route policy with the following settings.
    Source = LAN Subnets
    Destination = Create a new Object and call it X1 Ping Test (or X2 etc) This will be a WAN object with an IP of 8.8.8.8 (or appropriate IP)
    Service = ICMP
    Gateway = X1 Default Gateway (or appropriate interface Gateway)
    Interface = X1 (or appropriate interface e.g. X2)
    Metric = 10
    Comment = X1 Interface monitoring
    Disable route when interface disconnected = unchecked (This will mean that if the interface is down the probe will stop)
    Leave Allow VPN Path to take precedence,Permit TCP acceleration and Probe to None / unchecked
  3. Repeat this for each WAN interface that you have using the appropriate information in each option (Gateway, Interface and Probe IP) until you have added all the routes required
X1 and X2 Routes configured ready for monitoring

X1 and X2 Routes configured ready for monitoring

And thats all that is required. This setup will allow each probe to be routed correctly down the appropriate interface. By un-checking the option to disable the route when interface is disconnected means that the probe will start to fail once an interface is disconnected like the image below. This will also send an email alert advising that there is an issue on X1 interface.

During a test the X1 interface was disconnected to test that the reporting was working correctly

During a test the X1 interface was disconnected to test that the reporting was working correctly

While this type of monitoring works great to identify issues with a WAN interface it can also be adapted to catch high packet loss which might indicate that a connection is under load or something is wrong with the connection itself.