Capsa network analyzer: Full detection of DDoS attacks One of our clients turned to us for help that their network suffered a disconnection but they just failed to figure out the reason. With our assistance of Capsa network analyzer, it was concluded that the network was brought down by a compromised machine which was involved in a typical DDoS attack. In this article, we will see how the compromised machine works, and go further to see how the switches and routers work in this particular case. Network Infrastructure The client, in the line of chemical industry, is not a big network. There are just about 10 switches and a little more than 150 computers in their network. Some machines are working on IPX and some on TCP/IP. And there is no VLAN implementation. Like other small networks, they connect the computers all the way up from switches to the xDSL router. In this network, only a few machines are authorized to access to the Internet, which is connected from a xDSL router to multiple switches. The xDSL router has its own firewall enabled and all the Internet-accessible machines are equipped with anti-virus programs. Network Symptom 1. Sometimes the network becomes very lagging, which makes all the network-required applications work improperly. 2. All the switches’ indicator lights are wildly flashing. 3. The Ping command doesn’t work even between the machines in the network. 4. The connection will gradually recover a little when unplug some machines off. 5. The disconnection happens randomly. Possible Causes As all the switches’ indicator lights wildly flashing and no Ping reply got from any machine in the network, it’s first assumed that these are a big volume of broadcast traffics that consume the whole bandwidth. To prove it, we deploy a Capsa network analyzer to capture the traffic of the network immediately. The packets, however, show that there are a big volume of IP packets instead of the assumed excessive broadcast packets. Temporary Solution Our prediction is wrong. Anyway, we get hints that these IP packets are abnormal. After analysis, we understand that these packets are sent from machine 172.*.*.11, which has access to the Internet, to a remote public IP address with the estimated speed of 10,000 packets per second. The network goes back to normal when we told the network admin to isolate the machine from the network. In-depth Analysis Even the problem is solved and it’s the typical machine compromised in a DDoS attack, but we’ll take the change to go further. We all understand the packets to a public IP address should be captured by the SPAN port with Capsa network analyzer deployed. Apparently, these packets are broadcasted out, why? 1. The machine is compromised in a DDoS attack To figure out the real causes, we install a Capsa network analyzer on the compromised machine. After capturing some packets, we understand that the machine automatically initiates a connection to a FTP server and download a text file named ddos.txt from it, once it’s connected to the Internet. We can easily find the text file and see it only contains an IP address with port 80, and the IP address definitely is the targeted IP address. In order to penetrate the firewall, the packets are going out with port 80. This machine, unfortunately, has been subjected to a compromised client in a DDoS attack. And also we find that the machine randomly retries to download the text file when the downloaded one is empty, until it gets an IP address and port. Since the targeted address is received, the machine is geared to pour the targeted with massive packets. That also answers why the network-down happens irregularly. 2. xDSL router is overwhelmed Following the packets, we’ll see the packets are addressed to the Internet and they should go from the machine – switch – router – the Internet. How do we explain the broadcast phenomenon? There is only one answer that the switch can’t find the record of the router’s MAC address in its CAM table, so that it broadcast all the packets out to all physical ports. No doubt that the switch has this record at the first beginning. When the compromised machine starts its attack, the massive packets travel at the sequence of itself -> switch -> maybe another switch -> xDSL router -> the target. Now, only the related switches and the router are overwhelmed by the packets. All the machines except the compromised have trouble to communicate with the Internet. 3. Broadcast Storm Still we haven’t answered why the switch dropped the item recording the physical port information. There are two scenarios: A. If the switch doesn’t get data from a device, about defaulted 5 minutes, it recognizes the device as offline. The switch deletes the item recording the device’s physical address information immediately. B. When the switch detects the network with STP and finds the network topology changes, it locks all the outdated CAM records. Then it notifies all the switches connected and deletes the records in 15 seconds. The outdated means the switch hasn’t received any packet from its recorded physical address since last check time. Then all the packets targeted to the physical address will be broadcast out to all physical ports. 4. The Real Causes Throughout the analysis above, we’ll understand the cause of this network down. When the compromised machine starts its attack, the little xDSL router has to get more than 10,000 packets delivered in a second which makes it impossible to send any packets to any switch anymore. It means the router only receives packets but fails to send any packet into the Intranet. Then it’s reasonable to see the switch deletes the record of the router’s physical address by its time, maybe 15 seconds or at the longest 5 min. The attack carries on, and the destination addresses of the packets are the router’s address. After searching the CAM table, the switch failed to find the router’s address, and then it broadcast the packets to all physical ports. The traffic of the network grows by multiples suddenly, and everything goes wildly. No web pages, no email, no download and even no Ping replies. Final Solution Every time when we think the problem has been solved, we need to try to go deeper to see what the once-for-all solution for this kind of problem is. Apparently, this network problem is caused by a compromised DDoS client. It discloses the importance of examining the inappropriate network infrastructure as well. The final solutions for preventing this kind of network problem are listed below: 1. Disconnect the compromised client from the network at once. If possible, reload its operation system to eliminate the virus or worm. 2. Educate your users how to protect themselves from virus. 3. Adjust your network topology by adopting VLAN. Organize the machines with Internet access into a specific VLAN group to prevent affecting other users from other VLAN groups. 4. Configure the switch’s ports connecting to the computers as STP Portfast. In order that they don’t operate STP to prevent unnecessary topology changes.
