I spend two and a half hours last night chasing a ghost in our system... a larger customer of ours swore the system was not working at 10:30pm and he was so mad and going to leave us...blah blah blah.  Well I got on the system last night at 9:00pm and started gathering data...and about 45 minutes later, Rama got on as well...and started gathering data too.  So we checked everything...application servers, database servers, web servers, load balancers, internet throughput...nothing indicated there was the hint of a problem....this was of course backed up by our investigating of the logs which again showed no errors or anything to indicate there was a problem.  I've got two servers that monitor our application and if there's any problems with connecting I'll know in a matter of minutes.  I was able to employ a technique known as port cloning, which is a feature of the more advanced switches and routers.  So with cloning the port that dumps off to the internet I am able to connect that to a Linux server, put the interface in promiscuous mode and use several great tools to track down bandwidth consumption.  My favorite is iptraf, which will show me all the connections...allow me to sort by consumption and see what ports and IP's things are happening on.  I've used this technique in the past, but didn't have to clone the port on the switch because the network gateway was a Linux server so I just ran iptraf right there and could see the comings and goings...but now I can see EVERYthing.  Last week we had a bandwidth hog that was killing our bandwidth and it allowed me to track him down in less then a minute...I got the machine and closed the app that was doing it and once again..saved the day...damn I'm good!