Delivery-Date: Sun Apr 15 09:17:29 2007
Message-Id: <200704151617.l3FGHdkr005973@bourbon.usc.edu>
To: cs551@merlot.usc.edu
Subject: Re: [e2e] Simulator for wireless network 
Date: Sun, 15 Apr 2007 09:17:39 -0700
From: william@bourbon.usc.edu

Hi,

I'm just forwarding a post by S. Keshav to the e2e mailing
list.  He explained nicely why simulation of networks is
difficult.
--
Bill Cheng // bill.cheng@usc.edu <URL:http://merlot.usc.edu/william/usc/>


  -----Original Message-----
  Date: Sun, 15 Apr 2007 10:18:13 -0400
  From: "S. Keshav" <keshav@uwaterloo.ca>
  To: end2end-interest@postel.org
  Subject: Re: [e2e] Simulator for wireless network

  Durga,
  	The discussions on this topic should convince you, I hope, that  
  before using simulations, their role has to be clearly understood.
  
  For physical systems, such as planes, cars, and sailboats, the  
  primary  operational parameters are the laws of physics, which can be  
  modeled to as great a degree of accuracy as desired. In other words,  
  I can overlay a 2D or 3D grid on the underlying system and apply the  
  laws of physics at each grid point. This is why it's possible to  
  accurately design physical systems from computer simulations.
  
  For computer systems, where a single line of code can completely  
  change the behavior of the system, one has to confront the fact that  
  no simulation is ever going to be accurate. As has been pointed out  
  already, the issue is not just of radio propagation modeling, which  
  is hard enough, but the problem is that there are many layers of  
  software that intervene from the receipt of a radio signal and a user- 
  perceptible effect. A slight change in any of these can materially  
  affect the result. For instance, a slight change in the card firmware  
  can change the way in which packets are handed to the driver, which  
  can change the timing at which packets are received by an  
  application, which may result in user-perceptible audio effects for  
  VOIP over WLAN. It is practically impossible to model these with any  
  accuracy, and even if you do, a patch to the firmware, driver, OS, or  
  application will invalidate your results.
  
  This is the reason why 'proof by simulation', for computer systems,  
  at least, is farcical. Not only are simulators known to be buggy, but  
  they are also simulating a system that is too loosely coupled to be  
  adequately modeled.
  
  So, does this mean that simulation is useless? Not really.  
  Simulations are useful in helping to form intuitions about the  
  underlying system. They can also help explore the parameter space  
  systematically. But, one has to realize that it is a coarse tool, and  
  necessarily so. As long as you go in with your eyes open, simulations  
  are a reasonable first step (but only the first step).
  
  keshav