Title: Designing repeatable experiments on an Emulab testbed
Citation: Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering vol. 66 p. 28-39
Publisher: Springer
Publication Year: 2012
JRC N°: JRC59313
ISBN: 978-3-642-30375-3
URI: http://www.springerlink.com/content/978-3-642-30376-0#section=1067705&page=1&locus=0
DOI: 10.1007/978-3-642-30376-0_3
Type: Articles in periodicals and books
Abstract: Emulation testbeds are increasingly used in an effort to promote repeatable experiments in the area of distributed systems and networking. In this paper we are studying how different design choices, e.g. use of specific tools, can affect the repeatability of experiments of an emulation testbed (e.g. based on the Emulab software). Our study is based on multiple experiments that are checked for stability and consistency (e.g., repetition of the same experiment and measurement of the mean and standard deviation of our metrics). The results indicate that repeatability of quantitative results is possible, under a degree of expected statistical variation. The event scheduling mechanism of Emulab is proven to be accurate down to a sub-second granularity. On the other hand we demonstrate that there are significant differences between traffic generation tools in terms of consistent recreation of a predefined traffic pattern and therefore experiment repeatability. The main contribution of this study is that based on experimental results we provide scientific proofs that Emulab as a platform can be used for scientifically rigorous experiments for networking research. New users of Emulab can benefit from this study by understanding that Emulab¿s scheduling mechanism, it¿s built-in packet generators and Iperf can sufficiently support repeatable experiments while TCPreplay cannot and therefore an alternative tool, i.e. TCPivo should be used.
JRC Directorate:Space, Security and Migration

Files in This Item:
There are no files associated with this item.

Items in repository are protected by copyright, with all rights reserved, unless otherwise indicated.