This document summarize the author's experience over six years testing large-scale systems.We outline that experience in four points.First, we present a methodology for testing large-scale system. The methodology takes into account three dimensions of these systems: functionality, scalability, and volatility.The methodology proposes to execute tests in different workloads, from a small-scale static system up to a large-scale dynamic system.Experiments show that the alteration of the three dimensional aspects improves code coverage, thus improving the confidence on tests.Second, we introduce a distributed test architecture that uses both, a broadcast protocol to send messages from the test controller to testers and a converge cast protocol to send messages from testers back to the test controller. Experiments show that the architecture is more scalable than traditional centralized architectures when testing systems with more than um{1000} nodes.Third, we present an approach for using models as dynamic oracles for testing global properties of large-scale systems.This approach focuses on global, liveness, observable and controllable properties. We propose to efficiently keep updating a global model of the system during its execution. This model is then instantiated and evolved at runtime, by monitoring the corresponding distributed system, and serve as oracle for the distributed tests.We illustrate this approach by testing the reliability of two routing algorithms under churn. Results show common flaws in both algorithms.Finally, we present a model-driven approach for software artifacts deployment.We consider software artifacts as a product line and use feature models to represent their configurations and model-based techniques to handle automatic artifact deployment and reconfiguration.Experiments show that this approach reduces network traffic when deploying software on cloud environment.
展开▼
