PureEdgeSim: A simulation framework for performance evaluation of cloud, edge and mist computing environments

Mechalikh, C., Taktak, H., & Moussa, F. (2021). PureEdgeSim: A simulation framework for performance evaluation of cloud, edge and mist computing environments. Computer Science and Information Systems, 18(1), 43-66.

Bibtex:

@article{mechalikh2021pureedgesim,
  title={PureEdgeSim: A simulation framework for performance evaluation of cloud, edge and mist computing environments},
  author={Mechalikh, Charafeddine and Taktak, Hajer and Moussa, Faouzi},
  journal={Computer Science and Information Systems},
  volume={18}
  number={1},
  pages={43--66},
  year={2021}
}

PureEdgeSim is a simulation framework that enables to study Internet of Things on a large scale, as a distributed, dynamic, and highly heterogeneous infrastructure, as well as the applications that run on these things. It features realistic infrastructure models, allowing research on the edge-to-cloud continuum to be conducted. It encompasses all layers of edge computing modeling and simulation. It features a modular design in which each module addresses a particular aspect of the simulation. The Datacenters Manager module, for example, is concerned with the creation of data centers, servers, as well as end devices and their heterogeneity. Their geo-distribution and mobility, however, are handled by the Location Manager module. Similarly, the Network Module is in charge of bandwidth allocation and data transfer.

PureEdgeSim brings together analytical and numerical modeling techniques. The network, for example, is modeled by means of adjustable mathematical equations rather than at a low level. Such abstraction facilitates simulating thousands of edge devices and guarantees the evaluation of the simulation state at any moment. Finally, since PureEdgeSim is a discrete event simulator, the characteristics of the simulated environment, like network traffic and resources’ utilization, will evolve over time, thereby enabling support for online decision-making algorithms, a process that can be accomplished through the Workload Orchestrator module.

Each module provides a default implementation along with a set of adjustable settings to facilitate prototyping and experimentation. Thus, researchers can effortlessly implement their approaches without having to waste time specifying low-level details. Another key feature of PureEdgeSim is its extensibility, as it allows users to create and incorporate their own models without altering its codebase when the default models fail to satisfy their needs. In discrete event simulators, the temporal complexity of the simulation is dependent on the number of events produced at runtime. To reduce simulation duration, PureEdgeSim allows for balancing the precision of the simulation and its duration thanks to a set of dedicated parameters. How precise the simulation will rely on the user’s settings, particularly the update intervals. Thus, the simulation will be more precise and realistic the shorter these intervals are, but it will also last longer. You can read more in the WiKi

A simple representation of the simulation scenarios

• The development and maintenance of this project requires a considerable effort. Thus, any form of contribution is encouraged.

• If you are willing to use the framework to implement your own project on top of it, creating a fork is a bad solution. You are not supposed to modify the code base to implement your project, but extend it by creating some subclasses. Unless you plan to redistribute your changes, you will end up with an incompatible and obsolete version of the framework. The project is constantly evolving and bug fixes are a priority. Your fork with personal changes will miss these updates and high performance improvements.

• If you just came across this project and want to check it out later, don't forget to add a star ⭐ 😉.

PureEdgeSim offers many exclusive features:

• Realistic modeling of edge computing scenarios (and its related computing paradigms).
  • It supports devices heterogeneity (sensors, mobile devices, battery-powered..).
  • It supports the heterogeneity of their means of communication (WiFi, 5G, 4G, Ethernet..).
  • Realistic modeling of computing infrastructure (modeling computational tasks, infrastructure topology...).
  • Realistic network model (Latency, bandwidth allocation, support for peer to peer communications, routing..).
  • Realistic mobility model.
  • Realistic energy model.
• The support for online decision making (tasks orchestration, scheduling, etc.).
• The study of QoS-aware architectures and topologies.
• Efficiency and Scalability.
  • PureEdgeSim supports the simulation of thousands of devices (even 30000 devices).
  • PureEdgeSim supports scenarios with longer simualtion time (even 24 hours).
• A wide collection of metrics
  • Delays: execution time, waiting time, network time.
  • CPU utilization.
  • Network utilization: Average bandwidth, total traffic...
  • Energy consumption: The energy consumption of computing nodes (Cloud, edge/fog servers,edge devices), the remaining energy of battery-powered devices, the energy consumption of WAN, MAN, and LAN networks, and the enrgy consumption of WiFi, Cellular (5G, 4G,..) and Ethernet.
  • Task success rate and reason of failure (the rate of tasks failed due to the lack of resources, due to mobility, due to latency, or due to the death of edge devices when they run out of battery).
  • and many others.
• The visualization of the simulation environement (which helps to debug and understand the course of the simualtion, refer to the figure below).
• Extensibility: Users can implement their scenarios and integrate their models without modifying PureEdgeSim codebase.
• Versatiliy and Wide Applicability
• Ease of use.
• Reliability.
• Correctness.

The live visualization of the simulated environement

A system is said to be efficient when it can maintain a specified level of proficiency without utilizing an excessive amount of resources, i.e., execution time and memory. Determining the theoretical time complexity of the given simulation is not trivial because the execution time depends entirely on the user scenario (i.e., the devices and tasks count, the mobility model, the types of resources, etc.) and the number of events that occur during the simulation. Furthermore, some parameters strongly influence the number of simulations and, obviously, the number of runs also directly influences the execution time. To demonstrate the scalability of PureEdgeSim, a few experiments were conducted on a single Intel Core i7-8550U.

A 10-minute simulation scenario with 10000 devices (which generated 1131000 tasks) took 58 seconds.

A 24-hour simulation scenario with 200 devices (total of 3221280 generated tasks generated) took 51 seconds.

A 60-minute scenario with 100 devices took only 1 seconds.

Of course, this time is highly dependent on the complexity of the decision making algorithm itself and the number of generated events (as more devices = more tasks = more events), which is why it took longer with 10,000 devices. That said, it is still faster than real-time and outperforms all CloudSim/ CloudSim plus based simulators that either struggle to start due to the large number of devices or run out of memory due to the large number of events generated.

It is therefore easy to conclude that PureEdgeSim meets the “efficiency and scalability” criteria as it allows for simulated experiments involving hundreds if not thousands of devices and tasks and runs considerably faster than in wall-clock time. An important step to reduce the simulation duration and solve its dependency on the number of runs is to implement multi-threading to execute simulation runs in parallel, which is already supported by PureEdgeSim.

PureEdgeSim support for large-scale scenarios

As mentioned previously, PureEdgeSim offers the most realistic network, energy, and mobility models as compared to existing solutions. However, since it is a DES, the time complexity of the simulation is dependent on the number of events generated at runtime. To decrease simulation time, PureEdgeSim provides fast and complete control over the simulation environment by its collection of parameters, allowing users to compromise between simulation precision and length. Hence, how accurate the simulation is will be influenced by the user’s settings, particularly the update intervals. The shorter these intervals, the more precise and realistic the simulation, but also the longer it takes.

If we classify edge computing and its related paradigms based on their placement in the IoT architecture, we will have three categories, each with specific properties. For example, paradigms of the perception layer like mist computing rely on resource-constrained devices, that can join or leave the network at any moment.

The computing paradigms supported by PureEdgeSim

Accordingly, the model must be versatile enough to support all kinds of resources and topologies. For this, PureEdgeSim dedicates a configuration file for each layer. The computing nodes, be they mobile devices, servers, or data centers, are all defined in these files. For instance, this figure shows how a smartphone is modeled. As you can see, a set of attributes is also provided to specify the remaining characteristics, such as power consumption rates and mobility.

<?xml version="1.0"?>
<edge_devices>
	<device> <!-- this is a smartphone, for example -->
		<mobility>true</mobility><!-- the device is mobile or fixed --> 
		<connectivity>4g</connectivity><!--the connectivity type 4g, ethernet, Wi-Fi-->
		<speed>1.4</speed><!-- the speed of the device in meters per second : 1.4m/s = 5km/h, 0 = non mobile-->
		<minPauseDuration>100</minPauseDuration><!-- the minimum delay before moving to a new location-->
		<maxPauseDuration>400</maxPauseDuration><!-- the maximum delay before moving to a new location-->
		<minMobilityDuration>60</minMobilityDuration><!-- the minimum delay before stopping-->
		<maxMobilityDuration>100</maxMobilityDuration><!-- the maximum delay before stopping-->
		<battery>true</battery> <!-- relies on battery? -->
		<percentage>30</percentage> <!-- percentage of this device type -->
		<batteryCapacity>18.75</batteryCapacity> <!-- battery capacity in Watt-Hour -->
		<idleConsumption>0.078</idleConsumption><!-- idle energy consumption/ 
			second-->
		<maxConsumption>3.3</maxConsumption><!-- max energy consumption/second, 
			when device cpu is use at 100%-->
		<isOrchestrator>false</isOrchestrator> <!-- whether it orchestrates tasks or not -->
		<canGenerateTasks>true</canGenerateTasks> <!-- whether it generates data/tasks or not -->
		<canProcessTasks>true</canProcessTasks> <!-- whether it can process data/tasks or not -->
		<hosts>
			<host>
				<core>8</core> <!-- Number of CPU cores -->
				<mips>25000</mips> <!-- Processing power in MIPS -->
				<ram>4096</ram> <!-- Ram capacity in MBytes -->
				<storage>128000</storage> <!-- Storage in MBytes -->
				<VMs
					<VM>
						<core>8</core>
						<mips>25000</mips>
						<ram>4096</ram>
						<storage>128000</storage> 
					</VM>
				</VMs>
			</host>
		</hosts>
	</device>
...
</edge_devices>

How a smartphone can be modeled in PureEdgeSim

Simulators can be used to determine which factors have the greatest impact on performance, by evaluating different configurations, for example, the number of generated entities, the types of resources, the network settings, etc. Dealing with all these parameters programmatically can be challenging. Software is said to be usable if it can be easily comprehended and run by any user. While the usability of PureEdgeSim may be limited by the fact that a lot of specific foundational knowledge is required, from a technical standpoint, it is implemented as a simple application, with only a few steps to follow to run the simulation.The provided Javadoc documentation and simulation examples offer proper guidance to the user, and the console directly informs the user of any type of incorrect input. To save time and effort, each module comes with a default implementation (e.g., the Default Mobility Model, the Default Computing Node class, etc.). These off-the-shelf models also provide a highly customizable environment with a multitude of settings, so users can adjust their behavior without changing their original code. These parameters are preset, which also decreases the complexity of the tool.

Another key feature of PureEdgeSim is its high extensibility. It provides code reusability and ensures adherence to software engineering guidelines and principles for improvements, accuracy, and extensibility. Although it uses its pre-built models by default, users can still create and incorporate their customized ones into their simulation scenarios if any of these default implementations do not meet their needs, all while not changing the PureEdgeSim code base. An example is provided below to show how custom models can be effortlessly integrated to the simulation.

//Create a PureEdgeSim simulation
Simulation sim = new Simulation();

// To use your custom mobility model instead of the default one:
sim.setCustomMobilityModel(YourCustomMobilityModel.class);

// To change the tasks orchestrator:
sim.setCustomEdgeOrchestrator(YourCustomOrchestrator.class);
		
// To change the computing node class:
sim.setCustomComputingNode(YourCustomComputingNode.class);
		
// To change the tasks generator:
sim.setCustomTaskGenerator(YourCustomTaskGenerator.class); 
		
// To change the network model
sim.setCustomNetworkModel(YourCustomNetworkModel.class); 
		
// To change the simulation manager
sim.setCustomSimulationManager(YourCustomSimulationManager.class); 

// To change the topology
sim.setCustomTopologyCreator(YourCustomTopologyCreator.class); 
		
/* to use the default models you can simply delete or comment those lines */

// Finally, you can launch the simulation
sim.launchSimulation();

The tool is said to be correct if it fully complies with the stated requirements, which can be verified by checking test cases. To verify PureEdgeSim’s correctness through unit and integration testing, several test cases have been implemented. Nevertheless, this alone does not provide an assessment of which portions of the code were tested and which were not, neither gives a percentage of the overall coverage of tests alongside the project source code. To evaluate the amount of code that the available unit tests cover, code coverage reports were added to the project with the help of the Java Code Coverage Library (JaCoCo). It is important to clarify that the concern of such test cases is the technical feasibility of the scenarios and not the evaluation of the simulation outcome. It is simply a matter of testing whether the given results are logically reasonable and not of actually validating them. To validate the software, a case study was conducted using the simulator. At least 90% of PureEdgeSim code is covered by those tests. This achievement was made possible by testing all the new features and also by the considerable decrease in code duplication since duplicating code brings neglect of testing.

In conclusion, we can say that the proposed simulator performs as intended. Every function has finished without abortions and gave the expected output; All the input files were parsed successfully, and all the personalized models were integrated and verified with the predicted results. The plotting of the simulation results gave a reasonable graphical output. In terms of intended errors, such as incorrect inputs, proper error messages were displayed, and correct information was written into the log file. Moreover, the parameters can be modified, stored, and loaded without any error.

Reliability is the ability of a system to operate consistently under specified conditions while meeting all requirements. When it comes to the components of PureEdgeSim, reliability is ensured by extensive exception handling that catches any error occurring at runtime. Consequently, the errors will not cause the execution to stop (unless they are related to critical parameters without which the simulation cannot continue). While testing it, the proposed simulator never froze and satisfied all functional requirements. Furthermore, data reliability is high since the models are read-only and unmodifiable, meaning no data loss is to be feared. The only files that can be modified are the output files and the configuration files, which can be restored easily to the default values. In addition, PureEdgeSim uses two continuous integration services: Travis and AppVeyor, that automate tests and builds execution on Linux and Windows virtual machines, respectively. PureEdgeSim’s users will benefit from such services as any test or build failure will be communicated promptly, thus, ensuring greater reliability.

There are several ways to use PureEdgeSim; however, it is strongly advisable to run PureEdgeSim via a Java development environment, like Eclipse IDE. A set of predefined examples is provided under the “/examples“ directory, which should allow anyone to become familiar with PureEdgeSim.

The simplest and recommended method to run this project is to use an IDE like Eclipse, IntelliJ IDEA, or NetBeans. The required steps to get started with the project are listed below:

1. Downloading and extracting the project source code archive from the project repository : This can be done using the web browser or the command line git clone https://github.com/CharafeddineMechalikh/PureEdgeSim.git.

2. Importing the project to the IDE:

• In NetBeans, simply select the "Open project" option and chose the project directory.

• In Eclipse or IntelliJ IDEA, the project must be imported by selecting the location where it was extracted or cloned.

3. The imported project consists of ten packages: the main application package, the above-mentioned modules, and the examples package. The main application and the modules are the simulator source code that normally should not be modified. The examples, however, are where the users can start.

4. It is necessary to convert the project into a Maven project in order to download all the required libraries.

5. Once all the necessary libraries are downloaded, users can start with the most basic examples by running any of the classes located in the “examples“ package.

6. To build a new simulation scenario, the simplest way is to create another class in this package.

Importing PureEdgeSim project (click to zoom)

It is possible to use PureEdgeSim as a dependency in a Maven project, by inserting the dependency below into the pom.xml file:

<dependency>
	<groupId>com.mechalikh</groupId>
	<artifactId>pureedgesim</artifactId>
	<version>5.1.0</version>
</dependency>

Or on Gradle :

dependencies {
 implementation 'com.mechalikh:pureedgesim:5.1.0'
}

Assuming that git and maven are already installed, PureEdgeSim can be run from the command line as follows:

1. First, the project source code must be downloaded by cloning the repository via the command git clone https://github.com/CharafeddineMechalikh/PureEdgeSim.git.
2. Now that the project is cloned, it can be built using Maven by executing the mvn clean install command in the directory where it was cloned.
3. Now, the examples can be executed on Windows, Linux, or Mac operating systems, using the command mvn exec:java -Dexec.mainClass="package.Class_Name" . For instance, to execute “Example1”, the command is mvn exec:java -Dexec.mainClass="examples.Example1"

Here are some works that are based on PureEdgeSim to give an idea about its wider applicability and the possibilities it offers. Feel free to edit the readme file and add yours!

1. Mechalikh, C., Taktak, H., & Moussa, F. (2020, April). A fuzzy decision tree based tasks orchestration algorithm for edge computing environments. In International Conference on Advanced Information Networking and Applications (pp. 193-203). Springer, Cham. The algorithm can be found here.

2. Safavifar, Z., Ghanadbashi, S., & Golpayegani, F. (2021, November). Adaptive Workload Orchestration in Pure Edge Computing: A Reinforcement-Learning Model. In 2021 IEEE 33rd International Conference on Tools with Artificial Intelligence (ICTAI) (pp. 856-860). IEEE.

3. Neelakantam, G., Onthoni, D. D., & Sahoo, P. K. (2020). Reinforcement learning based passengers assistance system for crowded public transportation in fog enabled smart city. Electronics, 9(9), 1501.

4. Iftikhar, S., Golec, M., Chowdhury, D., Gill, S. S., & Uhlig, S. (2022). FogDLearner: A Deep Learning-based Cardiac Health Diagnosis Framework using Fog Computing. In Australasian Computer Science Week 2022 (pp. 136-144).

5. Wei, J., Cao, S., Pan, S., Han, J., Yan, L., & Zhang, L. (2020, June). SatEdgeSim: A toolkit for modeling and simulation of performance evaluation in satellite edge computing environments. In 2020 12th International Conference on Communication Software and Networks (ICCSN) (pp. 307-313). IEEE.

6. Qin, J., Guo, X., Ma, X., Li, X., & Yang, J. (2022). Application and Performance Evaluation of Resource Pool Architecture in Satellite Edge Computing. Aerospace, 9(8), 451.

7. Mechalikh, C., Taktak, H., & Moussa, F. (2019, July). Towards a scalable and QoS-aware load balancing platform for edge computing environments. In 2019 International Conference on High Performance Computing & Simulation (HPCS) (pp. 684-691). IEEE.

8. Epifâneo, L., Correia, C., & Rodrigues, L. (2021, November). Cathode: A Consistency-Aware Data Placement Algorithm for the Edge. In 2021 IEEE 20th International Symposium on Network Computing and Applications (NCA) (pp. 1-10). IEEE.

9. Tiburski, R. T. (2021). Task scheduling and security for edge devices in internet of things applications.

10. Mechalikh, C., Taktak, H., & Moussa, F. (2019, June). A scalable and adaptive tasks orchestration platform for IoT. In 2019 15th International Wireless Communications & Mobile Computing Conference (IWCMC) (pp. 1557-1563). IEEE.

11. Mordacchini, M., Ferrucci, L., Carlini, E., Kavalionak, H., Coppola, M., & Dazzi, P. (2021, September). Self-organizing Energy-Minimization Placement of QoE-Constrained Services at the Edge. In International Conference on the Economics of Grids, Clouds, Systems, and Services (pp. 133-142). Springer, Cham.

12. Okwuibe, J., Haavisto, J., Kovacevic, I., Harjula, E., Ahmad, I., Islam, J., & Ylianttila, M. (2021). SDN-Enabled Resource Orchestration for Industrial IoT in Collaborative Edge-Cloud Networks. IEEE Access, 9, 115839-115854.

13. Khosroabadi, F., Fotouhi-Ghazvini, F., & Fotouhi, H. (2021). SCATTER: service placement in real-time fog-assisted IoT networks. Journal of Sensor and Actuator Networks, 10(2), 26.

14. Ferrucci, L., Mordacchini, M., Coppola, M., Carlini, E., Kavalionak, H., & Dazzi, P. (2020, June). Latency preserving self-optimizing placement at the edge. In Proceedings of the 1st Workshop on Flexible Resource and Application Management on the Edge (pp. 3-8).

15. Kavalionak, H., Coppola, M., & Dazzi, P. Self-organizing Energy-Minimization Placement of QoE-Constrained Services at the Edge. In Economics of Grids, Clouds, Systems, and Services: 18th International Conference, GECON 2021, Virtual Event, September 21–23, 2021, Proceedings (p. 133). Springer Nature.

16. Mordacchini, M., Ferrucci, L., Carlini, E., Kavalionak, H., Coppola, M., & Dazzi, P. (2022). Energy and QoE aware Placement of Applications and Data at the Edge.

17. Kaneko, Y., Yokoyama, Y., Monma, N., Terashima, Y., Teramoto, K., Kishimoto, T., & Saito, T. (2020, September). A Microservice-Based Industrial Control System Architecture Using Cloud and MEC. In International Conference on Edge Computing (pp. 18-32). Springer, Cham.

18. Javaid, H., Saleem, S., Wajid, B., & Khan, U. G. (2021, May). Diagnose a Disease: A Fog Assisted Disease Diagnosis Framework with Bidirectional LSTM. In 2021 International Conference on Digital Futures and Transformative Technologies (ICoDT2) (pp. 1-6). IEEE.

This project is licensed under GNU GPLv3.

⬆️