front: the file contains class Metric for evaluating all posible solution (population of obtained fronts).
  pfront (Pareto front): the file contains class Metric for evaluating the obtained front from each test case.
  tpfront: (True pareto front): the file contains class Metric for evaluating the obtained front and True pareto front
 (Reference front). Means, you need to pass the Reference front in this class.

  True pareto front (Reference front) can be obtained by:
    1) You provide it (If you know the True Pareto front for your problem)
    2) Calculate from all possible fronts obtained from all test case.
          Assumption you have N1 algorithms to test. 
          Each algorithm give you a Obtained front. 
          Each algorithm you run N2 independent trials --> Number of all possible fronts: N1 * N2 
          Pass all N1*N2 front in our function to calculate the Non-donminated Solutions (Reference front
 - Approximate Pareto front - True Pareto front)


import pfevaluator

## Some avaiable performance metrics for evaluate each type of Pareto front.
pfront_metrics = ["UD", "NDC"]
tpfront_metrics = ["ER", "ONVG", "MS", "GD", "IDG", "MPFE", "S", "STE"]
volume_metrics = ["HV", "HAR"]

pm = pfevaluator.metric_pfront(obtained_front, pfront_metrics)              # Evaluate for each algorithm in each trial
tm = pfevaluator.metric_tpfront(obtained_front, reference_front, tpfront_metrics)        # Same above
vm = pfevaluator.metric_volume(obtained_front, reference_front, volume_metrics, None, all_fronts=matrix_fitness)

## obtained_front: is your front you found in each test case (each trial of each algorithm)
## reference_front (True Pareto front): is your True Pareto front of your problem.
##      If you don't know your True Pareto front, do the above step to get it from population of obtained fronts.
##      Using this function: reference_front = pfevaluator.find_reference_front(matrix_fitness)
##          matrix_fitness is all of your fronts in all test cases.

## The results is dict such as:     pm = { "UD": 0.2, "NDC": 0.1 } 

@article{nguyen2019efficient,
  title={Efficient Time-Series Forecasting Using Neural Network and Opposition-Based Coral Reefs Optimization},
  author={Nguyen, Thieu and Nguyen, Tu and Nguyen, Binh Minh and Nguyen, Giang},
  journal={International Journal of Computational Intelligence Systems},
  volume={12},
  number={2},
  pages={1144--1161},
  year={2019},
  publisher={Atlantis Press}
}