evaluators

collection of evaluators for performing experiments, optimization, and robust optimization.

class ema_workbench.em_framework.evaluators.BaseEvaluator(msis: AbstractModel | list[AbstractModel])

evaluator for experiments using a multiprocessing pool.

Parameters:

msis (collection of models)

Raises:

ValueError –

evaluate_all(jobs: list[EvaluateSolution], **kwargs)

Makes ema_workbench evaluators compatible with platypus evaluators.

abstractmethod evaluate_experiments(experiments: Iterable[Experiment], callback: Callable, **kwargs)

Used by ema_workbench.

abstractmethod finalize()

Finalize the evaluator.

abstractmethod initialize()

Initialize the evaluator.

optimize(algorithm: type[AbstractGeneticAlgorithm] = EpsNSGAII, nfe: int = 10000, searchover: str = 'levers', reference: Sample | None = None, constraints: Iterable[Constraint] | None = None, convergence_freq: int = 1000, logging_freq: int = 5, variator: type[Variator] | None = None, rng: Iterable[StdlibSeedLike] | None = None, initial_population: Iterable[Sample] | None = None, filename: str | None = None, directory: str | None = None, **kwargs) → list[tuple[DataFrame, DataFrame]]
optimize(algorithm: type[AbstractGeneticAlgorithm] = EpsNSGAII, nfe: int = 10000, searchover: str = 'levers', reference: Sample | None = None, constraints: Iterable[Constraint] | None = None, convergence_freq: int = 1000, logging_freq: int = 5, variator: type[Variator] | None = None, rng: StdlibSeedLike | None = None, initial_population: Iterable[Sample] | None = None, filename: str | None = None, directory: str | None = None, **kwargs) → tuple[DataFrame, DataFrame]

Convenience method for outcome optimization.

A call to this method is forwarded to :func:optimize, with evaluator and models arguments added in.

perform_experiments(scenarios: int | Iterable[Sample] | Sample = 0, policies: int | Iterable[Sample] | Sample = 0, reporting_interval: int | None = None, reporting_frequency: int | None = 10, uncertainty_union: bool = False, lever_union: bool = False, outcome_union: bool = False, uncertainty_sampling: AbstractSampler | SamplerTypes = Samplers.LHS, uncertainty_sampling_kwargs: dict | None = None, lever_sampling: AbstractSampler | SamplerTypes = Samplers.LHS, lever_sampling_kwargs: dict | None = None, callback: type[AbstractCallback] | None = None, combine: Literal['full_factorial', 'sample', 'cycle'] = 'full_factorial', **kwargs)

Convenience method for performing experiments.

A call to this method is forwarded to :func:perform_experiments, with evaluator and models arguments added in.

robust_optimize(robustness_functions: list[~ema_workbench.em_framework.outcomes.ScalarOutcome], scenarios: int | ~collections.abc.Iterable[~ema_workbench.em_framework.points.Sample], algorithm: type[~platypus.algorithms.AbstractGeneticAlgorithm] = <class 'platypus.algorithms.EpsNSGAII'>, nfe: int = 10000, convergence_freq: int = 1000, logging_freq: int = 5, rng: ~ema_workbench.em_framework.util.StdlibSeedLike | ~collections.abc.Iterable[~ema_workbench.em_framework.util.StdlibSeedLike] | None = None, **kwargs) → tuple[DataFrame, DataFrame]

Convenience method for robust optimization.

A call to this method is forwarded to :func:robust_optimize, with evaluator and models arguments added in.

class ema_workbench.em_framework.evaluators.Samplers(*values)

Enum for different kinds of samplers.

class ema_workbench.em_framework.evaluators.SequentialEvaluator(msis: AbstractModel | list[AbstractModel])

Sequential evaluator.

evaluate_experiments(experiments: Iterable[Experiment], callback: Callable, **kwargs)

Evaluate experiments.

finalize()

Finalizer.

initialize()

Initializer.

ema_workbench.em_framework.evaluators.optimize(model: AbstractModel, algorithm: type[AbstractGeneticAlgorithm] = EpsNSGAII, nfe: int = 10000, searchover: str = 'levers', evaluator: BaseEvaluator | None = None, reference: Sample | None = None, constraints: Iterable[Constraint] | None = None, convergence_freq: int = 1000, logging_freq: int = 5, variator: Variator = None, rng: Iterable[StdlibSeedLike] | None = None, initial_population: Iterable[Sample] | None = None, filename: str | None = None, directory: str | None = None, **kwargs) → list[tuple[DataFrame, DataFrame]]
ema_workbench.em_framework.evaluators.optimize(model: AbstractModel, algorithm: type[AbstractGeneticAlgorithm] = EpsNSGAII, nfe: int = 10000, searchover: str = 'levers', evaluator: BaseEvaluator | None = None, reference: Sample | None = None, constraints: Iterable[Constraint] | None = None, convergence_freq: int = 1000, logging_freq: int = 5, variator: Variator = None, rng: StdlibSeedLike | None = None, initial_population: Iterable[Sample] | None = None, filename: str | None = None, directory: str | None = None, **kwargs) → tuple[DataFrame, DataFrame]

Optimize the model.

Parameters:

  • model (1 or more Model instances)

  • algorithm (a valid Platypus optimization algorithm)

  • nfe (int)

  • searchover ({'uncertainties', 'levers'})

  • evaluator (evaluator instance)

  • reference (Sample instance, optional) – overwrite the default scenario in case of searching over levers, or default policy in case of searching over uncertainties

  • constraints (list, optional)

  • convergence_freq (int) – nfe between convergence check

  • logging_freq (int) – number of generations between logging of progress

  • variator (platypus GAOperator instance, optional) – if None, it falls back on the defaults in platypus-opts which is SBX with PM

  • rng (seed for initializing the global python random number generator as used by platypus-opt) – because platypus-opt uses the global random number generator, full reproducibility cannot be guaranteed in case of threading. If seed is an iterable, the optimization is run for each seed. The seed is added to the filename to ensure separate tarballs for each run of the optimization.

  • initial_population (A collection of samples with which to initialize the optimization algorithm)

  • filename (filename for storing intermediate archives/results) – Every convergence_freq, the archive or results at that moment are added to this file. The file itself will be a tarball.

  • directory (directory in which to store tarball of intermediate archives/results.)

  • kwargs (any additional arguments will be passed on to algorithm)

Returns:

  • pandas DataFrame with pareto front

  • pandas DataFrame with runtime convergence information

Raises:

ValueError if searchover is not one of 'uncertainties' or 'levers' –

ema_workbench.em_framework.evaluators.perform_experiments(models: AbstractModel | list[AbstractModel], scenarios: int | Iterable[Sample] | Sample = 0, policies: int | Iterable[Sample] | Sample = 0, evaluator: BaseEvaluator | None = None, reporting_interval: int | None = None, reporting_frequency: int | None = 10, uncertainty_union: bool = False, lever_union: bool = False, outcome_union: bool = False, uncertainty_sampling: AbstractSampler | SamplerTypes = Samplers.LHS, uncertainty_sampling_kwargs: dict | None = None, lever_sampling: AbstractSampler | SamplerTypes = Samplers.LHS, lever_sampling_kwargs: dict | None = None, callback: type[AbstractCallback] | None = None, return_callback: bool = False, combine: Literal['full_factorial', 'sample', 'cycle'] = 'full_factorial', log_progress: bool = False, **kwargs) → DefaultCallback | tuple[DataFrame, dict[str, ndarray]]

Sample uncertainties and levers, and perform the resulting experiments on each of the models.

Parameters:

  • models (one or more AbstractModel instances)

  • scenarios (int or iterable of Sample instances, optional)

  • policies (int or iterable of Sample instances, optional)

  • evaluator (Additional keyword arguments are passed on to evaluate_experiments of the)

  • reporting_interval (int, optional)

  • reporting_frequency (int, optional)

  • uncertainty_union (boolean, optional)

  • lever_union (boolean, optional)

  • outcome_union (boolean, optional)

  • uncertainty_sampling ({LHS, MC, FF, SOBOL, MORRIS, FAST}, optional)

  • uncertainty_sampling_kwargs (dict, optional)

  • lever_sampling ({LHS, MC, FF, SOBOL, MORRIS, FAST}, optional TODO:: update doc)

  • lever_sampling_kwargs (dict, optional)

  • callback (Callback instance, optional)

  • return_callback (boolean, optional)

  • log_progress (bool, optional)

  • combine ({'factorial', 'sample'}, optional) – how to combine uncertainties and levers? In case of ‘factorial’, both are sampled separately using their respective samplers. Next the resulting designs are combined in a full factorial manner. In case of ‘sample’, both are sampled separately and then combined by cycling over the shortest of the the two sets of designs until the longest set of designs is exhausted.

  • evaluator

Returns:

the experiments as a dataframe, and a dict with the name of an outcome as key, and the associated values as numpy array. Experiments and outcomes are aligned on index.

Return type:

tuple