ema_ipyparallel

This module provides functionality for combining the EMA workbench with IPython parallel.

class ema_workbench.em_framework.ema_ipyparallel.Engine(engine_id, msis, cwd)

class for setting up ema specific stuff on each engine also functions as a convenient namespace for workbench relevant variables

Parameters

• engine_id (int) –

• msis (list) –

• cwd (str) –

cleanup_working_directory()

remove the root working directory of the engine

run_experiment(experiment)

run the experiment, the actual running is delegated to an ExperimentRunner instance

class ema_workbench.em_framework.ema_ipyparallel.EngingeLoggerAdapter(logger, topic)

LoggerAdapter that inserts EMA as a topic into log messages

Parameters

• logger (logger instance) –

• topic (str) –

process(msg, kwargs)

Process the logging message and keyword arguments passed in to a logging call to insert contextual information. You can either manipulate the message itself, the keyword args or both. Return the message and kwargs modified (or not) to suit your needs.

Normally, you’ll only need to override this one method in a LoggerAdapter subclass for your specific needs.

class ema_workbench.em_framework.ema_ipyparallel.LogWatcher(**kwargs: Any)

A simple class that receives messages on a SUB socket, as published by subclasses of zmq.log.handlers.PUBHandler, and logs them itself.

This can subscribe to multiple topics, but defaults to all topics.

log_message(raw)

receive and parse a message, then log it.

subscribe()

Update our SUB socket’s subscriptions.

ema_workbench.em_framework.ema_ipyparallel.cleanup(client)

cleanup directory tree structure on all engines

ema_workbench.em_framework.ema_ipyparallel.get_engines_by_host(client)

returns the engine ids by host

Parameters

client (IPython.parallel.Client instance) –

Returns

a dict with hostnames as keys, and a list of engine ids

Return type

dict

ema_workbench.em_framework.ema_ipyparallel.initialize_engines(client, msis, cwd)

initialize engine instances on all engines

Parameters

• client (IPython.parallel.Client) –

• msis (dict) – dict of model structure interfaces with their names as keys

• cwd (str) –

ema_workbench.em_framework.ema_ipyparallel.set_engine_logger()

Updates EMA logging on the engines with an EngineLoggerAdapter This adapter injects EMA as a topic into all messages

ema_workbench.em_framework.ema_ipyparallel.start_logwatcher()

convenience function for starting the LogWatcher

Returns

• LogWatcher – the log watcher instance

• Thread – the log watcher thread

• .. note (there can only be one log watcher on a given url.)

ema_workbench.em_framework.ema_ipyparallel.update_cwd_on_all_engines(client)

updates the current working directory on the engines to point to the same working directory as this notebook

currently only works if engines are on same machine.

Parameters

client (IPython.parallel.Client instance) –