Peer configuration tutorial

Here we describe how to create OpenBCI peer configuration files and use them. Then we show how to run configured peers without using launcher.

Configuration file

Peer configuration is stored in one or more files, in INI-like format. The most important (and mandatory) config file is the one stored in the same directory as the peer source code file. Its name is the same as the peer it configures: for a peer implemented in file peer_a.py you define peer_a.ini. This file is called basic config. Treat it as a part of peer implementation. Basic config is always parsed and is the reference for validation of any configuration overrides.

Configuration file contains four sections: [config_sources], [local_params], [external_params] and [launch_dependencies].

[local_params] is the most straightforward section. Here we define config parameters which are owned by the peer – its properties. For an amplifier such properties would be sampling rate or number of channels.:

[local_params]

cookies_to_eat = 128
super_important_param = 0

A peer may also need parameters of other peers. For example a signal filter would need sampling rate of an amplifier from which it takes the signal. How to represent this in a config file? With the new configuration system we introduced peer_id’s for peers, which should be unique in the scope of a running experiment. Configuration file though has to be usable in many experiments, so we cannot hardcode exact peer ID’s there. Instead, in a section [config_sources] we define symbollic names for peers, from which the configured peer should take parameters.:

[config_sources]
signal_properties_source=
; or maybe just
amplifier=

In run-time real peer_ids are assigned to these config sources.

In a similar manner we define [launch_dependencies]. Launch dependencies are peers we need to synchronize with: a peer will not start its actual work until all the peers it depends on report that they are ready.

[launch_dependencies]
amplifier=

Both config and launch dependencies’ names are visible in the scope of the configuration file. So ‘amplifier’ from the example above is the same peer as the one in [config_sources].

Now the [external_params] section. Let’s say a peer needs a parameter ‘sampling_rate’ from some other peer which we symbolicly named ‘amplifier’. It will store the parameter as ‘amp_sampling_rate’. We can represent this as follows:

[external_params]
amp_sampling_rate = amplifier.sampling_rate

All parameter names we define in a config file should be unique. You can move parameters between [external_params] and [local_params] in config files loaded after the basic config. This may be useful during module development when we want to quickly test how the module works without loading other peers. Also the other way may be useful if in some experiment we want to change the way parameters are passed.

Config overriding

As mentioned above, you can pass custom configuration files to the peer. The one restriction is that you cannot define any parameter names not defined in basic config. You can move a parameter form local to external section, add a launch or config dependecy. If a config dependency is not referenced in external_params configuration, the peer will not require passing the dependency’s real peer_id on launch.

Command line - peer invocation

Below is the somewhat messy usage help generated by the config processing module using argparse:

usage: some_peer.py peer_id [options]

positional arguments:
  peer_id               Unique name for this instance of this peer

optional arguments:
  -h, --help            show this help message and exit
  -p LOCAL_PARAMS LOCAL_PARAMS, --local_params LOCAL_PARAMS LOCAL_PARAMS
                        Local parameter override value: param_name, value.
  -e EXTERNAL_PARAMS EXTERNAL_PARAMS, --external_params EXTERNAL_PARAMS EXTERNAL_PARAMS
                        External parameter override value: param_name value_def .
  -c CONFIG_SOURCES CONFIG_SOURCES, --config_sources CONFIG_SOURCES CONFIG_SOURCES
                        Config source ID assignment: src_name peer_id
  -d LAUNCH_DEPENDENCIES LAUNCH_DEPENDENCIES, --launch_dependencies LAUNCH_DEPENDENCIES LAUNCH_DEPENDENCIES
                        Launch dependency ID assignment: dep_name peer_id
  -f CONFIG_FILE, --config_file CONFIG_FILE
                        Additional configuration file (overrides):
                        path_to_file.

When starting a peer with config support you need to provide a peer_id for it and peer_ids of its launch/config dependecies. Custom config files – option -f. You can also override some parameters: using option -p / –local-params param_name value or -e / –external-params param_name value_definitions.

Assume we want to run two peers: peer_a.py with default configuration (defined in peer_a.ini in source directory)

[config_sources]
amp1_signal=
peerb=

[launch_dependencies]
peerb=

[external_params]
ext_txt = peerb.text

[local_params]
my_param = 1234
p = some text here

and peer_b.py with configuration peer_b.ini

[config_sources]
some_peer=

[external_params]
ext_p = some_peer.p

[launch_dependencies]

[local_params]
text = text text tralala

peer_a takes parameter ‘text’ from a peer ‘peerb’, and peer_b takes parameter ‘p’ from a peer ‘some_peer’. Peer_a also waits for ‘peerb’ readiness. We want them to take those parameters from each other.

Invocation of those peers with just assigning them peer_id’s would look like this:

python peer_a.py i_am_roger -c peerb sue
python peer_b.py sue -c some_peer i_am_roger

We do not need to provide launch depenency ID for peer_a because this time it’s the same peer as in config dependencies - assignment will be automatic.

Programming OpenBCI peers with configuration support

To enable configuration processing in a peer, import modules:

import peer.peer_config_control
import common.config_message as cmsg

The second module is not necessary for config initialization but may be needed for processing configuration messages later.

In the peer initialization code you need to also initialize config processing:

self.config = peer.peer_config_control.PeerControl(self)

# this will parse command line arguments, process all configuration and
# acquire external params. You need to provide a connection to Multiplexer.
self.config.initialize_config(self.conn)

# ...some other initialization

# inform other peers that this peer is ready to work
self.config.send_peer_ready(self.conn)

# wait for launch dependencies' readiness
self.config.synchronize_ready(self.conn)

Later, when you need a parameter value, call get_param(param_name_str) on your config object.

An example code for peer_a.py (in test directory) which does nothing apart from initialization:

#!/usr/bin/env python3
# -*- coding: utf-8 -*-

from multiplexer.multiplexer_constants import peers, types
from multiplexer.clients import BaseMultiplexerServer
import settings

import peer.peer_config_control

class TestServer2(BaseMultiplexerServer):
    def __init__(self, addresses):
        super(TestServer2, self).__init__(addresses=addresses, type=peers.ETR_SERVER)
        self.config = peer.peer_config_control.PeerConfigControl(self)
        self.config.initialize_config(self.conn)
        self.config.send_peer_ready(self.conn)
        self.config.synchronize_ready(self.conn)

    def handle_message(self, mxmsg):
        # handle something
        self.no_response()

if __name__ == "__main__":
    srv = TestServer2(settings.MULTIPLEXER_ADDRESSES)
    srv.loop()

[TEMPORARY] Example experiment definition in k2launcher

This will soon be replaced by a new launcher. Defining new tasks in captain for every experiment would be daunting.

# this is the intermediate for exchanging parameters between peers
task("python ../obci_control/peer/config_server.py", "config_server")

task("python ../obci_control/test/peer_a.py i_am_roger -c peerb sue, "A")
task("python ../obci_control/test/peer_b.py  sue -c some_peer i_am_roger", "B")

start("cftest", "config_server")
start("cftest", "A")
start("cftest", "B")

Other features

Todo

Support for updating and distributing configuration parameters during runtime will be added (using Multiplexer messages). Take note, though, that updating config parameters during runtime is a more complicated issue: peers need custom methods for handling parameter changes to prevent possible program crashes.