leap_ec.executable_rep package

Submodules

leap_ec.executable_rep.cgp module

leap_ec.executable_rep.executable module

This module provides executable object representations. An Executable in LEAP represents problem solutions as functions, agent controllers, etc.

A LEAP Executable is a kind of phenotype, typically constructed when we use a Decoder to convert a genotypic representation of the object into an executable phenotype.

Executable are also just callable functors, so you can use them in your code like any other function.

class leap_ec.executable_rep.executable.ArgmaxExecutable(wrapped_executable)

Bases: Executable

Wraps another Executable with logic that returns the index of the highest output.

For example, we can use this to convert the class selection distribution output by a softmax layer to an integer representing the index of the most likely class:

>>> executable = lambda x: [ x[0] ^ x[1], x[0] & x[1], x[0] + x[1] ]
>>> wrapped = ArgmaxExecutable(executable)

>>> executable([1, 1])
[0, 1, 2]

>>> wrapped([1, 1])
2

class leap_ec.executable_rep.executable.Executable: Bases: ABC

class leap_ec.executable_rep.executable.KeyboardExecutable(input_space, output_space, keymap=<function KeyboardExecutable.<lambda>>)

Bases: Executable

A non-autonomous Executable phenotype that allows users to control an agent via the keyboard.

Parameters

input_space – space of possible inputs (ignored)
output_space – the space of possible actions to sample from, satisfying the Space interface used by OpenAI Gym
keymap – dict mapping keys to elements of the output space

key_press(key, mod): You’ll need to assign this function to your environment’s key_press handler.

key_release(key, mod): You’ll need to assign this function to your environment’s key_release handler.

class leap_ec.executable_rep.executable.RandomExecutable(input_space, output_space)

Bases: Executable

A trivial Executable phenotype that samples a random value from its output space.

Parameters

input_space – space of possible inputs (ignored)
output_space – the space of possible actions to sample from, satisfying the Space interface used by OpenAI Gym

class leap_ec.executable_rep.executable.WrapperDecoder(wrapped_decoder, decorator)

Bases: Decoder

A decoder that takes an executable object output by the wrapped Decoder, and then wrapps that Executable with an additional decorator function.

For example, if we have a Decoder that produces Executable objects whose output is governed by a softmax layer (i.e. a distribution), we can use this class to decorate them with an ArgmaxExecutable to transform their output into an integer.

decode(genome, *args, **kwargs)

Parameters: genome – a genome you wish to convert
Returns: the phenotype associated with that genome

leap_ec.executable_rep.neural_network module

leap_ec.executable_rep.problems module

class leap_ec.executable_rep.problems.EnvironmentProblem(runs: int, steps: int, environment, fitness_type: str, gui: bool, stop_on_done=True, maximize=True)

Bases: ScalarProblem

Defines a fitness function over Executable by evaluating them within a given environment.

Parameters

runs (int) – The number of independent runs to aggregate data over.
steps (int) – The number of steps to run the simulation for within each run.
environment – A simulation environment corresponding to the OpenAI Gym environment interface.
behavior_fitness – A function

evaluate(phenome): Run the environmental simulation using executable phenotype as a controller, and use the resulting observations & rewards to compute a fitness value.

property num_inputs: Return the number of dimensions in the environment’s input space.

property num_outputs: Return the number of dimensions in the environment’s action space.

static space_dimensions(observation_space) → int

Helper to get the number of dimensions (variables) in an OpenAI Gym space.

The point of this helper is that it works on simple spaces:

>>> from gymnasium import spaces
>>> discrete = spaces.Discrete(8)
>>> EnvironmentProblem.space_dimensions(discrete)
1

Box spaces:

>>> box = spaces.Box(low=-1.0, high=2.0, shape=(3, 4), dtype=np.float32)
>>> EnvironmentProblem.space_dimensions(box)
12

And Tuple spaces:

>>> tup = spaces.Tuple([discrete, box])
>>> EnvironmentProblem.space_dimensions(tup)
13

class leap_ec.executable_rep.problems.ImageXYProblem(path, maximize=False)

Bases: ScalarProblem

A problem that takes a function that generates an image defined over (x, y) coordinates and computed its fitness based on its match to an externally-defined image.

evaluate(phenome)

Evaluate the given phenome.

Practitioners must over-ride this member function.

Note that by default the individual comparison operators assume a maximization problem; if this is a minimization problem, then just negate the value when returning the fitness.

Parameters: phenome – the phenome to evaluate (this will not be modified)
Returns: the fitness value

static generate_image(executable, width, height)

class leap_ec.executable_rep.problems.TruthTableProblem(boolean_function, num_inputs, num_outputs, name: Optional[str] = None, pad_inputs=False, maximize=True)

Bases: ScalarProblem

Defines a fitness function over a Executable by evaluating it against each row of a given Boolean function’s truth table.

Both the executable we receive and the boolean_function we compare against should return a list of 1 or more outputs.

evaluate(phenome)

Say our object function is $$(x_0 \wedge x_1) \vee x_3$:$

>>> problem = TruthTableProblem(lambda x: [ (x[0] and x[1]) or x[2] ], num_inputs=3, num_outputs=1)

The truth table for this Boolean function has eight entries:

F F F=F F F T=T F T F=F F T T=T T F F=F T F T=T T T F=T T T T=T

Now consider a different function, $$(x_0 \wedge x_1) \oplus x_3$$.

>>> executable = lambda x: [ (x[0] and x[1]) ^ x[2] ]

This function’s truth table differs from the first one by exactly one entry (in the second one, TTT=F). So we expect a fitness value of $7/8 = 0.875$:

>>> from leap_ec import Individual
>>> problem.evaluate(executable)
0.875

Note that we our lambda functions above return a list that contains a computed value, rather than just the value directly. This is because this framework allows us to work with functions of more than one output:

>>> problem = TruthTableProblem(lambda x: [ x[0] and x[1], x[0] or x[1] ], num_inputs=3, num_outputs=2)
>>> problem.evaluate(lambda x: [ x[0] and x[1], x[0] or x[1] ])
1.0

leap_ec.executable_rep.rules module

Pitt-approach rule systems are one of the two basic approach to evolving rule-based programs (alongside Michigan-approach systems). In Pitt systems, every individual encodes a complete set of rules for producing an output given a set of inputs.

Evolutionary rule systems (also known as learning classifier systems) are often used to create controller for agents (i.e. for reinforcement learning problems), or to evolve classifiers for pattern recognition (i.e. supervised learning).

This module provides a basic Pitt-approach system that uses the spaces API from OpenAI Gym to define input and output spaces for rule conditions and actions, respectively.

class leap_ec.executable_rep.rules.PittRulesDecoder(input_space, output_space, memory_space=None, priority_metric=None)

Bases: Decoder

A Decoder that contructs a Pitt-approach rule system phenotype (PittRulesExecutable) out of a real-valued genome.

We use the OpenAI Gym spaces API to define the types and dimensionality of the rule system’s inputs and outputs.

Parameters

input_space – an OpenAI-gym-style space defining the inputs
output_space – an OpenAI-gym-style space defining the outputs
priority_metric – a PittRulesExecutable.PriorityMetric enum value defining how matching rules are deconflicted within the controller
num_memory_registers – the number of stateful memory registers that each rule considers as additional inputs

If, for example, we want to evolve controllers for a robot that has 3 real-valued sensor inputs and 4 mutually exclusive actions to choose from, we might use a Box and Discrete space, respectively, from gym.spaces:

>>> from gymnasium import spaces
>>> in_ = spaces.Box(low=0, high=1.0, shape=(1, 3), dtype=np.float32)
>>> out_ = spaces.Discrete(4)
>>> decoder = PittRulesDecoder(input_space=in_, output_space=out_)

property action_bounds

The bounds of permitted values on action genes within each rule.

For example, the following decoder

>>> from gymnasium import spaces
>>> in_ = spaces.Box(low=0, high=1.5, shape=(1, 3), dtype=np.float32)
>>> out_ = spaces.Discrete(4)
>>> decoder = PittRulesDecoder(input_space=in_, output_space=out_)

allows just one output value gene in each rule, with a maximum value of 4.

Bounds are inclusive, so they look like this:

>>> decoder.action_bounds
[(0, 3)]

bounds(num_rules)

Return the (low, high) bounds that it makes sense for each gene to vary within.

>>> from gymnasium import spaces
>>> in_ = spaces.Box(low=0, high=1.0, shape=(1, 3), dtype=np.float32)
>>> out_ = spaces.Discrete(4)
>>> decoder = PittRulesDecoder(input_space=in_, output_space=out_)
>>> decoder.bounds(num_rules=4)
[[(0.0, 1.0), (0.0, 1.0), (0.0, 1.0), (0.0, 1.0), (0.0, 1.0), (0.0, 1.0), (0, 3)], [(0.0, 1.0), (0.0, 1.0), (0.0, 1.0), (0.0, 1.0), (0.0, 1.0), (0.0, 1.0), (0, 3)], [(0.0, 1.0), (0.0, 1.0), (0.0, 1.0), (0.0, 1.0), (0.0, 1.0), (0.0, 1.0), (0, 3)], [(0.0, 1.0), (0.0, 1.0), (0.0, 1.0), (0.0, 1.0), (0.0, 1.0), (0.0, 1.0), (0, 3)]]

property condition_bounds

The bounds of permitted values on condition genes within each rule.

For example, the following decoder

>>> from gymnasium import spaces
>>> in_ = spaces.Box(low=0, high=1.5, shape=(1, 3), dtype=np.float32)
>>> out_ = spaces.Discrete(4)
>>> decoder = PittRulesDecoder(input_space=in_, output_space=out_)

produces bounds that restrict the low and high value of each condition’s range between 0 and 1.5:

>>> decoder.condition_bounds
[(0.0, 1.5), (0.0, 1.5), (0.0, 1.5), (0.0, 1.5), (0.0, 1.5), (0.0, 1.5)]

decode(genome, *args, **kwargs)

Decodes a real-valued genome into a PittRulesExecutable.

For example, say we have a Decoder that takes continuous inputs from a 2-D box and selects between two discrete actions:

>>> import numpy as np
>>> from gymnasium import spaces
>>> in_ = spaces.Box(low=np.array((0, 0)), high=np.array((1.0, 1.0)), dtype=np.float32)
>>> out_ = spaces.Discrete(2)
>>> decoder = PittRulesDecoder(input_space=in_, output_space=out_)

Now we can take genomes that represent each rule as as segment of the form [low, high, low, high, action] and converts them into executable controllers:

>>> genome = [ [ 0.0,0.6, 0.0,0.4, 0],
...            [ 0.4,1.0, 0.6,1.0, 1] ]
>>> decoder.decode(genome)
<leap_ec.executable_rep.rules.PittRulesExecutable object at ...>

genome_to_rules(genome)

Convert a genome into a list of Rules.

Usage example:

>>> import numpy as np
>>> from gymnasium import spaces
>>> in_ = spaces.Box(low=np.array((0, 0)), high=np.array((1.0, 1.0)), dtype=np.float32)
>>> out_ = spaces.Discrete(2)
>>> decoder = PittRulesDecoder(input_space=in_, output_space=out_)

Now we can take genomes that represent each rule as as segment of the form [low, high, low, high, action] and converts them into Rule objects:

>>> genome = [ [ 0.0,0.6, 0.0,0.4, 0],
...            [ 0.4,1.0, 0.6,1.0, 1] ]
>>> decoder.genome_to_rules(genome)
[Rule(conditions=[(0.0, 0.6), (0.0, 0.4)], actions=[0]), Rule(conditions=[(0.4, 1.0), (0.6, 1.0)], actions=[1])]

initializer(num_rules: int)

Returns an initializer function that can generate genomes according to the segmented scheme that we use for rule sets—i.e. with the appropriate number of segments, inputs, outputs, and hidden registers.

For instance, if we have the following decoder:

>>> from gymnasium import spaces
>>> in_ = spaces.Box(low=0, high=1.0, shape=(1, 3), dtype=np.float32)
>>> out_ = spaces.Discrete(4)
>>> decoder = PittRulesDecoder(input_space=in_, output_space=out_)

Then we can get an initializer like so that creates genomes compatible with the decoder when called:

>>> initialize = decoder.initializer(num_rules=4)
>>> initialize()
[array(...), array(...), array(...), array(...)]

Notice that it creates four top-level segments (one for each rule), and that the condition bounds for each input within a rule are wrapped in tuple sub-segments.

mutator(condition_mutator, action_mutator)

Returns a mutation operator that properly handles the segmented genome representation used for rule sets.

This wraps two different mutation operators you provide, so that mutation can be configured differently for rule conditions and rule actions, respectively.

Parameters

condition_mutator – a mutation operator to use for the condition genes in each rule.
action_mutator – a mutation operator to use for the action genes in each rule.

For example, often we’ll apply a rule system to a real-valued observation space and an integer-valued action space.

>>> from gymnasium import spaces
>>> in_ = spaces.Box(low=0, high=1.0, shape=(1, 3), dtype=np.float32)
>>> out_ = spaces.Discrete(4)
>>> decoder = PittRulesDecoder(input_space=in_, output_space=out_)

These two spaces call for different mutation strategies:

>>> from leap_ec.real_rep.ops import genome_mutate_gaussian
>>> from leap_ec.int_rep.ops import individual_mutate_randint
>>> mutator = decoder.mutator(
...                     condition_mutator=genome_mutate_gaussian,
...                     action_mutator=individual_mutate_randint
... )

property num_genes_per_rule

This property reports the total number of genes that specify each rule.

For example, the following decoder

>>> from gymnasium import spaces
>>> in_ = spaces.Box(low=0, high=1.0, shape=(1, 3), dtype=np.float32)
>>> out_ = spaces.Discrete(4)
>>> decoder = PittRulesDecoder(input_space=in_, output_space=out_)

takes rule genomes that have 7 values in each segment: 6 to specify the condition ranges ((low, high) for each of 3 inputs), and 1 to specify the output action.

>>> decoder.num_genes_per_rule
7

property num_inputs

This property reports the number of dimensions in the system’s input space.

For example, the following decoder

>>> from gymnasium import spaces
>>> in_ = spaces.Box(low=0, high=1.0, shape=(1, 12), dtype=np.float32)
>>> out_ = spaces.Discrete(4)
>>> decoder = PittRulesDecoder(input_space=in_, output_space=out_)

has a 12-dimensional input space:

>>> decoder.num_inputs
12

property num_memory_registers

property num_outputs

This property reports the number of dimensions in the system’s output space.

For example, the following decoder

>>> from gymnasium import spaces
>>> in_ = spaces.Box(low=0, high=1.0, shape=(1, 12), dtype=np.float32)
>>> out_ = spaces.Discrete(4)
>>> decoder = PittRulesDecoder(input_space=in_, output_space=out_)

has a 1-dimensional output space:

>>> decoder.num_outputs
1

class leap_ec.executable_rep.rules.PittRulesExecutable(input_space, output_space, rules, priority_metric, init_mem=[])

Bases: Executable

An Executable phenotype that interprets a Pittsburgh-style ruleset and outputs the appropriate action.

Parameters

input_space – an OpenAI-gym-style space defining the inputs
output_space – an OpenAI-gym-style space defining the outputs
init_memory – a list of initial values for the memory registers
rules – a list of Rule objects
priority_metric – the rule prioritization strategy used to resolve conflicts

Rulesets are lists of rules. Rules are lists of the form [ c1 c1’ c2 c2’ … cn cn’ a1 … am m1 … mr], where (cx, cx’) are are the min and max bounds that the rule covers, a1 .. am are the output actions, and m1 … mr are values to write to the memory registers.

For example, this ruleset has two rules. The first rule covers the square bounded by (0.0, 0.6)’ and `(0.0, 0.4), returning the output action 0 if the input falls within that range:

>>> rules = [ Rule(conditions=[(0.0, 0.6), (0.0, 0.4)], actions=[0]),
...           Rule(conditions=[(0.4, 1.0), (0.6, 1.0)], actions=[1])
...         ]

The input and output spaces are defined in the style of OpenAI gym. For example, here’s how you would set up a PittRulesExecutable with the above ruleset that takes two continuous input variables on (0.0, 1.0), and outputs discrete values in {0, 1}:

>>> import numpy as np
>>> from gymnasium import spaces
>>> input_space = spaces.Box(low=np.array((0, 0)), high=np.array((1.0, 1.0)), dtype=np.float32)
>>> output_space = spaces.Discrete(2)
>>> rules = PittRulesExecutable(input_space, output_space, rules,
...                             priority_metric=PittRulesExecutable.PriorityMetric.RULE_ORDER)

class PriorityMetric(value)

Bases: Enum

An enumeration.

GENERALITY = 2

PERIMETER = 3

RULE_ORDER = 1

class leap_ec.executable_rep.rules.PlotPittRuleProbe(decoder, plot_dimensions: (<class 'int'>, <class 'int'>) = (0, 1), ax=None, xlim=(0, 1), ylim=(0, 1), modulo=1, context={'leap': {'distrib': {'non_viable': 0}}})

Bases: object

A visualization operator that takes the best individual in the population and plots the condition bounds for each rule, i.e. as boxes over the input space.

Parameters

num_inputs (int) – the number of inputs in the sensor space
num_outputs (int) – the number of output actions
plot_dimensions ((int, int)) – which two dimensions of the input space to visualize along the x and y axes; defaults to the first two dimensions, (0, 1)
ax – the matplotlib axis to plot to; if None (the default), new Axes are created
xlim ((float, float)) – bounds for the horizontal axis
ylim ((float, float)) – bounds for the vertical axis
modulo (int) – the interval (in generations) to go between each visualization; i.e. if set to 10, then the visualization will be updated every 10 generations
context – the context objected that the generation count is read from (should be updated by the algorithm at each generation)

This probe requires a decoder, which it uses to parse individual genomes into sets of rules that it can visualize:

>>> import numpy as np
>>> from gymnasium import spaces
>>> in_ = spaces.Box(low=np.array((0, 0)), high=np.array((1.0, 1.0)), dtype=np.float32)
>>> out_ = spaces.Discrete(2)
>>> decoder = PittRulesDecoder(input_space=in_, output_space=out_)

Now we can create the probe itself:

>>> probe = PlotPittRuleProbe(decoder)

If we feed it a population of a single individual, we’ll see all that individual’s rules visualized. Like all LEAP probes, it returns the population unmodified. This allows the probe to be inserted into an EA’s operator pipeline.

>>> from leap_ec.individual import Individual
>>> ruleset = np.array([[0.0, 0.6, 0.0, 0.5, 0],
...                     [0.4, 1.0, 0.3, 1.0, 1],
...                     [0.1, 0.2, 0.1, 0.2, 0],
...                     [0.5, 0.6, 0.8, 1.0, 1]])
>>> pop = [Individual(genome=ruleset)]
>>> probe(pop)
[Individual<...>(...)]

(Source code)

class leap_ec.executable_rep.rules.Rule(conditions, actions)

Bases: tuple

property actions: Alias for field number 1

property conditions: Alias for field number 0

leap_ec.executable_rep package

Submodules

leap_ec.executable_rep.cgp module

leap_ec.executable_rep.executable module

leap_ec.executable_rep.neural_network module

leap_ec.executable_rep.problems module

leap_ec.executable_rep.rules module

Module contents