Links

Link interfaces

class chainerrl.links.StatelessRecurrent

Stateless recurrent link interface.

This class defines the interface of a recurrent link ChainerRL can handle.

In most casese, you can just use ChainerRL’s existing containers like chainerrl.links.StatelessRecurrentChainList, chainerrl.links.StatelessRecurrentSequential, and chainerrl.links.StatelessRecurrentBranched to define a recurrent link. You can use Chainer’s recurrent links such as L.NStepLSTM inside the containers.

To write your own recurrent link, you need to implement the interface.

concatenate_recurrent_states(split_recurrent_states)

Concatenate recurrent states into a batch.

This method can be used to make a batched recurrent state from separate recurrent states obtained via the get_recurrent_state_at method.

Parameters:split_recurrent_states (object) – Recurrent states to concatenate. Returns:Batched recurrent_state. Return type:object

get_recurrent_state_at(recurrent_state, indices)

Get a recurrent state at given indices.

This method can be used to save a recurrent state so that you can reuse it when you replay past sequences.

Parameters:indices (int or array-like of ints) – Which recurrent state to get. Returns:Recurrent state of given indices. Return type:object

mask_recurrent_state_at(recurrent_state, indices)

Return a recurrent state masked at given indices.

This method can be used to initialize a recurrent state only for a certain sequence, not all the sequences.

Parameters:

• recurrent_state (object) – Batched recurrent state.
• indices (int or array-like of ints) – Which recurrent state to mask.

Returns:

New batched recurrent state.

Return type:

object

n_step_forward(x, recurrent_state)

Multi-step batch forward computation.

This method sequentially applies layers as chainer.Sequential does.

Parameters:

• x (list) – Input sequences. Each sequence should be a variable whose first axis corresponds to time or a tuple of such variables.
• recurrent_state (object) – Batched recurrent state. If set to None, it is initialized.
• output_mode (str) – If set to ‘concat’, the output value is concatenated into a single large batch, which can be suitable for loss computation. If set to ‘split’, the output value is a list of output sequences.

Returns:

Output sequences. See the description of the output_mode

argument.

object: New batched recurrent state.

Return type:

object

Link implementations

class chainerrl.links.Branched(*links)

Link that calls forward functions of child links in parallel.

When either the __call__ method of this link are called, all the argeuments are forwarded to each child link’s __call__ method.

The returned values from the child links are returned as a tuple.

Parameters:*links – Child links. Each link should be callable.

class chainerrl.links.EmpiricalNormalization(shape, batch_axis=0, eps=0.01, dtype=<class 'numpy.float32'>, until=None, clip_threshold=None)

Normalize mean and variance of values based on emprical values.

Parameters:

• shape (int or tuple of int) – Shape of input values except batch axis.
• batch_axis (int) – Batch axis.
• eps (float) – Small value for stability.
• dtype (dtype) – Dtype of input values.
• until (int or None) – If this arg is specified, the link learns input values until the sum of batch sizes exceeds it.

class chainerrl.links.FactorizedNoisyLinear(mu_link, sigma_scale=0.4)

Linear layer in Factorized Noisy Network

Parameters:

• mu_link (L.Linear) – Linear link that computes mean of output.
• sigma_scale (float) – The hyperparameter sigma_0 in the original paper. Scaling factor of the initial weights of noise-scaling parameters.

class chainerrl.links.MLP(in_size, out_size, hidden_sizes, nonlinearity=<function relu>, last_wscale=1)

Multi-Layer Perceptron

class chainerrl.links.MLPBN(in_size, out_size, hidden_sizes, normalize_input=True, normalize_output=False, nonlinearity=<function relu>, last_wscale=1)

Multi-Layer Perceptron with Batch Normalization.

Parameters:

• in_size (int) – Input size.
• out_size (int) – Output size.
• hidden_sizes (list of ints) – Sizes of hidden channels.
• normalize_input (bool) – If set to True, Batch Normalization is applied to inputs.
• normalize_output (bool) – If set to True, Batch Normalization is applied to outputs.
• nonlinearity (callable) – Nonlinearity between layers. It must accept a Variable as an argument and return a Variable with the same shape. Nonlinearities with learnable parameters such as PReLU are not supported.
• last_wscale (float) – Scale of weight initialization of the last layer.

class chainerrl.links.NIPSDQNHead(n_input_channels=4, n_output_channels=256, activation=<function relu>, bias=0.1)

DQN’s head (NIPS workshop version)

class chainerrl.links.NatureDQNHead(n_input_channels=4, n_output_channels=512, activation=<function relu>, bias=0.1)

DQN’s head (Nature version)

class chainerrl.links.Sequence(*layers)

Sequential callable Link that consists of other Links.

class chainerrl.links.StatelessRecurrentBranched(*links)

Stateless recurrent parallel link.

This is a recurrent analog to chainerrl.links.Branched. It bundles multiple links that implements StatelessRecurrent.

Parameters:*links – Child links. Each link should be recurrent and callable.

class chainerrl.links.StatelessRecurrentChainList(*links)

ChainList that auutomatically handles recurrent states.

This link extends chainer.ChainList by adding the recurrent_children property that returns all the recurrent child links and implementing recurrent state manimulation methods required for the StatelessRecurrent interface.

A recurrent state for this link is defined as a tuple of recurrent states of child recurrent links.

class chainerrl.links.StatelessRecurrentSequential(*layers)

Sequential model that can contain stateless recurrent links.

This link a stateless recurrent analog to chainer.Sequential. It supports the stateless recurrent interface by automatically detecting recurrent links and handles recurrent states properly.

For non-recurrent layers (non-link callables or non-recurrent callable links), this link automatically concatenates the input to the layers for efficient computation.

Parameters:*layers – Callable objects.

Link utility functions

chainerrl.links.to_factorized_noisy(link, *args, **kwargs)

Add noisiness to components of given link

Currently this function supports L.Linear (with and without bias)