Links¶
Link interfaces¶

class
chainerrl.links.
StatelessRecurrent
[source]¶ Stateless recurrent link interface.
This class defines the interface of a recurrent link ChainerRL can handle.
In most casese, you can just use ChainerRL’s existing containers like chainerrl.links.StatelessRecurrentChainList, chainerrl.links.StatelessRecurrentSequential, and chainerrl.links.StatelessRecurrentBranched to define a recurrent link. You can use Chainer’s recurrent links such as L.NStepLSTM inside the containers.
To write your own recurrent link, you need to implement the interface.

concatenate_recurrent_states
(split_recurrent_states)[source]¶ Concatenate recurrent states into a batch.
This method can be used to make a batched recurrent state from separate recurrent states obtained via the get_recurrent_state_at method.
Parameters: split_recurrent_states (object) – Recurrent states to concatenate. Returns: Batched recurrent_state. Return type: object

get_recurrent_state_at
(recurrent_state, indices)[source]¶ Get a recurrent state at given indices.
This method can be used to save a recurrent state so that you can reuse it when you replay past sequences.
Parameters: indices (int or arraylike of ints) – Which recurrent state to get. Returns: Recurrent state of given indices. Return type: object

mask_recurrent_state_at
(recurrent_state, indices)[source]¶ Return a recurrent state masked at given indices.
This method can be used to initialize a recurrent state only for a certain sequence, not all the sequences.
Parameters: Returns: New batched recurrent state.
Return type:

n_step_forward
(x, recurrent_state)[source]¶ Multistep batch forward computation.
This method sequentially applies layers as chainer.Sequential does.
Parameters:  x (list) – Input sequences. Each sequence should be a variable whose first axis corresponds to time or a tuple of such variables.
 recurrent_state (object) – Batched recurrent state. If set to None, it is initialized.
 output_mode (str) – If set to ‘concat’, the output value is concatenated into a single large batch, which can be suitable for loss computation. If set to ‘split’, the output value is a list of output sequences.
Returns:  Output sequences. See the description of the output_mode
argument.
object: New batched recurrent state.
Return type:

Link implementations¶

class
chainerrl.links.
Branched
(*links)[source]¶ Link that calls forward functions of child links in parallel.
When either the __call__ method of this link are called, all the argeuments are forwarded to each child link’s __call__ method.
The returned values from the child links are returned as a tuple.
Parameters: *links – Child links. Each link should be callable.

class
chainerrl.links.
EmpiricalNormalization
(shape, batch_axis=0, eps=0.01, dtype=<class 'numpy.float32'>, until=None, clip_threshold=None)[source]¶ Normalize mean and variance of values based on emprical values.
Parameters:

class
chainerrl.links.
FactorizedNoisyLinear
(mu_link, sigma_scale=0.4)[source]¶ Linear layer in Factorized Noisy Network
Parameters:  mu_link (L.Linear) – Linear link that computes mean of output.
 sigma_scale (float) – The hyperparameter sigma_0 in the original paper. Scaling factor of the initial weights of noisescaling parameters.

class
chainerrl.links.
MLP
(in_size, out_size, hidden_sizes, nonlinearity=<function relu>, last_wscale=1)[source]¶ MultiLayer Perceptron

class
chainerrl.links.
MLPBN
(in_size, out_size, hidden_sizes, normalize_input=True, normalize_output=False, nonlinearity=<function relu>, last_wscale=1)[source]¶ MultiLayer Perceptron with Batch Normalization.
Parameters:  in_size (int) – Input size.
 out_size (int) – Output size.
 hidden_sizes (list of ints) – Sizes of hidden channels.
 normalize_input (bool) – If set to True, Batch Normalization is applied to inputs.
 normalize_output (bool) – If set to True, Batch Normalization is applied to outputs.
 nonlinearity (callable) – Nonlinearity between layers. It must accept a Variable as an argument and return a Variable with the same shape. Nonlinearities with learnable parameters such as PReLU are not supported.
 last_wscale (float) – Scale of weight initialization of the last layer.

class
chainerrl.links.
NIPSDQNHead
(n_input_channels=4, n_output_channels=256, activation=<function relu>, bias=0.1)[source]¶ DQN’s head (NIPS workshop version)

class
chainerrl.links.
NatureDQNHead
(n_input_channels=4, n_output_channels=512, activation=<function relu>, bias=0.1)[source]¶ DQN’s head (Nature version)

class
chainerrl.links.
Sequence
(*layers)[source]¶ Sequential callable Link that consists of other Links.

class
chainerrl.links.
StatelessRecurrentBranched
(*links)[source]¶ Stateless recurrent parallel link.
This is a recurrent analog to chainerrl.links.Branched. It bundles multiple links that implements StatelessRecurrent.
Parameters: *links – Child links. Each link should be recurrent and callable.

class
chainerrl.links.
StatelessRecurrentChainList
(*links)[source]¶ ChainList that auutomatically handles recurrent states.
This link extends chainer.ChainList by adding the recurrent_children property that returns all the recurrent child links and implementing recurrent state manimulation methods required for the StatelessRecurrent interface.
A recurrent state for this link is defined as a tuple of recurrent states of child recurrent links.

class
chainerrl.links.
StatelessRecurrentSequential
(*layers)[source]¶ Sequential model that can contain stateless recurrent links.
This link a stateless recurrent analog to chainer.Sequential. It supports the stateless recurrent interface by automatically detecting recurrent links and handles recurrent states properly.
For nonrecurrent layers (nonlink callables or nonrecurrent callable links), this link automatically concatenates the input to the layers for efficient computation.
Parameters: *layers – Callable objects.