Links¶
Link interfaces¶
-
class
chainerrl.links.
StatelessRecurrent
[source]¶ Stateless recurrent link interface.
This class defines the interface of a recurrent link ChainerRL can handle.
In most casese, you can just use ChainerRL’s existing containers like chainerrl.links.StatelessRecurrentChainList, chainerrl.links.StatelessRecurrentSequential, and chainerrl.links.StatelessRecurrentBranched to define a recurrent link. You can use Chainer’s recurrent links such as L.NStepLSTM inside the containers.
To write your own recurrent link, you need to implement the interface.
-
concatenate_recurrent_states
(split_recurrent_states)[source]¶ Concatenate recurrent states into a batch.
This method can be used to make a batched recurrent state from separate recurrent states obtained via the get_recurrent_state_at method.
Parameters: split_recurrent_states (object) – Recurrent states to concatenate. Returns: Batched recurrent_state. Return type: object
-
get_recurrent_state_at
(recurrent_state, indices)[source]¶ Get a recurrent state at given indices.
This method can be used to save a recurrent state so that you can reuse it when you replay past sequences.
Parameters: indices (int or array-like of ints) – Which recurrent state to get. Returns: Recurrent state of given indices. Return type: object
-
mask_recurrent_state_at
(recurrent_state, indices)[source]¶ Return a recurrent state masked at given indices.
This method can be used to initialize a recurrent state only for a certain sequence, not all the sequences.
Parameters: Returns: New batched recurrent state.
Return type:
-
n_step_forward
(x, recurrent_state)[source]¶ Multi-step batch forward computation.
This method sequentially applies layers as chainer.Sequential does.
Parameters: - x (list) – Input sequences. Each sequence should be a variable whose first axis corresponds to time or a tuple of such variables.
- recurrent_state (object) – Batched recurrent state. If set to None, it is initialized.
- output_mode (str) – If set to ‘concat’, the output value is concatenated into a single large batch, which can be suitable for loss computation. If set to ‘split’, the output value is a list of output sequences.
Returns: - Output sequences. See the description of the output_mode
argument.
object: New batched recurrent state.
Return type:
-
Link implementations¶
-
class
chainerrl.links.
Branched
(*links)[source]¶ Link that calls forward functions of child links in parallel.
When either the __call__ method of this link are called, all the argeuments are forwarded to each child link’s __call__ method.
The returned values from the child links are returned as a tuple.
Parameters: *links – Child links. Each link should be callable.
-
class
chainerrl.links.
EmpiricalNormalization
(shape, batch_axis=0, eps=0.01, dtype=<class 'numpy.float32'>, until=None, clip_threshold=None)[source]¶ Normalize mean and variance of values based on emprical values.
Parameters:
-
class
chainerrl.links.
FactorizedNoisyLinear
(mu_link, sigma_scale=0.4)[source]¶ Linear layer in Factorized Noisy Network
Parameters: - mu_link (L.Linear) – Linear link that computes mean of output.
- sigma_scale (float) – The hyperparameter sigma_0 in the original paper. Scaling factor of the initial weights of noise-scaling parameters.
-
class
chainerrl.links.
MLP
(in_size, out_size, hidden_sizes, nonlinearity=<function relu>, last_wscale=1)[source]¶ Multi-Layer Perceptron
-
class
chainerrl.links.
MLPBN
(in_size, out_size, hidden_sizes, normalize_input=True, normalize_output=False, nonlinearity=<function relu>, last_wscale=1)[source]¶ Multi-Layer Perceptron with Batch Normalization.
Parameters: - in_size (int) – Input size.
- out_size (int) – Output size.
- hidden_sizes (list of ints) – Sizes of hidden channels.
- normalize_input (bool) – If set to True, Batch Normalization is applied to inputs.
- normalize_output (bool) – If set to True, Batch Normalization is applied to outputs.
- nonlinearity (callable) – Nonlinearity between layers. It must accept a Variable as an argument and return a Variable with the same shape. Nonlinearities with learnable parameters such as PReLU are not supported.
- last_wscale (float) – Scale of weight initialization of the last layer.
-
class
chainerrl.links.
NIPSDQNHead
(n_input_channels=4, n_output_channels=256, activation=<function relu>, bias=0.1)[source]¶ DQN’s head (NIPS workshop version)
-
class
chainerrl.links.
NatureDQNHead
(n_input_channels=4, n_output_channels=512, activation=<function relu>, bias=0.1)[source]¶ DQN’s head (Nature version)
-
class
chainerrl.links.
Sequence
(*layers)[source]¶ Sequential callable Link that consists of other Links.
-
class
chainerrl.links.
StatelessRecurrentBranched
(*links)[source]¶ Stateless recurrent parallel link.
This is a recurrent analog to chainerrl.links.Branched. It bundles multiple links that implements StatelessRecurrent.
Parameters: *links – Child links. Each link should be recurrent and callable.
-
class
chainerrl.links.
StatelessRecurrentChainList
(*links)[source]¶ ChainList that auutomatically handles recurrent states.
This link extends chainer.ChainList by adding the recurrent_children property that returns all the recurrent child links and implementing recurrent state manimulation methods required for the StatelessRecurrent interface.
A recurrent state for this link is defined as a tuple of recurrent states of child recurrent links.
-
class
chainerrl.links.
StatelessRecurrentSequential
(*layers)[source]¶ Sequential model that can contain stateless recurrent links.
This link a stateless recurrent analog to chainer.Sequential. It supports the stateless recurrent interface by automatically detecting recurrent links and handles recurrent states properly.
For non-recurrent layers (non-link callables or non-recurrent callable links), this link automatically concatenates the input to the layers for efficient computation.
Parameters: *layers – Callable objects.