Using recurrent models

Recurrent model interface

class chainerrl.recurrent.Recurrent[source]

Interface of recurrent and stateful models.

This is an interface of recurrent and stateful models. ChainerRL supports recurrent neural network models as stateful models that implement this interface.

To implement this interface, you need to implement three abstract methods of it: get_state, set_state and reset_state.

get_state()[source]

Get the current state of this model.

Returns:Any object that represents a state of this model.
reset_state()[source]

Reset the state of this model to the initial state.

For typical RL models, this method is expected to be called before every episode.

set_state(state)[source]

Overwrite the state of this model with a given state.

Parameters:state (object) – Any object that represents a state of this model.
update_state(*args, **kwargs)[source]

Update this model’s state as if self.__call__ is called.

Unlike __call__, stateless objects may do nothing.

Utilities

chainerrl.recurrent.state_kept(link)[source]

Keeps the previous state of a given link.

This is a context manager that saves saves the current state of the link before entering the context, and then restores the saved state after escaping the context.

This will just ignore non-Recurrent links.

# Suppose the link is in a state A
assert link.get_state() is A

with state_kept(link):
    # The link is still in a state A
    assert link.get_state() is A

    # After evaluating the link, it may be in a different state
    y1 = link(x1)
    assert link.get_state() is not A

# After escaping from the context, the link is in a state A again
# because of the context manager
assert link.get_state() is A
chainerrl.recurrent.state_reset(link)[source]

Reset the state while keeping the previous state of a given link.

This is a context manager that saves saves the current state of the link and reset it to the initial state before entering the context, and then restores the saved state after escaping the context.

This will just ignore non-Recurrent links.

# Suppose the link is in a non-initial state A
assert link.get_state() is A

with state_reset(link):
    # The link's state has been reset to the initial state
    assert link.get_state() is InitialState

    # After evaluating the link, it may be in a different state
    y1 = link(x1)
    assert link.get_state() is not InitialState

# After escaping from the context, the link is in a state A again
# because of the context manager
assert link.get_state() is A