cached#

Source code: tianshou/data/buffer/cached.py

class CachedReplayBuffer(main_buffer: ReplayBuffer, cached_buffer_num: int, max_episode_length: int)[source]#

CachedReplayBuffer contains a given main buffer and n cached buffers, cached_buffer_num * ReplayBuffer(size=max_episode_length).

The memory layout is: | main_buffer | cached_buffers[0] | cached_buffers[1] | ... | cached_buffers[cached_buffer_num - 1] |.

The data is first stored in cached buffers. When an episode is terminated, the data will move to the main buffer and the corresponding cached buffer will be reset.

Parameters:

  • main_buffer – the main buffer whose .update() function behaves normally.

  • cached_buffer_num – number of ReplayBuffer needs to be created for cached buffer.

  • max_episode_length – the maximum length of one episode, used in each cached buffer’s maxsize.

See also

Please refer to ReplayBuffer for other APIs’ usage.

add(batch: RolloutBatchProtocol, buffer_ids: ndarray | list[int] | None = None) → tuple[ndarray, ndarray, ndarray, ndarray][source]#

Add a batch of data into CachedReplayBuffer.

Each of the data’s length (first dimension) must equal to the length of buffer_ids. By default the buffer_ids is [0, 1, …, cached_buffer_num - 1].

Return (current_index, episode_reward, episode_length, episode_start_index) with each of the shape (len(buffer_ids), …), where (current_index[i], episode_reward[i], episode_length[i], episode_start_index[i]) refers to the cached_buffer_ids[i]th cached buffer’s corresponding episode result.