A replay buffer (or experience replay memory) stores past transitions that an agent has experienced. During training, a random batch of transitions is sampled from the buffer to break temporal correlations and stabilize learning. This technique is used in DQN, SAC, DDPG, and many other off-policy algorithms.
Given a buffer of transitions, a batch size, and a random seed, sample a batch of transitions uniformly at random without replacement.
Set the random seed for reproducibility
Sample batch_size transitions from the buffer uniformly at random without replacement
Return the sampled transitions as a list ordered by ascending sampled index.
Input: buffer = [[0, 0, 1, 1, 0], [1, 1, 0.5, 2, 0], [2, 0, -1, 3, 1], [3, 1, 2, 4, 0], [4, 0, 0, 0, 1]], batch_size = 3, seed = 42
Output: [[1, 1, 0.5, 2, 0], [2, 0, -1, 3, 1], [4, 0, 0, 0, 1]]
Explanation: The seeded NumPy generator selects three distinct indices, which are sorted before retrieving transitions.
Input: buffer = [[0, 0, 1, 1, 0], [1, 1, 0.5, 2, 0], [2, 0, -1, 3, 1], [3, 1, 2, 4, 0], [4, 0, 0, 0, 1]], batch_size = 1, seed = 7
Output: [[0, 0, 1, 1, 0]]
Create a local generator with np.random.RandomState(seed).
Choose indices without replacement, sort them, and retrieve those buffer entries.
Accepts: array
Accepts: number
Accepts: number
A replay buffer (or experience replay memory) stores past transitions that an agent has experienced. During training, a random batch of transitions is sampled from the buffer to break temporal correlations and stabilize learning. This technique is used in DQN, SAC, DDPG, and many other off-policy algorithms.
Given a buffer of transitions, a batch size, and a random seed, sample a batch of transitions uniformly at random without replacement.
Set the random seed for reproducibility
Sample batch_size transitions from the buffer uniformly at random without replacement
Return the sampled transitions as a list ordered by ascending sampled index.
Input: buffer = [[0, 0, 1, 1, 0], [1, 1, 0.5, 2, 0], [2, 0, -1, 3, 1], [3, 1, 2, 4, 0], [4, 0, 0, 0, 1]], batch_size = 3, seed = 42
Output: [[1, 1, 0.5, 2, 0], [2, 0, -1, 3, 1], [4, 0, 0, 0, 1]]
Explanation: The seeded NumPy generator selects three distinct indices, which are sorted before retrieving transitions.
Input: buffer = [[0, 0, 1, 1, 0], [1, 1, 0.5, 2, 0], [2, 0, -1, 3, 1], [3, 1, 2, 4, 0], [4, 0, 0, 0, 1]], batch_size = 1, seed = 7
Output: [[0, 0, 1, 1, 0]]
Create a local generator with np.random.RandomState(seed).
Choose indices without replacement, sort them, and retrieve those buffer entries.
Accepts: array
Accepts: number
Accepts: number