Calculate effective communication bandwidth under the supplied ring-equivalent model. Let S be payload_bytes, R be world_size, and t be duration_seconds. The pass count c is two for all_reduce and one for all_gather, reduce_scatter, or all_to_all. The modeled bytes per rank A and bandwidth B are:
A=cSRR−1. B=tA.A single rank requires no communication. Return a Python dictionary with algorithm_bytes and bandwidth_bytes_per_second, both finite Python floats.
Input: payload_bytes = 1024, world_size = 4, duration_seconds = 0.001, collective = "all_reduce"
Output: {"algorithm_bytes":1536,"bandwidth_bytes_per_second":1536000}
Explanation: Two ring passes move three quarters of the payload each, and dividing 1536 bytes by 0.001 seconds gives the bandwidth.
Input: payload_bytes = 1200, world_size = 3, duration_seconds = 0.002, collective = "all_gather"
Output: {"algorithm_bytes":800,"bandwidth_bytes_per_second":400000}
Input: payload_bytes = 2048, world_size = 2, duration_seconds = 0.004, collective = "reduce_scatter"
Output: {"algorithm_bytes":1024,"bandwidth_bytes_per_second":256000}
Python / performs floating-point division.
float converts a numeric value to a Python float.
Accepts: number
Accepts: number
Accepts: number
Accepts: string
Calculate effective communication bandwidth under the supplied ring-equivalent model. Let S be payload_bytes, R be world_size, and t be duration_seconds. The pass count c is two for all_reduce and one for all_gather, reduce_scatter, or all_to_all. The modeled bytes per rank A and bandwidth B are:
A=cSRR−1. B=tA.A single rank requires no communication. Return a Python dictionary with algorithm_bytes and bandwidth_bytes_per_second, both finite Python floats.
Input: payload_bytes = 1024, world_size = 4, duration_seconds = 0.001, collective = "all_reduce"
Output: {"algorithm_bytes":1536,"bandwidth_bytes_per_second":1536000}
Explanation: Two ring passes move three quarters of the payload each, and dividing 1536 bytes by 0.001 seconds gives the bandwidth.
Input: payload_bytes = 1200, world_size = 3, duration_seconds = 0.002, collective = "all_gather"
Output: {"algorithm_bytes":800,"bandwidth_bytes_per_second":400000}
Input: payload_bytes = 2048, world_size = 2, duration_seconds = 0.004, collective = "reduce_scatter"
Output: {"algorithm_bytes":1024,"bandwidth_bytes_per_second":256000}
Python / performs floating-point division.
float converts a numeric value to a Python float.
Accepts: number
Accepts: number
Accepts: number
Accepts: string
