Implement cache-aware request routing. Track full assigned tokens and uncached prefill load independently for every worker. A worker is feasible when adding the full request length remains within capacity. Rank feasible workers by longest cached prefix, lowest projected uncached load, greatest remaining capacity after assignment, then lowest worker ID. Assign -1 when no worker is feasible. Return a dictionary with assignments and worker_uncached_loads in worker input order.
Input: request_token_counts = [10,8], cached_prefix_lengths = [[8,0],[0,6]], worker_ids = [2,1], worker_capacities = [20,20]
Output: {"assignments":[2,1],"worker_uncached_loads":[2,2]}
Explanation: Each request selects the worker holding its longest reusable prefix.
Input: request_token_counts = [4], cached_prefix_lengths = [[0,0]], worker_ids = [5,2], worker_capacities = [10,10]
Output: {"assignments":[2],"worker_uncached_loads":[0,4]}
Input: request_token_counts = [6], cached_prefix_lengths = [[3,4]], worker_ids = [0,1], worker_capacities = [5,5]
Output: {"assignments":[-1],"worker_uncached_loads":[0,0]}
Track assigned_tokens and uncached_loads separately.
Build one lexicographic key for the four routing priorities.
Accepts: array
Accepts: array
Accepts: array
Accepts: array
Implement cache-aware request routing. Track full assigned tokens and uncached prefill load independently for every worker. A worker is feasible when adding the full request length remains within capacity. Rank feasible workers by longest cached prefix, lowest projected uncached load, greatest remaining capacity after assignment, then lowest worker ID. Assign -1 when no worker is feasible. Return a dictionary with assignments and worker_uncached_loads in worker input order.
Input: request_token_counts = [10,8], cached_prefix_lengths = [[8,0],[0,6]], worker_ids = [2,1], worker_capacities = [20,20]
Output: {"assignments":[2,1],"worker_uncached_loads":[2,2]}
Explanation: Each request selects the worker holding its longest reusable prefix.
Input: request_token_counts = [4], cached_prefix_lengths = [[0,0]], worker_ids = [5,2], worker_capacities = [10,10]
Output: {"assignments":[2],"worker_uncached_loads":[0,4]}
Input: request_token_counts = [6], cached_prefix_lengths = [[3,4]], worker_ids = [0,1], worker_capacities = [5,5]
Output: {"assignments":[-1],"worker_uncached_loads":[0,0]}
Track assigned_tokens and uncached_loads separately.
Build one lexicographic key for the four routing priorities.
Accepts: array
Accepts: array
Accepts: array
Accepts: array
