Story details

A preprint presents CEMA-Router, an infrastructure-aware method for routing batches of language-model inference requests across cloud and edge resources. The design considers differences in compute capacity, energy efficiency, network conditions and model quality.

The goal is to balance response quality, cost, energy use and batch completion time. It is a proposed routing system evaluated by its authors, not a deployed service or a universal routing result.