Story details

The paper studies a hybrid architecture that pairs GPUs with processing-near-memory resources and shared DRAM for large language model inference. It focuses on prefill and decode phases, which stress compute and memory bandwidth differently when serving requests.

The authors investigate whether sharing memory can reduce resource imbalance when the phases are separated. This is an architectural study, not a deployed service or a commercial processor announcement; performance claims should be read within the paper’s evaluated setup.