Predictive Pipelined Decoding: A Compute-Latency Trade-off for Exact LLM Decoding

Language Model / By

Seongjun Yang, Gibbeum Lee, Jaewoong Cho, Dimitris Papailiopoulos, Kangwook Lee