News Tag
LLM inference
The latest Data Phoenix news tagged LLM inference.
Latest
NewsAMD details automatic FlashInfer kernel routing on ROCm 10.1
NewsAWS adds distributed KV caching to ElastiCache for self-hosted LLMs
NewsAWS adds KV cache-aware routing to SageMaker real-time endpoints
NewsAWS launches GPU-aware routing for SageMaker HyperPod
NewsCohere details dynamic speculative decoding and contributes the method to vLLM