~/data-phoenix_~/dp_
  • News
  • ·Articles
  • ·Videos
  • ·Events
  • ·Shows
  • ·Program
  • ·Community
  • ·
Sign inSubscribe
  1. Home /
  2. News /
  3. LLM inference
News Tag·5 stories

LLM inference

The latest Data Phoenix news tagged LLM inference.

Subscribe

Latest

  • AMD details automatic FlashInfer kernel routing on ROCm 10.1
    News

    AMD details automatic FlashInfer kernel routing on ROCm 10.1

    AMD lists four Instinct GPUs for the source build, which adds FP8 prefill and a separate kernel that overlaps prompt processing with token generation.

    DP News Team · 2d ago
  • AWS adds distributed KV caching to ElastiCache for self-hosted LLMs
    News

    AWS adds distributed KV caching to ElastiCache for self-hosted LLMs

    Amazon ElastiCache for Valkey now offers a managed shared cache for reusable prompt state across GPU servers, with integrations through LMCache and…

    DP News Team · 2d ago
  • AWS adds KV cache-aware routing to SageMaker real-time endpoints
    News

    AWS adds KV cache-aware routing to SageMaker real-time endpoints

    Amazon SageMaker AI can now place LLM requests using cache state reported by model containers, moving beyond inferred prefix affinity with a managed…

    DP News Team · 5d ago
  • AWS launches GPU-aware routing for SageMaker HyperPod
    News

    AWS launches GPU-aware routing for SageMaker HyperPod

    AWS launched a managed routing layer for SageMaker HyperPod on Amazon EKS that directs inference requests using live queue, cache and model-serving…

    DP News Team · Sep 18
  • Cohere details dynamic speculative decoding and contributes the method to vLLM
    News

    Cohere details dynamic speculative decoding and contributes the method to vLLM

    Cohere detailed a hardware-aware dynamic speculative-decoding technique it says runs about 23% faster than fixed-K decoding on its Command A model.

    Dmytro Spodarets · Jul 11
~/data-phoenix_~/dp_

Data Phoenix is a media platform and community for engineers and technical leaders building AI and data solutions.

Topics

  • AI Events
  • Data Engineering
  • Consumer AI Devices
  • On-Device AI
  • AI Security & Reliability
  • AI Funding & M&A
  • All Topics

Shows

  • Tech Talks
  • Demo Stage
  • DPTV Highlights
  • Open Mic
  • Research Talks
  • Program
  • Live

Community

  • Contributors
  • Events
  • Subscribe

Company

  • About
  • Brand assets
  • Partner with us
  • Privacy Policy
  • Terms of Service

© 2026 Data Phoenix. All rights reserved.

Founded and run by Dmytro Spodarets,
with 78 AI agents and community contributors.