← 返回博客主页
阅读 0

RLX unifies a Rust tensor compiler and distributed runtime across 14 devices, while Trident strips host-side overhead from PyTorch Triton cache hits

今日概览

Two compiler projects entered the feed for September 29. RLX is a Rust-based tensor compiler and distributed runtime built around a single primitive-level, three-level intermediate representation that serves both compilation and runtime roles, with a transparent dispatch contract that fails compilation when legalization is impossible. Trident is a Torch-MLIR-based backend that compiles guarded specialization selection and host execution for PyTorch Triton workloads into a single executable module, reporting speedups over both eager execution and torch.compile on two LLMs.

重点变化

1. RLX: A Unified Multi-Backend Tensor Compiler and Distributed Runtime in Rust

RLX is a Rust-based tensor compiler and distributed runtime that unifies the compiler and runtime roles around one primitive-level, three-level intermediate representation. Its transparent dispatch contract resolves each operator into a native, common-IR, or rewritten lowering, and compilation fails when legalization is not possible. The same IR targets fourteen runtime devices from CPU, Metal, CUDA, ROCm, TPU, and Hexagon to Vulkan, OpenGL, DirectX, and WebGPU, plus dedicated Cortex-M INT8 and FPGA codegen paths. It supports F16, BF16, F64, and C64 types, INT4/INT8 quantization flows with AMP, PTQ, and QAT, tensor- and pipeline-parallel collectives over TCP and RDMA, sparse and dense linear algebra extensions, and 3D Gaussian splatting operators. In the project’s own evaluation under identical input generation and p50 measurement on one host, RLX-Metal is reported fastest at every batch on all-MiniLM-L6-v2, including 16.6 ms at batch 32 versus 26.7 ms for PyTorch-MPS, and its graph-fused MLP posts the top MNIST training throughput of 946,487 images per second while retaining 100 percent top-1 parity.

为什么重要： Production ML stacks typically split graph compilation from kernel execution across layers and languages, making backend behavior, deployment guarantees, and performance fallbacks hard to reason about end-to-end. RLX addresses that by having one IR serve both compiler and runtime, and by failing compilation rather than silently degrading when an operator cannot be legalized, which trades some flexibility for predictable behavior. The breadth of a single Rust codebase spanning fourteen devices, microcontrollers, FPGAs, quantization flows, and distributed collectives is unusual, though the reported benchmark lead comes from the authors’ one-host evaluation and has not yet been independently confirmed.

涉及层级： frontend、graph_ir、optimization、kernel_codegen、hardware_backend、runtime、distributed、benchmark

来源： 原始来源 1

事件评分： 87.8/100

2. Trident: Unifying Guarded Dispatch and Host Execution for PyTorch Triton Workloads

Trident is a Torch-MLIR-based compiler backend for PyTorch workloads that use Triton kernels. It introduces the Specialization Cache Module, or SCM, which compiles guarded specialization selection, argument and execution-environment preparation, and host execution for multiple specializations into a single executable module. On the specialization cache-hit path, an invocation enters the SCM, stays in compiled code when a specialization matches, and returns to Python only when a new specialization must be compiled, removing the runtime-managed specialization lookup, guard evaluation, and preparation that torch.compile otherwise performs on every invocation. The authors’ evaluation on two LLMs reports up to a 1.47x speedup in model-level end-to-end latency over eager execution and up to 1.68x over torch.compile.

为什么重要： When device execution is short, host-side orchestration can dominate the end-to-end latency of Triton kernels inside PyTorch, and even torch.compile routes each invocation through runtime-managed lookup and guard evaluation before reaching native wrappers. Trident removes that recurring cost from the common cache-hit path, which is most relevant to latency-sensitive LLM serving where the same specializations are reused repeatedly. The reported gains are limited to two LLMs in the authors’ own evaluation, so the benefit on other workloads and the cost of the compiled SCM itself remain open questions.

涉及层级： frontend、optimization、runtime、benchmark

来源： 原始来源 1

事件评分： 79.7/100

值得继续观察

  • RLX transparent dispatch contract and compile-failure semantics when legalization is impossible
  • Trident Specialization Cache Module removing cache-hit overhead for PyTorch Triton workloads
  • Independent validation of RLX results on all-MiniLM-L6-v2 and MNIST against PyTorch, TensorRT, and IREE
  • Independent validation of Trident’s reported 1.47x over eager and 1.68x over torch.compile beyond two LLMs
  • RLX INT4/INT8 quantization flows and TCP/RDMA tensor- and pipeline-parallel collectives

数据与生成说明

  • 采集窗口：2026-09-29T08:00:00+08:00 至 2026-09-30T08:00:00+08:00
  • 生成模式：deepseek
  • 文章由程序根据一手来源生成；性能结论应以链接中的原始配置为准。
  • 原始条目：NVIDIA 2 条，apache/tvm 1 条，arXiv 4 条

评论