TensorFold LLM inference engine written in Zig
First seen Nov 12 Updated Nov 12, 11:39 UTC Heat 32 medium evidence
TensorFold is an LLM inference engine written in Zig that runs Metal or CUDA kernels directly. Its repository also lists Vulkan in its description. It reads checkpoints and tokenizes requests itself, with no Python or MLX needed for serving.
- What to do
- Clone the repository and try serving a model on Metal or CUDA.
- Who should care
- Developers running local LLM inference on Macs or NVIDIA machines.
Why it matters: The Apache-2.0 engine serves models on Apple Silicon and NVIDIA GPUs without Python or MLX installed.
Key facts
- License
- Apache-2.0
- Contributor tooling requires ATC 0.9.4 or later
Timeline
Coverage
-
ashhart/TensorFold: LLM Inference Engine for Metal, CUDA and Vulkan.github.com · GitHub · 1.8k stars
Evidence
Medium evidence: several community reports point the same way, but there is no official confirmation yet.
Details written Nov 12, 11:37 UTC from the sources listed above.