Overview

tinygrad is a deep learning framework sitting between PyTorch and micrograd, maintained by tiny corp. It pursues minimalism and hackability — the entire framework is just ~200 Python source files yet supports the full DL workflow from training to inference.

Python CUDA Metal OpenCL WebGPU

Core Components

  • Tensor Library: PyTorch-like Eager API with autograd
  • IR Compiler: Kernel fusion & lowering, JIT + Graph execution
  • Training Utils: nn, optim, datasets — complete training suite
  • LLM Inference: GGUF format support for open-source LLMs

Hardware Backends

NVIDIA CUDA/PTX, AMD ROCm/CDNA/RDNA3/RDNA4, Apple Metal, OpenCL, Qualcomm QCOM, WebGPU, LLVM IR, ONNX import.

Key Features

  • Lazy Execution & Kernel Fusion (DEBUG=3/4 to see fused kernels)
  • BEAM Search for optimal kernel configs
  • Process Replay testing across changes
Deep AI Open