Tutorials

Interactive Jupyter notebooks to learn Nabla’s features and capabilities through hands-on examples. All notebooks are preliminary and subject to change.

Interactive Notebooks

• A Primer on Automatic Differentiation
  • The Jacobian: The Full Derivative
  • 1. Jacobian-Vector Product (JVP): The “Pushforward”
  • 2. Vector-Jacobian Product (VJP): The “Pullback”
• Training an MLP (CPU and GPU)
  • 1. Imports and Setup
  • 2. Configuration
  • 3. Model and Loss Function
  • 4. Data Generation and Parameter Initialization
  • 5. Optimizer (AdamW)
  • 6. Defining the Core Training Step
  • 7. Defining the Execution Modes
  • 8. Training and Benchmarking Loop
  • 9. Performance Comparison
  • 10. Final Evaluation & Visualization
  • 11. Conclusion
• Training a Transformer (CPU and GPU)
  • 1. Imports and Configuration
  • 2. Positional Encoding
  • 3. Scaled Dot-Product Attention
  • 4. Multi-Head Attention
  • 5. Position-wise Feed-Forward Network
  • 6. Layer Normalization
  • 7. Encoder and Decoder Layers
  • 8. Embedding Lookup
  • 9. Full Transformer Forward Pass
  • 10. Loss Function
  • 11. Parameter Initialization
  • 12. Data Generation
  • 13. Optimizer (AdamW)
  • 14. Defining the Training Step & Inference
  • 15. Exploring Nabla’s Execution Modes
  • 16. Training and Benchmarking
  • 17. Performance Comparison
  • 18. Final Evaluation
• Program Transformations Uncovered
  • 1. Defining and Visualizing a Python Function
  • 3. Gradient Transformation
  • 4. Vectorization Transformation
  • 5. Compilation Transformation with MAX