You Don't Need Matrix Calculus

Rather than mastering complex matrix calculus rules, physicists discovered a simpler approach: write out formulas with explicit indices and use ordinary differentiation, which requires only basic calculus knowledge and works universally. The author demonstrates this technique by deriving the gradient of a neuron activation function, showing that index notation yields results faster and more clearly than formal matrix calculus, which only applies to specific formula types and requires memorizing numerous specialized rules.