Many scientists use computers for important calculations, but who makes sure they produce the correct answer? For this reason, the design and validation of computer hardware has become a topic of intensive research. This talk is a case study demonstrating the importance of rigorous mathematics for this field.

In March 1993, Intel introduced the Pentium processor; it contained a record 3.1 million transistors and could perform over 100 million operations per second. Within two years, a bug in the floating point division unit (FDIV) would cost the company an estimated $475 million dollars.

The subsequent investigation covered topics in algorithm development, number theory, and statistics. I will discuss the SRT division algorithm used by the processor and the problems that it poses in validation, how the bug was discovered and contained, and the interesting statistical distribution of errors.