ISR latency refers to the time delay from when an interrupt is generated by hardware to when the interrupt service routine (ISR) begins execution in a system. Minimizing ISR latency is crucial in real-time operating systems to ensure that critical tasks are performed within the required timelines for system stability and performance.