Big O notation is a mathematical concept used in computer science to describe the upper bound of an algorithm's running time or space requirements in terms of input size. It provides a high-level understanding of the algorithm's efficiency and scalability, allowing for the comparison of different algorithms regardless of hardware or implementation specifics.