Floating point arithmetic is a method of representing real numbers in a way that can support a wide range of values by using a fixed number of digits. It is essential in computing for handling very large or very small numbers, but it introduces rounding errors and precision limitations due to its finite representation of infinite real numbers.