Integer factorization is the mathematical process of decomposing a composite number into a product of smaller integers, specifically prime numbers. This problem is computationally challenging and forms the basis of the security of many cryptographic systems, such as RSA encryption, due to the difficulty of factoring large numbers efficiently.