Rank deficiency occurs in a matrix when its rank is less than the maximum possible, indicating linear dependence among its rows or columns. This can lead to issues such as the inability to find unique solutions to linear systems or to invert the matrix.