Competitive learning is an unsupervised learning algorithm where neurons in a neural network compete to become active, leading to specialization among neurons for different input patterns. This mechanism is often used for clustering and feature selection, enabling the network to discover patterns in the input data without explicit labels.