Gaussian distributions, also known as normal distributions, are fundamental in statistics due to their symmetric, bell-shaped curve characterized by mean and standard deviation. They are central to the Central Limit Theorem, which states that the distribution of sample means approaches a normal distribution as the sample size increases, regardless of the original distribution's shape.