Cave formation, or speleogenesis, occurs primarily through the dissolution of soluble rocks such as limestone, gypsum, and dolomite by natural acids present in groundwater. This process can take millions of years, resulting in intricate structures like stalactites and stalagmites, which contribute to the complexity and beauty of caves worldwide.