Secondary storage refers to non-volatile devices that store data persistently, allowing for data retrieval even when a computer is powered off. It is essential for storing large volumes of data and complements primary storage by providing additional space and long-term data retention.