A secondary index is an additional data structure that improves the performance of query operations by allowing quick access to records based on non-primary key attributes. It enhances database efficiency by providing alternative paths to data retrieval, but it can also introduce overhead due to the need for index maintenance during data modifications.