A cost matrix is a table used in decision-making processes to quantify the cost associated with various actions and outcomes, often in the context of classification problems in machine learning. It helps in evaluating the trade-offs between different types of errors and optimizing the decision rules to minimize the overall cost.