A non-convex function is a type of function where the line segment between any two points on the graph may not lie entirely above or on the graph, leading to multiple local minima and maxima. This characteristic makes optimization problems involving non-convex functions more challenging due to the possibility of getting trapped in local optima instead of finding the global optimum.