The bias of an estimator is the difference between the expected value of the estimator and the true value of the parameter being estimated. A biased estimator systematically overestimates or underestimates the parameter, while an unbiased estimator has an expected value equal to the true parameter value.