The 'Count to Infinity Problem' is a routing issue in distance vector protocols where routers continuously increment path costs to a destination that has become unreachable, potentially leading to routing loops. This problem arises because routers only share information with their immediate neighbors, lacking global network awareness to detect unreachable nodes efficiently.