Horizontal scaling, also known as scaling out, involves adding more machines or instances to a system to handle increased load, rather than upgrading existing hardware. This method is cost-effective and enhances fault tolerance and redundancy by distributing workloads across multiple servers.