This article describes Haven – a system for holistic load balancing and auto scaling in a multi-tenant cloud environment. It takes into account the utilization levels of different resources as part of its load balancing and auto scaling algorithms. Haven is able to provide performance at par with a hardware load balancer while still providing the flexibility and customizability of a software load balancer.