Startup and not want to be bothered about physical servers and related headaches? You need to signup with #AWS or one of the other cloud providers.
Even when we had physical servers it was not an on-premise device. Leased from a provider. But not build around cloud principles so no elasticity etc.
One summer month when in-bound traffic went up 3x was sufficient to quickly realize the value of moving to Amazon EC2 instances.