When you get to a point where you feel like this whole thing is going to fly, I'd recommend starting to think if paying the "cloud tax" (resources spent around EC2 stability issues and the cloud-specific stuff) a good idea in your particular case. There are some companies that benefit greatly from the elasticity of the cloud (the ability to scale up and down along with their specific load demands), but many companies aren't like that. If your traffic is relatively stable and predictable (you do not have 10-100x traffic surges) and your infrastructure load does not grow linearly with the traffic, using real hardware over-provisioned to handle 2-5-10x traffic spikes without huge decrease in performance may be a better idea in terms of the cost.

Of course, you could start the company based on all of the PaaS magic sauce (databases, queues, caches, etc) provided by Amazon nowadays and only use EC2 to run your application code (AFAIU that's the ideal use-case for AWS) and just kill misbehaving nodes when an issue occurs, but then you need to factor AWS costs into your business plan because migrating away from a PaaS is almost impossible at any large scale, so you are going to stay with Amazon for a very long time.

There's three types of workloads that make sense to run on EC2:

a) Extremely spiky/seasonal loads (batch jobs, event/campaign traffic)

b) Loads that can be structured as to run entirely from spot-instances (worker-pools)

c) Loads so small that the markup versus rented/dedicated hardware just doesn't matter

Maybe I'd just add one more case here: some users are OK with locking themselves up to AWS by treating it as a platform from the day one and building on top of AWS database/queue/etc services. For those people using EC2 just to run the app code and replacing instances when they misbehave may be a good idea.

And to your point somewhere else in here, it is a hell of a thing to try and move away from that platform. Yeah, it's super easy to beat EC2 on cost of compute resources, but really if you're running everything yourself on top of compute at AWS then you're doing it wrong.

A lot of AWS services can be used by real hardware though, so it's not all or nothing.

For example, where I work we use S3 to store an archive of files but keep the working set of data cached on our web servers which are at codero.

We have video rendering servers which turned out to be much cheaper to do with a cluster of desktop-class hardware in a server closet at our office as opposed to the server grade GPU instances on EC2. The monthly cost of a single GPU instance at EC2 is more than the total cost of the hardware off of newegg.

However, for outages we have a script that spins up GPU instances on EC2 which is much more economical than having a separate set of servers somewhere just in case.

AWS is a success because there are no upfront costs, it lets you scale up very quickly, and you don't need in-house hardware expertise to maintain your machines. People are willing to pay a premium for these advantages.

1) you can't get new hardware delivered and get it up and running, all in under 10 minutes.

2) also, assuming the previously gathered hardware is not needed anymore, you can't just return it and say "i used it only for two days because i had a traffic spike, take this 200$ and we're okay.

3) you can't programmatically install, configure, reinstall and reconfigure hardware configurations, networking and services on phisycal services. At least, not as easily.

Many others, but these are very valid points.

Of course, Amazon is not the solution to all of the problems you could ever have, but still it solves a great deal of problems.

2) SL has hourly physical server rental now (turn up is quoted at 20-30 minutes though)

3) SL has an API for ordering changes, you can setup a script to run on first boot (and probably system images too). What are you thinking for network configuration? Really the only thing I've had to configure on SL is port speed (somewhat API accessible, but not if they need to drop in a 10G card/put you on a 10G rack, etc), and disabling the private ports (API accessible, real time changes).

With AWS I can scale up easy, not have to worry about doing things like replacing failed hard disks, and most importantly I can be in multiple geographic sites for no additional cost. That to me right now is worth a 50% premium as the cost for doing that would be higher than that savings.

I think if you reach a certain scale, and have predictable usage, it is not a bad thing to setting up cabinets in 2 or 3 locations. We have found too that a lot of Colos are getting bought up and then will not lease you a few cabinets. They want to sell only to people who want a cage, or entire room. It is hard for small to medium sized businesses.

Colocation is a very different beast and I certainly would not encourage anybody to do that until a very large scale when rented hardware economics stop working for them.

Sorry, but that is mostly a lie.

Running a non-trivial app on EC2 is significantly more complex than doing the same on (rented) bare metal. Scaling to a massive size can be easier on EC2, but only after you paid a significant upfront cost in terms of dollars and development complexity.

Is your app prepared to deal with spontaneous instance hangs, (drastic) temporary instance slowdowns, sudden instance or network failures?

Did you know that ELBs can only scale up by a certain, sparsely documented amount per hour?

Or that you need a process to deal with "Zombie" instances that got stuck while being added/removed to ELBs (e.g. the health-check never succeeds).

Or that average uptime (between forced reboots) for EC2 instances is measured in months, for physical servers in years?

Or that Autoscaling Groups with Spot instances can run out of instances even if your bid amount is higher than the current price in all but one of the availability zones that it spans?

The list of counter-intuitive gotchas grows very long very quickly once you move an EC2 app to production.

This is a surprise to me, given that I work at an AWS shop doing things other people would call "DevOps". AWS doesn't automate provisioning or provide a (worthwhile) deployment pipeline, andAWS doesn't react (except in crude and fairly stupid ways) when something goes wrong or out-of-band.

No, they don't. They provide the tools, its still up to you to orchestrate it.

But wouldn't that apply also to SoftLayer?

The main benefits of Amazon is that it: a) allows you to scale down i.e. buy services in smaller portions than complete physical servers and b) APIs c) integrated features

You could probably pay for one devops position once your infrastructure gets to 10 physical servers.

The best resource to research different providers are the webhostingtalk.com forums. You can also contact me and I will do my best to advise you based on your desired criteria.

*Full Disclosure: I'm the founder/owner of a dedicated hosting company

I could go on and on about those, but other options were even more painful.

I've been running three 32GB servers (each with 3TB storage) with them for 2+ years now and the only outage I've experienced is the switch (5 port GBit) dying once. Hetzner tech replaced it in under an hour.

These three servers cost me €263/month (that's total, not each). Included in that monthly price is an additional IPv4 for each server, a private 5port Gbit switch, remote console access and 300GB of DC backup space.

There are probably better deals available now (i.e. more RAM at the same price) than the one I'm on since it's old and not offered on their site any more (/makes note to self to call Hetzner sales)

I wouldn't want to do it, which is why I'd rather work for somebody who'd pay for AWS, but I think there's a thing in there somewhere for those who want to dig.