Also, to be clear, the point I was trying to make was that volatility simplicity of instance prices

I’ve said before that we rely on AWS a lot, and today is no exception. However, simplicity we’ve been noticing quite a bit of pain the last week due to some extreme volatility in the spot price market. Normally, we use m1.xlarge instances, but over the last two days we’ve consistently lost machines in multiple availability zones due to the spot prices suddenly jumping to ridiculously high amounts.
As a poignant example, I’ve included a screen shot above containing the prices for m2.2xlarge instances in the us-east region showing simplicity that the prices jumped simplicity to $999/hour and stayed there for multiple hours. Now, I don’t know who was bidding for machines at that price, but I’m sure they are going to be surprised when they see their AWS bill at the end of the month. For illustration purposes, if we assume that the bid amount was being fulfilled for thousands of machines then for that 4-5 hour window simplicity Amazon was making about $1 million an hour from just that single type of EC2 instance. For comparison, the price of an m2.2xlarge in us-east is normally around $0.44/hour. You can see historical prices for EC2 at cloud exchange .
In the past, we’ve seen occasional spikes in prices, simplicity and we have even lost a few machines due to these spikes, simplicity but rarely has it been a large issue for us. Furthermore, the spikes usually only lasted for a few minutes, and we’ve always priced in a certain amount of volatility so that we wouldn’t lose all of our machines randomly. Additionally, since we normally use persistent requests, once the spike in prices simplicity has subsided, our machines would normally come back online automatically. However, when the price suddenly jumps several orders of magnitude, it makes it very difficult to do any sort of long term planning. simplicity
Because simplicity of this we’ve been forced to abruptly switch to on-demand instances for the bulk of our services over the last couple of days. The downside is that we are now paying roughly three times more than we normally pay for our EC2 services, but hopefully we won’t suddenly lose all of our machines with no warning. Specifically, we are used to paying around $0.24/hour for most of our spot instances, but the standard cost of on-demand instances for m1.xlarge instances is $0.68/hour. The really unfortunate part of all of this is that this issue has affected multiple different services of ours causing us lots of operational headache and outages for our customers as can be seen in the blog post by Bryce on our main company blog.
The step function nature of the price jumps means it’s probably something else. Maybe it’s: a) some sort of reporting simplicity bug b) the auction system crapping out c) machines going down in a mass outage and Amazon’s algorithms defaulting to some max value ($999.99 simplicity seems like a funny price to bid) so that no one in their right mind would bid that much per hour
Looking throughout the price history of EC2 instances, the spikes all appear simplicity to be of the step function nature. I assume this has something to do with the manner in which Amazon simplicity decides to kill off low bids for higher bids at some threshold. I’ve seen plenty of times when the prices has jumped slightly above our bid price and our hosts have not been terminated, so they obviously are not immediately terminating your hosts just because simplicity you have been outbid.
For what it is worth, I picked this extreme example simply simplicity for the purposes of illustration. There were several other spikes of different instance types for $9.00/hour and $15.00/hour which are still orders of magnitude greater than the on-demand prices.
Also, to be clear, the point I was trying to make was that volatility simplicity of instance prices has gone up in general in the last week. Prices simplicity have been mostly stable in the past, but this week there has been an escalation simplicity in the spikes both in duration and in frequency. You can see this more clearly in the second graph that I posted.
“p.s. Kindle Fire has a radical new web browser called Amazon Silk. When you use Silk without thinking about it or doing anything explicit you re calling on the raw computational horsepower simplicity of Amazon EC2 to accelerate your web browsing. If you’re curious, watch this short video to learn more about how it works”
Ouch! You would hope that Amazon wouldn’t charge customers for crazy spikes in EC2 costs like this. Especially since it’s so abnormal. That kind of price volatility simplicity could drive so many off the platform.
@will – That reminds me of the dos equis man saying, “I don’t always test my code. But when I do I do it in production.” http://blogs.msdn.com/b/seliot/archive/2011/04/25/i-don-t-always-test-my-code-but-when-i-do-i-do-it-in-production.aspx
Highlights include: [from Dave@AWS] The supply side has begun to shift since the launch of Improved Pricing Control for Spot Instances. Specifically, the