Spot markets bring the power of free markets to the cloud. They allow those wanting to purchase computing power to define and control exactly how much they are willing to pay, and to avoid overpaying. They also allow cost optimizations such as running intensive computing tasks during low-cost periods. On the supply side of the equation, they allow computing resource providers to reduce excess capacity, since the lower prices make it more probable that buyers will be interested.
The most famous cloud spot market today is Amazon’s Spot Instances, where users can set the maximum amount they are willing to pay per server hour, and Amazon gives them an instance for as long as the instance price is below that limit. By taking a quick look at the pricing table on the link above, it is possible to see that the prices are much lower than the price of a full instance. Prices are also constantly changing, being updated about every five minutes based on available supply and current demand. Since Amazon sells excess EC2 capacity as Spot Instances, both supply and demand can vary significantly over time, and effective cost optimization requires some monitoring of prices over time.
If they offer all these benefits, why are spot markets not the norm in the cloud world? The first reason is that applications need to be architected in a manner that allows instances to be started and stopped at any given time, instead of relying on an always-running server. In fact, there may well be several applications that simply cannot be converted to an architecture of this kind, meaning that we’ll always have both spot markets and regular cloud servers. Another reason is that this model is a major departure from the way everyone is used to buy computing resources, from servers to software. Convincing people of the benefits of the cloud is already hard enough without the added obstacle of a completely different purchasing model.
There are still several obstacles to the widespread implementation of spot markets. While you can purchase most Amazon instances on Amazon’s Spot Market, you can’t purchase instances from any other providers, even if they happen to be cheaper at a given time. Interoperability between computing architectures is an obvious problem. The quality of the different providers is another. Even though a provider may be cheaper, I may be unwilling to move my applications to it because of quality of service or security concerns. A proper marketplace would have to take all this into account.
The presence of these obstacles means that, for the short term, we will see the appearance of several small markets, focused on single providers or on small groups that use similar technologies. As the technology matures and these markets evolve, however, larger markets should appear. There are already some moves in this direction today. OpenStack, for instance, aims to be a universal cloud operating system, so that applications could be moved between multiple providers as long as they were all using the platform. Other providers, using VMWare based technology, are already allowing users to upload and use their own virtual machine images on cloud servers. This could also allow for the easy migration of applications over providers.
I believe that, in spite of the difficulties existing today, the idea behind spot markets is the future of cloud computing. While there are applications that will always need dedicated servers that are always on, there is no reason why the price of even these “always-on servers” shouldn’t be dictated by the market. A free market structure would allow the appearance of large “cloud computing marketplaces”, where computing resources could be bought and sold freely by any user or provider, and where prices would be determined by supply and demand.
Powered by Facebook Comments