After years of prepping for this moment, the world’s first ARM-based servers have been deployed in a production environment. Chinese search giant Baidu is using Marvell’s chips in a cloud storage application.
Chinese search engine giant Baidu is using ARM-based servers from Marvell making it the first company to depend on servers using the cell-phone chip in a production environment. Baidu is using the new ARM servers in its cloud storage application named Baidu Pan.
ARM, which licenses its IP to a variety of chip makers, had stated its intentions to enter the data center market back in 2010, as worries about energy efficiency increased and the needs of webscale computing customers changed. While less powerful than their Intel counterparts, a cluster of lower-power ARM chips is more power efficient on a performance per watt basis and some workloads don’t even need the performance characteristics of a big Intel core.
The combination of these two trends has led to a plethora of vendors from big names like Marvell and AMD to startups such as Calxeda to license ARM’s cores with an eye toward making servers. Holding ARM back so far has been the delay in building out 64-bit capable cores (they are expected later this year) as well as a lack of enterprise software running on the ARM platform.
But given the economics of these so-called wimpy cores and the limits of using ARM cores in the enterprise server market today, the use of ARM-based servers in the storage arena is not surprising. Storage usage scenarios are perfect in many ways because they don’t need a lot of raw performance, nor do they require 64-bit capable cores.
Thus, Baidu using ARM for storage makes sense. It’s also an area where Calxeda expects to see its first production deployments sometime this year, according to a conversation I had with Karl Freund, the VP of marketing of Calxeda last December. As for the Baidu deployment, it’s using the quad-core Armada CPU, Marvell’s storage controller, and a 10Gb Ethernet switch all integrated on a single system on a chip.
Marvell’s release says the chip firm customized the ARM servers specifically for Baidu’s cloud storage requirements, taking the concept of server customization common in webscale deployments to the chip level. Marvell says the platform is designed to increase the amount of storage for conventional 2U chassis up to 96 TB, and to lower the total cost of ownership by 25 percent, compared with previous x86-based server solutions. The end result should cut Baidu’s power in its data center by half according to the release.
more details for those with a tech interest:
Marvell notches another design win after Dell and Codethink boxes
By Timothy Prickett Morgan • Get more from this author
Posted in Servers , 20th February 2013 21:05 GMT
Free whitepaper – HP 3PAR storage efficiencies that unlock green savings
Chip maker Marvell has notched up its third public design win for an ARM server, this one at Baidu, one of the two big search engine giants in China.
Avi Liebermensch, manager of server products at the maker of ARM processors and other kinds of chips, tells El Reg that the Baidu server is based on the company's Armada XP MV78460. This is the same quad-core processor that Dell chose for its "Copper" ARM sled servers announced last May and purchased by unspecified customers of its Data Center Solutions bespoke server business unit. The Baserock slab ARM server from Codethink also uses this Marvell processor as its main engine.
Baidu has deployed an ARM server for cheap and dense storage with local processing
Liebermensch was not at liberty to give a lot of details on the Baidu machines, and China is celebrating its New Year right now so people in Beijing are not around to answer calls. But Marvell did sneak us a picture and gave us some insight into the machine that Baidu has worked with Marvell and an unspecified ODM to create.
As you can see, the Baidu ARM box has six sleds that slide into a 2U chassis, and Liebermensch says that this box has a total of 24 drives and supports a maximum of 96TB. That almost certainly means it uses 4TB SATA drives. The Dell copper sled crammed four of these Marvell Armada XP processors, four SATA drives, and a baby Ethernet switch linking the four processors to each other and to the outside world for the Copper sleds, and it could be that the Baidu box is built by Dell or is a knockoff of the Copper design.
The processor count on the Baidu server sled was not divulged. What Liebermensch did say is that Marvell had grabbed a storage controller that can handle SAS or SATA drives over a PCI-Express controller built into the chip and an Ethernet switch that implements two 10 Gigabit Ethernet ports and six Gigabit Ethernet ports. This sounds like the Baidu box has a slightly different network and storage setup than the Dell Copper box.
Block diagram of Marvell's Armada XP MV78460 chip
The Armada XP MV78460 uses Marvell "Sheeva" PJ4B cores, which are a variant of the 32-bit ARMv7-MP design that has the 40-bit memory addressing added to it. That means each socket can address up to 16GB of main memory, in this case, which is more than enough for a lot of workloads.
Baidu has opted for a version of the chip running at 1.6GHz, just like Dell. The Armada XP chip has a crossbar interconnect that links all of the elements of the SoC together, which might be out there on a much larger motherboard if this was not an SoC. Each Sheeva CPU has a floating point unit and is linked over a coherency fabric to a 2MB on-die L2 cache. The memory controller has ECC scrubbing and runs at up to 1.6GHz, matching the clock speed if you want.
There are four PCI-Express 2.0 controllers on the chip, as well as controllers to drive two SATA peripheral ports and four Gigabit Ethernet ports. There is also a 4Gb/sec packet processor (handy for some things, not so much for others) and a security engine for encrypting and decrypting data. All in a 15 watt thermal envelope.
It is not clear which "Cheetah" switch module that Baidu has embedded on the sleds in the server, but it could be the same as the "Cheetah 3" 98DX5156 Layer 2/3 network switching that Codethink is using in its Baserock Slab, an eight-node system that comes in a 1U chassis that is based on the same processor from Marvell.
What Baidu was looking for in this particular server was to get computing power right next to the storage, not to create a JBOD that then hooks into an external server. Given this, it is reasonable to assume that the Baidu box has four Armada XP chips nodes talking to four drives on each sled.
The servers run Ubuntu Server 12.10, which is the Linux from Canonical and which is the second release to formally support ARM servers. Marvel has also cooked up a systems management tool for Baidu that speaks IMPI 2.0 that can work with its existing control freakage.
Marvell is very excited, calling this the first real deployment of ARM in the data center, and says that Baidu is not just experimenting with machines, but actually installing these machines to run some of its "cold storage" software, known internally as Pan, after months of development and testing.
When pressed about how many machines Baidu had installed, Marvell would not say. That would irk its customer during the holiday. But what Marvell could divulge is that the setup had a 25 per cent lower total cost of ownership compared to previous x86-based servers used by Baidu for the same purpose.
But it doesn't appear that large web companies are contemplating a wholesale switch to microservers, which will limit the size of the opportunity for ARM. Jason Waxman, general manager in Intel's datacentre and connected systems group, said that, while these companies were interested in microservers, their overriding demand is for improved efficiency among higher-powered server processors; those sitting in the 95W-130W range.
"Across the large number of datacentres that sit at big web scale, far and away the propensity over the past couple of years has been to deliver more performance for that power envelope versus doing something radical like cutting power usage by 90 percent and taking a hit in terms of performance," he said.
This isn't a server in the classic sense. It's a specific function within a data center. There does seem to be a change in focus/language from the ARM camp:
Calxeda, the Austin, Texas-based startup that is building out highly dense, low power ARM-based servers has a new market in the storage world. During a visit last week to the company’s headquarters, company executives shared that in addition to web hosting and big data applications it sees a near-term opportunity in the storage world and that is has fielded more than 20 requests for proposals for systems using ARM-based processors.
Karl Freund, the VP of marketing for Calxeda, says the company has shipped about 3,000 nodes and 130 systems although none are deployed in production environments yet. He expects the first production deployments to occur at the end of the second quarter of 2013. But most of the conversation was about how ARM-based systems could be used today in the storage market. Not just for cold storage such as Amazon’s Glacier or Facebook’s photo storage effort, but even for the big storage systems for scale out storage and enterprise class storage appliances. Named customers who are evaluating the systems include Scale.io, Gluster and Inktank, the storage startup backed by Mark Shuttleworth of Ubuntu fame that is commercializing Ceph.
There are more, notes Freund, (pictured) who says that when Calxeda servers make it into production environments, they will likely be deployed first in a storage capacity, as storage customers don’t care if the chips are 64-bit compatible. For now, ARM-based systems are stuck only able to address less memory because ARM only has a 32-bit capable core design. Next year ARM will have a 64-bit capable design and systems will be built around them in 2014 (maybe even late 2013). Calxeda plans its 64-bit capable SoC for 2014.
But Calxeda isn’t waiting and in storage, it’s also not focusing on power consumption — the initial draw for ARM-based servers in the scale out data center. For the storage world, where spinning hard drives tends to suck huge quantities of electricity, adding a low-power has a negligable affect on the consumption of an overall system. However, Calxeda boasts that popping in more of its systems on a chip (SoC) are both cheaper and make for faster information transfer and retrieval.
Its tests show roughly a 4X improvement in IOPs for a rack of Calxeda SoCs versus x86-based systems. Adding Calexeda’s SoCs also cuts complexity because the entire system of processing and networking components are integrated on the SoC, and the terabit-plus fabric between cores also offers more network capacity between cores in a system –the so-called east-west networking traffic.
As the market for scale out computing, storage and networking changes the demands made on IT equipment, Calxeda and others are seeing an opportunity that may have begun in servers and the cloud computing environment, but certainly isn’t stopping there. No wonder Intel is trying to catch up with chips of its own. So far, it’s recently announced new Atom-based chips haven’t made the cut for most customers I’ve spoken with (the lack of integration of the entworking and processing hardware is a problem), but in 2014 it will have a new, integrated SoC as well. Then, the competition will really get interesting.
There is a certain percent of the market that is eager to do business with someone other than Intel. I'm all for new competition, if it's good. But here you have a new technology that is going up a well entrenched technology. If ARM is to sell on the merits of it's technology it may stand a decent chance in making headway. But a what I don't know is what the service contracts look like at a fair number of the larger corporations. If you're an IT manager that specs these servers you could find yourself in a bind and unemployed fairly quickly The real business world is often quite ugly.
On the other hand, cell phones and tablets don't have the same teeth. If a superior technology comes along it can replace the status quo over time. Blackberry was the king of the market, until it wasn't.
One large, vertical integrated CPU manufacture and designer poses a huge threat. If this is ARMH's future, get out of the stock today!
"a new technology that is going up a well entrenched technology"
" If a superior technology comes along it can replace the status quo over time."
"Blackberry was the king of the market, until it wasn't. "
"The real business world is often quite ugly. "
What you just don't get is that the world is going ARM !!!!
And ARM64 hasn't even started...
Keep looking backwards...