AWS Graviton2: What it means for Arm in the data center, cloud, enterprise, AWS
AWS re:Invent
Amazon Web Services launched its Graviton2 processors, which promise up to 40% better performance from comparable x86-based instances for 20% less. Graviton2, based on the Arm architecture, may have a big impact on cloud workloads, AWS' cost structure, and Arm in the data center.
Graviton2 was unveiled at AWS' re:Invent 2019 conference and ZDNet was debriefed by the EC2 team in an exclusive. Unlike the Graviton effort and A1 instances unveiled a year ago, Graviton2 ups the ante for processor makers such as Intel and AMD. With Graviton2, AWS is making it clear that it is serious about Arm processors in the data center as well as moving cloud infrastructure innovation at its pace.
"We're going big for our customers and our internal workloads," said Raj Pai, vice president of AWS EC2. AWS is launching new Arm-based versions of Amazon EC2 M, R, and C instance families.
Indeed, Graviton2, which is optimized for cloud-native applications, is based on 64-bit Arm Neoverse cores and a custom system on a chip designed by AWS. Graviton2 boasts 2x faster floating-point performance per core for scientific and high-performance workloads, support for up to 64 virtual CPUs, 25Gbps of networking, and 18Gbps of EBS Bandwidth.
AWS CEO Andy Jassy said the new Graviton2 instances illustrate the benefits of designing your own chips. "We decided that we were going to design chips to give you more capabilities. While lots of companies have been working with x86 for a long time, we wanted to push the price to performance ratio for you," said Jassy during his keynote. Jassy added that Intel and AMD remain key partners to AWS.
We'll get into the specs and performance comparisons in a bit, but here are a few strategic takeaways from Graviton2:
- AWS isn't going to wait for the tech supply chain to innovate for it and is making a statement with performance comparisons against an Intel Xeon-based instance. The EC2 team was clear that Graviton2 sends a message to vendors that they need to move faster and AWS is not going to hold back its cadence based on suppliers. Also: AWS launches EC2 instance based on custom-built Inferentia chip
- The first installment of AWS' next-generation EC2 instances will be based on Graviton2. Yes, Intel and AMD will power next-gen instances in the months ahead, but AWS said Graviton2 is offering enough performance to be considered next-generation EC2.
- AWS is looking to become the Apple of the cloud. AWS will have a choice of processors, software providers as well as services and instances. But AWS is putting its weight behind the 7nm Graviton2 processor because it can offer more value to customer via its own silicon and integrated stack. AWS' decision to build its own multi-purpose CPU rhymes with Apple's call to make its own chips.
- Graviton2 is built under a 7nm process and that makes AWS one of the few chipmakers to garner that access to that manufacturing technology. Graviton2 has 30 million transistors. For comparison, AMD's Epyc Rome has 39.5 million.
- The software ecosystem around Arm has developed to the point where AWS Graviton2 can take on more advanced workloads. The core of this ecosystem is AWS' Nitro platform that abstracts the underpinning hardware via a light hypervisor. The short version is that workloads can be moved to various hardware architectures easily.
- And AWS' initial strategy is to move its internal services to Graviton2-based infrastructure. Graviton2 required significant investment, but AWS can garner returns and improve its operating margins due to the ability to cut out middlemen involved with procuring processors, power savings due to Arm and efficiency gains from optimizing its own infrastructure.
In other words, AWS' Graviton2 launch may indicate that Arm in the data center is becoming a reality after years of false starts, delays, and hopes.
AWS services like Amazon Elastic Load Balancing, Amazon ElastiCache, and Amazon Elastic Map Reduce have tested the AWS Graviton2 instances and plan to move them into production in 2020.
"I think AWS custom built this for its infrastructure with plans to take it mainstream," explained Patrick Moorhead, the principle of Moor Insights and Strategy. "People will move a bunch of workloads over, but it will take years and years. AWS wants multi-vendor and it's not anti-AMD or anti-Intel, but wants the healthy competition. The other message here is that AWS is not going to commoditize IaaS. AWS is taking it to the next level."
How Graviton2 came together
The big splash of Graviton2 contrasts with last year's almost understated launch of the first Graviton instances, which were designed for a limited set of workloads such as messaging and serverless AWS Lambda functions. Turns out the original Graviton was more of an effort to work out issues, build a developer ecosystem and lay the groundwork for Graviton2. "We didn't oversell it. Graviton at first was a narrow single product," said Pai.
Yes, the first Graviton A1 instances did land a few customers such as Smugmug and Ancestry, but 2nd Watch noted that few customers were using those instances. Flexera's RightScale also didn't have A1 instance users in its customer base either. On the first Graviton chip-based instances, Jassy said "we've been pleasantly surprised" with adoption.
AWS certainly didn't oversell the first Graviton. What the first Graviton effort did is allow a software ecosystem to develop. The new Graviton2 instances are supported by several open-source software distributions and services. Here's the list of support for Graviton2 instances:
- Amazon Linux 2, Ubuntu 16.04, 18.04 and newer, Red Hat Enterprise Linux 7.6 and 8.0, SUSE, Fedora, Debian, FreeBSD, NetBSD Linux operating systems, as well as the Amazon Corretto distribution of OpenJDK.
- Docker Desktop, Amazon ECS, and Amazon EKS for containers.
- Amazon CloudWatch, AWS Systems Manager, AWS CodeCommit, Cloud9, CodePipeline, and Amazon Inspector for tools.
- And AWS Code Suite and Jenkins for developer tools.
Simply put, Graviton2 has more momentum behind it out of the gate, but none of it would be possible without Nitro, an AWS platform that abstracts the underlying hardware via a thin hypervisor. Nitro has been around for years, but AWS has rearchitected its infrastructure based on it. The upshot is that AWS can move workloads to any architecture and processor.
Must read:
- Can AWS be caught? Here's how its cloud computing rivals can improve their chances
- Amazon AWS: Complete business guide to the world's largest provider of cloud services
- What is cloud computing? Everything you need to know from public and private cloud to software as a service
Pai explained that Nitro is foundational to the Graviton effort as well as AWS overall.
"Nitro is how we are using virtualization and optimizing the stack across hardware and software," said Pai. "Nitro allows us to modularize and build new platforms easier. ASICs take functions off of core hardware and are optimized for tasks."
Nitro also includes a series of cards that take care of various functions. In a nutshell, Nitro is the hypervisor created for AWS and also allows the cloud provider to swap out processors and architectures more easily. "There is no customization required at the hypervisor level," he said.
A few core facts:
- Nitro launched in November 2017.
- Started development in 2013.
- The Nitro hypervisor is built for AWS.
- Nitro cards cover networking, storage, system controllers.
- A Nitro security chip is integrated into the motherboard.
- Nitro hypervisor has bare-metal-like performance.
- Nitro got a boost (as did Graviton) from Amazon's purchase of chip designer Annapurna Labs in 2016.
"AWS has been working on Nitro for years and likely has tens of millions of Arm processors serving as the front end to Xeons," said Moorhead. "There's nothing like Nitro out there. It's a tiny hypervisor that means it's not as much work to move to any architecture without lag."
Nitro opened the door for more Arm in AWS infrastructure, the cloud giant was focused on customized systems on chips not necessarily cores. The cores need to be standard and Arm just so happened to be pondering a move into the data center. ZDNet was chronicling Arm's data center prospects back in 2013 here, here and here. Let's just say the revolution basically tripped over the starting line. Startups came and went. Efforts were started and then scrapped. And the software lift wasn't exactly easy given most workloads were on x86.
Cloud vendors were into the power savings of Arm even if the raw performance wasn't there. Distributed workloads made sense. However, the market didn't quite develop.
But in 2018, Arm unveiled Neoverse, its brand designed for the data center to the edge. Neoverse for Arm represented a core that could be utilized by the likes of Qualcomm, Marvell, and Nvidia. With Arm's development of Neoverse and recent improvements, AWS could focus on the system on a chip, or SoC. Graviton2 has fp16 and int8 support to accelerate machine learning workloads.
Also: Arm unveils Neoverse N1, E1 platforms for cloud-to-edge infrastructure
Another development was that Softbank acquired Arm and freed it up to invest more in data center opportunities over the last three years.
Pai and the EC2 team noted that AWS worked closely with Arm on tools such as context switching. AWS was an early licensee and helped with Neoverse by providing a feedback loop in what was termed a "tight relationship." "We are providing feedback based on workloads. That benefits the industry, but all the features are available for us," said Pai.
Must read:
- Cloud cost control becoming a leading issue for businesses
- Cloud cost control also a challenge for small businesses and freelancers
In the end, AWS' move to fully embrace Arm was more about software than silicon, but the tooling, developer platforms, and applications are more developed than ever. Yes, customers will have to create multi architecture approaches with their applications, but AWS and its customers are likely to move that direction.
Performance and Arm
The EC2 team at AWS focused on Graviton2's performance instead of things like power consumption and cost that have dominated arguments for Arm-based data center processes.
In many ways, the AWS argument for Graviton2 was Tesla-esque. Yes, there are environmental benefits to the Tesla, but the autos can blow past other cars on the road. Design and performance sells cars. Similarly, Graviton2 will likely consume less power, but the performance argument is the one that'll take Arm mainstream.
AWS went through a series of slides and workloads benchmarking Graviton2 instances versus the most recent EC2 instances powered by Intel. The workloads were varied, but the one constant is Graviton2 instances performed better.
There is one key caveat in that Graviton2 is optimized to running AWS compute so the comparisons revolved around AWS instances. I get weary of benchmarking processors due to different conditions, specs, and plenty of footnotes. Moorhead, another veteran of processor benchmarking spats, said AWS' comparisons were notable because they were pure. "AWS' performance marks were on its infrastructure. And that's as pure as you can make it. It's not a lab. It's not a different compiler. This is performance on AWS in its environment," said Moorhead.
In other words, AWS with Graviton doesn't have to account for every workload. AWS has a much smaller number of applications to run relative to AMD and Intel, which have to simulate performance against thousands of legacy applications.
"I tried to poke holes in this, but each time I came away with one question: Is AWS going to be able to make enough Graviton2 chips," quipped Moorhead.
Perhaps the biggest takeaway from the Graviton2 performance is that Arm in the data center will be consumed primarily through the cloud. Sure, Arm may get some enterprise traction should Qualcomm, Marvell and others make server inroads. But using Arm in the enterprise is going to require a heavy software lift that may not be worth it.
Here are the key benchmarks highlighted by AWS.
AWS as the Apple of cloud computing
On one level, AWS' Graviton2 isn't all that unique. The big cloud vendors all have some efforts with custom processors. Google Cloud Platform has largely focused on inference, machine learning, and artificial intelligence with its TPUs. Microsoft Azure has Arm efforts with partners as well as programmable silicon.
What's different with AWS' approach to Graviton2 is that it is an all-in approach to competing with Intel and AMD even as it supports those instances.
In addition, AWS is taking a downright Apple approach to infrastructure as a service. This approach will also prevent IaaS, an AWS stronghold, to be commoditized. Here's a look at the Apple and AWS similarities.
Apple has invested heavily in silicon innovation to power its iPhones and iPads. The AWS investment in Arm-based Graviton2 processors is similar except the platform is a cloud, not a mobile device.
By being vertically integrated, Apple can better meld its hardware and software in a seamless experience. AWS' Nitro is another example of melding software, hardware, and platform together to optimize any workload.
The returns are way better without a middleman involved. Apple is paying Qualcomm but doesn't like it. Apple was hoping Intel could deliver on 5G but ended up buying the unit to control its own fate. AWS could also rely on Intel, AMD, Nvidia, or some other vendor, but wants to control its own fate and potential profit margins.
What if Graviton2 takes on a third of AWS workloads? AWS can cut its Intel spending and that can't be good for the chip giant's data center business. Given that AWS accounts for the bulk of Amazon's profit, the returns on Graviton could be magnified should more workloads go to Arm.
"What AWS is really doing is fighting the commoditization of IaaS. They are so big and investing so much that they are increasing the barriers to entry," said Moorhead. "There's strategic value in silicon."