A few weeks back I set a goal of trying to find out which processor was the most performant.

I created a repeatable approach to delivering a simple WordPress environment that we could manipulate the type of CPU and we had a way to measure the performance of the instances.

Now we need to add pricing into the equation and do some comparisons.

Calculations

While the distributed testing tool does give us a nice UI to interface with and review results, to gather results from several tests would’ve been a little cumbersome. Thankfully it is stored in DynamoDB and with a little bit of python I could extract the data into CSV output.

#!/usr/bin/env python3

import boto3

client = boto3.client('dynamodb')


testIds = ['ZgRIW1e68', 'rqKqImReS', 'NVdSNqwFd', '32b6ZO6Og', 'PqiiWG8nd', 'QSFzcJaE8']

for testId in testIds:
    response = client.get_item(
        TableName='LoadTester-ScenariosTable-8Q70ANCHY15K',
        Key={'testId': { 'S': testId } }
    )
    
    line=''
    
    line = response['Item']['testName']['S'] + ','
    # AvgRt
    line = line + response['Item']['results']['M']['avg_rt']['S'] + ','
    # AvgLt
    line = line + response['Item']['results']['M']['avg_lt']['S'] + ','
    # AvgCt
    line = line + response['Item']['results']['M']['avg_ct']['S'] + ','
    # AvgBw
    line = line + str(int(response['Item']['results']['M']['bytes']['S']) / 1024) + ','
    # Total
    line = line + response['Item']['results']['M']['throughput']['N'] + ','
    # Success
    line = line + response['Item']['results']['M']['succ']['N'] + ','
    # Error
    line = line + response['Item']['results']['M']['fail']['N'] + ','
    # Req./Sec
    line = line + str(int(response['Item']['results']['M']['throughput']['N']) / int(response['Item']['results']['M']['testDuration']['S']))
    
    print(line)

Now that we have the data from all the tests we need to get pricing. I used the AWS Calculator to gather this data.

Then I created an Excel Spreadsheet to manipulate the data, create graphs and calculate performance vs cost.

Outcomes

First off, let’s look at how many requests per second each setup could manage. Higher is better.

Winner: ARM EC2 and Intel RDS

Next let’s compare the response time and latency. Lower is better.

Winner: ARM EC2 and Intel RDS

Next let’s compare the connection time. Lower is better.

Winner: AMD EC2 and Intel RDS

Next let’s look at the request counts.

• Total Higher is better.
• Successes Higher is better.
• Errors Lower is better.

Winner: ARM EC2 and Intel RDS

Finally let’s bring price into the comparison.

To do this I calculated the price per month of the instances in USD with no storage, on demand and single Instance and AZ.

To calculate the price performance (requests per dollar per month) I used the following calculation:

Requests/Second * Seconds in a Month / Price/Month

• Price per month Lower is better.
• 1000 req/$/month Higher is better.
• Performance as a percentage (comapred to ARM EC2 and ARM RDS) Higher is better.

Winner: ARM EC2 and ARM RDS

Conclusion

While this analysis wasn’t significantly scientific it at least gives us a starting to point that we should really consider looking at the new Graviton. AWS are suggesting a 40% price performance increase. I’ve noted a few articles talk about a 40% increase in performance and 20% reduction in cost. If we compare ARM EC2 & RDS with Intel EC2 & RDS, we see about 30% increase in price/performance. An ARM to Intel price comparison I found to be about 13%. On a pure performance comparison of all ARM compared to Intel, I saw about 30% difference.

An interesting take away was the best performing solution was when the EC2 instance was ARM and the RDS instance was Intel in almost every comparison. AMD EC2 instance was marginally faster at connections, but that is likely due to the fact that it was dealing with less connections overall. My guess is that the MySQL code is optimised for Intel over ARM, while PHP isn’t tied so tightly, but without digging into the depths of the source code of either, I can only guess.

Ultimately it seems that ARM Graviton2 processors are really worth considering if you have workloads that can make use of these processors. - This is basically anything that’s open source or uses interpreted languages such as Python, PHP or Ruby. Java is even available for ARM and so is Corretto (AWS’s distribution of Java). Unless you have compiled applications with no source code or run Windows, you really should be considering Graviton2 processors.