Google Tensor SoC And Pixel 6 Benchmarks Show Google’s Intentions For Building Its Own SoC

We recently covered some of the details around Google’s Tensor SoC at the launch of the Pixel 6 and Tensor SoC that Google had. But now the embargo for reviews has lifted and I’ve had an opportunity to test the Pixel 6 and Pixel 6 Pro’s tensor SoC. Many people believe that Google’s Tensor SoC is more akin to a Samsung Exynos semi-custom design and according to some code that Anandtech’s Andrei Frumusanu found, it probably is along those lines. It seems odd that Google would try to claim the SoC as their own even though Samsung is heavily involved in the manufacturing and modem and likely some of the chip design as well. Nevertheless, Google’s intentions with the Tensor SoC is to derive some of the AI performance and intelligence that it has created with the TPU and bring that down into a mobile SoC. Google claimed that nobody in the market was creating chips that satisfied Google’s needs for AI performance, so they created their own. 

It has been covered numerous times, but just as a refresher the Tensor SoC is a combination of off-the-shelf ARM CPU cores and GPU cores combined with Google’s own TPU and ISP. Google hasn’t really given details about the TPU or ISP’s full capabilities, but that’s what benchmarks and reviews are for. I had a chance to use the Pixel 6 Pro and Pixel 6 for the last week or so and the camera performance genuinely lives up to what people expect out of a flagship smartphone. However, I still wanted to benchmark the device to assess the chip’s actual performance against some of the most notable Android competition. One note, the Galaxy S21 Ultra in the US ships with a Snapdragon 888 and not an Exynos 2100 like many other parts of the world. Additionally, the RedMagic 6S Pro that I tested is the only Snapdragon 888+ device.

Benchmark tests & Methodology

For my testing, I ran GeekBench, GeekBench ML, 3DMark Wildlife, 3DMark Wildlife Stress Test and PCMark. I chose GeekBench because it’s a simple CPU benchmark and can show how Google’s decision to go with two Cortex X1 cores and two A76 cores instead of one X1 and three A78s affected overall CPU performance. To Google’s point, the company has said that it does not actually care about individual SoC component tests but rather a complete system AI test. And when you consider how few AI benchmarks are out there and ones that are easy to run, GeekBench ML was an easy decision. The important part in choosing GeekBench ML was not to test CPU or GPU performance individually, but instead to test them together using the NNAPI test which is Google’s own API that it uses for ML acceleration. This feels like the perfect test to use to compare AI performance across devices, especially since it does multiple types of ML workloads and tests FP32, INT16 and INT8 performance. 3DMark was important because it is cross-platform and the standard for 3D graphics performance, especially in gaming. I also chose to use the stress test because thermal throttling is a real problem with some mobile SoCs and sustained performance is more relevant than a 60 second benchmark. After that, I wanted to run something that was more of a system-level benchmark that considers all the different types of usages a user might have so I ran PCMark for Android. All benchmarks were run at least 3 times to normalize scores for any kind of variance except for 3DMark Wildlife Stress Test since it runs a 20-loop test. Also, the ZTE/Nubia RedMagic 6S Pro has an active cooled fan that automatically runs whenever graphically intensive applications are open, so it did turn on the fan for the 3DMark and PCMark tests but not for the GeekBench results. Additionally, I turned off benchmark boosting mode on the ASUS and Snapdragon Insider devices. Final note, all Pixel devices are running the latest version of Android 12 while the others are still on Android 11, which may represent a small performance difference.

GeekBench 5

Geekbench runs a single core and multi-core benchmark to test CPU performance. In this benchmark, we see that the Pixel 5 by far performs the worst in all tests, which sets Google up to show a significant improvement in performance from one generation to the next if the Tensor SoC is competitive with the rest of the market. Otherwise, Single Core performance across all the devices is roughly within 10% of each other, which makes sense since the Snapdragon 888, Snapdragon 888+ and Google Tensor all use the same ARM Cortex X1 big core. The multicore results are also not surprising, showing that a single X1 with three A78 cores generally does better than two X1 cores with two A76 cores. In our testing, the majority of Snapdragon 888 and 888+ devices scored a solid 20% faster CPU score.

GeekBench ML

This benchmark was the benchmark I was most interested in running because GeekBench ML does test both CPU and GPU, neither of which I was interested in testing. Instead, I ran the NNAPI benchmark which uses TensorFlow Lite as the ML framework and is Google’s default API for machine learning applications. This would also serve as a good test of Google’s claim that the market simply was not producing devices that meet its requirements for AI performance. However, we didn’t really see Google’s claims really stack up, unless you consider Google’s decision to downgrade AI performance on the Pixel 5 when going from the Pixel 4’s Snapdragon 855. The ‘AI Engine’ in the Snapdragon 855 is capable of up to 7 TOPS while the ‘AI Engine’ in the 765G has 5.5 so, it comes as no surprise that Google’s Tensor SoC would look like a huge improvement compared to a downgrade in performance. Additionally, while there is no doubt that Google’s Tensor SoC delivered the fastest NNAPI performance across all the Snapdragon 888 devices, the difference in performance was between 2% and 7% depending on the device. To me, this small of a difference in AI performance indicates that Google’s efforts to build a Tensor SoC with Samsung feels like more of a cost-saving and roadmap certainty decision than a performance one. Additionally, it might not even be a cost-saving measure if Google is spending too much money building the Tensor SoC and doesn’t sell enough devices to amortize that cost across enough chips. Nonetheless, Google’s Tensor is now the AI performance top dog for Android. Also, ASUS needs to do something about its NNAPI performance because both the ROG Phone 5 and Snapdragon Insider Phone are vastly underperforming against the other Snapdragon 888 devices.

3DMark Wildlife

3DMark Wildlife produced some surprising results and some not so surprising results. One of the surprising results was that the Pixel 5 really is performing quite poorly despite running Android 12 just like the Pixel 6 and Pixel 6 Pro and the GPU is simply being outclassed. Almost all the Snapdragon devices performed the same within a few percentage points, but the Pixel 6 Pro and Pixel 6 performed exceptionally well with benchmarks as high as almost 20% higher than the fastest Snapdragon device.  This was somewhat unexpected, so I decided to see if this was ‘bursty’ performance or sustained, which led me to run the 3DMark Wildlife Stress Test.

3DMark Wildlife Stress Test

This test was designed to figure out which devices have great peak GPU performance which devices have great sustained GPU performance. After all, nobody is playing games on their phones for only a minute or two and sustained GPU performance is relevant for applications like augmented reality as well. So, it came as a huge surprise to me to see how much the Google Pixel 6 and Pixel 6 Pro throttled compared to the rest of the devices. As you can see from the graph, the first bar is the initial test run and the second bar is the final test result after 20 loops. Unsurprisingly, the Redmagic 6S Pro had nearly no throttling which means that the little fan in that really does do its job in cooling the phone. If you look at the devices, they have been ordered from lowest to highest final test result, which indicates their ranking based on sustained performance. The difference in performance was so great that I actually had to notate these percentages and graph them.

As you can sort of tell from the graph, the Pixel 6 Pro throttled its GPU performance by over 50% at 58.3% while the Pixel 6 throttled at a more moderate 45%. The other Snapdragon 888 devices throttled anywhere from 24 to 36%. However, the Pixel 5 only throttled by 0.7% and the Redmagic 6S Pro with its active cooling only throttled an equally impressive 1%. To me, this means that Google’s GPU performance, while initially impressive, does have issues with longer term use and isn’t really viable for anything other than limited bursty GPU workloads. This means that gaming performance on the Pixel 6 Pro is worse than most of the flagship phones and the Pixel 6 is roughly on par.

PCMark Android

PCMark for Android is a good test because it tries to emulate a series of day-to-day tasks that a user might do on a device and tests them. As you can see, the Google Pixel 5, 6 and 6 Pro score the lowest across all the other devices. The Pixel 5 is the most obvious underperformer, but the ROG Phone 5 also performs roughly as well as the Pixel 6 devices do. That said, the performance difference between the Pixel 6 devices ranges anywhere between 1% to 37% depending on the device.

Conclusion

In the end, Google’s Tensor SoC inside of the Pixel 6 and Pixel 6 Pro appears to be a very big first step for Google in partnership with Samsung. While it remains to be seen where Google goes with the Tensor SoC in the future and how much more customization the company does, it is quite clear that the company for the most part has a flagship-class SoC that comes close to beating the competition in some aspects and wins in others. If Google’s goal was to go out and build the fastest chip for ML on an Android device, it seems like they may have done it. The performance difference between Qualcomm’s Snapdragon chips does make one wonder if the Tensor project’s goal was really performance oriented or if it was more about cost and roadmap control. If Google doesn’t ship enough Pixels or other devices with the Tensor SoC, I’m not really sure how much of a cost saving exercise it would be to build a semi-custom chip.

Note: Moor Insights & Strategy writers and editors may have contributed to this article. 

Moor Insights & Strategy, like all research and analyst firms, provides or has provided paid research, analysis, advising, or consulting to many high-tech companies in the industry, including 8x8, Advanced Micro Devices, Amazon, Applied Micro, ARM, Aruba Networks, AT&T, AWS, A-10 Strategies, Bitfusion, Blaize, Box, Broadcom, Calix, Cisco Systems, Clear Software, Cloudera, Clumio, Cognitive Systems, CompuCom, Dell, Dell EMC, Dell Technologies, Diablo Technologies, Digital Optics, Dreamchain, Echelon, Ericsson, Extreme Networks, Flex, Foxconn, Frame (now VMware), Fujitsu, Gen Z Consortium, Glue Networks, GlobalFoundries, Google (Nest-Revolve), Google Cloud, HP Inc., Hewlett Packard Enterprise, Honeywell, Huawei Technologies, IBM, Ion VR, Inseego, Infosys, Intel, Interdigital, Jabil Circuit, Konica Minolta, Lattice Semiconductor, Lenovo, Linux Foundation, MapBox, Marvell, Mavenir, Marseille Inc, Mayfair Equity, Meraki (Cisco), Mesophere, Microsoft, Mojo Networks, National Instruments, NetApp, Nightwatch, NOKIA (Alcatel-Lucent), Nortek, Novumind, NVIDIA, Nuvia, ON Semiconductor, ONUG, OpenStack Foundation, Oracle, Poly, Panasas, Peraso, Pexip, Pixelworks, Plume Design, Poly, Portworx, Pure Storage, Qualcomm, Rackspace, Rambus, Rayvolt E-Bikes, Red Hat, Residio, Samsung Electronics, SAP, SAS, Scale Computing, Schneider Electric, Silver Peak, SONY, Springpath, Spirent, Splunk, Sprint, Stratus Technologies, Symantec, Synaptics, Syniverse, Synopsys, Tanium, TE Connectivity, TensTorrent, Tobii Technology, T-Mobile, Twitter, Unity Technologies, UiPath, Verizon Communications, Vidyo, VMware, Wave Computing, Wellsmith, Xilinx, Zebra, Zededa, and Zoho which may be cited in blogs and research.