NPU 101: Why the Neural Processing Unit Is the Key to Faster, Smarter AI-Friendly Devices
Discover the power of the NPU, a specialized processor for AI, enhancing speed and efficiency in machine learning.
Generative AI is here to stay, it seems, whether we like it or not. In fact, Microsoft has recently ushered in a new era of PCs with its Copilot+ laptops. These are the first of what will be known as AI PCs, and they all have one thing in common. These AI PCs are equipped with a neural processing unit or NPU. So, if you’re wondering what this new contraption, the NPU, is, we’ve got the answer.
The NPU Isn’t New, Even If Manufacturers Suggest It Is
The first thing we need to do is correct the misconception that the NPU is a new development in computing. It isn’t. In fact, Apple has included an NPU in its devices since the A11 Bionic chip in the iPhone 8 in 2017. The first Apple Silicon processor for Mac, the M1, also featured an NPU called the Neural Engine.
On an iPhone, iPad, or Mac, the Neural Engine handles all facial recognition tasks, like using Face ID or registering your facial expressions for animated Memoji. It also helps Siri understand your voice and is responsible for anything related to Machine Learning.
That being said, the NPU wasn’t heralded as such a “next big thing” to consumers until the advent of generative AI. Now, though, it’s getting the spotlight and is the deciding factor of what constitutes an AI PC.
What the NPU Does for Your PC
An NPU is a highly specialized processor designed explicitly to carry out AI tasks. It’s optimized to make the complex mathematical computations needed for machine learning algorithms and artificial neural networks.
Before the development of the NPU, all AI-related tasks had to be done by either the CPU or GPU (graphics processing unit). The processors are okay when it comes to AI algorithms, but because that’s not their intended use, they don’t excel in that arena.
What kinds of tasks are we talking about? One example is blurring your background when you’re on a video call. Another use of the NPU could be object detection in video or photo editing. The NPU can also help apply AI filters to your pictures. Without the aid of AI, these tasks are impossible, require a large amount of RAM, or risk bogging down your GPU.
The NPU is designed specifically to execute deep learning algorithms and other AI tasks. Since that’s all the NPU does, it can be tightly optimized for those tasks. After all, the NPU doesn’t need to draw the pretty pictures on your display like a GPU does. It also isn’t required to be a general-purpose jack-of-all-trades like your CPU.
So, the brunt of the AI work that your computer needs to do is best handled by the NPU. Apple’s been doing that for some time with its Neural Engine. Google does the same in its servers with the Tensor Processing Unit.
The NPU Arrives in the PC to Make Your AI Experience Better
Since AI tasks are handled so much better with an NPU, more chip manufacturers are including them. Intel’s Meteor Lake chips, introduced in December 2023, were probably the most prominent. However, Qualcomm has just jumped into the arena with its Snapdragon X Elite chips.
These are the chips powering the new Copilot+ PCs. The Snapdragon X Elite features an integrated Qualcomm Hexagon NPU, capable of carrying out 45 tera operations per second, or TOPS. TOPS is used to measure how powerful a processor is at handling AI tasks.
Since Windows 11 has AI built in, the NPU helps make those features faster and more responsive. Microsoft has declared that an AI PC needs to provide at least 45 TOPS of performance to be called such.
Currently, NPUs for consumer use are mainly in the laptop market. In time, we should see the same technology and branding in desktops.
Not Just for Computers: Other Uses of the Neural Processing Unit
NPUs are gaining traction in a variety of devices that traditionally lacked advanced processors, such as TVs and cameras. The fact is, electronic devices today increasingly function as standalone computers, and NPUs are being integrated into numerous household gadgets.
Let’s look at your smart TV, for instance. Many newer televisions use an NPU to upscale the resolution of older content to modern 4K quality. In cameras, NPUs help improve image stabilization, enhance the quality of your photography, and facilitate features like auto-focus and facial recognition.
Smart home devices are also leveraging the NPU to handle machine learning tasks on edge devices. This includes voice recognition and security information processing, which many consumers prefer to keep local due to the sensitive nature of the data, avoiding the need to send it offsite to cloud servers.
The NPU: A Useful Innovation in AI Across All of Tech
As we move further into a future driven by AI, the proliferation of NPUs will only gain momentum. With key players like Intel, AMD, and Qualcomm integrating NPUs into their latest processors, we are entering an era where AI processing is becoming more streamlined, efficient, and widespread.
Devices with NPUs can already execute AI tasks faster, resulting in quicker data processing times and greater convenience for users. The NPU can help you speed up your video editing or apply advanced AI filters in applications. It efficiently handles AI tasks in smartphones, so NPUs are leading the way toward a smarter, more efficient computing experience.
Ultimately, NPUs represent a major leap forward in AI and machine learning at the consumer level. By specializing in neural network operations and AI tasks, NPUs reduce the burden on traditional CPUs and GPUs. This not only leads to more efficient computing systems overall but also gives developers the tools needed to continue innovating AI-driven software.
Whatever task you’re performing on your PC or mobile device, there’s an excellent chance that NPUs will eventually affect how those tasks are processed.
Leave a Reply
Leave a Reply
