CPU vs. GPU vs. TPU for AI and Machine Learning

Hardware selection starts with the work the system must do: run application logic, prepare data, train a model, or serve predictions within a latency budget. CPUs, GPUs, and TPUs can participate in the same pipeline. Their names alone do not determine which configuration will be fastest or cheapest.

A central processing unit (CPU) runs general-purpose programs. A graphics processing unit (GPU) developed from hardware for rendering graphics into a programmable parallel processor. A tensor processing unit (TPU) is Google's family of accelerators designed for machine learning, especially tensor and matrix operations.

How the architectures differ

CPU: flexible execution and control flow

A CPU core contains registers, execution units, and nearby caches. Multiple cores communicate through an interconnect and access main memory through memory controllers. Cache layouts vary: some caches belong to individual cores, while others are shared. The diagram is a conceptual layout, not a specification for every processor.