Current Single Instruction, Multiple Data (SIMD) extensions, such as AVX used by both Intel and AMD, can do matrix multiplication, but their math multiplication is nowhere near that of GPUs, which are built for it. So ACE brings that same technology to the x86 CPU. ACE is not a replacement for AVX, but an extension of the instruction set.
But while this closes the gap between CPU and GPU for both inference and training, it still doesn’t make the CPU truly competitive with the GPU, says Jim McGregor, principal analyst with TIRIAS Research.
“The CPU will never be more efficient than the GPU/AI accelerator,” he said. “However, it does allow you to offload some AI workloads to the CPU and/or use the CPU for AI workloads in applications that may not have a GPU or AI accelerator, such as embedded/edge applications.”
