The response from various firms has been to design processors from the ground up with AI in mind. The result of Graphcore’s efforts is called an intelligent processing unit (IPU). This name is not just marketing: on GPUs, memory (the staging area for data) and brain (where they are processed) are kept separate — meaning that data constantly have to be ferried back and forth between the two areas, creating a bottleneck with data-heavy AI applications. To do away with it, Graphcore’s chips do not just have hundreds of mini-brains, but the memory is placed right next to it, minimising data traffic.
“Computer architectures need to follow the structure of the data they’re processing,” says Nigel Toon, one of Graphcore’s co-founders. The most basic feature of AI workloads is that they are “embarrassingly parallel”, which means they can be cut into thousands of chunks which can all be worked on at the same time. Graphcore’s chips, for instance, have more than 1,200 individual number-crunching “cores”, and can be linked together to provide still more power.
Unlike many tasks, says Mr Toon at Graphcore, ultra-precise calculations are not needed in AI. That means chip designers can save energy by reducing the fidelity of the numbers their creations are juggling.
All that can add up to big gains. Mr Toon reckons that Graphcore’s current chips are anywhere between ten and 50 times more efficient than GPUs. They have already found their way into specialised computers sold by Dell, as well as into Azure, Microsoft’s cloud-computing service.
Read the full article