PERFORMANCE BENCHMARKS

Our Poplar SDK accelerates machine learning training and inference with high-performance optimisations delivering world leading performance on IPUs across models such as  natural language processing,  probabilistic modelling, computer vision, recommenders and more. We have provided a selection of the latest IPU performance results on this page and will update it regularly. To replicate our benchmarks, visit the Graphcore GitHub site for public code examples and applications.

BERT (Bidirectional Encoder Representations from Transformers) is one of the most well known NLP models in use today. The IPU accelerates both training and inference on BERT, delivering 25x faster time to train with 2x faster inference at extremely low latency. 

Bert-Base Inference Click to Zoom
Go to public example
Bert-Base Training Click to Zoom
Go to public example

Deep Voice from Baidu is a prominent text-to-speech (TTS) model family for high-quality, end-to-end speech synthesis. The IPU’s capacity to significantly accelerate fully convolutional TTS models like Deep Voice 3, with 6.8x higher throughput than a GPU opens up the opportunity to create entirely new classes of TTS models.

Benchmarks Click to Zoom
Go to public example

Female: "He likes the taste of worcestershire sauce"

Male: "She demonstrated great leadership on the field"

IPU excels with models designed to leverage small, group convolutions due to its fine grained architecture and specific features in the Poplar SDK. We deliver performance gains for both training and inference for newer computer vision models like EfficientNet and ResNeXt.

Benchmarks Click to Zoom
Go to public example
EfficientNet-80- Training Click to Zoom
Go to public example
ResNext-101- Inference Click to Zoom
Go to public example
ResNext-50- Training Click to Zoom
Go to public example

Probabilistic models using the Markov Chain Monte Carlo (MCMC) method use iterative sampling of an implicit distribution with Hamiltonian Monte Carlo (HMC) schemes to manage noise and uncertainty in data. The IPU delivers 15x faster time to train for MCMC using standard TensorFlow Probability.

MCMC Probabilistic Model- Training TF Click to Zoom
Go to public example

Variational Inference (VI) is another common way of managing probabilistic inference, by introducing an approximate distribution, which is then sampled and optimised to get as close as possible to the target. In a TensorFlow-based Variational Autoencoder (VAE) Model combining both approaches, the IPU sees over 4.8x faster time to train.

VAE Probabilistic Model- Training Click to Zoom
Go to public example
Time Series Analysis- Training Click to Zoom
Go to public example
Dense Autoencoder- Training Click to Zoom
Go to public example
See all benchmarks
×