PopART C++ API Reference

Introduction

The Poplar Advanced Run Time (PopART) is part of the Poplar SDK for implementing and running algorithms on networks of Graphcore IPU processors.

This document describes the PopART C++ API. For more information about PopART, please refer to the PopART User Guide.

PopART C++ API

Sessions

class InferenceSession : public popart::Session

Public Static Functions

static std::unique_ptr<InferenceSession> createFromOnnxModel(const std::string &model, const DataFlow &dataFlow, std::shared_ptr<DeviceInfo> deviceInfo, const std::vector<Loss *> &losses = {}, const InputShapeInfo &inputShapeInfo = InputShapeInfo(), const SessionOptions &userOptions = SessionOptions (), const Patterns &patterns = Patterns())

Create a runtime class for executing an ONNX graph on a set of IPU hardware for inference.

 

Parameters
  • model: Either an ONNX model protobuf, or the name of a file containing an ONNX model protobuf
  • inputShapeInfo: Information about the shapes of input and output tensors
  • losses: An optional list of loss layers to use after inference
  • dataFlow: Configuration for the data feeds and fetches
  • userOptions: String to configure session options
  • patterns: Optimization patterns to apply

 

class TrainingSession : public popart::Session

Public Functions

void updateOptimizer(const Optimizer *optimizer)

Update the optimizer.

Note that the optimizer passed in must be compatible with that passed to the constructor. For example, you cannot update to an Optimizer which uses momentum here, if the Optimizer passed to the constructor did not have momentum. Reason: The Ir would need to change to incorporate momentum, but the Ir is frozen once constructed. NB: Must call optimizerFromHost for this update to take effect on the device.

 

Parameters
  • optimizer: A pointer to a popart::Optimizer

 

void optimizerFromHost()

write whatever optimizer tensors (learning rates, momentum, initial momentum tensors (zero)) there are to device

const std::vector<std::string> &getHostReduceStreamIds() const

Access the stream IDs for variables that are involved in host side reductions on the host.

Only populated if hostAllReduce is enabled in the SessionOptions

const std::map<std::string, poplar::RemoteBuffer> &getHostReduceRemoteBuffers() const

Access the remote buffers associated with gradient and weight streams that are used in host side all reduce operations.

Only populated if hostAllReduce and hostAllReduceRemoteBuffer are enabled.

void connectStreamToCallback(const std::string &streamHandle, std::function<void(void *)> callback, unsigned index = 0, )

Connect Poplar stream callbacks.

In conjunction with getGradAndVarStreamIds the streams can be used to copy gradients to the host to perform collective operations after which the variables can be streamed back after they have been updated to the device. index referes to the replica index when using replicated graphs.

void copyFromRemoteBuffer(const poplar::RemoteBuffer &buffer, void *w, int repeat_index, unsigned replication_index = 0)

Read from a RemoteBuffer object into a user space pointer w.

This can be useful when we run larger models with host side reductions since HEXOPT is currently limited to 128 MB

void copyToRemoteBuffer(void *w, const poplar::RemoteBuffer &buffer, int repeat_index, unsigned replication_index = 0)

Write to a RemoteBuffer object from a user space pointer w.

This can be useful when we run larger models with host side reductions since HEXOPT is currently limited to 128 MB

Public Static Functions

static std::unique_ptr<TrainingSession> createFromOnnxModel(const std::string &model, const DataFlow &dataFlow, const std::vector<Loss *> &losses, const Optimizer &optimizer, std::shared_ptr<DeviceInfo> deviceInfo, const InputShapeInfo &inputShapeInfo = InputShapeInfo(), const SessionOptions &userOptions = SessionOptions (), const Patterns &patterns = Patterns())

Create a runtime class for executing an ONNX graph on a set of IPU hardware for training.

 

Parameters
  • model: Either an ONNX model protobuf, or the name of a file containing an ONNX model protobuf
  • inputShapeInfo: Information about the shapes of input and output tensors
  • dataFlow: Configuration for the data feeds and fetches
  • losses: A list of loss layers to use when training
  • optimizer: The name of an optimizer to use when training
  • userOptions: String to configure session options
  • patterns: Optimization patterns to apply

 

Session option flags

struct SessionOptions

A structure containing user configuration options for the Session class.

Public Members

std::string logDir

A directory for log traces to be written into.

std::set<DotCheck> dotChecks = {}

When to write ‘.dot’ files during Ir construction.

int firstDotOp = 0

The ops to write to the .dot file will be a continuous interval of the schedule, controlled by firstDotOp and finalDotOp.

In particular, it will be [min(0, firstDotOp), max(N ops in Ir, finalDotOp))

bool dotOpNames = false

Include the Op name in the .dot file (the Op type is always exported)

bool exportPoplarComputationGraph = false

Export Poplar computation graph.

bool exportPoplarVertexGraph = false

Export Poplar vertex graph.

bool enableOutlining = true

Controls caching of identical sections of the graph.

bool enableOutliningCopyCostPruning = true

Controls whether the cost of copying of cached sections should be included in the outlining cost model.

float outlineThreshold = 1.0f

The incremental value that a sub-graph requires, relative to its nested sub-graphs (if any), to be eligible for outlining.

A high threshold results in fewer sub-graphs being outlined, a negative value results in all being outlined. The gross value of a sub-graph is the sum of its constituent Ops’ getSubgraphValue() values. To disable outlining, it is better to set enableOutlining to false than to set this value to infinity. The default value of 1.0f results in all high Value operations such as convolution being cached, but standalone low Value operations such as Relu will not be.

RecomputationType autoRecomputation = RecomputationType::None

Enable recomputation of operations in the graph in the backwards pass to reduce model size at the cost of computation cycles.

MergeVarUpdateType mergeVarUpdate = MergeVarUpdateType::None

Enable merging of VarUpdates into groups of VarUpdates, by flattening and concatenating Variable Tensors and Updating Tensors.

int64_t mergeVarUpdateMemThreshold = 1000000

The AutoLoose and AutoTight VarUpdate merging algorithm has a threshold on the total memory of Variable Tensors to merge for updating.

Memory in bytes.

int64_t looseThresholdAtPeak = 8000

The AutoLoose VarUpudate merging algorithm has absolute threshold defined by min(mergeVarUpdateMemThreshold, liveAtPeak - liveCurrently + looseThresholdAtPeak), where liveAtPeak is an estimate of the maximum live memory of the computation, and liveCurrently is an estimate of the live memory where the threshold is being used to determine whether to schedule or postpone a VarUpdate.

bool rearrangeAnchorsOnHost = true

Before anchor tensors are streamed from device to host, they are not necessarily arranged in memory as required when they are to be copied from host stream to host.

This can be done on the device or on the host. Done on host by default to save memory, but often at the expense of cycles, especially for larger anchor tensors.

bool enablePrefetchDatastreams = true

By default, we will use prefetching for input data streams.

Poplar will speculative read data for a stream before is is required to allow the ‘preparation’ of the data to occur in parallel with compute

bool enableNonStableSoftmax = false

By default, we use the stable-softmax poplar function.

This input tensor to softmax, x, is preprocessed by subtracting max(x) to each element before computing the exponentials, ensuring numerical stability. If you are sure the inputs to your softmax operations are small enough to not cause overflow when computing the exponential, you can enable the non-stable version instead for speed-up

bool enableVirtualGraphs = false

Enable placement of operations on individual IPUs by creating a ‘virtual graph’ for each IPU.

bool enableReplicatedGraphs = false

Enable replication of graphs.

bool enableGradientAccumulation = false

Enable gradient accumulation.

int64_t replicatedGraphCount = 1

If enableReplicatedGraphs is true, replicatedGraphCount will set the number of model replications.

E.g. if your model uses 1 IPU, a replicatedGraphCount of 2 will use 2 IPUs. If your model is pipelined across 4 IPUs, a replicatedGraphCount of 4 will use 16 IPUs total. Therefore the number of IPUs you request must be a multiple of replicatedGraphCount.

int64_t accumulationFactor = 1

Specify the number of micro-batches to accumulate before applying the varUpdate.

bool autoVirtualGraph = false

Enable transformation pass that attempts to automatically place ops on virtual graphs to achieve model parallelism.

bool enablePipelining = false

Enable pipelining of virtual graphs.

bool ignoreData = false

Use synthetic data i.e.

disable data transfer to/from the host Set to ‘true’ to use synthetic data, ‘false’ to use real data. This option is deprecated, please use syntheticDataMode.

SyntheticDataMode syntheticDataMode = SyntheticDataMode::Off

Use synthetic data i.e.

disable data transfer to/from the host Set to ‘Off’ to use real data Note: this will be overriden by the legacy option ‘ignoreData’.

bool instrumentWithHardwareCycleCounter = false

Add instrumentation to your program to count the number of device cycles (a single tile, on a single IPU) that your main program takes to execute.

Expect this to have a small detrimental impact on performance.

bool disableGradAccumulationTensorStreams = false

If true, the weight gradient tensors are not saved off the device when devicex.weightsFromHost() is called.

Note: this option is overridden if syntheticDataMode is not Off.

bool compileEngine = true

when false, the backend will build the Poplar graph, but do not compile it into an Engine.

When this option is set, no execution can be performed, and nothing can be transferred to the device. Functions which retrieve information from the graph building stage will be ok (tile mapping).

bool enableEngineCaching = false

Enable poplar executable caching.

std::string cachePath = "session_cache"

Path to save the poplar::Executable to.

bool enableGroupedMatmuls = false

Enable/disable the grouping of matmuls that are the same shape.

bool enableSerializedMatmuls = true

Enable/disable the serializing of matmuls.

bool enableStableNorm = false

If true, computes the mean first and subtracts the activations from it before computing the variance.

The implementation with this flag set to true is slower than when set to false. The stable version requires the first order moment to be estimated and applied to the sample set before the second order central moment is calculated.

bool hostAllReduce = false

Perform AllReduce operation on the host. Only useful for training session.

bool hostWeightUpdate = false

Perform weight update on the host. Only useful for training session.

bool hostAllReduceRemoteBuffer = false

Enable the use of poplar::RemoteBuffers for hostAllReduce operations.

std::map<std::string, std::string> engineOptions

Poplar engine options.

std::map<std::string, std::string> convolutionOptions

Poplar convolution options.

std::map<std::string, std::string> reportOptions

Poplar reporting options.

std::vector<std::string> customCodelets

List of codelets (with filetype) to be added to the poplar graph.

See the poplar documentation for more information.

std::string customCodeletCompileFlags

Compile flags for the custom codelets.

For example -g to generate debug info.

double timeLimitScheduler = 1e9

The maximum allowed time that can be spent searching for a good Graph schedule before a solution must be returned.

int64_t swapLimitScheduler = static_cast<int64_t>(1e9)

The maximum number of improving steps allowed by the scheduling algorithm before a solution must be returned.

std::string serializedPoprithmsAnnealGraphsDir = {}

PopART uses Poprithms for scheduling PopART Graphs.

The Poprithms Graphs created for scheduling can be optionally serialized (written to file). The string below specified the directory to serialize Poprithms Graphs to. If it is empty, then the Graphs will not be serialized. The names of serialization files will be poprithms_anneal_graph_`i’.json for the lowest non-existing `i’s. The directory must already exist, PopART will not create it.

std::string kahnTieBreaker = "greedy"

The initial scheduling is done with Kahn’s algorithm.

When several Ops are free to be scheduled, this controls which method is used

bool decomposeGradSum = false

Replaces single sums of partial gradients with a tree of additions.

This can reduce max liveness at the cost of extra cycles. A typical use case for this would be if a large weight tensor is used as an input to many operations

Training operations

Losses

class L1Loss : public popart::Loss
class NllLoss : public popart::Loss

Optimisers

class ConstSGD : public popart::SGD
class SGD : public popart::Optimizer

Subclassed by popart::ConstSGD

Builder

class Builder

An interface for a Builder, used for creating ONNX graphs.

Public Functions

Builder &createSubgraphBuilder()

Return a Builder for a graph which is nested inside this Builder’s graph.

TensorId addInputTensor(const TensorInfo &tensorInfo, const std::string &debugPrefix = "")

Add a new input tensor to the model.

 

Return
The unique name of the input tensor
Parameters
  • tensorInfo: The shape and type of the input tensor
  • debugPrefix: A string to prepend to the name of the tensor

 

TensorId addUntypedInputTensor(const std::string &debugPrefix = "")

Add a new input tensor without a type or shape to the model.

 

Return
The unique name of the input tensor
Parameters
  • debugPrefix: A string to prepend to the name of the tensor

 

void addInputTensorFromHigherScope(const TensorId &tensorId)

Add a new named input tensor to the model.

 

Parameters
  • tensorId: The identifier string of the input tensor. This identifier must already exist in the parent GraphProto’s name scope and must appear topologically before this sub-graph.

 

TensorId addInitializedInputTensor(const ConstVoidData &initData, const std::string &debugPrefix = "")

Add a new preinitialized input tensor to the model.

 

Return
The unique name of the input tensor
Parameters
  • initData: The initial data of the input tensor
  • debugPrefix: A string to prepend to the name of the tensor

 

void addOutputTensor(const TensorId &arg0)

Adds one of the outputs from a node in the graph into the list of output tensors.

AiOnnxOpset6 aiOnnxOpset6()

Return the builder interface for ai.onnx opset 6.

AiOnnxOpset7 aiOnnxOpset7()

Return the builder interface for ai.onnx opset 7.

AiOnnxOpset8 aiOnnxOpset8()

Return the builder interface for ai.onnx opset 7.

AiOnnxOpset9 aiOnnxOpset9()

Return the builder interface for ai.onnx opset 9.

AiOnnxOpset10 aiOnnxOpset10()

Return the builder interface for ai.onnx opset 10.

AiOnnxOpset11 aiOnnxOpset11()

Return the builder interface for ai.onnx opset 11.

AiOnnxMlOpset1 aiOnnxMlOpset1()

Return the builder interface for ai.onnx.ml opset 1.

AiGraphcoreOpset1 aiGraphcoreOpset1()

Return the builder interface for ai.graphcore opset 1.

template <class T>
TensorId reshape_const(T &t, const std::vector<TensorId> &args, const std::vector<int64_t> &shape, const std::string &name = {})

This is a helper function that will add a constant and a reshape using the provided domain.

void recomputeOutputInBackwardPass(const TensorId &nodeOutputName, RecomputeType value = RecomputeType::RECOMPUTE)

Enable/disable recomputation of the output of the node in the backward pass.

 

Parameters
  • nodeOutputName: Name of the output tensor of the ONNX node
  • value: If the recompute is enabled/disabled

 

void recomputeOutputInBackwardPass(const std::set<TensorId> &nodeOutputNames, RecomputeType value = RecomputeType::RECOMPUTE)

Enable/disable recomputation of the output of the node in the backward pass.

 

Parameters
  • nodeOutputNames: Names of the output tensors of the ONNX node
  • value: If the recompute is enabled/disabled

 

bool getRecomputeOutputInBackwardPass(const TensorId &nodeOutputName)

Get whether the given node will have its output recomputed in the backward pass.

 

Parameters
  • nodeOutputName: Name of the output tensor of the ONNX node used to find the node in the ONNX model.

 

bool getRecomputeOutputInBackwardPass(const std::set<TensorId> &nodeOutputNames)

Get whether the given node will have its output recomputed in the backward pass.

 

Parameters
  • nodeOutputNames: Names of the output tensors of the ONNX node used to find the node in the ONNX model.

 

void virtualGraph(const TensorId &nodeOutputName, int64_t value = 0)

Set the virtual graph that computes the given node.

Applies when creating a graph for a multi-IPU configuration.

 

Parameters
  • nodeOutputName: Name of the output tensor of the ONNX node
  • value: The index of the virtual graph that computes this node

 

void pingPongPhase(const TensorId &nodeOutputName, int64_t value = 0)

Set the ping pong phase that computes the given node.

 

Parameters
  • nodeOutputName: Name of the output tensor of the ONNX node
  • value: The index of the virtual graph that computes this node

 

void setSerializeMatMul(const std::set<TensorId> &nodeOutputNames, std::string mode, int64_t factor, bool keep_precision)

Set the settings for matmuls that should be serialized.

This option will split a matmul into seperate smaller matmuls that will be excuted in series. This will also serialize the grad operations if training.

 

Parameters
  • nodeOutputNames: Name of the output matmul tensors of the ONNX node
  • mode: Which dimension of the mat mul to serialize on.
  • factor: The number of serialised matmuls, must be a factor of the dimentions to serialise on.

 

void setPartialsType(const TensorId &nodeOutputName, const std::string partialsType)

Set the partials type for the given node.

Used on the convolution op.

 

Parameters
  • nodeOutputName: Name of the output tensor of the ONNX node
  • partialsType: The type for the partials. Can be either FLOAT or HALF.

 

std::string getPartialsType(const TensorId &nodeOutputName)

Get the partials type for the given node.

 

Parameters
  • nodeOutputName: Name of the output tensor of the ONNX node

 

void setAvailableMemoryProportion(const TensorId &nodeOutputName, const float availableMemoryProportion)

Set the available memory for the given node.

Used on the convolution op.

 

Parameters
  • nodeOutputName: Name of the output tensor of the ONNX node
  • availableMemoryProportion: The available memory proportion 0 < x <= 1.

 

void setAttribute(const std::string &attribute, boost::any value)

Set an attribute that will be set on all subsequent operations.

boost::any getAttribute(const std::string attribute) const

Get an attribute that has been set for all subsequent operations.

void clearAttribute(const std::string &attribute)

Unset an attribute that will be set on all subsequent operations.

bool hasAttribute(const std::string &attribute)

Check if attribute is set.

boost::any getAttribute(const std::string &attribute)

Get current attribute value.

int64_t getPipelineStage() const

A convenience function for the pipeline stage attribute.

int64_t getPingPongPhase() const

A convenience function for the ping pong phase attribute.

int64_t getVirtualGraph() const

A convenience function for the virtual graph attribute.

void virtualGraph(const std::set<TensorId> &nodeOutputNames, int64_t value = 0)

Set the virtual graph that computes the given node.

Applies when creating a graph for a multi-IPU configuration.

 

Parameters
  • nodeOutputNames: Names of the output tensors of the ONNX node
  • value: The index of the virtual graph that computes this node

 

void addNodeAttribute(const std::string &attributeName, const int64_t &attributeValue, const std::set<TensorId> &nodeOutputNames)

Add an attribute to the ONNX node which is uniquely identified by the outputs.

This functions will throw an exception if it can’t find the unique node or the attribute already exists.

 

Parameters
  • attributeName: The name of the attribute to add.
  • attributeValue: An int64_t value of the attribute to add.
  • nodeOutputNames: Names of the output tensors of the ONNX node used to find the node in the ONNX model.

 

void addNodeAttribute(const std::string &attributeName, const std::vector<int64_t> &attributeValue, const std::set<TensorId> &nodeOutputNames)

Add an attribute to the ONNX node which is uniquely identified by the outputs.

This functions will throw an exception if it can’t find the unique node or the attribute already exists.

 

Parameters
  • attributeName: The name of the attribute to add.
  • attributeValue: An std::vector<int64_t> value of the attribute to add.
  • nodeOutputNames: Names of the output tensors of the ONNX node used to find the node in the ONNX model.

 

void addNodeAttribute(const std::string &attributeName, const float &attributeValue, const std::set<TensorId> &nodeOutputNames)

Add an attribute to the ONNX node which is uniquely identified by the outputs.

This functions will throw an exception if it can’t find the unique node or the attribute already exists.

 

Parameters
  • attributeName: The name of the attribute to add.
  • attributeValue: A float value of the attribute to add.
  • nodeOutputNames: Names of the output tensors of the ONNX node used to find the node in the ONNX model.

 

void addNodeAttribute(const std::string &attributeName, const std::vector<float> &attributeValue, const std::set<TensorId> &nodeOutputNames)

Add an attribute to the ONNX node which is uniquely identified by the outputs.

This functions will throw an exception if it can’t find the unique node or the attribute already exists.

 

Parameters
  • attributeName: The name of the attribute to add.
  • attributeValue: An std::vector<float> value of the attribute to add.
  • nodeOutputNames: Names of the output tensors of the ONNX node used to find the node in the ONNX model.

 

void addNodeAttribute(const std::string &attributeName, const std::string &attributeValue, const std::set<TensorId> &nodeOutputNames)

Add an attribute to the ONNX node which is uniquely identified by the outputs.

This functions will throw an exception if it can’t find the unique node or the attribute already exists.

 

Parameters
  • attributeName: The name of the attribute to add.
  • attributeValue: A std::string value of the attribute to add.
  • nodeOutputNames: Names of the output tensors of the ONNX node used to find the node in the ONNX model.

 

void addNodeAttribute(const std::string &attributeName, const std::vector<std::string> &attributeValue, const std::set<TensorId> &nodeOutputNames)

Add an attribute to the ONNX node which is uniquely identified by the outputs.

This functions will throw an exception if it can’t find the unique node or the attribute already exists.

 

Parameters
  • attributeName: The name of the attribute to add.
  • attributeValue: An std::vector<std::string> value of the attribute to add.
  • nodeOutputNames: Names of the output tensors of the ONNX node used to find the node in the ONNX model.

 

void addNodeAttribute(const std::string &attributeName, const bool attributeValue, const std::set<TensorId> &nodeOutputNames)

Add an attribute to the ONNX node which is uniquely identified by the outputs.

This functions will throw an exception if it can’t find the unique node or the attribute already exists.

 

Parameters
  • attributeName: The name of the attribute to add.
  • attributeValue: An bool value of the attribute to add
  • nodeOutputNames: Names of the output tensors of the ONNX node used to find the node in the ONNX model.

 

void addNodeAttribute(const std::string &attributeName, const ConstVoidData &attributeValue, const std::set<TensorId> &nodeOutputNames)

Add an attribute to the ONNX node which is uniquely identified by the outputs.

This functions will throw an exception if it can’t find the unique node or the attribute already exists.

 

Parameters
  • attributeName: The name of the attribute to add.
  • attributeValue: An constant tensor initializer
  • nodeOutputNames: Names of the output tensors of the ONNX node used to find the node in the ONNX model.

 

bool nodeHasAttribute(const std::string &attributeName, const std::set<TensorId> &nodeOutputNames)

Check whether the ONNX node has an attribute set.

This functions will throw an exception if it can’t find the unique node.

 

Parameters
  • attributeName: The name of the attribute to find.
  • nodeOutputNames: Names of the output tensors of the ONNX node used to find the node in the ONNX model.

 

int64_t getInt64NodeAttribute(const std::string &attributeName, const std::set<TensorId> &nodeOutputNames)

Get the int64_t value of the attribute for the ONNX node.

This functions will throw an exception if it can’t find the unique node or the attribute does not exist or it has not been set to the int64_t type.

 

Return
Value of the attribute
Parameters
  • attributeName: The name of the attribute to find.
  • nodeOutputNames: Names of the output tensors of the ONNX node used to find the node in the ONNX model.

 

std::vector<int64_t> getInt64VectorNodeAttribute(const std::string &attributeName, const std::set<TensorId> &nodeOutputNames)

Get the std::vector<int64_t> value of the attribute for the ONNX node.

This functions will throw an exception if it can’t find the unique node or the attribute does not exist or it has not been set to the std::vector<int64_t> type.

 

Return
Value of the attribute
Parameters
  • attributeName: The name of the attribute to find.
  • nodeOutputNames: Names of the output tensors of the ONNX node used to find the node in the ONNX model.

 

float getFloatNodeAttribute(const std::string &attributeName, const std::set<TensorId> &nodeOutputNames)

Get the float value of the attribute for the ONNX node.

This functions will throw an exception if it can’t find the unique node or the attribute does not exist or it has not been set to the float type.

 

Return
Value of the attribute
Parameters
  • attributeName: The name of the attribute to find.
  • nodeOutputNames: Names of the output tensors of the ONNX node used to find the node in the ONNX model.

 

std::vector<float> getFloatVectorNodeAttribute(const std::string &attributeName, const std::set<TensorId> &nodeOutputNames)

Get the std::vector<float> value of the attribute for the ONNX node.

This functions will throw an exception if it can’t find the unique node or the attribute does not exist.

 

Return
Value of the attribute
Parameters
  • attributeName: The name of the attribute to find.
  • nodeOutputNames: Names of the output tensors of the ONNX node used to find the node in the ONNX model.

 

std::string getStringNodeAttribute(const std::string &attributeName, const std::set<TensorId> &nodeOutputNames)

Get the std::string value of the attribute for the ONNX node.

This functions will throw an exception if it can’t find the unique node or the attribute does not exist or it has not been set to the std::string type.

 

Return
Value of the attribute
Parameters
  • attributeName: The name of the attribute to find.
  • nodeOutputNames: Names of the output tensors of the ONNX node used to find the node in the ONNX model.

 

std::vector<std::string> getStringVectorNodeAttribute(const std::string &attributeName, const std::set<TensorId> &nodeOutputNames)

Get the std::vector<std::string> value of the attribute for the ONNX node.

This functions will throw an exception if it can’t find the unique node or the attribute does not exist.

 

Return
Value of the attribute
Parameters
  • attributeName: The name of the attribute to find.
  • nodeOutputNames: Names of the output tensors of the ONNX node used to find the node in the ONNX model.

 

void removeNodeAttribute(const std::string &attributeName, const std::set<TensorId> &nodeOutputNames)

Remove an attribute from the ONNX node.

This functions will throw an exception if it can’t find the unique node or the attribute does not exist.

 

Parameters
  • attributeName: The name of the attribute to find.
  • nodeOutputNames: Names of the output tensors of the ONNX node used to find the node in the ONNX model.

 

std::vector<std::string> getAllNodeAttributeNames(const std::set<TensorId> &nodeOutputNames)

Get all the attribute names from the ONNX node.

This functions will throw an exception if it can’t find the unique node.

 

Parameters
  • nodeOutputNames: Names of the output tensors of the ONNX node used to find the node in the ONNX model.

 

int64_t getVirtualGraph(const TensorId &nodeOutputName)

Get the index of the virtual graph that computes this node.

This applies in a multi IPU system.

 

Parameters
  • nodeOutputName: Name of the output tensor of the ONNX node used to find the node in the ONNX model.

 

int64_t getVirtualGraph(const std::set<TensorId> &nodeOutputNames)

Get the index of the virtual graph that computes this node.

This applies in a multi IPU system.

 

Parameters
  • nodeOutputNames: Names of the output tensors of the ONNX node used to find the node in the ONNX model.

 

std::string getModelProto() const

Retrieve the ONNX serialized ModelProto.

 

Return
A serialized ONNX ModelProto

 

void saveModelProto(const std::string &fn)

Save the builder’s ONNX ModelProto into the builder and validate it.

 

Parameters
  • fn: The name of a file containing an ONNX model protobuf.

 

void saveInitializersExternally(const std::vector<TensorId> &ids, const std::string &fn)

The model data cannot exceed 2GB - the maximum size of a Protobuf message.

To prevent this for large models, ONNX tensor data can be saved separately.

 

Parameters
  • ids: The names of tensors whose data is to be saved externally.
  • fn: The name of a file containing the binary tensor data.

 

std::vector<TensorId> getInputTensorIds() const

Return a list of ONNX graph input tensor ids.

 

Return
A vector of input tensor names

 

std::vector<TensorId> getOutputTensorIds() const

Return a list of ONNX graph output tensor ids.

 

Return
A vector of output tensor names

 

std::vector<TensorId> getValueTensorIds() const

Return a list of ONNX graph value tensor ids.

These tensors are stored in the value_info section of the ONNX GraphProto structure.

 

Return
A vector of output tensor names

 

std::vector<int64_t> getTensorShape(const TensorId id)

Return an ONNX graph tensor shape, from either the input, output, or value_info lists in the GraphProto.

 

Return
A vector of tensor dimensions
Parameters
  • id: Tensor id

 

bool isInitializer(const TensorId id) const

Returns true if the ONNX tensor is in the initializer list of the GraphProto.

 

Return
A boolean
Parameters
  • id: Tensor id

 

std::string getTensorDtypeString(const TensorId id)

Return an ONNX graph tensor type as a lower case string, from either the input, output, or value_info lists in the GraphProto.

 

Return
A lower case string of tensor type
Parameters
  • id: Tensor id

 

void pushNameScope(const std::string &name)

Push a name onto the name scope stack.

The names of tensors and nodes added to the ONNX graph will be prefixed with a concatenation of the names in the name stack.

void popNameScope()

Remove the last entry in the name scope stack.

std::string getNameScope(const std::string &name = "") const

Get the current namescope stack using the default delimiter.

 

Return
A string of the concatenated namescope stack.
Parameters
  • name: Optional string to concatenate to the end of the stack

 

void setGraphName(const std::string &name)

Specifies a graph name.

 

Parameters
  • name: string to name the graph

 

Public Static Functions

static std::unique_ptr<Builder> create()

Create a builder for an ONNX model.

static std::unique_ptr<Builder> createFromOnnxModel(const std::string &modelProtoOrFilename)

Create a builder which loads a serialized ONNX ModelProto into the builder and validates it.

 

Parameters
  • modelProtoOrFilename: Either an ONNX model protobuf, or the name of a file containing an ONNX model protobuf.

 

class AiGraphcoreOpset1 : public popart::DomainOpSet

Public Functions

std::vector<TensorId> groupnormalization(const std::vector<TensorId> &args, int64_t num_groups, float epsilon = 1e-05f, const std::string &name = {})

Add a groupnormalization operation to the model.

This is a poplar extension

The group will be created from a strided input

 

Return
A vector of tensors (y, mean, var)
Parameters
  • args: A vector of input tensors (x, scale, bias)
  • num_groups: The number of groups to separate the channels into
  • epsilon: The epsilon value to use to avoid division by zero.
  • name: Optional identifier for operation

 

TensorId subsample(const std::vector<TensorId> &args, const std::vector<int64_t> &strides, const std::string &name = {})

Add a subsample operation to the model.

This is a poplar extension

If multiple tensors are provided that strides will applied to them all

 

Return
The name of the result tensor
Parameters
  • args: Tensor T
  • strides: The strides
  • name: Optional identifier for operation

 

TensorId printtensor(const std::vector<TensorId> &args, int64_t print_gradient = 1, const std::string &name = {})

Add a print tensor operation to the model.

This is a poplar extension

TensorId scale(const std::vector<TensorId> &args, float scale, const std::string &name = {})

Add a scale operation to the model.

This is a poplar extension, to replace the experimental scale operator that has been removed

 

Return
The name of the result tensor
Parameters
  • args: Tensor T
  • scale: The scale to apply
  • name: Optional identifier for operation

 

TensorId gelu(const std::vector<TensorId> &args, const std::string &name = {})

Add a gelu operation to the model.

This is a poplar extension, to replace the experimental scale operator that has been removed

 

Return
The name of the result tensor
Parameters
  • args: Tensor T
  • name: Optional identifier for operation

 

TensorId init(Attributes::Ints shape, Attributes::Int data_type, Attributes::Int init_type, const std::string &name = {})

Add an init operation to the model.

 

Return
The name of the result tensor
Parameters
  • shape: Shape of the tensor to initialise
  • data_type: Data type to initialise tensor with
  • init_type: Mode of tensor initialisations
  • name: Optional identifier for operation

 

TensorId dynamicslice(const std::vector<TensorId> &args, Attributes::Ints axes, Attributes::Ints sizes, Attributes::Int noOverlap, const std::string &name = {})

Add a dynamic slice operation to the model.

Creates a new slice tensor, e.g. slice = tensor[offset]

 

Return
The name of the result tensor
Parameters
  • args: [tensor, offset]
  • axes: Axes along which to slice
  • sizes: Size of the slice in each axis
  • name: Optional identifier for operation

 

TensorId dynamicupdate(const std::vector<TensorId> &args, Attributes::Ints axes, Attributes::Ints sizes, Attributes::Int noOverlap, const std::string &name = {})

Add a dynamic update operation to the model.

Creates a copy of “tensor” with “slice” inserted at “offset”, e.g. out = tensor, out[offset] = slice

 

Return
The name of the result tensor
Parameters
  • args: [tensor, offset, slice]
  • axes: Axes along which to update
  • sizes: Size of the slice in each axis
  • name: Optional identifier for operation

 

TensorId dynamiczero(const std::vector<TensorId> &args, Attributes::Ints axes, Attributes::Ints sizes, const std::string &name = {})

Add a dynamic zero operation to the model.

Creates a copy of “tensor” with a slice at “offset” set to zero, e.g. out = tensor, out[offset] = 0.0

 

Return
The name of the result tensor
Parameters
  • args: [tensor, offset]
  • axes: Axes along which to erase
  • sizes: Size of the slice in each axis
  • name: Optional identifier for operation

 

TensorId dynamicadd(const std::vector<TensorId> &args, Attributes::Ints axes, Attributes::Ints sizes, const std::string &name = {})

Add a dynamic add operation to the model.

Creates a copy of “tensor” with “slice” added at “offset”, e.g. out = tensor, out[offset] += slice

 

Return
The name of the result tensor
Parameters
  • args: [tensor, offset, slice]
  • axes: Axes along which to add
  • sizes: Size of the slice in each axis
  • name: Optional identifier for operation

 

std::vector<TensorId> call(const std::vector<TensorId> &args, unsigned num_outputs, const Builder &callee, const std::string &name = {})

Add a call operation to the model.

This is a poplar extension, to expose manual code re-use to the builder

 

Return
A vector of tensors; the subgraph outputs
Parameters
  • args: Tensor T
  • callee: The subgraph to call into
  • name: Optional identifier for operation

 

class BuilderImpl

An implementation of a Builder.

Public Functions

void op(const OperatorIdentifier &opid, int opsetVersion, const std::vector<TensorId> &inputs, const std::vector<TensorId> &outputs, const std::map<std::string, boost::any> &opAttributes, const std::string &name, std::function<void(std::vector<TensorId>, std::map<std::string, boost::any>)> validateInput = nullptr)

Add an op to the model.

 

Parameters
  • opid: The operator identifier
  • opsetVersion: The opset for the domain of the op
  • inputs: The input tensor ids
  • outputs: The output tensor ids
  • opAttributes: The attributes of the op
  • name: Debug name
  • validateInput: Callback function to validate the inputs & attributes

 

std::vector<TensorId> op(const OperatorIdentifier &opid, int opsetVersion, const std::vector<TensorId> &inputs, const std::map<std::string, boost::any> &opAttributes, const std::string &name, std::function<void(std::vector<TensorId>, std::map<std::string, boost::any>)> validateInput = nullptr)

Add an op to the model.

 

Return
A list of output tensor ids. Size is given by numberOfOutputs
Parameters
  • opid: The operator identifier
  • opsetVersion: The opset for the domain of the op
  • inputs: The input tensor ids
  • opAttributes: The attributes of the op
  • name: Debug name
  • validateInput: Callback function to validate the inputs & attributes

 

std::vector<TensorId> op(const OperatorIdentifier &opid, int opsetVersion, const std::vector<TensorId> &inputs, unsigned int numOutputs, const std::map<std::string, boost::any> &opAttributes, const std::string &name, std::function<void(std::vector<TensorId>, std::map<std::string, boost::any>)> validateInput = nullptr)

Add an op to the model.

 

Return
A list of output tensor ids. Size is given by numberOfOutputs
Parameters
  • opid: The operator identifier
  • opsetVersion: The opset for the domain of the op
  • inputs: The input tensor ids
  • numOutputs: The number if output tensors
  • opAttributes: The attributes of the op
  • name: Debug name
  • validateInput: Callback function to validate the inputs & attributes

 

Data flow

enum popart::AnchorReturnTypeId

Values:

FINAL = 0
EVERYN
ALL
SUM
class AnchorReturnType
class DataFlow

Device manager

enum popart::DeviceType

Values:

IpuModel = 0
Cpu
Ipu
Sim
class DeviceInfo

Represents a device.

Subclassed by popart::popx::DevicexInfo

Public Functions

virtual bool attach() = 0

Attach to the IPU.

 

Return
Returns true if successfully attaches to the device

 

virtual void detach() = 0

Detach from the IPU.

DeviceType getType() const

Get the type of the device.

std::string toString() const

Return a description of the device.

virtual int getId() const = 0

Get the device id.

virtual std::string getVersion() const = 0

Get the version of the software on the ipu.

virtual int getNumIpus() const = 0

Get the number of ipus.

virtual int getTilesPerIpu() const = 0

Get the number tiles per ipu.

virtual int getNumWorkerContexts() const = 0

Get the number work contexts per tile.

class DeviceManager

The class for.

Public Functions

void registerDeviceProvider(DeviceProvider *provider)

Used to register a device provider.

 

Parameters
  • provider: A provider

 

std::shared_ptr<DeviceInfo> getDevice(SyncPattern syncPattern = SyncPattern::Full, uint32_t deviceManagerId = 0)

Get the Device object of a device by ID.

 

Return
List of requested IPUs.
Parameters
  • syncPattern: Sync pattern
  • deviceManagerId: Number of IPUs to request.

 

std::vector<std::shared_ptr<DeviceInfo>> enumerateDevices(SyncPattern pattern = SyncPattern::Full, uint32_t replication_factor = 1, int numIpus = 1, DeviceType deviceType = DeviceType::Ipu)

Get the list of all devices fulfilling the specified criteria.

 

Return
List of requested IPUs.
Parameters
  • pattern: Sync pattern.
  • replication_factor: Number of times to replicate.
  • numIpus: Number of IPUs to request.
  • deviceType: Type of device required.

 

std::shared_ptr<DeviceInfo> acquireAvailableDevice(int numIpus = 1, int tilesPerIpu = 1216, SyncPattern pattern = SyncPattern::Full, uint32_t replication_factor = 1)

Finds the first available hardware device, that a certain number of IPUs.

This method will attach to the device.

Return
A device, which can be used with a session. Will return nullptr if no device is available
Parameters
  • numIpus: The number of IPUs on the device [=1]
  • tilesPerIpu: The number of tiles on the IPU [=1216]

 

std::shared_ptr<DeviceInfo> acquireDeviceById(int id, SyncPattern pattern = SyncPattern::Full, uint32_t replication_factor = 1)

Allocates the hardware device by id.

This id can be found running ‘gc-info -l’ This method will attach to the device

Return
A device. Will return nullptr if the device is not available
Parameters
  • id: The index of the IPU to be used

 

std::shared_ptr<DeviceInfo> createCpuDevice()

Create a ‘simulated’ CPU device.

 

Return
A device

 

std::shared_ptr<DeviceInfo> createIpuModelDevice(std::map<std::string, std::string> &options)

Create a ‘simulated’ IPU Model device The following options are supported : | numIPUs | The number of ipus to simulate | | tilesPerIPU | The number of tiles per ipu | | compileIPUCode | Whether or note to compile read Ipu code for modelling|.

 

Return
A device
Parameters
  • options: Configuration settings for the IPU Model

 

std::shared_ptr<DeviceInfo> createSimDevice(std::map<std::string, std::string> &options)

Create a ‘simulated’ Sim device The following options are supported : | numIPUs | The number of ipus to simulate | | tilesPerIPU | The number of tiles per ipu |.

 

Return
A device
Parameters
  • options: Configuration settings for the Sim

 

Public Static Functions

static DeviceManager &createDeviceManager()

Accessor for the device manager.

 

Return
A reference to the DeviceManager

 

class DeviceProvider

The interface for device provides which are registered with the device manager.

Subclassed by popart::popx::DevicexManager

Public Functions

virtual void enumerate(std::vector<std::shared_ptr<DeviceInfo>> &devices, uint32_t requiredNumIPUs, SyncPattern syncPattern, DeviceType type) = 0

Get the list of all devices fulfilling the specified criteria.

 

Parameters
  • devices: Devices to get
  • requiredNumIPUs: Number of IPUs to request.
  • syncPattern: Sync pattern

 

virtual std::shared_ptr<DeviceInfo> createHostDevice(DeviceType type, const std::map<std::string, std::string> &options) = 0

Create a host device for testing.

std::ostream &popart::operator<<(std::ostream&, VirtualGraphMode)
std::ostream &popart::operator<<(std::ostream&, const ConvPartialsType&)
std::ostream &popart::operator<<(std::ostream&, VirtualGraphMode)
std::ostream &popart::operator<<(std::ostream &os, const DeviceInfo &di)
std::ostream &popart::operator<<(std::ostream &os, const DeviceType &dt)

Write a representation of a DeviceType to an output stream.

 

Return
the same output stream for chaining
Parameters
  • os: output stream
  • dt: device type reference

 

std::ostream &popart::operator<<(std::ostream&, const GradInOutMapper&)
std::ostream &popart::operator<<(std::ostream&, const GradOpInType&)
std::ostream &popart::operator<<(std::ostream&, const GraphId&)
std::ostream &popart::operator<<(std::ostream &ss, const Half &v)
std::ostream &popart::operator<<(std::ostream &os, const OperatorIdentifier &opid)
std::ostream &popart::operator<<(std::ostream &os, const Patterns &patterns)
std::ostream &popart::operator<<(std::ostream&, const Scope&)
std::ostream &popart::operator<<(std::ostream &stream, const TensorInfo &ti)
std::ostream &popart::operator<<(std::ostream &os, const TensorType &tt)
std::ostream &popart::operator<<(std::ostream &os, const TopoCons &tc)
std::ostream &popart::operator<<(std::ostream &ss, const std::vector<std::size_t> &v)
template <typename T>
std::ostream &popart::operator<<(std::ostream &os, const NDArrayWrapper<T> &array)

Error handling

enum popart::ErrorSource

Values:

popart = 0
popart_internal
poplar
poplibs
unknown
class error : public runtime_error

Exception class for popart.

Subclassed by popart::internal_error, popart::memory_allocation_err

Public Functions

template <typename... Args>
error(const char *s, const Args&... args)

Variadic constructor for error which allows the user to use a fmt string for the message.

As the fmt::format function can throw an exception itself and it is used in the initilization list we have to use the unusally C++ syntax to catch that exception and convert it to a popart exception

throw error(“This is an error reason {}”, 42);

class memory_allocation_err : public popart::error