PopART C++ API Reference

Introduction

The Poplar Advanced Run Time (PopART) is part of the Poplar SDK for implementing and running algorithms on networks of Graphcore IPU processors.

This document describes the PopART C++ API. For more information about PopART, please refer to the PopART User Guide.

PopART C++ API

Sessions

class InferenceSession : public popart::Session

Public Static Functions

static std::unique_ptr<InferenceSession> createFromOnnxModel(const std::string &model, const DataFlow &dataFlow, std::shared_ptr<DeviceInfo> deviceInfo, const std::vector<Loss *> &losses = {}, const InputShapeInfo &inputShapeInfo = InputShapeInfo(), const SessionOptions &userOptions = SessionOptions(), const Patterns &patterns = Patterns())

Create a runtime class for executing an ONNX graph on a set of IPU hardware for inference.

 

Parameters
  • model: Either an ONNX model protobuf, or the name of a file containing an ONNX model protobuf

  • inputShapeInfo: Information about the shapes of input and output tensors

  • losses: An optional list of loss layers to use after inference

  • dataFlow: Configuration for the data feeds and fetches

  • userOptions: String to configure session options

  • patterns: Optimization patterns to apply

 

class TrainingSession : public popart::Session

Public Functions

void updateOptimizer(const Optimizer *optimizer)

Update the optimizer.

Note that the optimizer passed in must be compatible with that passed to the constructor. For example, you cannot update to an Optimizer which uses momentum here, if the Optimizer passed to the constructor did not have momentum. Reason: The Ir would need to change to incorporate momentum, but the Ir is frozen once constructed. NB: Must call optimizerFromHost for this update to take effect on the device.

 

Parameters
  • optimizer: A pointer to a popart::Optimizer

 

void optimizerFromHost()

write whatever optimizer tensors (learning rates, momentum, initial momentum tensors (zero)) there are to device

const std::vector<std::string> &getHostReduceStreamIds() const

Access the stream IDs for variables that are involved in host side reductions on the host.

Only populated if hostAllReduce is enabled in the SessionOptions

void connectStreamToCallback(const std::string &streamHandle, std::function<void(void *)> callback, unsigned index = 0, )

Connect Poplar stream callbacks.

In conjunction with getGradAndVarStreamIds the streams can be used to copy gradients to the host to perform collective operations after which the variables can be streamed back after they have been updated to the device. index referes to the replica index when using replicated graphs.

Public Static Functions

static std::unique_ptr<TrainingSession> createFromOnnxModel(const std::string &model, const DataFlow &dataFlow, const std::vector<Loss *> &losses, const Optimizer &optimizer, std::shared_ptr<DeviceInfo> deviceInfo, const InputShapeInfo &inputShapeInfo = InputShapeInfo(), const SessionOptions &userOptions = SessionOptions(), const Patterns &patterns = Patterns())

Create a runtime class for executing an ONNX graph on a set of IPU hardware for training.

 

Parameters
  • model: Either an ONNX model protobuf, or the name of a file containing an ONNX model protobuf

  • inputShapeInfo: Information about the shapes of input and output tensors

  • dataFlow: Configuration for the data feeds and fetches

  • losses: A list of loss layers to use when training

  • optimizer: The name of an optimizer to use when training

  • userOptions: String to configure session options

  • patterns: Optimization patterns to apply

 

Session option flags

struct SessionOptions

A structure containing user configuration options for the Session class.

Public Members

std::string logDir

A directory for log traces to be written into.

std::set<DotCheck> dotChecks = {}

When to write ‘.dot’ files during Ir construction.

int firstDotOp = 0

The ops to write to the .dot file will be a continuous interval of the schedule, controlled by firstDotOp and finalDotOp.

In particular, it will be [min(0, firstDotOp), max(N ops in Ir, finalDotOp))

bool dotOpNames = false

Include the Op name in the .dot file (the Op type is always exported)

bool exportPoplarComputationGraph = false

Export Poplar computation graph.

bool exportPoplarVertexGraph = false

Export Poplar vertex graph.

bool enableOutlining = true

Controls caching of identical sections of the graph.

bool enableOutliningCopyCostPruning = true

Controls whether the cost of copying of cached sections should be included in the outlining cost model.

float outlineThreshold = 1.0f

The incremental value that a sub-graph requires, relative to its nested sub-graphs (if any), to be eligible for outlining.

A high threshold results in fewer sub-graphs being outlined, a negative value results in all being outlined. The gross value of a sub-graph is the sum of its constituent Ops’ getSubgraphValue() values. To disable outlining, it is better to set enableOutlining to false than to set this value to infinity. The default value of 1.0f results in all high Value operations such as convolution being cached, but standalone low Value operations such as Relu will not be.

RecomputationType autoRecomputation = RecomputationType::None

Enable recomputation of operations in the graph in the backwards pass to reduce model size at the cost of computation cycles.

MergeVarUpdateType mergeVarUpdate = MergeVarUpdateType::None

Enable merging of VarUpdates into groups of VarUpdates, by flattening and concatenating Variable Tensors and Updating Tensors.

int64_t mergeVarUpdateMemThreshold = 1000000

The AutoLoose and AutoTight VarUpdate merging algorithm has a threshold on the total memory of Variable Tensors to merge for updating.

Memory in bytes.

int64_t looseThresholdAtPeak = 8000

The AutoLoose VarUpudate merging algorithm has absolute threshold defined by min(mergeVarUpdateMemThreshold, liveAtPeak - liveCurrently + looseThresholdAtPeak), where liveAtPeak is an estimate of the maximum live memory of the computation, and liveCurrently is an estimate of the live memory where the threshold is being used to determine whether to schedule or postpone a VarUpdate.

bool rearrangeAnchorsOnHost = true

Before anchor tensors are streamed from device to host, they are not necessarily arranged in memory as required when they are to be copied from host stream to host.

This can be done on the device or on the host. Done on host by default to save memory, but often at the expense of cycles, especially for larger anchor tensors.

bool enablePrefetchDatastreams = true

By default, we will use prefecting for input data streams.

Poplar will speculative read data for a stream before is is required to allow the ‘preparation’ of the data to occur in parrallel with compute

bool enableNonStableSoftmax = false

By default, we use the stable-softmax poplar function.

This input tensor to softmax, x, is preprocessed by subtracting max(x) to each element before computing the exponentials, ensuring numerical stability. If you are sure the inputs to your softmax operations are small enough to not cause overflow when computing the exponential, you can enable the non-stable version instead for speedup

bool enableVirtualGraphs = false

Enable placement of operations on individual IPUs by creating a ‘virtual graph’ for each IPU.

bool enableReplicatedGraphs = false

Enable replication of graphs.

bool enableGradientAccumulation = false

Enable gradient accumulation.

int64_t replicatedGraphCount = 1

If enableReplicatedGraphs is true, replicatedGraphCount will set the number of replicated graphs - must be a factor of the number of IPU’s (CHECK)

int64_t accumulationFactor = 1

Specify the number of micro-batches to accumulate before applying the varUpdate.

bool autoVirtualGraph = false

Enable transformation pass that attempts to automatically place ops on virtual graphs to achieve model parallelism.

bool enablePipelining = false

Enable pipelining of virtual graphs.

bool ignoreData = false

Use synthetic data i.e.

disable data transfer to/from the host Set to ‘true’ to use synthetic data, ‘false’ to use real data To be removed: T13474

SyntheticDataMode syntheticDataMode = SyntheticDataMode::Off

Use synthetic data i.e.

disable data transfer to/from the host Set to ‘Off’ to use real data Note: this will be overriden by the legacy option ‘ignoreData’ until it is removed (T13474)

bool instrumentWithHardwareCycleCounter = false

Add instrumentation to your program to count the number of device cycles (a single tile, on a single IPU) that your main program takes to execute.

Expect this to have a small detrimental impact on performance.

bool disableGradAccumulationTensorStreams = false

If true, the weight gradient tensors are not saved off the device when devicex.weightsFromHost() is called.

Note: this option is overridden if syntheticDataMode is not Off.

bool compileEngine = true

when false, the backend will build the Poplar graph, but do not compile it into an Engine.

When this option is set, no execution can be performed, and nothing can be transferred to the device. Functions which retrieve information from the graph building stage will be ok (tile mapping).

bool enableEngineCaching = false

Enable poplar executable caching.

std::string cachePath = "session_cache"

Path to save the poplar::Executable to.

bool enableGroupedMatmuls = false

Enable/disable the grouping of matmuls that are the same shape.

bool enableSerializedMatmuls = true

Enable/disable the serializing of matmuls.

bool hostAllReduce = false

Perform AllReduce operation on the host. Only useful for training session.

bool hostWeightUpdate = false

Perform weight update on the host. Only useful for training session.

std::map<std::string, std::string> engineOptions

Poplar engine options.

std::map<std::string, std::string> convolutionOptions

Poplar convolution options.

std::map<std::string, std::string> reportOptions

Poplar reporting options.

std::vector<std::string> customCodelets

List of codelets (with filetype) to be added to the poplar graph.

See the poplar documentation for more information.

std::string customCodeletCompileFlags

Compile flags for the custom codelets.

For example -g to generate debug info.

Training operations

Losses

class L1Loss : public popart::Loss
class NllLoss : public popart::Loss

Optimisers

class ConstSGD : public popart::SGD
class SGD : public popart::Optimizer

Subclassed by popart::ConstSGD

Builder

class Builder

An interface for a Builder, used for creating ONNX graphs.

Public Functions

Builder &createSubgraphBuilder()

Return a Builder for a graph which is nested inside this Builder’s graph.

TensorId addInputTensor(const TensorInfo &tensorInfo, const std::string &debugPrefix = "")

Add a new input tensor to the model.

 

Return

The unique name of the input tensor

Parameters
  • tensorInfo: The shape and type of the input tensor

  • debugPrefix: A string to prepend to the name of the tensor

 

TensorId addUntypedInputTensor(const std::string &debugPrefix = "")

Add a new input tensor without a type or shape to the model.

 

Return

The unique name of the input tensor

Parameters
  • debugPrefix: A string to prepend to the name of the tensor

 

void addInputTensorFromHigherScope(const TensorId &tensorId)

Add a new named input tensor to the model.

 

Parameters
  • tensorId: The identifier string of the input tensor. This identifier must already exist in the parent GraphProto’s name scope and must appear topologically before this sub-graph.

 

TensorId addInitializedInputTensor(const ConstVoidData &initData, const std::string &debugPrefix = "")

Add a new preinitialized input tensor to the model.

 

Return

The unique name of the input tensor

Parameters
  • initData: The initial data of the input tensor

  • debugPrefix: A string to prepend to the name of the tensor

 

void addOutputTensor(const TensorId &arg0)

Adds one of the outputs from a node in the graph into the list of output tensors.

AiOnnxOpset6 aiOnnxOpset6()

Return the builder interface for ai.onnx opset 6.

AiOnnxOpset7 aiOnnxOpset7()

Return the builder interface for ai.onnx opset 7.

AiOnnxOpset8 aiOnnxOpset8()

Return the builder interface for ai.onnx opset 7.

AiOnnxOpset9 aiOnnxOpset9()

Return the builder interface for ai.onnx opset 9.

AiOnnxOpset10 aiOnnxOpset10()

Return the builder interface for ai.onnx opset 10.

AiOnnxOpset11 aiOnnxOpset11()

Return the builder interface for ai.onnx opset 11.

AiOnnxMlOpset1 aiOnnxMlOpset1()

Return the builder interface for ai.onnx.ml opset 1.

AiGraphcoreOpset1 aiGraphcoreOpset1()

Return the builder interface for ai.graphcore opset 1.

template<class T>
TensorId reshape_const(T &t, const std::vector<TensorId> &args, const std::vector<int64_t> &shape, const std::string &name = {})

This is a helper function that will add a constant and a reshape using the provided domain.

void recomputeOutputInBackwardPass(const TensorId &nodeOutputName, RecomputeType value = RecomputeType::RECOMPUTE)

Enable/disable recomputation of the output of the node in the backward pass.

 

Parameters
  • nodeOutputName: Name of the output tensor of the ONNX node

  • value: If the recompute is enabled/disabled

 

void recomputeOutputInBackwardPass(const std::set<TensorId> &nodeOutputNames, RecomputeType value = RecomputeType::RECOMPUTE)

Enable/disable recomputation of the output of the node in the backward pass.

 

Parameters
  • nodeOutputNames: Names of the output tensors of the ONNX node

  • value: If the recompute is enabled/disabled

 

bool getRecomputeOutputInBackwardPass(const TensorId &nodeOutputName)

Get whether the given node will have its output recomputed in the backward pass.

 

Parameters
  • nodeOutputName: Name of the output tensor of the ONNX node used to find the node in the ONNX model.

 

bool getRecomputeOutputInBackwardPass(const std::set<TensorId> &nodeOutputNames)

Get whether the given node will have its output recomputed in the backward pass.

 

Parameters
  • nodeOutputNames: Names of the output tensors of the ONNX node used to find the node in the ONNX model.

 

void virtualGraph(const TensorId &nodeOutputName, int64_t value = 0)

Set the virtual graph that computes the given node.

Applies when creating a graph for a multi-IPU configuration.

 

Parameters
  • nodeOutputName: Name of the output tensor of the ONNX node

  • value: The index of the virtual graph that computes this node

 

void pingPongPhase(const TensorId &nodeOutputName, int64_t value = 0)

Set the ping pong phase that computes the given node.

 

Parameters
  • nodeOutputName: Name of the output tensor of the ONNX node

  • value: The index of the virtual graph that computes this node

 

void setSerializeMatMul(const std::set<TensorId> &nodeOutputNames, std::string mode, int64_t factor, bool keep_precision)

Set the settings for matmuls that should be serialized.

This option will split a matmul into seperate smaller matmuls that will be excuted in series. This will also serialize the grad operations if training.

 

Parameters
  • nodeOutputNames: Name of the output matmul tensors of the ONNX node

  • mode: Which dimension of the mat mul to serialize on.

  • factor: The number of serialised matmuls, must be a factor of the dimentions to serialise on.

 

void setPartialsType(const TensorId &nodeOutputName, const std::string partialsType)

Set the partials type for the given node.

Used on the convolution op.

 

Parameters
  • nodeOutputName: Name of the output tensor of the ONNX node

  • partialsType: The type for the partials. Can be either FLOAT or HALF.

 

std::string getPartialsType(const TensorId &nodeOutputName)

Get the partials type for the given node.

 

Parameters
  • nodeOutputName: Name of the output tensor of the ONNX node

 

void setAvailableMemoryProportion(const TensorId &nodeOutputName, const float availableMemoryProportion)

Set the available memory for the given node.

Used on the convolution op.

 

Parameters
  • nodeOutputName: Name of the output tensor of the ONNX node

  • availableMemoryProportion: The available memory proportion 0 < x <= 1.

 

void setAttribute(const std::string &attribute, boost::any value)

Set an attribute that will be set on all subsequent operations.

boost::any getAttribute(const std::string attribute) const

Get an attribute that has been set for all subsequent operations.

void clearAttribute(const std::string &attribute)

Unset an attribute that will be set on all subsequent operations.

bool hasAttribute(const std::string &attribute)

Check if attribute is set.

boost::any getAttribute(const std::string &attribute)

Get current attribute value.

int64_t getPipelineStage() const

A convenience function for the pipeline stage attribute.

int64_t getPingPongPhase() const

A convenience function for the ping pong phase attribute.

int64_t getVirtualGraph() const

A convenience function for the virtual graph attribute.

void virtualGraph(const std::set<TensorId> &nodeOutputNames, int64_t value = 0)

Set the virtual graph that computes the given node.

Applies when creating a graph for a multi-IPU configuration.

 

Parameters
  • nodeOutputNames: Names of the output tensors of the ONNX node

  • value: The index of the virtual graph that computes this node

 

void addNodeAttribute(const std::string &attributeName, const int64_t &attributeValue, const std::set<TensorId> &nodeOutputNames)

Add an attribute to the ONNX node which is uniquely identified by the outputs.

This functions will throw an exception if it can’t find the unique node or the attribute already exists.

 

Parameters
  • attributeName: The name of the attribute to add.

  • attributeValue: An int64_t value of the attribute to add.

  • nodeOutputNames: Names of the output tensors of the ONNX node used to find the node in the ONNX model.

 

void addNodeAttribute(const std::string &attributeName, const std::vector<int64_t> &attributeValue, const std::set<TensorId> &nodeOutputNames)

Add an attribute to the ONNX node which is uniquely identified by the outputs.

This functions will throw an exception if it can’t find the unique node or the attribute already exists.

 

Parameters
  • attributeName: The name of the attribute to add.

  • attributeValue: An std::vector<int64_t> value of the attribute to add.

  • nodeOutputNames: Names of the output tensors of the ONNX node used to find the node in the ONNX model.

 

void addNodeAttribute(const std::string &attributeName, const float &attributeValue, const std::set<TensorId> &nodeOutputNames)

Add an attribute to the ONNX node which is uniquely identified by the outputs.

This functions will throw an exception if it can’t find the unique node or the attribute already exists.

 

Parameters
  • attributeName: The name of the attribute to add.

  • attributeValue: A float value of the attribute to add.

  • nodeOutputNames: Names of the output tensors of the ONNX node used to find the node in the ONNX model.

 

void addNodeAttribute(const std::string &attributeName, const std::vector<float> &attributeValue, const std::set<TensorId> &nodeOutputNames)

Add an attribute to the ONNX node which is uniquely identified by the outputs.

This functions will throw an exception if it can’t find the unique node or the attribute already exists.

 

Parameters
  • attributeName: The name of the attribute to add.

  • attributeValue: An std::vector<float> value of the attribute to add.

  • nodeOutputNames: Names of the output tensors of the ONNX node used to find the node in the ONNX model.

 

void addNodeAttribute(const std::string &attributeName, const std::string &attributeValue, const std::set<TensorId> &nodeOutputNames)

Add an attribute to the ONNX node which is uniquely identified by the outputs.

This functions will throw an exception if it can’t find the unique node or the attribute already exists.

 

Parameters
  • attributeName: The name of the attribute to add.

  • attributeValue: A std::string value of the attribute to add.

  • nodeOutputNames: Names of the output tensors of the ONNX node used to find the node in the ONNX model.

 

void addNodeAttribute(const std::string &attributeName, const std::vector<std::string> &attributeValue, const std::set<TensorId> &nodeOutputNames)

Add an attribute to the ONNX node which is uniquely identified by the outputs.

This functions will throw an exception if it can’t find the unique node or the attribute already exists.

 

Parameters
  • attributeName: The name of the attribute to add.

  • attributeValue: An std::vector<std::string> value of the attribute to add.

  • nodeOutputNames: Names of the output tensors of the ONNX node used to find the node in the ONNX model.

 

void addNodeAttribute(const std::string &attributeName, const bool attributeValue, const std::set<TensorId> &nodeOutputNames)

Add an attribute to the ONNX node which is uniquely identified by the outputs.

This functions will throw an exception if it can’t find the unique node or the attribute already exists.

 

Parameters
  • attributeName: The name of the attribute to add.

  • attributeValue: An bool value of the attribute to add

  • nodeOutputNames: Names of the output tensors of the ONNX node used to find the node in the ONNX model.

 

void addNodeAttribute(const std::string &attributeName, const ConstVoidData &attributeValue, const std::set<TensorId> &nodeOutputNames)

Add an attribute to the ONNX node which is uniquely identified by the outputs.

This functions will throw an exception if it can’t find the unique node or the attribute already exists.

 

Parameters
  • attributeName: The name of the attribute to add.

  • attributeValue: An constant tensor initializer

  • nodeOutputNames: Names of the output tensors of the ONNX node used to find the node in the ONNX model.

 

bool nodeHasAttribute(const std::string &attributeName, const std::set<TensorId> &nodeOutputNames)

Check whether the ONNX node has an attribute set.

This functions will throw an exception if it can’t find the unique node.

 

Parameters
  • attributeName: The name of the attribute to find.

  • nodeOutputNames: Names of the output tensors of the ONNX node used to find the node in the ONNX model.

 

int64_t getInt64NodeAttribute(const std::string &attributeName, const std::set<TensorId> &nodeOutputNames)

Get the int64_t value of the attribute for the ONNX node.

This functions will throw an exception if it can’t find the unique node or the attribute does not exist or it has not been set to the int64_t type.

 

Return

Value of the attribute

Parameters
  • attributeName: The name of the attribute to find.

  • nodeOutputNames: Names of the output tensors of the ONNX node used to find the node in the ONNX model.

 

std::vector<int64_t> getInt64VectorNodeAttribute(const std::string &attributeName, const std::set<TensorId> &nodeOutputNames)

Get the std::vector<int64_t> value of the attribute for the ONNX node.

This functions will throw an exception if it can’t find the unique node or the attribute does not exist or it has not been set to the std::vector<int64_t> type.

 

Return

Value of the attribute

Parameters
  • attributeName: The name of the attribute to find.

  • nodeOutputNames: Names of the output tensors of the ONNX node used to find the node in the ONNX model.

 

float getFloatNodeAttribute(const std::string &attributeName, const std::set<TensorId> &nodeOutputNames)

Get the float value of the attribute for the ONNX node.

This functions will throw an exception if it can’t find the unique node or the attribute does not exist or it has not been set to the float type.

 

Return

Value of the attribute

Parameters
  • attributeName: The name of the attribute to find.

  • nodeOutputNames: Names of the output tensors of the ONNX node used to find the node in the ONNX model.

 

std::vector<float> getFloatVectorNodeAttribute(const std::string &attributeName, const std::set<TensorId> &nodeOutputNames)

Get the std::vector<float> value of the attribute for the ONNX node.

This functions will throw an exception if it can’t find the unique node or the attribute does not exist.

 

Return

Value of the attribute

Parameters
  • attributeName: The name of the attribute to find.

  • nodeOutputNames: Names of the output tensors of the ONNX node used to find the node in the ONNX model.

 

std::string getStringNodeAttribute(const std::string &attributeName, const std::set<TensorId> &nodeOutputNames)

Get the std::string value of the attribute for the ONNX node.

This functions will throw an exception if it can’t find the unique node or the attribute does not exist or it has not been set to the std::string type.

 

Return

Value of the attribute

Parameters
  • attributeName: The name of the attribute to find.

  • nodeOutputNames: Names of the output tensors of the ONNX node used to find the node in the ONNX model.

 

std::vector<std::string> getStringVectorNodeAttribute(const std::string &attributeName, const std::set<TensorId> &nodeOutputNames)

Get the std::vector<std::string> value of the attribute for the ONNX node.

This functions will throw an exception if it can’t find the unique node or the attribute does not exist.

 

Return

Value of the attribute

Parameters
  • attributeName: The name of the attribute to find.

  • nodeOutputNames: Names of the output tensors of the ONNX node used to find the node in the ONNX model.

 

void removeNodeAttribute(const std::string &attributeName, const std::set<TensorId> &nodeOutputNames)

Remove an attribute from the ONNX node.

This functions will throw an exception if it can’t find the unique node or the attribute does not exist.

 

Parameters
  • attributeName: The name of the attribute to find.

  • nodeOutputNames: Names of the output tensors of the ONNX node used to find the node in the ONNX model.

 

std::vector<std::string> getAllNodeAttributeNames(const std::set<TensorId> &nodeOutputNames)

Get all the attribute names from the ONNX node.

This functions will throw an exception if it can’t find the unique node.

 

Parameters
  • nodeOutputNames: Names of the output tensors of the ONNX node used to find the node in the ONNX model.

 

int64_t getVirtualGraph(const TensorId &nodeOutputName)

Get the index of the virtual graph that computes this node.

This applies in a multi IPU system.

 

Parameters
  • nodeOutputName: Name of the output tensor of the ONNX node used to find the node in the ONNX model.

 

int64_t getVirtualGraph(const std::set<TensorId> &nodeOutputNames)

Get the index of the virtual graph that computes this node.

This applies in a multi IPU system.

 

Parameters
  • nodeOutputNames: Names of the output tensors of the ONNX node used to find the node in the ONNX model.

 

std::string getModelProto() const

Retrieve the ONNX serialized ModelProto.

 

Return

A serialized ONNX ModelProto

 

std::vector<TensorId> getInputTensorIds() const

Return a list of ONNX graph input tensor ids.

 

Return

A vector of input tensor names

 

std::vector<TensorId> getOutputTensorIds() const

Return a list of ONNX graph output tensor ids.

 

Return

A vector of output tensor names

 

std::vector<TensorId> getValueTensorIds() const

Return a list of ONNX graph value tensor ids.

These tensors are stored in the value_info section of the ONNX GraphProto structure.

 

Return

A vector of output tensor names

 

std::vector<int64_t> getTensorShape(const TensorId id)

Return an ONNX graph tensor shape, from either the input, output, or value_info lists in the GraphProto.

 

Return

A vector of tensor dimensions

Parameters
  • id: Tensor id

 

void pushNameScope(const std::string &name)

Push a name onto the name scope stack.

The names of tensors and nodes added to the ONNX graph will be prefixed with a concatenation of the names in the name stack.

void popNameScope()

Remove the last entry in the name scope stack.

std::string getNameScope(const std::string &name = "") const

Get the current namescope stack using the default delimiter.

 

Return

A string of the concatenated namescope stack.

Parameters
  • name: Optional string to concatenate to the end of the stack

 

void setGraphName(const std::string &name)

Specifies a graph name.

 

Parameters
  • name: string to name the graph

 

Public Static Functions

static std::unique_ptr<Builder> create()

Create a builder for an ONNX model.

static std::unique_ptr<Builder> createFromOnnxModel(const std::string &modelProtoOrFilename)

Create a builder which loads a serialized ONNX ModelProto into the builder and validates it.

 

Parameters
  • modelProtoOrFilename: Either an ONNX model protobuf, or the name of a file containing an ONNX model protobuf.

 

class AiGraphcoreOpset1 : public popart::DomainOpSet

Public Functions

std::vector<TensorId> groupnormalization(const std::vector<TensorId> &args, int64_t num_groups, float epsilon = 1e-05f, const std::string &name = {})

Add a groupnormalization operation to the model.

This is a poplar extension

The group will be created from a strided input

 

Return

A vector of tensors (y, mean, var)

Parameters
  • args: A vector of input tensors (x, scale, bias)

  • num_groups: The number of groups to separate the channels into

  • epsilon: The epsilon value to use to avoid division by zero.

  • name: Optional identifier for operation

 

TensorId subsample(const std::vector<TensorId> &args, const std::vector<int64_t> &strides, const std::string &name = {})

Add a subsample operation to the model.

This is a poplar extension

If multiple tensors are provided that strides will applied to them all

 

Return

The name of the result tensor

Parameters
  • args: Tensor T

  • strides: The strides

  • name: Optional identifier for operation

 

TensorId printtensor(const std::vector<TensorId> &args, int64_t print_gradient = 1, const std::string &name = {})

Add a print tensor operation to the model.

This is a poplar extension

TensorId scale(const std::vector<TensorId> &args, float scale, const std::string &name = {})

Add a scale operation to the model.

This is a poplar extension, to replace the experimental scale operator that has been removed

 

Return

The name of the result tensor

Parameters
  • args: Tensor T

  • scale: The scale to apply

  • name: Optional identifier for operation

 

TensorId gelu(const std::vector<TensorId> &args, const std::string &name = {})

Add a gelu operation to the model.

This is a poplar extension, to replace the experimental scale operator that has been removed

 

Return

The name of the result tensor

Parameters
  • args: Tensor T

  • name: Optional identifier for operation

 

class BuilderImpl

An implementation of a Builder.

Public Functions

void op(const OperatorIdentifier &opid, int opsetVersion, const std::vector<TensorId> &inputs, const std::vector<TensorId> &outputs, const std::map<std::string, boost::any> &opAttributes, const std::string &name, std::function<void(std::vector<TensorId>, std::map<std::string, boost::any>)> validateInput = nullptr)

Add an op to the model.

 

Parameters
  • opid: The operator identifier

  • opsetVersion: The opset for the domain of the op

  • inputs: The input tensor ids

  • outputs: The output tensor ids

  • opAttributes: The attributes of the op

  • name: Debug name

  • validateInput: Callback function to validate the inputs & attributes

 

std::vector<TensorId> op(const OperatorIdentifier &opid, int opsetVersion, const std::vector<TensorId> &inputs, const std::map<std::string, boost::any> &opAttributes, const std::string &name, std::function<void(std::vector<TensorId>, std::map<std::string, boost::any>)> validateInput = nullptr)

Add an op to the model.

 

Return

A list of output tensor ids. Size is given by numberOfOutputs

Parameters
  • opid: The operator identifier

  • opsetVersion: The opset for the domain of the op

  • inputs: The input tensor ids

  • opAttributes: The attributes of the op

  • name: Debug name

  • validateInput: Callback function to validate the inputs & attributes

 

std::vector<TensorId> op(const OperatorIdentifier &opid, int opsetVersion, const std::vector<TensorId> &inputs, unsigned int numOutputs, const std::map<std::string, boost::any> &opAttributes, const std::string &name, std::function<void(std::vector<TensorId>, std::map<std::string, boost::any>)> validateInput = nullptr)

Add an op to the model.

 

Return

A list of output tensor ids. Size is given by numberOfOutputs

Parameters
  • opid: The operator identifier

  • opsetVersion: The opset for the domain of the op

  • inputs: The input tensor ids

  • numOutputs: The number if output tensors

  • opAttributes: The attributes of the op

  • name: Debug name

  • validateInput: Callback function to validate the inputs & attributes

 

Data flow

enum popart::AnchorReturnTypeId

Values:

FINAL = 0
EVERYN
ALL
class AnchorReturnType
class DataFlow

Device manager

enum popart::DeviceType

Values:

IpuModel
Cpu
Ipu
Sim
class DeviceInfo

Represents a device.

Subclassed by popart::popx::DevicexInfo

Public Functions

virtual bool attach() = 0

Attach to the IPU.

 

Return

Returns true if successfully attaches to the device

 

virtual void detach() = 0

Detach from the IPU.

DeviceType getType() const

Get the type of the device.

std::string toString() const

Return a description of the device.

virtual int getId() const = 0

Get the device id.

virtual std::string getVersion() const = 0

Get the version of the software on the ipu.

virtual int getNumIpus() const = 0

Get the number of ipus.

virtual int getTilesPerIpu() const = 0

Get the number tiles per ipu.

virtual int getNumWorkerContexts() const = 0

Get the number work contexts per tile.

class DeviceManager

The class for.

Public Functions

void registerDeviceProvider(DeviceProvider *provider)

Used to register a device provider.

 

Parameters
  • provider: A provider

 

std::vector<std::shared_ptr<DeviceInfo>> enumerateDevices(SyncPattern syncPattern = SyncPattern::FULL, uint32_t replication_factor = 1)

List all the connected hardware devices.

To use a device you must first attach the device before you can set it on the Session

Return

A list of devices.

 

std::shared_ptr<DeviceInfo> acquireAvailableDevice(int numIpus = 1, int tilesPerIpu = 1216, SyncPattern pattern = SyncPattern::FULL, uint32_t replication_factor = 1)

Finds the first available hardware device, that a certain number of IPUs.

This method will attach to the device.

Return

A device, which can be used with a session. Will return nullptr if no device is available

Parameters
  • numIpus: The number of IPUs on the device [=1]

  • tilesPerIpu: The number of tiles on the IPU [=1216]

 

std::shared_ptr<DeviceInfo> acquireDeviceById(int id, SyncPattern pattern, uint32_t replication_factor)

Allocates the hardware device by id.

This id can be found running ‘gc-info -l’ This method will attache to the device

Return

A device. Will return nullptr if the device is not available

Parameters
  • id: The index of the IPU to be used

 

std::shared_ptr<DeviceInfo> createCpuDevice()

Create a ‘simulated’ CPU device.

 

Return

A device

 

std::shared_ptr<DeviceInfo> createIpuModelDevice(std::map<std::string, std::string> &options)

Create a ‘simulated’ IPU Model device The following options are supported : | numIPUs | The number of ipus to simulate | | tilesPerIPU | The number of tiles per ipu | | compileIPUCode | Whether or note to compile read Ipu code for modelling|.

 

Return

A device

Parameters
  • options: Configuration settings for the IPU Model

 

std::shared_ptr<DeviceInfo> createSimDevice(std::map<std::string, std::string> &options)

Create a ‘simulated’ Sim device The following options are supported : | numIPUs | The number of ipus to simulate | | tilesPerIPU | The number of tiles per ipu |.

 

Return

A device

Parameters
  • options: Configuration settings for the Sim

 

Public Static Functions

static DeviceManager &createDeviceManager()

Accessor for the device manager.

 

Return

A reference to the DeviceManager

 

class DeviceProvider

The interface for device provides which are registered with the device manager.

Subclassed by popart::popx::DevicexManager

Public Functions

virtual void enumerate(std::vector<std::shared_ptr<DeviceInfo>> &devices, SyncPattern syncPattern, uint32_t replication_factor) = 0

Get a list of ipu devices.

virtual std::shared_ptr<DeviceInfo> createHostDevice(DeviceType type, const std::map<std::string, std::string> &options) = 0

Create a host device for testing.

std::ostream &popart::operator<<(std::ostream&, VirtualGraphMode)
std::ostream &popart::operator<<(std::ostream&, const ConvPartialsType&)
std::ostream &popart::operator<<(std::ostream&, VirtualGraphMode)
std::ostream &popart::operator<<(std::ostream &os, const DeviceInfo &di)
std::ostream &popart::operator<<(std::ostream &os, const DeviceType &dt)

Write a representation of a DeviceType to an output stream.

 

Return

the same output stream for chaining

Parameters
  • os: output stream

  • dt: device type reference

 

std::ostream &popart::operator<<(std::ostream&, const GradInOutMapper&)
std::ostream &popart::operator<<(std::ostream&, const GradOpInType&)
std::ostream &popart::operator<<(std::ostream&, const GraphId&)
std::ostream &popart::operator<<(std::ostream &ss, const Half &v)
std::ostream &popart::operator<<(std::ostream &os, const OperatorIdentifier &opid)
std::ostream &popart::operator<<(std::ostream &os, const Patterns &patterns)
std::ostream &popart::operator<<(std::ostream&, const Scope&)
std::ostream &popart::operator<<(std::ostream &stream, const TensorInfo &ti)
std::ostream &popart::operator<<(std::ostream &os, const TensorType &tt)
std::ostream &popart::operator<<(std::ostream &os, const TopoCons &tc)
std::ostream &popart::operator<<(std::ostream &ss, const std::vector<std::size_t> &v)
template<typename T>
std::ostream &popart::operator<<(std::ostream &os, const NDArrayWrapper<T> &array)

Error handling

enum popart::ErrorSource

Values:

popart = 0
popart_internal
poplar
poplibs
unknown
class error : public runtime_error

Exception class for popart.

Subclassed by popart::internal_error, popart::memory_allocation_err

Public Functions

template<typename ...Args>
error(const char *s, const Args&... args)

Variadic constructor for error which allows the user to use a fmt string for the message.

As the fmt::format function can throw an exception itself and it is used in the initilization list we have to use the unusally C++ syntax to catch that exception and convert it to a popart exception

throw error(“This is an error reason {}”, 42);

class memory_allocation_err : public popart::error