matcher

Subgraph Isomorphic Matcher

With uTensorGraphMatcher, performing isomorphic subgraph matching along with replacing or manipulating the matched subgraph(s) takes just a few line of code:

from utensor_cgen.matcher import uTensorGraphMatcher

# `pattrn_ugraph` is the pattern to match with
pattrn_ugraph = ...
matcher = uTensorGraphMatcher(pattrn_ugraph)

# a larget graph to perform subgraph match
subject_ugraph = ...

# matches is a list of `uTensorGraphMatch` objects
matches = matcher.match_all(subject_ugraph)
if matches:
  # do stuff with the matches

Use Case: Node Fusion

Note: we’ll use operation/node/layer interchangeably in the documentation

  • It’s commonly seen pattern in convolution neural network (CNN), conv -> relu -> pooling. That is, a 2D convolution followed by a relu layer and then a pooling down sampling layer.

  • With our uTensorGraphMatcher, you can locate such pattern in your CNN model and fuse/replace matched nodes into one optimized QuantizedFusedConv2DMaxpool node.

  • Left: original graph

  • Middle: matched convolution layer

  • Right: replace the matched layer with specialized QuantizedFusedConv2DMaxpool node

conv-pool-fuse

Use Case: Dropout Layer Removal

  • Though dropout is an effective technique to improve training performance of your model, it’s not necessary during inference phrase.

  • In the mainstream frameworks such as Tensorflow or `PyTorch`_, an dropout layer is typically implemented with other elementary operations/nodes. As a result, finding and removing those nodes for inference optimization (both in model size and prediciton time) is not trivial and error prone.

  • With our uTensorGraphMatcher, you can find and remove the dropout nodes as illustrated in the following picture.

    • Left: original graph with dropout Layers

    • Middle: matched dropout layers

    • Right: graph with dropout layers removed

cnn-dropout

We use mainly Tensorflow for declaring the pattern graph for matcher now.

High-level graph builder is on its way, see Future Works for detail.

Module members

class utensor_cgen.matcher.uTensorGraphMatch

A isomorphic subgraph match

See also uTensorGraphMatcher

Parameters

  • pattern_ugraph (uTensorGraph) – the parttern graph

  • subject_ugraph (uTensorGraph) – the subjective graph

  • patrn2subj_op_map (dict) – a dict with key as op name in the pattern_ugraph and value as the matched op in the subject_ugraph

  • subj2patrn_op_map (dict) – a dict with key as op name in the subject_ugraph and value as the matched op in the pattern_ugraph

  • patrn2subj_tensor_map (dict) – a dict with key as the tensor object in the pattern_ugraph and value as the tensor object in the subject_ugraph

  • subj2patrn_tensor_map (dict) – a dict with key as the tensor object in the subject_ugraph and value as the tensor object in the pattern_ugraph

replace_with(self, callback, suffix=None)

Replace matched subgraph with a given ugraph given by the callback, not in-place

Parameters

  • callback (callable) – a callable object which takes a uTensorGraphMatch and reutrn three values – a uTensorGraph object to replaced the matched subgraph with (the replacing graph), an input_map (dict) maps input tensors in pattern graph to the input tensors in replacing graph and an output_map (dict) which maps the output tensors

  • suffix (str) – (optional) the suffix to add to the name of ops/tensors in the replacing graph returned by callback. If not given, it will be a random string

Return type

uTensorGraph, a new graph with matched subgraph replaced

class utensor_cgen.matcher.uTensorGraphMatcher

Isomorphic Subgraph Matcher

Perform isomorphic subgraph match against given graph

A minimal example

# Example: match and replace
patrn_ugraph = ... # load the pattern uTensorGraph

# create a matcher
matcher = uTensorGraphMatcher(
  pattern_ugraph=patrn_ugraph,
  op_equality_delegate=delegate, # a subclass of :py:class:`OpEqualityDelegateBase`
)

# define a callback
def callback(match):
  # inspect the match object
  ....
  # return the replacing graph, input and output map
  return repl_ugraph, input_map, output_map

# load the subject uTensorGraph
subj_ugraph = ...

# search for 1 match
matches = matcher.match(subj_ugraph, n=1) # return a list of uTensorGraphMatch objects

if matches:
  match = matches[0]
  match.replace_with(callback)

See also uTensorGraphMatch

Parameters

pattern_ugraph (uTensorGraph) – a graph serve as pattern to look for

match(self, other_ugraph, n=1)

Match the pattern against the graph

Parameters

  • other_ugraph (uTensorGraph) – the graph where to search the pattern

  • n (int) – the maximum matches to return

Return type

List[uTensorGraphMatch]