Multilayer perceptron (MLP)

The model below is a simple multilayer perceptron (MLP) with 3 layers.

class MLP(nn.Module): def __init__(self, input_size, hidden_size, output_size): super().__init__() self.fc1 = nn.Linear(input_size, hidden_size) self.fc2 = nn.Linear(hidden_size, hidden_size) self.fc3 = nn.Linear(hidden_size, output_size) def forward(self, x): x = nn.functional.relu(self.fc1(x)) x = nn.functional.relu(self.fc2(x)) return self.fc3(x) mlp = MLP(256.0, 384.0, 128.0)

We demonstrate the model at a variety of batch sizes. The model has 295K parameters.

IA-840F: 3 big cores

ModelBatch sizeMean latency (μs)99th Percentile latency (μs)
mlp_b114.04.7
mlp_b444.45.2
mlp_b885.05.8

IA-420F: 6 small cores

ModelBatch sizeMean latency (μs)99th Percentile latency (μs)
mlp_b114.65.4
mlp_b444.95.6
mlp_b885.76.5