Multilayer perceptron (MLP)

The model below is a simple multilayer perceptron (MLP) with 3 layers.

class MLP(nn.Module):
    def __init__(self, input_size, hidden_size, output_size):
        super().__init__()
        self.fc1 = nn.Linear(input_size, hidden_size)
        self.fc2 = nn.Linear(hidden_size, hidden_size)
        self.fc3 = nn.Linear(hidden_size, output_size)

    def forward(self, x):
        x = nn.functional.relu(self.fc1(x))
        x = nn.functional.relu(self.fc2(x))
        return self.fc3(x)

mlp = MLP(256.0, 384.0, 128.0)

We demonstrate the model at a variety of batch sizes. The model has 295K parameters.

V80: 6 cores, block size 32

V80 PCIe optimisations underway, improvements coming in the next release

IA-840F: 3 cores, block size 64

IA-420F: 6 core, block size 32