In this article, you will learn how to train a Neural Network. Then you will learn how to use that network inside Maya to classify elements of your scene.

If you want to know what are Neural Networks, how they work and why they are particularly useful, check out this other article.

What you will learn in this article

How to train a simple Neural Network for pattern recognition inside the Keras framework

How to run the Keras framework inside Maya

How to feed Maya scene to your Neural Network and get predictions

You will need these resources to follow this tutorial

How to train a simple Neural Network in Keras

Keras is a Python interface for training Neural Networks using other frameworks as backends. Keras doesn’t train the networks itself but instead lets you define your models with a simple syntax an then translates those instructions into your framework of choice. There are a few reasons I chose Keras for this tutorial:

1. It works in Python 2.7
2. The syntax is simple and intuitive
3. It can use different ‘engines’ (backends) and provide consistent results

In Maya, we are stuck with Python 2.7. So, we cannot run the coolest frameworks in town (Pytorch and Tensorflow), but Keras let us define our model in an agnostic way, and we can then use the best engine for whatever developing environment we find ourselves in.

To train our network we don’t need to be in Maya specifically, so I am going to recommend you train your network using Google Collaboratory.

IPython Notebooks and Google Collaboratory

Google Collaboratory is a service that runs Python code online; it supports rich markups that make it very easy to follow, read, and experiment (this is inspired in IPython/Jupyter Notebooks if you don’t know what these are you should check this out). You can download the notebook for this tutorial in the resources section, but I will go through the code here.

The first step in our code is to load all the Python packages we need:

# Load everything we need
import keras                             # Our framework of choice. In Google Collab. Keras runs using Tensorflow as a backend
from keras.models import Sequential      # An object we need to define our model
from keras.layers import Dense           # This is the type of network
from keras import utils                  # Tools to process our data
import numpy as np                       # Types for structuring our data
from sklearn import datasets             # A library of interesting ML datasets

# Additionally, we set Numpy to show numbers without scientific notation that can be hard to read
np.set_printoptions(suppress=True)

Some thoughts about the previous code snippet. You should know that most Machine Learning frameworks express the inputs and outputs to their models as Numpy’s NDArray type. ND stands for N-Dimensional, that is, an array of any dimensions you want it to be. You’ll see that in this tutorial our problem has 4 input features and 3 output features, and our dataset has 150 samples of information. Therefore our NDArrays will be of sizes 150,4 (input) and 150,3 (output).

Another thing to notice is models can be defined in Keras using the Sequential API or the Functional API. We will use Sequential for the time being, for it is simpler to grasp.

Loading and Pre-Processing the Iris Dataset

Now we will load our dataset. Most of the times you will store datasets in CSV files and manipulate them accordingly to fit an NDArray. For this tutorial, we will load a dataset from the Sklearn package.

irisData = datasets.load_iris()

The Iris dataset (Fischer, 1936) is one of the best-known sample datasets for pattern recognition. According to the UCI ML Repo (https://archive.ics.uci.edu/ml/datasets/iris): “The dataset contains 3 classes of 50 instances each, where each class refers to a type of iris plant. One class is linearly separable from the other 2; the latter are NOT linearly separable from each other.” These is what the plants look like in real life:

The dataset contains 5 plant’s attributes:

1. sepal length in cm
2. sepal width in cm
3. petal length in cm
4. petal width in cm
5. class

We will use the first 4 features (inputs) to predict the class of the plant (output). Our dataset is partially pre-processed, and you can see the inputs and targets running the following lines in your notebook:

irisData.data
irisData.target

You will notice that the target classes are integers (0, 1, 2). But that formatting would lead the Neural Network to infer order from the classes, which would not be true. So we have to convert the target formatting to something known as One-Hot Encoding. In this encoding the first category will be represented as a vector [1, 0, 0], the second will be vector [0, 1, 0], and so on. Doing that is this simple:

inputs = irisData.data
targets = irisData.target
targetsVec = utils.to_categorical(targets)

Finally, when we train our model part of the samples will be used for learning, and the other part will be used to test the accuracy of the model. Therefore, we need to shuffle our dataset, so all categories are evenly represented in both training and test sets.

randIds = range(0,150)
np.random.shuffle(randIds)
randInputs = np.ndarray(shape=(150,4), dtype=float)
randTargets = np.ndarray(shape=(150,3), dtype=float)
i = 0
for j in randIds:
  randInputs[i] = (inputs[j])
  randTargets[i] = (targetsVec[j])
  i += 1

Our first pattern recognition Neural Network

Keras Sequential API makes defining Neural Nets easy. We will define our network with three lines of code:

model = Sequential()                                       # We use Keras' Sequential API
model.add(Dense(20, input_dim=4, activation='sigmoid'))    # Add a hidden dense layer with 20 neurons and sig. activation
model.add(Dense(3, activation='softmax'))                  # Add 3 outputs (one per class) with softmax activation

Don’t worry if you have not yet read our article on how a Neural Networks work. For now, all you need to know is that we will use our model to find the correlation between the petal and sepal data and the classes of the plants. The hidden layer can be understood as a set of ‘filters’ (20 filters for that matter) that will find different correlations between the four input features. The output layer will sort out the pattern that exists between the activation of such filters and the annotated categories. The layers are said to be dense because, in each layer, all inputs affect all outputs.

You might be curious about the topology we have just built. This is what it looks like:

Now we have to actually train our model, so it learns the patterns in the data. First, we set the learning parameters using these two lines:

adam = keras.optimizers.Adam(lr=0.05)
model.compile(loss='binary_crossentropy', optimizer=adam, metrics=['accuracy'])

Here we are telling Keras what should be the optimizer (Adam) and the learning rate (lr). We will talk about both of these in a future article. The loss function is the method Keras will use to determine how far, and in which direction, the model is from the correct answer so it knows how to ‘learn’. The accuracy metric will help us determine how well our model is performing on the test set.

Now we train our network:

model.fit(randInputs, randTargets, epochs=10, validation_split=0.3, batch_size=None)

Notice the parameter validation_split (at 0.3). That means 30% of all our samples will be used for testing accuracy only, while the other 70% will be actually used to train the network. We do this to avoid a problem called overfitting, that we will talk about in a future article (feel free to Google it).

After the 10 training epochs, you should have an accuracy of +95%. Notice that Neural Networks are have randomly initiated weights, so every time you run this notebook, you will get slightly different results.

Exporting your model

Saving a Neural Network in Keras is as easy as:

model.save('iris.h5')

You can then download the file using these two lines:

from google.colab import files
files.download( 'iris.h5' )

How to run Keras inside Maya

Running Keras inside Maya is not as easy as it could be. You might know different package managers that make the installation of Python packages as easy as ‘pip install keras’ or ‘conda install -c conda-forge keras’. These will not work in Maya.

A few words about Maya, Python and binary compatibility

Maya has its own Python interpreter (aka. Mayapy.exe) that is not binary compatible with many Python libraries. We have a detailed post about binary compatibility here.

For now, all you need to know is that some Python libraries we’ll use are not compatible with Maya. These are: Numpy, Sciy, and H5py. You can download the Maya 2018 compatible versions of these with the resources for this article.

Installing Keras backend and other dependencies

Before installing Keras and its dependencies we highly recommend that you add ‘mayapy.exe’ to your environments Path Variable. This is so you can easily access mayapy from any prompt without the need for typing the whole path for mayapy (c:\program files\etc…). Just press the windows button and search for ‘Edit environment variables for your account’.

Add the path to your Maya binaries, as shown:

Now install the PIP package manager. Notice that Maya lives inside your Program Files, so installing anything in mayapy requires administrator’s privilege. You will need a command prompt running as admin. Then download the get-pip python script from this address: https://bootstrap.pypa.io/get-pip.py and run the following command from the folder where you have downloaded the script:

mayapy get-pip.py

Now you can easily install everything else you’ll need. First we will install Theano, the backend we’ll use for Keras. Theano depends on Numpy and Scypi. So download both these packages from the resources in this article and run the following commands:

mayapy -m pip install PATH/numpy-1.13.1+mkl-cp27-none-win_amd64.whl
mayapy -m pip install PATH/scipy-0.19.1-cp27-none-win_amd64.whl

Now you can install Theano straight from the web, as it is not binary incompatible with Maya itself.

mayapy -m pip install theano

Before installing Keras we need to install its dependency H5py, a wrapper for the HDF5 library. I have built an H5PY wheel that is binary compatible with Maya, you can download it from the resources. But I could not bundle all the DLLs with the wheel, so you will need to install the HDF5 library for the Python package to work. The compiled library HDF5 library is also available in the resources.

mayapy -m pip install PATH/h5py-2.8.0rc1.post0-cp27-cp27m-win_amd64.whl

And now install Keras from the web

mayapy -m pip install keras

The first time you run Keras inside Maya (doing a ‘import keras’) you’ll get na error. That happens because Keras comes configured to run on Tensorflow and you do not have Tensorflow installed in mayapy. Keras will create a Json file in your Windows user directory (c:\users\yourName\.keras). Find that json file and substitute the word ‘tensorflow’ for the word ‘theano’.

A little gotcha. When you run mayapy.exe from a prompt your Python user directory is c:\users\yourName\, but when you run Python from within Maya itself the user directory is c:\users\yourName\documents. In that case, you will need to configure Keras backend again.

You are good to go.

When you run Keras again you’ll be warned about the fact that no compiler has been configured for your Theano installation. That means Theano will not run at full speed. But since we are not training our model inside Maya that will not be a real problem.

How to load your model classify new data

Now that you have Keras up and running you can use it to run your previously trained models. For this tutorial, I have prepared a sample scene for you.

The sample scene constitutes a procedural flower, for which you can control the length and width of petals and sepals. And, you guessed it, we will then used those values to classify what type of plant you have created.

Loading your trained Neural Network inside Maya

To classify your flower you will need the following Python packages: Pymel core, to extract scene attributes; Numpy to create a NDArray you will feed the model; and Keras to load the actual model.

import pymel.core as pmc
import numpy as np
from keras.models import load_model

Loading the model can be done with a single line of code

model = load_model("PATH\\iris.h5")

You can then create a function to extract scene values and get a prediction from your model:

def getPlantClass():
  # Get plant attributes from scene objects
  sl = pmc.general.getAttr('plant.sepalLength')
  sw = pmc.general.getAttr('plant.sepalWidth')
  pl = pmc.general.getAttr('plant.petalLength')
  pw = pmc.general.getAttr('plant.petalWidth')

  # Get prediciton from model
  plantData = np.array([sl,sw,pl,pw])
  plantData = plantData.reshape((1,4))
  prediction = model.predict(plantData)
  return prediction

After the function is declared you can:

getPlantClass()

You will not get a pretty name for your plant but instead a “probability” (this is quotation marks for Neural Networks are not actually probabilistic models). For example, the previous image returned a 0.77 chance of being a Setosa, a 0.22 chance of being a Virginica, and 0.005 chance of being Versicolor. In order to get a more readable result you can get the ID of the highest probability using:

np.argmax(getPlantClass())

And you can then use a dictionary to convert those values to the actual class names, as exemplified in the resources and in the image below:

In Conclusion

We have classified scalar values into categories, using the knowledge that was implicit in a dataset. Think how many things in a 3D scene are composed of scalar values: vertex positions, particle positions, transform matrices, joint orientations, the color values in textures and renders, and so on. That means we’ve got a method to classify any of these assets into categories, provided we have a dataset with sufficient information for the task at hand.

Here are the resources you need to follow this tutorial