Ever wondered how to do deep learning in C++? Ever want to do machine learning on one of the most used language that is not a script? tiny-dnn is your best solution.

I’ll assume that you are familiar with the basic concepts in a neural network and the (modern) C++ programming language.  Also you should be able to build and install library from source. I can’t teach how to do that here. Everyone’s setup is a bit different. It will take me forever to list all the possible methods of installing tiny-dnn.

Send me a mail or comment bellow if you need help with anything.

## Solving the xor problem in tiny-dnn

To start with , let’s solve one of the most basic problem in Neural Networks – the xor problem.

The problem is simple. You have the input `{{0,0}, {0,1}, {1,0}, {1,1}}` that have to be mapped to the output `{{0}, {1}, {1}, {0}}`. Effectively making the neural network act like an XOR gate. Hence the name.  Let’s start coding.

First, we include the header and the namespaces.

```#include "tiny_dnn/tiny_dnn.h"
using namespace tiny_dnn;
using namespace tiny_dnn::activation;
using namespace tiny_dnn::layers;
```

then construct and Neural Network

```network<sequential> net;
net << fully_connected_layer(2,3) << sigmoid_layer()
<< fully_connected_layer(3,1) << sigmoid_layer();
```

Conveniently, all the weights are randomly initialized for us.

Then prepare the dataset.

```std::vector<vec_t> trainIn = {{0,0}, {0,1}, {1,0}, {1,1}};
std::vector<vec_t> trainOut = {{0}, {1}, {1}, {0}};
```

Finally we can train our network with the `gradient_descent` optimizer (learning rate set to 0.53) set the loss function to MSE, batch size of 1 for 1000 epochs.

```gradient_descent optimizer(0.53f);
net.fit<mse>(optimizer, trainIn, trainOut, 1, 1000);
```

Now .save the network for future use.

```net.save("net"); //Load the network with net.load("net")
```

or use the trained network by calling  the `predict` method.

```vec_t result = net.predict({1,1});
std::cout << result << std::endl;
```

Something like `0.959534` should show up. That’s it. We just got an XOR network trained in tiny-dnn and saved it!

Full source code is available here:
https://gist.github.com/marty1885/dd648a1806348bf4cd2c2fd0feafae36

Note that you need a C++14 capable compiler to build any tiny-dnn code. Also you must link against the threading library on your system because tiny-dnn utilizes multi thread. For example to build the above code on clang: `clang++ tiny-dnn-xor.cpp -O tiny-dnn-xor -std=c++14 -O3 -pthread`.

## Building an autoencoder

Lets do something more advanced with tiny-dnn. How about build an autoencoder to encode  raw audio signals. I’ll use adamstark’s AudioFile library to load and write WAV files.

It’s not useful but good enough for demonstration purpose.

We start by declaring our some parameters we need later on and a RNG to create training samples out of.

```const int SAMPLE_NUM = 1024;
const int WINDOW_SIZE = 2048;
const int BATCH_SIZE = 256;
const int NUM_EPOCHS = 64;

vector<vec_t> samples(SAMPLE_NUM, vec_t(WINDOW_SIZE));
mt19937 rng;
```

Then load a audio file and create training samples.

```AudioFile<float> audioFile;
auto& data = audioFile.samples;
uniform_int_distribution<int> dist(0, data.size()-WINDOW_SIZE);

for(auto& v : samples)
{
int offset = dist(rng);
auto start = data.begin() + offset;
copy(start, start+WINDOW_SIZE, v.begin());
}
```

Then create an autoencoder. Tran it on the dataset we just created and save it,

```network<sequential> net;
net << fully_connected_layer(WINDOW_SIZE, 512) << tanh_layer()
<< fully_connected_layer(512, WINDOW_SIZE);

net.fit<absolute>(optimizer, samples, samples
, BATCH_SIZE, NUM_EPOCHS);
net.save("net");
```

Last, we can use the network to generated encoded and decoded audio (code not shown). The resulting audio won’t sound great. But it sure sounds like the original audio. It works!

Full source source with some addition is available here:
https://gist.github.com/marty1885/4b4a0cfa1bec0eb5fc99cbafcfb00716

## Advanced usage of tiny-dnn

### Build flags/ configuration macros

There are several  configuration macros in tiny-dnn that you can put before the header. Utilizing them proerly can make your neural network run and train faster.

Most flags start with a `CNN_` prefix.

1. Parallelization flags. `CNN_USE_OMP`, `CNN_USE_TBB` and `CNN_SINGLE_THREAD`.
• Selects what parallel framework tiny-dnn uses. If not specified, then C++ std::thread is used.
• `CNN_SINGLE_THREAD`disables parallel computing.
2. Backend flags. Ex: `CNN_USE_AVX`, `CNN_USE_SSE`, `CNN_USE_AVX2`, `USE_OPENCL` and `CNN_USE_NNPACK`, etc…
• Enables the corresponding computing backend and set it as default. Otherwise a “internal” backend (written in C/C++ only) is used to ensure compatibility on most  computer.
• Use `CNN_USE_AVX` if your are running on a modern x86. It is the best backend so far. The OpenCL backend is there but can’t do any training.

### Optimizers

tiny-dnn provides a list of optimizers. Including `gradient_descent`, `momentum`, `nesterov_momentum`, `adagrad`, `RMSProp`, `adamax` and `adam`.

All hyper parameters are adjustable from their constructor. Have fun experimenting with them.

### Accessing layers and weights

Assuming you have constructed a neural network called `net`. `net[n]` will let you access the N th layer in the network. `layer->layer_type()` will return a string that show what kind of the layers is. Ex:

```network<sequential> net;
net << fully_connected(2,1) << tanh_layer();
for(int i=0;i<net.depth();i++)
std::cout << net[i]->layer_type() << std::endl;
```

`weights()` returns a vector of flatten weights in the form of `vec_t`. For example, calling weights() of a fully connected layer will give you 2 set of weights. The first is the actual weight matrix, the other is the bias.

```network<sequential> net;
net << fully_connected(2,2);
std::vector<vec_t*> weights = net->weights();
std::cout << weights.size() << std::endl; //Prints 2
```

Other methods includes `set_trainable(bool)` sets if the layer’s weight is freezed. `trainable()` returns if the layer is trainable now.

tiny-dnn documents how to do more with it,

1. Osama khafagy says: