Stack ReLU layers

Each layer reads the output of the layer before it; layer 1 reads x. A layer with one unit computes a·ReLU(w·z + b) + c; with two units it adds a second term a′·ReLU(w′·z + b′). The thick line is the last layer's output; the faint lines are the earlier layers, and the dots mark the bends of the thick line. Every new layer starts as the tent t(z) = 2·ReLU(z) − 4·ReLU(z − 0.5), two units that fold [0, 1] in half, so each press of Add 1 layer doubles the pieces: 2, 4, 8, 16, …. Move the sliders to bend the layers in other ways. Switch a layer to one unit and its output only rises or only falls, so it adds at most one bend; Tent puts the fold back.