I have written a few simple keras layers. This post will summarise about how to write your own layers. It’s for beginners because I only know simple and easy ones 😉

## 1. Keras layer

https://keras.io/layers/about-keras-layers/ introduces some common methods. For beginners I don’t think it’s necessary to know these.

## 2. Keras Lambda layer

Lambda layer is an easy way to customise a layer to do simple arithmetics. As written in the page,

…an arbitrary Theano / TensorFlow expression…

we can use the operations supported by Keras backend such as `dot, transpose, max, pow, sign, etc `

as well as those are not specified in the backend documents but actually supported by Theano and TensorFlow – e.g., `**, /, //, %`

for Theano.

### 2.1 Lambda layer and output_shape

You might need to specify the output shape of your Lambda layer, especially your Keras is on Theano. Otherwise it just seems to infer it with `input_shape`

.

#### 2.1.1 With function

You can create a function that returns the output shape, probably after taking input_shape as an input. Here, the function returns the shape of the WHOLE BATCH.

#### 2.1.2 With tuple

If you pass tuple, it should be the shape of ONE DATA SAMPLE.

## 3. A Keras model as a layer

On high-level, you can combine some layers to design your own layer. For example, I made a Melspectrogram layer as below. (Complete codes are on keras_STFT_layer repo.) In this way, I could re-use `Convolution2D`

layer in the way I want.

Downside would be some overhead due to many layers.

## 4. Customising Layer

When Lego-ing known layers doesn’t get you what you want, write your own!

### 4.1 Read the document

https://keras.io/layers/writing-your-own-keras-layers/ Read this! Whether you fully understand it or not. I didn’t fully understand but later I got it thanks to @fchollet’s help.

### 4.2 Four methods

#### 4.2.1 `__init()__`

:

initiate the layer. Assign attributes to `self`

so that you can use them later.

#### 4.2.2`build(self, input_shape)`

:

- initiate the tensor variables (e.g.
`W`

,`bias`

, or whatever) using Keras backend functions (e.g.,`self.W = K.variable(an_init_numpy_array)`

). - set
`self.trainable_weights`

with a list of variables. e.g.,`self.trainable_weights=[self.W]`

.

**Remember : **trainable weights should be tensor variables so that machine can auto-differenciate them for you.

**Remember (2):** Check out the dtype of every variable! If you initiate a tensor variable with `float64`

a numpy array, the variable might be also `float64`

, which will get you an error. Usually it wouldn’t because by default `K.variable()`

casts the value into `float32`

. But, check check check! check it by simply printing `x.dtype`

.

#### 4.2.3 `call(self, x, mask=None)`

:

This is where you implement the **forward-pass operation**. You may want to dot product with one of the trainable weights and input (`K.dot(x, self.W)`

), wanna expand the dimensionality of a tensor variable (`K.expand_dims(var1, dim=2)`

), or whatever.

Again, **dtype**! For example, I had to use this line, `np.sqrt(2. * np.pi).astype('float32')`

, to make the constant to be `float32`

.

#### 4.2.4 `get_output_shape_for(self, input_shape)`

As the name says.

### 4.3 Examples

#### 4.3.1 Example 1 : Cropping2D Layer

It crops 2D input. Simple!

#### 4.3.2 Example 2. ParametricMel

### 4.4 Tips

Remember: you need to make the operation of layer differentiable w.r.t the input and trainable weights you set. Look up keras backend use them.

`tensor.shape.eval()`

returns an integer tuple. You would need to print them a lot 😉