view tvii/deep.py @ 80:3c7927f59b05

notes to self re deep neural networks
author Jeff Hammel <k0scist@gmail.com>
date Sun, 17 Dec 2017 13:43:42 -0800
parents
children
line wrap: on
line source

"""
Deep neural networks

Forward propagation for layer `l`

Input: a[l-1]

Output: a[l], cache(z[l] {w[l], b[1]})

z[l] = w[l] ...

---

Backward propagation for layer `l`:

Input: da[l]
Output: da[l-1], dW[l], db[l]

dz[l] = da[l]* g[l]'(z[l])
dw[l] = dz[l] a[l-1]
db[l] = dz[l]
dz[l-1] w[l].T dz[l]

dz[l] = w[l+1].T dz[l+1] * g[l]' ( z[l] )

=>

dZ[l] dZ[l] * g[l]' ( Z[l] )

dW[l] (1/m) dZ[l] A[l-1].T

db[l] = (1/m) np.dum(dZ[l], axis=1, keepdims=True)
dA[l-1] = W[l].T * dZ[l]


For the final layerL
da[l] = - (y/a) + (1 - y)/(1-a)
dA[l] = (-(y(1)/a(1) + (1 - y(1))/(1 - a(1)) # first training example
        ...)


The weight matrix for layer `l`, W[l] is
of the shape (n[l], n[l-1])
"""