Mercurial > hg > tvii
view tvii/logistic_regression.py @ 23:f34110e28a0a
[logistic regression] we have a working cost function
author | Jeff Hammel <k0scist@gmail.com> |
---|---|
date | Mon, 04 Sep 2017 09:14:25 -0700 |
parents | 3713c6733990 |
children | 89f46435a9e2 |
line wrap: on
line source
""" z = w'x + b a = sigmoid(z) L(a,y) = -(y*log(a) + (1-y)*log(1-a)) [| | | ] X = [x1 x2 x3] [| | | ] [z1 z2 z3 .. zm] = w'*X + [b b b b ] = [w'*x1+b + w'*x2+b ...] """ import numpy as np from .sigmoid import sigmoid def propagate(w, b, X, Y): """ Implement the cost function and its gradient for the propagation: Forward Propagation: - You get X - You compute $A = \sigma(w^T X + b) = (a^{(0)}, a^{(1)}, ..., a^{(m-1)}, a^{(m)})$ - You calculate the cost function: $J = -\frac{1}{m}\sum_{i=1}^{m}y^{(i)}\log(a^{(i)})+(1-y^{(i)})\log(1-a^{(i)})$ Here are the two formulas you will be using: $$ \frac{\partial J}{\partial w} = \frac{1}{m}X(A-Y)^T\tag{7}$$ $$ \frac{\partial J}{\partial b} = \frac{1}{m} \sum_{i=1}^m (a^{(i)}-y^{(i)})\tag{8}$$ Arguments: w -- weights, a numpy array of size (num_px * num_px * 3, 1) b -- bias, a scalar X -- data of size (num_px * num_px * 3, number of examples) Y -- true "label" vector (containing 0 if non-cat, 1 if cat) of size (1, number of examples) Return: cost -- negative log-likelihood cost for logistic regression dw -- gradient of the loss with respect to w, thus same shape as w db -- gradient of the loss with respect to b, thus same shape as b Tips: - Write your code step by step for the propagation. np.log(), np.dot() """ m = X.shape[1] A = sigmoid(w.T*X + b)# compute activation def cost_function(w, b, X, Y): """ Cost function for binary classification yhat = sigmoid(W.T*x + b) interpret yhat thhe probably that y=1 Loss function: y log(yhat) + (1 - y) log(1 - yhat) """ m = X.shape[1] A = sigmoid(np.dot(w.T, X) + b) cost = np.sum(Y*np.log(A) + (1 - Y)*np.log(1 - A)) return (-1./m)*cost def logistic_regression(_): """the slow way""" J = 0 dw1 =0 dw2=0 db=0 raise NotImplementedError('TODO') def logistic_regression(nx): dw = np.zeros(nx) # TODO # z = np.dot(wT, x) + b # "boradcasting raise NotImplementedError('TODO') # derivatives: # dz1 = a1 - y1 ; dz2 = a2 - y2 ; .... # dZ = [ dz1 dz2 ... dzm ] # Z = w'X + b = np.dot(w', X) + b # A sigmoid(Z) #dZ = A - Y #dw = (1./m)*X*dZ' #db = (1./m)*np.sum(dZ) # w -= alpha*dw # b -= alpha*db