Vectorizing across multiple examples 

University of Michigan - Ann Arbor

To compute logistic regression and its gradient descent on $$m$$ examples, using a for loop to accumulate errors and derivatives takes significant time on large datasets. Vectorization efficiently eliminates the need for explicit for loops. First, stack the $$m$$ examples horizontally into matrices $$X$$ and $$Y$$, so the shape of $$X$$ is $$(n_x, m)$$, where $$n_x$$ is the number of features, and the shape of $$Y$$ is $$(1, m)$$. Then, compute $$Z$$ and $$A$$:

$$Z = [z^{(1)} \dots z^{(m)}] = w^T X + b = [(w^T x^{(1)} + b) \dots (w^T x^{(m)} + b)]$$

$$A = [a^{(1)} \dots a^{(m)}] = \sigma(Z)$$

The shape of $$Z$$ and $$A$$ is $$(1, m)$$. The derivatives of the loss function $$\mathcal{L}$$ with respect to $$Z$$, $$w$$, and $$b$$ are:

$$\frac{d\mathcal{L}}{dZ} = A - Y$$

$$\frac{d\mathcal{L}}{dw} = \frac{1}{m} X \left(\frac{d\mathcal{L}}{dZ}\right)^T$$

$$\frac{d\mathcal{L}}{db} = \frac{1}{m} \sum \frac{d\mathcal{L}}{dZ}$$

Vectorized Equations for Logistic Regression on m Examples

https://www.coursera.org/learn/neural-networks-deep-learning?specialization=deep-learning

Neural Networks and Deep Learning

```python
for iter in range(10000): # Iterations of Gradient Descent
   Z = np.dot(w.T, X) + b   # Broadcasting
   A = sigma(Z)
   dLdZ = A - Y
   dLdW = (1/m) * X * dLdZ.T
   dLdb = (1/m) * np.sum(dLdZ)
   w = w - a * dLdW
   b = b - a * dLdb
```

Learn Before

Related

Learn After