# Deep Learning - Week 4 Lecture Notes

# What is Deep Neural Network?

as you can see, from the past 3 courses we have explored logistic regression as a **shallow network**.

## Notations

Notation | Description |
---|---|

$l$ | number of layers |

$n$ | number of hidden units |

$n^{[l}$ | number of hidden units at layer $l$ |

$a^{[l]}$ | activations in layer $l$ |

$a^{[l]} = g^{[l]}(z^{[l]})$ | activations in layer $l$ |

$W^{[l]}$ | weights in layer $l$ for $z^{[l]}$ |

## Forward Propagation in Deep Network

basically same from before, lol

# Getting your Matrix Dimension Right

to sum up:

Variable | Shape / Dimension |
---|---|

$w^{[l]}$ | $(n^{[l]}, n^{[l-1]})$ |

$b^{[l]}$ | $(n^{[l]}, 1)$ |

$dw^{[l]}$ | $(n^{[l]}, n^{[l-1]})$ |

$db^{[l]}$ | $(n^{[l]}, 1)$ |

Additional notes:

dimension of $z^{[l]}$ should be same as $a^{[l]}$

# Why Deep Neural Networks Works?

Circuit theory

There are functions you can compute with a “small” L-layer deep neural network that shallower networks require exponentially more hidden units to compute

means: if we try to compute a function $ \hat y $ with depth $ n $, and try to compute the same function with shallower network say with depth $ n - x$, we might ended up need to add **exponentially** more hidden units (not layers), for example we might have to add $2^{x}$ more units to the current.

**Deep learning** is just branding!

Start with logistic regression, then 1 hidden layer, then 2 hidden layer first!