Dataset (n=1024)
Variation of Batch-Sizes
| Learnrate | Batch-Size | |
|---|---|---|
| Batch-GD | 0.005 | 1024 |
| Mini-Batch-GD | 0.005 | 16 |
| Stochastic-GD | 0.005 | 1 |
Optimizer Visualization
| Learnrate | Momentum | Scaling Decay | Batch-Size | |
|---|---|---|---|---|
| Momentum | 0.005 | 0.9 | - | 16 |
| AdaGrad | 0.825 | - | - | 16 |
| RMSProp | 0.125 | - | 0.9 | 16 |
| Adam | 0.4 | 0.9 | 0.99 | 16 |
Tools: Python 3.6(Tensorflow, Matplotlib, Numpy)