diff options
-rw-r--r-- | README.md | 3 |
1 files changed, 1 insertions, 2 deletions
@@ -236,8 +236,7 @@ Two additional arguments are also possible: * gamma (default=0.01): determine the update rate of the metric for a minibatch setting, i.e., (1-gamma) * oldMetric + gamma newMetric. Smaller minibatches require a smaller gamma. A default value depending on the size of the minibatches is `gamma = 1. - torch.pow(1.-1./nTraining,miniBatchSize)` where `nTraining` is the number of training examples of the dataset and `miniBatchSize` is the number of training examples per minibatch. * qdFlag (default=true): Whether to use the quasi-diagonal reduction (true) or only the diagonal (false). The former should be better. -To implement a natural gradient descent, one should also use a module for generating the pseudo-labels. - +Replacing Linear by QDRiemmaNNLinear is a straightforward implementation of the outer product gradient descent. ## Requirements |