Welcome to mirror list, hosted at ThFree Co, Russian Federation.

github.com/clementfarabet/lua---nnx.git - Unnamed repository; edit this file 'description' to name the repository.
summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorGaetanMarceauCaron <gaetan.marceau-caron@inria.fr>2016-04-14 11:52:33 +0300
committerGaetanMarceauCaron <gaetan.marceau-caron@inria.fr>2016-04-14 11:52:33 +0300
commitdcab6930c5c0869248ecf3482a1bc64555f0c253 (patch)
tree3a62bbdfdd37c30ace36b95334b57d8d6adecb9a
parentf8c6839417cc50c866fc13c951105661941faa55 (diff)
Changing the default value for gamma
-rw-r--r--README.md2
1 files changed, 1 insertions, 1 deletions
diff --git a/README.md b/README.md
index 96947fe..dffeb17 100644
--- a/README.md
+++ b/README.md
@@ -233,7 +233,7 @@ The algorithm is defined in Riemannian metrics for neural networks I: feedforwar
To use this module, simply replace `nn.Linear(ninput,noutput)` with `nnx.QDRiemaNNLinear(ninput,noutput)`.
As always, the step-size must be chosen accordingly.
Two additional arguments are also possible:
-* gamma (default=0.01): determine the update rate of the metric for a minibatch setting, i.e., (1-gamma) * oldMetric + gamma newMetric. Should be set to 1/#minibatch
+* gamma (default=0.01): determine the update rate of the metric for a minibatch setting, i.e., (1-gamma) * oldMetric + gamma newMetric. Smaller minibatches require a smaller gamma.
* qdFlag (default=true): Whether to use the quasi-diagonal reduction (true) or only the diagonal (false). The former should be better.
To implement a natural gradient descent, one should also use a module for generating the pseudo-labels.