Welcome to mirror list, hosted at ThFree Co, Russian Federation.

github.com/clementfarabet/lua---nnx.git - Unnamed repository; edit this file 'description' to name the repository.
summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorGaetanMarceauCaron <gaetan.marceau-caron@inria.fr>2016-04-13 16:46:54 +0300
committerGaetanMarceauCaron <gaetan.marceau-caron@inria.fr>2016-04-13 16:46:54 +0300
commit9727ba29aa3e04c2e39c264fa93e45fb703f4021 (patch)
tree98930ae2cdb66d142909429ebd07cf1e8251831a
parentdad5a49aafccc7d0ca7f65c40b87b90f7c6e1330 (diff)
reformatting
-rw-r--r--README.md8
1 files changed, 4 insertions, 4 deletions
diff --git a/README.md b/README.md
index 5a7fa4e..8c17f5d 100644
--- a/README.md
+++ b/README.md
@@ -229,12 +229,12 @@ The re-sampled output:
### QDRiemaNNLinear ###
The Quasi-Diagonal Riemannian Neural Network Linear (QDRiemaNNLinear) module is an implementation
of the quasi-diagonal reduction of metrics, used for Riemannian gradient descent.
-The algorithm is defined in http://arxiv.org/abs/1303.0818 and an efficient implementation is described in http://arxiv.org/abs/1602.08007.
+The algorithm is defined in Riemannian metrics for neural networks I: feedforward networks by Yann Ollivier (http://arxiv.org/abs/1303.0818) and an efficient implementation is described in Practical Riemannian Neural Networks by Yann Ollivier and Gaetan Marceau-Caron (http://arxiv.org/abs/1602.08007).
To use this module, simply replace nn.Linear(ninput,noutput) with nnx.QDRiemaNNLinear(ninput,noutput).
As always, the step-size must be chosen accordingly.
-Two other arguments are also possible:
-gamma (default=0.01): determine the update rate of the metric for a minibatch setting, i.e., (1-gamma) * oldMetric + gamma newMetric. Should be set to 1/#minibatch
-qdFlag (default=true): Whether to use the quasi-diagonal reduction (true) or only the diagonal (false). The former should be better.
+Two additional arguments are also possible:
+* gamma (default=0.01): determine the update rate of the metric for a minibatch setting, i.e., (1-gamma) * oldMetric + gamma newMetric. Should be set to 1/#minibatch
+* qdFlag (default=true): Whether to use the quasi-diagonal reduction (true) or only the diagonal (false). The former should be better.
To implement a natural gradient descent, one should also use a module for generating the pseudo-labels.