From 9727ba29aa3e04c2e39c264fa93e45fb703f4021 Mon Sep 17 00:00:00 2001 From: GaetanMarceauCaron Date: Wed, 13 Apr 2016 15:46:54 +0200 Subject: reformatting --- README.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/README.md b/README.md index 5a7fa4e..8c17f5d 100644 --- a/README.md +++ b/README.md @@ -229,12 +229,12 @@ The re-sampled output: ### QDRiemaNNLinear ### The Quasi-Diagonal Riemannian Neural Network Linear (QDRiemaNNLinear) module is an implementation of the quasi-diagonal reduction of metrics, used for Riemannian gradient descent. -The algorithm is defined in http://arxiv.org/abs/1303.0818 and an efficient implementation is described in http://arxiv.org/abs/1602.08007. +The algorithm is defined in Riemannian metrics for neural networks I: feedforward networks by Yann Ollivier (http://arxiv.org/abs/1303.0818) and an efficient implementation is described in Practical Riemannian Neural Networks by Yann Ollivier and Gaetan Marceau-Caron (http://arxiv.org/abs/1602.08007). To use this module, simply replace nn.Linear(ninput,noutput) with nnx.QDRiemaNNLinear(ninput,noutput). As always, the step-size must be chosen accordingly. -Two other arguments are also possible: -gamma (default=0.01): determine the update rate of the metric for a minibatch setting, i.e., (1-gamma) * oldMetric + gamma newMetric. Should be set to 1/#minibatch -qdFlag (default=true): Whether to use the quasi-diagonal reduction (true) or only the diagonal (false). The former should be better. +Two additional arguments are also possible: +* gamma (default=0.01): determine the update rate of the metric for a minibatch setting, i.e., (1-gamma) * oldMetric + gamma newMetric. Should be set to 1/#minibatch +* qdFlag (default=true): Whether to use the quasi-diagonal reduction (true) or only the diagonal (false). The former should be better. To implement a natural gradient descent, one should also use a module for generating the pseudo-labels. -- cgit v1.2.3