Welcome to mirror list, hosted at ThFree Co, Russian Federation.

github.com/torch/torch.github.io.git - Unnamed repository; edit this file 'description' to name the repository.
summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authornicholas-leonard <nick@nikopia.org>2016-07-26 17:46:10 +0300
committernicholas-leonard <nick@nikopia.org>2016-07-26 17:46:10 +0300
commit4e7626f8c4f0274b6a1524d1017829b6f1c277fb (patch)
tree756821971f803d784e7c7be92ed22053284cea3f
parenta95d34ff2a69052babe2ebf2e8cab32197e69f45 (diff)
add Element-Research mention and updated RMVA
-rw-r--r--blog/_posts/2015-09-21-rmva.md10
-rw-r--r--blog/_posts/2016-07-25-nce.md6
-rw-r--r--blog/_posts/images/rnnlm-small.pngbin0 -> 42197 bytes
3 files changed, 11 insertions, 5 deletions
diff --git a/blog/_posts/2015-09-21-rmva.md b/blog/_posts/2015-09-21-rmva.md
index 68fc0c6..edcccc4 100644
--- a/blog/_posts/2015-09-21-rmva.md
+++ b/blog/_posts/2015-09-21-rmva.md
@@ -366,9 +366,13 @@ Here are some results for the Translated MNIST dataset :
For this dataset, the images are of size `1x60x60` where each image contains a randomly placed `1x28x28` MNIST digit.
The `3x12x12` glimpse uses a depth of 3 scales
where each successive patch is twice the height and width of the previous one.
-Training with this dataset was started about 3 days prior to this blog post.
-For 7 glimpses, after 193 epochs, we get 1.223% error. Note that the model is still training.
-The paper gets 1.22% and 1.2% error for 6 and 8 glimpses, respectively.
+After 683 epochs of training on the Translatted MNIST dataset, using 7 glimpses, we obtain 0.92% error.
+The paper reaches 1.22% and 1.2% error for 6 and 8 glimpses, respectively.
+The exact command used to obtain those results:
+
+```lua
+th examples/recurrent-visual-attention.lua --cuda --dataset TranslatedMnist --unitPixels 26 --learningRate 0.001 --glimpseDepth 3 --maxTries 200 --stochastic --glimpsePatchSize 12
+```
Note : you can evaluate your models with the [evaluation script](https://github.com/Element-Research/rnn/blob/master/scripts/evaluate-rva.lua).
It will generate a sample of glimpse sequences and print the confusion matrix results for the test set.
diff --git a/blog/_posts/2016-07-25-nce.md b/blog/_posts/2016-07-25-nce.md
index 89862a5..58980e5 100644
--- a/blog/_posts/2016-07-25-nce.md
+++ b/blog/_posts/2016-07-25-nce.md
@@ -4,7 +4,7 @@ title: Language modeling a billion words
comments: True
author: nicholas-leonard
excerpt: Noise contrastive estimation is used to train a multi-GPU recurrent neural network language model on the Google billion words dataset.
-picture: https://raw.githubusercontent.com/torch/torch.github.io/master/blog/_posts/images/rnnlm.png
+picture: https://raw.githubusercontent.com/torch/torch.github.io/master/blog/_posts/images/rnnlm-small.png
---
<!---# Language modeling a billion words -->
@@ -18,10 +18,12 @@ picture: https://raw.githubusercontent.com/torch/torch.github.io/master/blog/_po
* [Future work](#nce.future)
* [References](#nce.ref)
+In our last post, we presented a [recurrent model for visual attention](http://torch.ch/blog/2015/09/21/rmva.html)
+which combined reinforcement learning with recurrent neural networks.
In this Torch blog post, we use noise contrastive estimation (NCE) [[2]](#nce.ref)
to train a multi-GPU recurrent neural network language model (RNNLM)
on the Google billion words (GBW) dataset [[7]](#nce.ref).
-The work presented here is the result of many months of on-and-off work.
+The work presented here is the result of many months of on-and-off work at [Element-Research](https://www.discoverelement.com/research).
The enormity of the dataset caused us to contribute some novel open-source Torch modules, criteria and even a multi-GPU tensor.
We also provide scripts so that you can train and evaluate your own language models.
diff --git a/blog/_posts/images/rnnlm-small.png b/blog/_posts/images/rnnlm-small.png
new file mode 100644
index 0000000..0cbff63
--- /dev/null
+++ b/blog/_posts/images/rnnlm-small.png
Binary files differ