diff options
author | nicholas-leonard <nick@nikopia.org> | 2014-09-05 02:21:33 +0400 |
---|---|---|
committer | nicholas-leonard <nick@nikopia.org> | 2014-09-05 02:21:33 +0400 |
commit | 5187dad92f485207d2194b27cb48b4e832c06516 (patch) | |
tree | d07f00df114413c10adab948c4e293476098a26d /README.md | |
parent | 5f2c6f4b906427ac4db3aaa53ac0a145bcad2d59 (diff) |
library documentation++
Diffstat (limited to 'README.md')
-rw-r--r-- | README.md | 21 |
1 files changed, 14 insertions, 7 deletions
@@ -49,24 +49,31 @@ tests: ## Library Documentation ## This section includes documentation for the following objects: * [SoftMaxTree](#nnx.SoftMaxTree) : a hierarchical log-softmax Module; - * [TreeNLLCriterion](#nnx.TreeNLLCriterion) : a Negative log-likelihodd Criterion for the SoftMaxTree; + * [TreeNLLCriterion](#nnx.TreeNLLCriterion) : a negative log-likelihood Criterion for the SoftMaxTree; <a name='nnx.SoftMaxTree'/> ### SoftMaxTree ### A hierarchy of parameterized log-softmaxes. Used for computing the likelihood of a leaf class. -This Module should be used with the [TreeNLLCriterion](#nnx.TreeNLLCriterion). -Requires a Tensor mapping one `parent_id` to many `child_id`. -Greatly accelerates learning and testing for language models with large vocabularies. +This Module should be used in conjunction with the [TreeNLLCriterion](#nnx.TreeNLLCriterion). +Using this for large vocabularies (100,000 and more) greatly accelerates training and evaluation +of neural network language models (NNLM). A vocabulary hierarchy is provided via the [dp](https://github.com/nicholas-leonard/dp/blob/master/README.md) package's [BillionWords](https://github.com/nicholas-leonard/dp/blob/master/doc/data.md#dp.BillionWords) [DataSource](https://github.com/nicholas-leonard/dp/blob/master/doc/data.md#dp.DataSource). -Computes the log of a product of softmaxes in a path. -Returns an output tensor of size 1D. +The constructor takes 2 mandatory and 4 optional arguments : + * `inputSize` : the number of units in the input embedding representation; + * `hierarchy` : a Tensor mapping one `parent_id` to many `child_id` (a tree); + * `rootId` : a number identifying the root node in the hierarchy. Defaults to `-1`; + * `accUpdate` : when the intent is to use `backwardUpdate` or `accUpdateGradParameters`, set this to true to save memory. Defaults to false; + * `static` : when true (the defualt), returns parameters with keys that don't change from batch to batch; + * `verbose` : prints some additional information concerning the hierarchy during construction. + +Method `forward` returns an `output` Tensor of size 1D. <a name='nnx.TreeNLLCriterion''/> ### TreeNLLCriterion ### -Measures the Negative Log Likelihood (NLL) for [SoftMaxTrees](#nnx.SoftMaxTree). +Measures the Negative log-likelihood (NLL) for [SoftMaxTrees](#nnx.SoftMaxTree). Used for maximizing the likelihood of SoftMaxTree outputs. The SoftMaxTree Module outputs a column Tensor representing the log likelihood of each target in the batch. Thus SoftMaxTree requires the targets. |