diff options
author | Andreas Fidjeland <andreas@fidjeland.io> | 2013-12-05 23:47:15 +0400 |
---|---|---|
committer | Andreas Fidjeland <andreas@fidjeland.io> | 2013-12-05 23:47:15 +0400 |
commit | 40372a6e043f2bce905be1966f48983a2924fc2a (patch) | |
tree | 3a00c611763e8c4d17c2a8574cbba9d15180cfac | |
parent | 6bf15e408a423b13abe81f4566bb457331581134 (diff) |
Better formatting for docstrings in REPL
* fixed line-wrap issues
* fixed nested list issues
-rw-r--r-- | adagrad.lua | 14 | ||||
-rw-r--r-- | asgd.lua | 8 | ||||
-rw-r--r-- | cg.lua | 18 | ||||
-rw-r--r-- | fista.lua | 16 | ||||
-rw-r--r-- | lbfgs.lua | 20 | ||||
-rw-r--r-- | lswolfe.lua | 31 | ||||
-rw-r--r-- | rprop.lua | 14 | ||||
-rw-r--r-- | sgd.lua | 23 |
8 files changed, 72 insertions, 72 deletions
diff --git a/adagrad.lua b/adagrad.lua index a65279f..b23715c 100644 --- a/adagrad.lua +++ b/adagrad.lua @@ -1,17 +1,17 @@ --[[ ADAGRAD implementation for SGD ARGS: -- opfunc : a function that takes a single input (X), the point of +- `opfunc` : a function that takes a single input (X), the point of evaluation, and returns f(X) and df/dX -- x : the initial point -- state : a table describing the state of the optimizer; after each +- `x` : the initial point +- `state` : a table describing the state of the optimizer; after each call the state is modified - state.learningRate : learning rate - state.paramVariance : vector of temporal variances of parameters +- `state.learningRate` : learning rate +- `state.paramVariance` : vector of temporal variances of parameters RETURN: -- x : the new x vector -- f(x) : the function, evaluated before the update +- `x` : the new x vector +- `f(x)` : the function, evaluated before the update ]] function optim.adagrad(opfunc, x, config, state) @@ -17,10 +17,10 @@ ARGS: - `x` : the initial point - `state` : a table describing the state of the optimizer; after each call the state is modified - - `state.eta0` : learning rate - - `state.lambda` : decay term - - `state.alpha` : power for eta update - - `state.t0` : point at which to start averaging +- `state.eta0` : learning rate +- `state.lambda` : decay term +- `state.alpha` : power for eta update +- `state.t0` : point at which to start averaging RETURN: - `x` : the new x vector @@ -8,25 +8,25 @@ http://www.gatsby.ucl.ac.uk/~edward/code/minimize/example.html [x fx c] = minimize([0 0]', 'rosenbrock', -25) -Note that we limit the number of function evaluations only, it seems much more -important in practical use. +Note that we limit the number of function evaluations only, it seems much +more important in practical use. ARGS: - `opfunc` : a function that takes a single input, the point of evaluation. - `x` : the initial point - `state` : a table of parameters and temporary allocations. - - `state.maxEval` : max number of function evaluations - - `state.maxIter` : max number of iterations - - `state.df[0,1,2,3]` : if you pass torch.Tensor they will be used for temp storage - - `state.[s,x0]` : if you pass torch.Tensor they will be used for temp storage +- `state.maxEval` : max number of function evaluations +- `state.maxIter` : max number of iterations +- `state.df[0,1,2,3]` : if you pass torch.Tensor they will be used for temp storage +- `state.[s,x0]` : if you pass torch.Tensor they will be used for temp storage RETURN: - `x*` : the new x vector, at the optimal point -- `f` : a table of all function values: - - `f[1]` is the value of the function before any optimization - - `f[#f]` is the final fully optimized value, at x* +- `f` : a table of all function values where + `f[1]` is the value of the function before any optimization and + `f[#f]` is the final fully optimized value, at x* (Koray Kavukcuoglu, 2012) --]] @@ -5,13 +5,13 @@ - `pl` : minimizer of intermediate problem Q(x,y) - `xinit` : initial point - `params` : table of parameters (**optional**) - - `params.L` : 1/(step size) for ISTA/FISTA iteration (0.1) - - `params.Lstep` : step size multiplier at each iteration (1.5) - - `params.maxiter` : max number of iterations (50) - - `params.maxline` : max number of line search iterations per iteration (20) - - `params.errthres`: Error thershold for convergence check (1e-4) - - `params.doFistaUpdate` : true : use FISTA, false: use ISTA (true) - - `params.verbose` : store each iteration solution and print detailed info (false) +- `params.L` : 1/(step size) for ISTA/FISTA iteration (0.1) +- `params.Lstep` : step size multiplier at each iteration (1.5) +- `params.maxiter` : max number of iterations (50) +- `params.maxline` : max number of line search iterations per iteration (20) +- `params.errthres`: Error thershold for convergence check (1e-4) +- `params.doFistaUpdate` : true : use FISTA, false: use ISTA (true) +- `params.verbose` : store each iteration solution and print detailed info (false) On output, `params` will contain these additional fields that can be reused. @@ -36,7 +36,7 @@ Algorithm is published in Title = {A Fast Iterative Shrinkage-Thresholding Algorithm for Linear Inverse Problems}, Volume = {2}, Year = {2009}} ---]] +]] function optim.FistaLS(f, g, pl, xinit, params) local params = params or {} @@ -13,22 +13,22 @@ rate allows a reduction of confidence in the step size. ARGS: -- `opfunc` : a function that takes a single input (X), the point of +- `opfunc` : a function that takes a single input (X), the point of evaluation, and returns f(X) and df/dX -- `x` : the initial point -- `state` : a table describing the state of the optimizer; after each +- `x` : the initial point +- `state` : a table describing the state of the optimizer; after each call the state is modified - - `state.maxIter` : Maximum number of iterations allowed - - `state.maxEval` : Maximum number of function evaluations - - `state.tolFun` : Termination tolerance on the first-order optimality - - `state.tolX` : Termination tol on progress in terms of func/param changes - - `state.lineSearch` : A line search function - - `state.learningRate` : If no line search provided, then a fixed step size is used +- `state.maxIter` : Maximum number of iterations allowed +- `state.maxEval` : Maximum number of function evaluations +- `state.tolFun` : Termination tolerance on the first-order optimality +- `state.tolX` : Termination tol on progress in terms of func/param changes +- `state.lineSearch` : A line search function +- `state.learningRate` : If no line search provided, then a fixed step size is used RETURN: - `x*` : the new `x` vector, at the optimal point - `f` : a table of all function values: - `f[1]` is the value of the function before any optimization + `f[1]` is the value of the function before any optimization and `f[#f]` is the final fully optimized value, at `x*` (Clement Farabet, 2012) diff --git a/lswolfe.lua b/lswolfe.lua index fe9b767..d7beb4f 100644 --- a/lswolfe.lua +++ b/lswolfe.lua @@ -1,25 +1,24 @@ --[[ A Line Search satisfying the Wolfe conditions ARGS: -- `opfunc` : a function (the objective) that takes a single input (X), +- `opfunc` : a function (the objective) that takes a single input (X), the point of evaluation, and returns f(X) and df/dX -- `x` : initial point / starting location -- `t` : initial step size -- `d` : descent direction -- `f` : initial function value -- `g` : gradient at initial location -- `gtd` : directional derivative at starting location -- `options`: - - `c1` : sufficient decrease parameter - - `c2` : curvature parameter - - `tolX` : minimum allowable step length - - `maxIter` : maximum nb of iterations +- `x` : initial point / starting location +- `t` : initial step size +- `d` : descent direction +- `f` : initial function value +- `g` : gradient at initial location +- `gtd` : directional derivative at starting location +- `options.c1` : sufficient decrease parameter +- `options.c2` : curvature parameter +- `options.tolX` : minimum allowable step length +- `options.maxIter` : maximum nb of iterations RETURN: -- `f` : function value at x+t*d -- `g` : gradient value at x+t*d -- `x` : the next x (=x+t*d) -- `t` : the step length +- `f` : function value at x+t*d +- `g` : gradient value at x+t*d +- `x` : the next x (=x+t*d) +- `t` : the step length - `lsFuncEval` : the number of function evaluations ]] function optim.lswolfe(opfunc,x,t,d,f,g,gtd,options) @@ -1,17 +1,17 @@ --[[ A plain implementation of RPROP ARGS: -- `opfunc` : a function that takes a single input (X), the point of +- `opfunc` : a function that takes a single input (X), the point of evaluation, and returns f(X) and df/dX - `x` : the initial point - `state` : a table describing the state of the optimizer; after each call the state is modified - - `state.stepsize` : initial step size, common to all components - - `state.etaplus` : multiplicative increase factor, > 1 (default 1.2) - - `state.etaminus` : multiplicative decrease factor, < 1 (default 0.5) - - `state.stepsizemax` : maximum stepsize allowed (default 50) - - `state.stepsizemin` : minimum stepsize allowed (default 1e-6) - - `state.niter` : number of iterations (default 1) +- `state.stepsize` : initial step size, common to all components +- `state.etaplus` : multiplicative increase factor, > 1 (default 1.2) +- `state.etaminus` : multiplicative decrease factor, < 1 (default 0.5) +- `state.stepsizemax` : maximum stepsize allowed (default 50) +- `state.stepsizemin` : minimum stepsize allowed (default 1e-6) +- `state.niter` : number of iterations (default 1) RETURN: - `x` : the new x vector @@ -1,23 +1,24 @@ --[[ A plain implementation of SGD ARGS: -- `opfunc` : a function that takes a single input (X), the point of - evaluation, and returns f(X) and df/dX + +- `opfunc` : a function that takes a single input (X), the point + of a evaluation, and returns f(X) and df/dX - `x` : the initial point - `config` : a table with configuration parameters for the optimizer - - `config.learningRate` : learning rate - - `config.learningRateDecay` : learning rate decay - - `config.weightDecay` : weight decay - - `config.momentum` : momentum - - `config.dampening` : dampening for momentum - - `config.nesterov` : enables Nesterov momentum +- `config.learningRate` : learning rate +- `config.learningRateDecay` : learning rate decay +- `config.weightDecay` : weight decay +- `config.momentum` : momentum +- `config.dampening` : dampening for momentum +- `config.nesterov` : enables Nesterov momentum - `state` : a table describing the state of the optimizer; after each call the state is modified - - `state.learningRates` : vector of individual learning rates +- `state.learningRates` : vector of individual learning rates RETURN: -- x : the new x vector -- f(x) : the function, evaluated before the update +- `x` : the new x vector +- `f(x)` : the function, evaluated before the update (Clement Farabet, 2012) ]] |