Age | Commit message (Collapse) | Author |
|
Don't load all references, read them line by line.
Corpora with millions of sentences can now be evaluated without consuming gigabytes of RAM.
|
|
|
|
metrics need getReferenceLength (for background smoothing) to work with kbmira
|
|
|
|
|
|
|
|
|
|
|
|
As in Nakov et al (Coling 2012)
|
|
For storing the word vocabulary used in computation of
BLEU scores. This change will reduce the running time
of extractor about 2-3 seconds (9% reduction).
|
|
Using directives should be put into the implementation files.
|
|
Compiles, not tested.
Conflicts:
Jamroot
OnDiskPt/PhraseNode.h
OnDiskPt/TargetPhrase.cpp
OnDiskPt/TargetPhrase.h
OnDiskPt/TargetPhraseCollection.cpp
mert/BleuScorer.cpp
mert/Data.cpp
mert/FeatureData.cpp
moses-chart-cmd/src/Main.cpp
moses/src/AlignmentInfo.h
moses/src/ChartManager.cpp
moses/src/LM/Ken.cpp
moses/src/LM/Ken.h
moses/src/LMList.h
moses/src/LexicalReordering.h
moses/src/PhraseDictionaryTree.h
moses/src/ScoreIndexManager.h
moses/src/StaticData.h
moses/src/TargetPhrase.h
moses/src/Word.cpp
scripts/ems/experiment.meta
scripts/ems/experiment.perl
scripts/training/train-model.perl
|
|
|
|
must double check with barry or lexi
|
|
|
|
and fear translations, add variations to 1slack formulation
|
|
Regression tests all check out, and kbmira seems to work fine
on a Hansard French->English task.
HypPackEnumerator class may be of interest to pro.cpp and future
optimizers, as it abstracts a lot of the boilerplate involved in
enumerating multiple k-best lists.
MiraWeightVector is not really mira-specific - just a weight vector
that enables efficient averaging. Could be useful to a perceptron
as well. Same goes for MiraFeatureVector.
Interaction with sparse features is written, but untested.
|
|
|
|
|
|
|
|
- Move a definition of sentenceLevelBleuPlusOne() from pro.cpp
to BleuScorer.cpp.
- Add check for the length of an input vector.
|
|
- Make BleuScorer::setReferenceFiles() more testable by
adding OpenReference() and OpenReferenceStream().
|
|
This commit also includes
- Fix typo.
- Fix indentations.
- Add 'const' to Scorer::applyFactors().
|
|
- Add an unit test for Reference.
- Move functions to calculate the reference length from
BleuScorer to Reference.
|
|
The reason is that we want to add the unit test.
|
|
To avoid defining the similar variables twice to calculate
document-wise BLEU and sentence-wise BLEU scores.
|
|
Use std::ostream instead of directly using std::cerr.
|
|
|
|
|
|
|
|
This change might be useful to avoid duplicating the names.
The reason is that although MERT programs are standalone
applications, some header files such as data.h and
point.h have common guard macro names like "DATA_H" and
"POINT_H", and this is not good naming conventions
when you want to include external headers.
Some files actually include headers in Moses and KenLM's util.
|
|
|
|
The reason is that the type is used as internal purpose.
|
|
Declared const_iterator was not *const* actually.
|
|
Declared const_iterator was not *const* actually.
|
|
Should use it in .cpp files.
|
|
|
|
|
|
|
|
Using public const members is not good idea.
It should be initialized in private by constructors.
|
|
|
|
|
|
Functions comments should be placed in their declarations.
|
|
|
|
|
|
M mert/BleuScorer.h
M mert/ScorerFactory.h
M mert/Scorer.h
M mert/PerScorer.h
M mert/TerScorer.h
M mert/Makefile.am
AM scripts/training/mert-moses-multi.pl
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4299 1f5c12ca-751b-0410-a591-d2e778427230
|
|
given candidate and reference files:
evaluator --sctype PER --reference ref.file --candidate cand.file
usage: evaluator [options] --reference ref1[,ref2[,ref3...]] --candidate cand1[,cand2[,cand3...]]
[--sctype|-s] the scorer type (default BLEU)
[--scconfig|-c] configuration string passed to scorer
This is of the form NAME1:VAL1,NAME2:VAL2 etc
[--reference|-R] comma separated list of reference files
[--candidate|-C] comma separated list of candidate files
[--bootstrap|-b] number of booststraped samples (default 0 - no bootstraping)
[--rseed|-r] the random seed for bootstraping (defaults to system clock)
[--help|-h] print this message and exit
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@4153 1f5c12ca-751b-0410-a591-d2e778427230
|
|
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@3899 1f5c12ca-751b-0410-a591-d2e778427230
|
|
average, closest (default) and with case information (default is preserving case). Pay attention that both defaults are different from the previous version (which were shortest reflen and case-insensitive).
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@2459 1f5c12ca-751b-0410-a591-d2e778427230
|
|
git-svn-id: https://mosesdecoder.svn.sourceforge.net/svnroot/mosesdecoder/trunk@1905 1f5c12ca-751b-0410-a591-d2e778427230
|