Age | Commit message (Collapse) | Author |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Compiles, not tested.
Conflicts:
Jamroot
OnDiskPt/PhraseNode.h
OnDiskPt/TargetPhrase.cpp
OnDiskPt/TargetPhrase.h
OnDiskPt/TargetPhraseCollection.cpp
mert/BleuScorer.cpp
mert/Data.cpp
mert/FeatureData.cpp
moses-chart-cmd/src/Main.cpp
moses/src/AlignmentInfo.h
moses/src/ChartManager.cpp
moses/src/LM/Ken.cpp
moses/src/LM/Ken.h
moses/src/LMList.h
moses/src/LexicalReordering.h
moses/src/PhraseDictionaryTree.h
moses/src/ScoreIndexManager.h
moses/src/StaticData.h
moses/src/TargetPhrase.h
moses/src/Word.cpp
scripts/ems/experiment.meta
scripts/ems/experiment.perl
scripts/training/train-model.perl
|
|
|
|
must double check with barry or lexi
|
|
assert(false). Update Jamfile
|
|
|
|
and fear translations, add variations to 1slack formulation
|
|
|
|
Regression tests all check out, and kbmira seems to work fine
on a Hansard French->English task.
HypPackEnumerator class may be of interest to pro.cpp and future
optimizers, as it abstracts a lot of the boilerplate involved in
enumerating multiple k-best lists.
MiraWeightVector is not really mira-specific - just a weight vector
that enables efficient averaging. Could be useful to a perceptron
as well. Same goes for MiraFeatureVector.
Interaction with sparse features is written, but untested.
|
|
|
|
evaluator
|
|
|
|
- Create a test module for Point.
- Add test cases.
|
|
- Add const.
- Add a virtual destructor to abstract class.
- Add some TODOs to some constant values.
Create files for "overlapping" classes.
|
|
|
|
Add error check when gzopen() fails.
|
|
The test contains only ngram counting. More tests for calculating
BLEU score are reuiqred.
|
|
- Add Vocabulary factory and the unit test.
- Remove Scorer::ClearVocabulary().
|
|
- Introduce the namespace to avoid naming collisions. The class name
is used in KenLM.
- Add the unit test.
|
|
|
|
- Add an unit test for Reference.
- Move functions to calculate the reference length from
BleuScorer to Reference.
|
|
The reason is that we want to add the unit test.
|
|
|
|
|
|
- Add a high resolution timing function to measure the
wall-clock time by gettimeofday().
- Now the Timer class use getrusage() to measure the elapsed
CPU time as KenLM does.
- Revive Timer::restart().
- Add Timer::ToString() for reporting the detail statistics
as well as for debugging.
- Add a simple unit test for Timer.
|
|
example: to interpolate BLEU and CDER use --sctype=BLEU,CDER
to specify weights use --scconfig=weights:0.3+0.7
This scorer should replace MergeScorer (which requires mert-moses-multi.pl) soon.
Interpolated scorer is more universal and is used in the same way as other scorers.
|
|
This change might be useful to avoid duplicating the names.
The reason is that although MERT programs are standalone
applications, some header files such as data.h and
point.h have common guard macro names like "DATA_H" and
"POINT_H", and this is not good naming conventions
when you want to include external headers.
Some files actually include headers in Moses and KenLM's util.
|
|
When tokenizing a string delimited by spaces (say, "9 9 8 7 ")
with Tokenize(), resulting a sequence of strings are
{"9", "9", "8", "7", "" }, which is different
from we have expected. We are not interested in empty strings.
This commit fix this issue, and add unit tests for
the tokenize functions.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|