| Age | Commit message (Collapse) | Author |
|---|
|
The code uses two mechanisms for generating random numbers: srand()/rand(),
which is not thread-safe, and srandom()/random(), which is POSIX-specific.
Here I add a util/random.cc module that centralizes these calls, and unifies
some common usage patterns. If the implementation is not good enough, we can
now change it in a single place.
To keep things simple, this uses the portable srand()/rand() but protects them
with a lock to avoid concurrency problems.
The hard part was to keep the regression tests passing: they rely on fixed
sequences of random numbers, so a small code change could break them very
thoroughly. Util::rand(), for wide types like size_t, calls std::rand() not
once but twice. This behaviour was generalized into utils::wide_rand() and
friends.
|
|
Some places in mert use srandom()/random(), but these are POSIX-specific.
The standard alternative, srand()/rand(), is not thread-safe. This module
wraps srand()/rand() in mutexes (very short-lived, so should not cost much)
so that it relies on just Boost and the C standard library, not on a Unix-like
environment.
This may reduce the width of the random numbers on some platforms: it goes
from "long int" to just "int". If that is a problem, we may have to use
Boost's randomizer utilities, or eventually, the C++ ones.
|
|
2. Code cleanup in IOWrapper and a bit elsewhere.
|
|
|
|
|
|
Compiles, not tested.
Conflicts:
Jamroot
OnDiskPt/PhraseNode.h
OnDiskPt/TargetPhrase.cpp
OnDiskPt/TargetPhrase.h
OnDiskPt/TargetPhraseCollection.cpp
mert/BleuScorer.cpp
mert/Data.cpp
mert/FeatureData.cpp
moses-chart-cmd/src/Main.cpp
moses/src/AlignmentInfo.h
moses/src/ChartManager.cpp
moses/src/LM/Ken.cpp
moses/src/LM/Ken.h
moses/src/LMList.h
moses/src/LexicalReordering.h
moses/src/PhraseDictionaryTree.h
moses/src/ScoreIndexManager.h
moses/src/StaticData.h
moses/src/TargetPhrase.h
moses/src/Word.cpp
scripts/ems/experiment.meta
scripts/ems/experiment.perl
scripts/training/train-model.perl
|
|
|
|
Conflicts:
Jamroot
mert/Data.cpp
mert/Data.h
mert/FeatureArray.cpp
mert/FeatureArray.h
mert/FeatureData.cpp
mert/FeatureData.h
mert/FeatureStats.cpp
mert/FeatureStats.h
mert/mert.cpp
moses-chart-cmd/src/IOWrapper.h
moses-chart-cmd/src/Main.cpp
moses-cmd/src/IOWrapper.cpp
moses-cmd/src/IOWrapper.h
moses-cmd/src/Main.cpp
moses/src/GlobalLexicalModel.cpp
moses/src/Jamfile
moses/src/Parameter.cpp
moses/src/PhraseDictionary.cpp
moses/src/ScoreIndexManager.h
moses/src/TargetPhrase.h
regression-testing/tests/phrase.lexicalized-reordering-bin/truth/results.txt
regression-testing/tests/phrase.lexicalized-reordering-cn/truth/results.txt
regression-testing/tests/phrase.lexicalized-reordering/truth/results.txt
regression-testing/tests/phrase.multiple-translation-system-lr/truth/results.txt
regression-testing/tests/phrase.show-weights.lex-reorder/truth/results.txt
regression-testing/tests/phrase.show-weights/truth/results.txt
scripts/ems/experiment.meta
scripts/ems/experiment.perl
scripts/training/filter-model-given-input.pl
scripts/training/mert-moses.pl
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Squashed commit of the following:
- Clean up PRO.
- Clean up ScoreStats.
- Clean up ScoreData.
- Clean up ScoreArray.
- Remove unnecessary headers.
- Clean up ScopedVector.
- Clean up Point.
- Clean up PerScorer.
- Clean up Optimizer.
- Clean up MergeScorer.
- Clean up InterpolatedScorer.
- Clean up FileStream.
- Clean up FeatureStats.
- Remove inefficient string concatenation.
- Clean up FeatureData.
- Clean up FeatureArray.
- Clean up Data.
|
|
Squashed commit of the following:
- Clean up PRO.
- Clean up ScoreStats.
- Clean up ScoreData.
- Clean up ScoreArray.
- Remove unnecessary headers.
- Clean up ScopedVector.
- Clean up Point.
- Clean up PerScorer.
- Clean up Optimizer.
- Clean up MergeScorer.
- Clean up InterpolatedScorer.
- Clean up FileStream.
- Clean up FeatureStats.
- Remove inefficient string concatenation.
- Clean up FeatureData.
- Clean up FeatureArray.
- Clean up Data.
|
|
|
|
|
|
|
|
|
|
In the commit 4b6232b757542e7dfd05b6c494d5f97fdb80c448,
I thought I had fixed the bug around the tuning on a subset of
features by checking whether pdim and the length of the
active features which you want to optimize in the tuning.
However, it was wrong. I should set Point::optindices
appropriately according to specified the subset.
|
|
In the commit 4b6232b757542e7dfd05b6c494d5f97fdb80c448,
I thought I had fixed the bug around the tuning on a subset of
features by checking whether pdim and the length of the
active features which you want to optimize in the tuning.
However, it was wrong. I should set Point::optindices
appropriately according to specified the subset.
|
|
mert will check whether the dimension and the number of
fetures are equal.
|
|
mert will check whether the dimension and the number of
fetures are equal.
|
|
This commit is a temporary bug fix.
|
|
This commit is a temporary bug fix.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
'D' and 'O' is hard to find when you might want to
search by command line tools such as grep.
|
|
'D' and 'O' is hard to find when you might want to
search by command line tools such as grep.
|
|
Default filenames in usage() were differen from the actual ones.
|
|
Default filenames in usage() were differen from the actual ones.
|
|
|
|
|
|
Adding 'virtual' keyword for an inherited virtual function is good
practice because it will let the users know where the function
comes from.
|
|
Adding 'virtual' keyword for an inherited virtual function is good
practice because it will let the users know where the function
comes from.
|
|
|
|
|
|
|
|
|
