diff options
author | Matthias Huck <mhuck@inf.ed.ac.uk> | 2015-05-01 00:26:30 +0300 |
---|---|---|
committer | Matthias Huck <mhuck@inf.ed.ac.uk> | 2015-05-01 00:26:30 +0300 |
commit | 4ee8f2dec1c200858c4ed82becc95ed1f7c1017f (patch) | |
tree | 07ecd1b03f6aa7926bfa3fdeaf176833b836ed7b /mert/Reference.h | |
parent | 1d86b8fde7f62dbd0897815479d44df981de5d84 (diff) |
sentence-bleu less greedy regarding memory
Don't load all references, read them line by line.
Corpora with millions of sentences can now be evaluated without consuming gigabytes of RAM.
Diffstat (limited to 'mert/Reference.h')
-rw-r--r-- | mert/Reference.h | 5 |
1 files changed, 5 insertions, 0 deletions
diff --git a/mert/Reference.h b/mert/Reference.h index 2c12f2ed7..a7878f3e7 100644 --- a/mert/Reference.h +++ b/mert/Reference.h @@ -59,6 +59,11 @@ public: int CalcClosest(std::size_t length) const; int CalcShortest() const; + void clear() { + m_length.clear(); + m_counts->clear(); + } + private: NgramCounts* m_counts; |