Welcome to
mirror list
, hosted at
ThFree Co
, Russian Federation.
github.com/moses-smt/mosesdecoder.git - Unnamed repository; edit this file 'description' to name the repository.
index
:
github.com/moses-smt/mosesdecoder.git
BEFORE_FF_FRAMEWORK
CheckTargetNgrams
CompactPT
DPR_MOSES
FF_ttptr
GHK
GHKMStruct
QE
RELEASE-0.91
RELEASE-1.0
RELEASE-2.1
RELEASE-2.1.1
RELEASE-3.0
RELEASE-3.0-cpp14
RELEASE-4.0
abhi_discr
alvations-patch-1
alvations-patch-2
alvations-patch-normalization
alvations-patch-regexes
alvations-python-tokenizer
amittai
andre_lynum
android-ui
apply-coding-standard
async-factors
augmLMResult
auli_constraint
bertoldi_onlinecommand
biased-bitext-sampling
bilingual-lm
bilingual-lm-paul
braune_longdist
cdyer-multifactor
chardmeier-coref
chardmeier-ldc
chardmeier-lminterp
chart_alignments
clang-error
conditional-oxlm
config-switching
dalsa
damt-phrase2master
damt_common
damt_hiero
damt_phrase
dumb_psd
dynamic-models
dynamic-phrase-tables
dynsapt
eva_cube_pruning
eva_maxent
factored_hacks
fethi
fix-wchar-subscripts
fscorer
grace_cube_pruning_gen
hierarchical-reo
hieu_ftemplate_2
hieu_iphone
jjmoses
jointlm_improvements
josiah
jozef_pipeline
kenlm-prefetch
lane-constraint
lane-mtm3
lane-multi
lane-syntax
lane_moses2
lexi-smoothing
lexi_RELEASE-1.0
local_lms
maria_HeadDrivenFeature
maria_MTM14
maria_SelPref
maria_SyntaxFeatures
master
matecat_async
mbotTestedDecoder
mbotdecoder
merge-cmd
mert-mtm5
mert-otherMetrics
mert-other_metrics
mingw-release-2.1.1
minphr
mira-mtm5
miramerge
mjdenkowski
mmt-dev
mmt-merge
moses+batchedlm
moses-batchedlm
moses-ebay
moses-ebay.premerge
moses-server
moses_cachebased
mt3_chart
mt3_constraint
mtm15-multipass
mtm6-parallel-mert
mtm6_cruise_control
mtm_lm
multiple-models
multiple-models@3284
neuralpt
ni_DPR_reordering_model
ni_DPR_reordering_model2
nmt-hybrid
norestart
nplm
oovpt
patch-detokenizer-ko
perf_baseline0
perf_moses2.baseline
perf_moses2.gpulm
prefix-private-cderscorer-member
probing
revert-176-RELEASE-3.0-cpp14
samplerank
span-length
sparse-reordering
suffix_array
syntaxContext
ug-debug
ug-m4m
ug-mct-find
vw_integration
vw_tgtcontext
wipo
wipoNew
wipoNewMerger
yanggao-softdep-hiero
yanggao_maxent
Unnamed repository; edit this file 'description' to name the repository.
www-data
summary
refs
log
tree
commit
diff
log msg
author
committer
range
path:
root
/
scripts
/
tokenizer
/
tokenizer.perl
Age
Commit message (
Expand
)
Author
2016-12-23
Merge pull request #168 from tofula/master
Hieu Hoang
2016-07-31
Separate comma after a number end sentence
Antoine Dusséaux
2016-02-23
.' at end of sentence is missed
Jim Regan
2015-10-12
Named group added for the safer 'protected patterns' recognition regexp.
Tomáš Fulajtár
2015-09-23
ga (mostly) behaves more like fr/it
Jim Regan
2015-05-29
Add license notices to scripts.
Jeroen Vermeulen
2015-05-17
Fix a lot of lint, mostly trailing whitespace.
Jeroen Vermeulen
2015-04-13
add use warnings to all perl scripts
Hieu Hoang
2015-04-02
consistently use 'env perl' command for environments where the 1st perl in PA...
Hieu Hoang
2015-04-02
Conditional import of Thread package for perl installations that don't suppor...
Hieu Hoang
2015-01-30
"just put it in. I'll verify it if i can be bovvered" --Hieu /usr/bin/env
Kenneth Heafield
2015-01-30
Revert "env perl shebang"
Matthias Huck
2015-01-28
env perl shebang
Kenneth Heafield
2015-01-16
move normalisation of quotes into normalize-punctuation.perl /Tom Hoar
Hieu Hoang
2014-08-04
changes to protecting specified patterns (with example patterns)
Philipp Koehn
2014-02-21
- added option -no-escape to skip escaping of special characters
Ulrich Germann
2013-10-17
correct usage
Barry Haddow
2013-10-07
Add option to protect expressions - eg URLs
Barry Haddow
2013-08-16
Fixed bug in tokenizer.perl where comma separated lists of single
Jeremy Gwinnup
2013-07-24
Add option to do Penn Treebank style tokenization
Phil Williams
2013-02-27
fixed typo
amittai
2012-06-30
add multi-threading feature to the tokenizer.perl
Pidong Wang
2012-06-26
lock m_vocab variable access in Encode() and Lookup(). Other functions are st...
Hieu Hoang
2012-06-26
Change Bin to RealBin. Thanks to Tom Hoar
Hieu Hoang
2012-06-26
escape bar character with proper html escape sequence
phikoehn
2012-05-30
more escaping in tokenizer; wrapper for berkeley parser (german)
phikoehn
2012-05-26
minor fixes
phikoehn
2012-03-23
bug fix to filter hierarchical
phikoehn
2012-03-20
lotsa minor changes: mostly bug fixes, tokenizer now esacapes special Moses c...
phikoehn
2012-01-18
Option to disable buffering (from Tom Hoar)
Barry Haddow
2010-05-05
official release of experiment.perl
phkoehn