Welcome to
mirror list
, hosted at
ThFree Co
, Russian Federation.
github.com/moses-smt/mosesdecoder.git - Unnamed repository; edit this file 'description' to name the repository.
index
:
github.com/moses-smt/mosesdecoder.git
BEFORE_FF_FRAMEWORK
CheckTargetNgrams
CompactPT
DPR_MOSES
FF_ttptr
GHK
GHKMStruct
QE
RELEASE-0.91
RELEASE-1.0
RELEASE-2.1
RELEASE-2.1.1
RELEASE-3.0
RELEASE-3.0-cpp14
RELEASE-4.0
abhi_discr
alvations-patch-1
alvations-patch-2
alvations-patch-normalization
alvations-patch-regexes
alvations-python-tokenizer
amittai
andre_lynum
android-ui
apply-coding-standard
async-factors
augmLMResult
auli_constraint
bertoldi_onlinecommand
biased-bitext-sampling
bilingual-lm
bilingual-lm-paul
braune_longdist
cdyer-multifactor
chardmeier-coref
chardmeier-ldc
chardmeier-lminterp
chart_alignments
clang-error
conditional-oxlm
config-switching
dalsa
damt-phrase2master
damt_common
damt_hiero
damt_phrase
dumb_psd
dynamic-models
dynamic-phrase-tables
dynsapt
eva_cube_pruning
eva_maxent
factored_hacks
fethi
fix-wchar-subscripts
fscorer
grace_cube_pruning_gen
hierarchical-reo
hieu_ftemplate_2
hieu_iphone
jjmoses
jointlm_improvements
josiah
jozef_pipeline
kenlm-prefetch
lane-constraint
lane-mtm3
lane-multi
lane-syntax
lane_moses2
lexi-smoothing
lexi_RELEASE-1.0
local_lms
maria_HeadDrivenFeature
maria_MTM14
maria_SelPref
maria_SyntaxFeatures
master
matecat_async
mbotTestedDecoder
mbotdecoder
merge-cmd
mert-mtm5
mert-otherMetrics
mert-other_metrics
mingw-release-2.1.1
minphr
mira-mtm5
miramerge
mjdenkowski
mmt-dev
mmt-merge
moses+batchedlm
moses-batchedlm
moses-ebay
moses-ebay.premerge
moses-server
moses_cachebased
mt3_chart
mt3_constraint
mtm15-multipass
mtm6-parallel-mert
mtm6_cruise_control
mtm_lm
multiple-models
multiple-models@3284
neuralpt
ni_DPR_reordering_model
ni_DPR_reordering_model2
nmt-hybrid
norestart
nplm
oovpt
patch-detokenizer-ko
perf_baseline0
perf_moses2.baseline
perf_moses2.gpulm
prefix-private-cderscorer-member
probing
revert-176-RELEASE-3.0-cpp14
samplerank
span-length
sparse-reordering
suffix_array
syntaxContext
ug-debug
ug-m4m
ug-mct-find
vw_integration
vw_tgtcontext
wipo
wipoNew
wipoNewMerger
yanggao-softdep-hiero
yanggao_maxent
Unnamed repository; edit this file 'description' to name the repository.
www-data
summary
refs
log
tree
commit
diff
log msg
author
committer
range
path:
root
/
scripts
/
tokenizer
Age
Commit message (
Expand
)
Author
2021-03-13
Add tokenisation support for the Tetun language
Raphael Merx
2020-07-31
adding rules for Catalan
Cristina España i Bonet
2020-02-20
Revert "line buffering for tokeniser and truecaser"
Kenneth Heafield
2020-02-17
line buffering for tokeniser and truecaser
William Waites
2019-12-17
Modernized
HjalmarrSv
2019-11-25
Single quotes should be escaped as single quotes.
alvations-patch-normalization
alvations
2019-10-14
Update replace-unicode-punctuation.perl
Kevin Canwen Xu
2018-11-12
Merge branch 'master' of github.com:moses-smt/mosesdecoder
Hieu Hoang
2018-11-12
removing python port. Sacremoses is newer
Hieu Hoang
2018-11-10
Add option "-b" (unbuffer output) to tokenizer scripts
Loïc Vial
2018-11-09
rename directory to work with python import
Hieu Hoang
2018-11-09
python wrapper works
Hieu Hoang
2018-11-07
start borging Luis Gomes code
Hieu Hoang
2018-11-07
tokenizer.perl: split final dots unconditionally
Ozan Caglayan
2018-09-06
Handle glottal stops in Somalian
Barry Haddow
2018-04-10
Contributing MosesTokenizer from NLTK to Moses
alvations-python-tokenizer
alvations
2018-02-14
add fi/sv-specific colon handling in tokenizer.perl
Scherrer Yves
2018-01-19
Korean words has spaces =)
patch-detokenizer-ko
alvations
2016-12-23
Merge pull request #168 from tofula/master
Hieu Hoang
2016-07-31
Separate comma after a number end sentence
Antoine Dusséaux
2016-04-19
add script for acquis cleaning
Hieu Hoang
2016-02-23
.' at end of sentence is missed
Jim Regan
2015-10-12
Named group added for the safer 'protected patterns' recognition regexp.
Tomáš Fulajtár
2015-09-23
ga (mostly) behaves more like fr/it
Jim Regan
2015-09-23
ga (mostly) behaves more like fr/it
Jim Regan
2015-05-29
Add license notices to scripts.
Jeroen Vermeulen
2015-05-17
Fix a lot of lint, mostly trailing whitespace.
Jeroen Vermeulen
2015-05-16
Fix more Python lint.
Jeroen Vermeulen
2015-04-26
Merge branch 'master' of https://github.com/moses-smt/mosesdecoder
alvations
2015-04-19
add pre tokenization cleaning script. In case training has bad, overlying lon...
Hieu Hoang
2015-04-13
add use warnings to all perl scripts
Hieu Hoang
2015-04-07
examples
Flammie Pirinen
2015-04-07
full set of cases and caps
Flammie Pirinen
2015-04-07
add fi to list to silence warnings
Flammie Pirinen
2015-04-07
also lowercase if case fail
Flammie Pirinen
2015-04-07
fix detokenising : in abbrev. case suffix case
Flammie Pirinen
2015-04-02
consistently use 'env perl' command for environments where the 1st perl in PA...
Hieu Hoang
2015-04-02
Conditional import of Thread package for perl installations that don't suppor...
Hieu Hoang
2015-03-20
added Gacha Filter from WMT14
alvations
2015-02-13
Remove debug
Barry Haddow
2015-01-30
"just put it in. I'll verify it if i can be bovvered" --Hieu /usr/bin/env
Kenneth Heafield
2015-01-30
Revert "env perl shebang"
Matthias Huck
2015-01-28
env perl shebang
Kenneth Heafield
2015-01-16
don't normalise quotes if tokenizing like Penn /Phil Williams
Hieu Hoang
2015-01-16
move normalisation of quotes into normalize-punctuation.perl /Tom Hoar
Hieu Hoang
2014-11-21
makemteval and small change to tokenizer. /Tom Hoar and Tomas Fulajtar
Hieu Hoang
2014-10-10
Penn Tree Bank compliant versions of preprocessing
Philipp Koehn
2014-08-04
changes to protecting specified patterns (with example patterns)
Philipp Koehn
2014-08-01
grrrr...
Philipp Koehn
2014-08-01
added flush switch (-b) to normalize-punctuation.perl
Philipp Koehn
[next]