Welcome to mirror list, hosted at ThFree Co, Russian Federation.

github.com/marian-nmt/sentencepiece.git - Unnamed repository; edit this file 'description' to name the repository.
summaryrefslogtreecommitdiff
path: root/src/sentencepiece_processor.cc
AgeCommit message (Collapse)Author
2018-09-01Introduced new Filesystem API.Taku Kudo
2018-08-14Added make_py_wheel_mac.pyTaku Kudo
2018-08-03Added JoinPath and StrCatTaku Kudo
2018-08-01Minor fixes for windowsTaku Kudo
2018-07-26Added --unk_surface option to allow user to change unknown surface string.Taku Kudo
2018-07-12Added new API to get bos/eos/unk/pad idsTaku Kudo
2018-06-22Add LoadFromSerialiedProtoTaku Kudo
2018-06-18Introduced minimum string_wrapper to remove extra string copyTaku Kudo
2018-06-18Uses abs::string_view instead of StringPieceTaku Kudo
2018-06-17Minor fixesTaku Kudo
2018-06-16Minor fixes.Taku Kudo
2018-06-11Support an empty normalziation and other minor fixesTaku Kudo
2018-06-06Support vocab restriction feature in BPE model.Taku Kudo
2018-06-06Support vocab restriction featureTaku Kudo
2018-06-04Minor style fixesTaku Kudo
2018-05-06CHECK to Status migration for Trainer.Taku Kudo
2018-04-30Reimplement Trainer with Proto reflectionTaku Kudo
2018-04-28Uses util::Status to propagate error messagesTaku Kudo
2018-04-09Support to change ids of <unk>, <s>, </s>Taku Kudo
2018-02-28Add Sample/NBestEncodeTaku Kudo
2017-07-23use pointer instead of not-const referenceresec
2017-07-20Impl std::istream variant to Load and LoadOrDieresec
Implemented for SentencePieceProcessor: bool SentencePieceProcessor::Load(std::istream &istream) void SentencePieceProcessor::LoadOrDie(std::istream &istream)
2017-03-07Initialize repositoryTaku Kudo
    generated by cgit v1.2.3 (git 2.25.1) at 2024-07-21 12:18:24 +0300