diff options
| author | Taku Kudo <taku910@users.noreply.github.com> | 2018-04-09 13:02:16 +0300 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2018-04-09 13:02:16 +0300 |
| commit | c7caf71a379d2fca859f8d35bc8d7704644b2609 (patch) | |
| tree | 2c04023a698d2b6eca5c620f90cb940e8422f896 /README.md | |
| parent | 557ed16b0561d91d67eaad0455c79798fa44f803 (diff) | |
Update README.md
Diffstat (limited to 'README.md')
| -rw-r--r-- | README.md | 2 |
1 files changed, 1 insertions, 1 deletions
@@ -200,7 +200,7 @@ You can find that the original input sentence is restored from the vocabulary id ``` ```<output file>``` stores a list of vocabulary and emission log probabilities. The vocabulary id corresponds to the line number in this file. -## Refine special meta tokens +## Redefine special meta tokens By default, SentencePiece uses Unknown (<unk>), BOS (<s>) and EOS (</s>) tokens which have the ids of 0, 1, and 2 respectively. We can redefine these mappings in training phase as follows. ``` |