diff options
| author | Taku Kudo <taku910@users.noreply.github.com> | 2018-04-09 15:04:38 +0300 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2018-04-09 15:04:38 +0300 |
| commit | 21df2635237d10bf5435a320f7d3f390c9e647ba (patch) | |
| tree | 433e6f1ff0bd34bd31f8074ad02abc7fd20cec1a /README.md | |
| parent | 90c409fecc2aeae72cf14a791902d2ca6869d757 (diff) | |
Update README.md
Diffstat (limited to 'README.md')
| -rw-r--r-- | README.md | 2 |
1 files changed, 1 insertions, 1 deletions
@@ -206,7 +206,7 @@ You can find that the original input sentence is restored from the vocabulary id ``` % spm_train --bos_id=0 --eos_id=1 --unk_id=2 --input=... --model_prefix=... ``` -When setting -1 id e.g., ```bos_id=-1```, this special token is ignored. Note that the unknow id cannot be removed and these ids must start with 0 and be continous. In addition, we can define an id for padding (<pad>). Padding id is disabled by default. You can assign an id as ```--pad_id=3```. +When setting -1 id e.g., ```bos_id=-1```, this special token is ignored. Note that the unknow id cannot be removed. In addition, these ids must start with 0 and be continous. In addition, we can define an id for padding (<pad>). Padding id is disabled by default. You can assign an id as ```--pad_id=3```. If you want to assign another special tokens, please see [Use custom symbols](doc/special_symbols.md). |