Module: text/WordpieceTokenizer
Classes
Type aliases
WordPieceTokenizerConfig
Ƭ WordPieceTokenizerConfig: Object
Type declaration
| Name | Type |
|---|
lowercase? | boolean |
neverSplit? | string[] |
unknownToken? | string |
vocab | string |
Defined in
text/WordpieceTokenizer.ts:12