找到 71 个包，花费 4 毫秒

gpt-tokenizer789.2kMIT3.0.1

A pure JavaScript implementation of a BPE tokenizer (Encoder/Decoder) for GPT-2 / GPT-3 / GPT-4 and other OpenAI models

niieani3 months 前BPE, encoder, decoder, tokenizer

npmGitHub

gpt-3-encoder379.6kMIT1.1.4

Javascript BPE Encoder Decoder for GPT-2 / GPT-3

AIDungeonover 2 years 前

npmGitHub

tiktoken-node17.9kMIT0.0.7

tiktoken is a fast [BPE](https://en.wikipedia.org/wiki/Byte_pair_encoding) tokeniser for use with OpenAI's models.

ceifaover 1 year 前

npmGitHub

@nick.heiner/gpt-3-encoder4.6kMIT1.2.0

Javascript BPE Encoder Decoder for GPT-2 / GPT-3

NickHeinerover 2 years 前

npmGitHub

gptoken3.9kMIT0.1.0

Javascript BPE Encoder Decoder for GPT-2 / GPT-3. The "gpt-3-encoder" module provides functions for encoding and decoding text using the Byte Pair Encoding (BPE) algorithm. It can be used to process text data for input into machine learning models, or to