Skip to content

Pull requests: openai/tiktoken

New pull request New
68 Open 124 Closed
68 Open 124 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Reviews
Filter by reviews
No reviews Review required Approved review Changes requested
Assignee
Filter by who’s assigned
Assigned to nobody Loading
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Pull requests list

Add CI support for Windows ARM64
#580 opened Jul 14, 2026 by chinazhangchao Loading…
Return ValueError instead of panicking on duplicate mergeable_ranks
#579 opened Jul 13, 2026 by devYRPauli Loading…
docs: add community health files and README updates
#576 opened Jul 9, 2026 by Mukller Loading…
fix: apply surrogate-pair fixup in encode_with_unstable
#575 opened Jul 8, 2026 by steps-re Loading…
1
Update PyO3 to 0.29
#574 opened Jul 6, 2026 by musicinmybrain Loading…
fix: reject encoders missing single-byte tokens to prevent reachable panic (#568)
#570 opened Jun 14, 2026 by yen0304 Loading…
4 tasks done
1
fix(model): add exact mapping for gpt-4.5
#569 opened Jun 12, 2026 by yen0304 Loading…
fix(py): make encode_with_unstable handle surrogates like encode (#541)
#567 opened Jun 7, 2026 by MasterOfLogic1 Loading…
1
build: remove outdated arm64 test-skip (fixes #562)
#563 opened Jun 1, 2026 by CryptoSentinelAI Loading…
1
Remove test-skip comment for macOS arm64
#562 opened May 27, 2026 by AvinZarlez Loading…
perf: speed up batch tokenization and decoding hot paths
#561 opened May 27, 2026 by eonr Loading…
Fix CoreBPE::new type inference failure
#558 opened May 24, 2026 by fallintoplace Loading…
1
fix(py): make encode_with_unstable handle surrogates like encode (#541)
#556 opened May 24, 2026 by LeSingh1 Loading…
1
Recognise gpt-5.1 model identifier (Fixes #464)
#554 opened May 23, 2026 by jbbqqf Loading…
1
Repair surrogates in encode_with_unstable (Fixes #541)
#553 opened May 23, 2026 by jbbqqf Loading…
1
Add lexer fast path for o200k_base, cl100k_base, and gpt-2 pretokenization patterns; custom regexes fall back to fancy_regex
#552 opened May 22, 2026 by augustasio Loading…
Add 2-byte pair lookup table to skip hashmap for initial BPE merge scan
#551 opened May 22, 2026 by augustasio Loading…
bench: add tail-latency benchmark for encode_ordinary_batch
#548 opened May 20, 2026 by alobroke Loading…
1
build: enable line tables in release builds so backtraces show file:line (#103)
#546 opened May 18, 2026 by LeSingh1 Loading…
1
fix(py): report element count in buffer.shape, not byte length (#405)
#545 opened May 18, 2026 by LeSingh1 Loading…
1
1
Use user-specific cache directory by default
#544 opened May 15, 2026 by MukundaKatta Contributor Loading…
1
Add HTTP timeout to read_file network fetch
#539 opened May 9, 2026 by Spectual Loading…
1 of 3 tasks
perf: 3.3x faster encode_ordinary_batch for single-threaded workloads
#531 opened Apr 22, 2026 by homanp Loading…
build: add PyInstaller hooks for dynamic encodings
#529 opened Apr 19, 2026 by MukundaKatta Contributor Loading…
1
Honor offline cache for encoding downloads
#525 opened Apr 18, 2026 by MukundaKatta Contributor Loading…
1
ProTip! Adding no:label will show everything without a label.