fix: use user-private cache directory to prevent cache poisoning on shared systems

[yen0304]

Summary

Fixes #500. Addresses two security vulnerabilities in tiktoken/load.py:

1. Cache poisoning via shared temp directory (CWE-377/CWE-379): The default cache directory was /tmp/data-gym-cache/, which is world-writable on multi-user Linux systems. An attacker could pre-populate poisoned BPE data with predictable filenames (sha1(url)), causing subsequent users to load attacker-controlled tokenizer data.

2. Silent cache loading without hash verification (CWE-345): When expected_hash is None (e.g., custom encodings), cached files were loaded without any integrity check or warning.

Changes

• Default cache directory changed from tempfile.gettempdir() + "/data-gym-cache" to a user-private location:
  • Linux/macOS: $XDG_CACHE_HOME/tiktoken or ~/.cache/tiktoken
  • Windows: %LOCALAPPDATA%\tiktoken\cache
• Cache directory permissions set to 0o700 (owner-only) on creation
• Warning logged when loading cached files without hash verification
• Full backward compatibility: TIKTOKEN_CACHE_DIR and DATA_GYM_CACHE_DIR env vars continue to work as before

Why not migrate the old cache?

Files in /tmp/data-gym-cache/ could already be poisoned, so we intentionally do not copy them. Users will re-download encodings once into the secure location.

Test plan

• New tests/test_cache_security.py with 8 tests covering:
  • XDG_CACHE_HOME respected
  • Default dir not in /tmp
  • Cache dir created with restricted permissions (no group/other access)
  • Hash verification: valid hash, invalid hash, tampered cache re-fetch
  • Warning logged when loading without hash
• All existing tests pass (test_misc.py, test_simple_public.py, test_helpers.py)
• Manual verification: tiktoken.get_encoding('cl100k_base') works correctly with new cache location

[yen0304]

Closing as duplicate of #544 which addresses the same issue. Apologies for the duplicate — my search missed the existing PR due to different keywords.