fix: remove dead null-language scoring config#1275
Merged
anderdc merged 8 commits intoMay 23, 2026
Merged
Conversation
…skipping them Extensions in programming_languages.json with no language field (no tree-sitter parser) were silently scored as skipped-unsupported because the routing gate in calculate_token_score_from_file_changes only checked the NON_CODE_EXTENSIONS constant, not the JSON config. Affected extensions: env, gitattributes, gitignore, gql, graphql, ipynb, move, nim, pde, puml, tex. Fix: - Add TokenConfig.is_line_count_extension() that returns True when an extension is in NON_CODE_EXTENSIONS OR is configured in programming_languages.json with language=None. - Replace the ext in NON_CODE_EXTENSIONS gate with weights.is_line_count_extension(ext). - Remove the now-unused NON_CODE_EXTENSIONS import from tree_sitter_scoring. Tests: - Add TestNullLanguageLineCountScoring with regressions for graphql and gitignore (positive), unknown extension (negative control), graphql weight-accuracy pin, and a config-coverage guard that iterates every null-language entry in programming_languages.json and asserts it reaches line-count — catching future config/code drift before it zeroes miner scores.
MkDev11
force-pushed
the
fix/issue-1274-null-language-line-count-scoring
branch
from
58c64ce to
845a2a7
Compare
anderdc
requested changes
May 22, 2026
Collaborator
anderdc
left a comment
anderdc
left a comment
There was a problem hiding this comment.
The entries this routes to line-count score skipped-unsupported (0) by design — any extension not on the NON_CODE_EXTENSIONS allowlist scores 0; that's the default, not a bug. Line-count rewards raw line volume with no structural check, which is wrong for real languages (nim, move, pde) and undeserved for plumbing files (gitignore, env). The inconsistency #1274 describes is fixed by deleting the dead config, not by honoring it.
Drop the is_line_count_extension helper, the routing and import changes, and TestNullLanguageLineCountScoring. The PR should be two changes:
- Remove these 10 entries from
programming_languages.json:env,gitattributes,gitignore,gql,graphql,ipynb,move,nim,pde,puml. - Add
textoNON_CODE_EXTENSIONSingittensor/constants.py— LaTeX is document markup; keep itstexentry inprogramming_languages.jsonfor the weight.
MkDev11
changed the title
anderdc
approved these changes
May 22, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
programming_languages.jsonentries that intentionally route toskipped-unsupported:env,gitattributes,gitignore,gql,graphql,ipynb,move,nim,pde,puml.textoNON_CODE_EXTENSIONSso LaTeX keeps its configured weight and scores through the existing line-count path.Related Issues
Closes #1274
Type of Change
Testing
Automated checks run:
uv run ruff check gittensor/constants.py gittensor/validator/utils/load_weights.py gittensor/validator/utils/tree_sitter_scoring.py tests/validator/test_token_scoring_integration.py-> PASSuv run pytest tests/validator/test_token_scoring_integration.py tests/validator/test_load_weights.py -q-> PASS, 108 passedgit diff --check-> PASSChecklist
No docs update needed; this keeps the existing unsupported-extension scoring contract and removes dead config.