Feature: Add alias-based Solr rebuild and cutover workflow to cron scripts (ALT)#126
Draft
mdorf wants to merge 5 commits into
Draft
Feature: Add alias-based Solr rebuild and cutover workflow to cron scripts (ALT)#126mdorf wants to merge 5 commits into
mdorf wants to merge 5 commits into
Conversation
Replaces the old -c URL flag with -c COLLECTION_NAME. Full re-index (-a) now creates a new collection, indexes into it, then atomically swaps the alias and deletes the old collection. Individual ontology indexing (-o) goes through the live alias unchanged. Removes the Goo.configure/add_search_backend hack, valid_url? check, and search_index_all_url references. The -c flag is now required with -a to enforce explicit collection naming. Usage: ./bin/ncbo_ontology_index -a -c term_search_20260413 ./bin/ncbo_ontology_property_index -a -c prop_search_20260413 ./bin/ncbo_ontology_index -o SNOMEDCT,GO
These settings are no longer used — full re-indexing now uses SolrCloud aliases with explicit collection names passed via the -c flag on the index scripts.
Index scripts now support three workflows: -a -c NAME rebuild without alias swap (existing) -a -c NAME -p rebuild then promote alias on success --promote-only -c promote alias to existing collection, no rebuild New ncbo_search_aliases script inspects active aliases and their backing collections for term_search and prop_search.
2 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Reworks
ncbo_ontology_indexandncbo_ontology_property_indexto use the new SolrCloud alias re-index workflow, adds flexible alias promotion flags, and introduces anncbo_search_aliasesinspection script.Prerequisites
This PR depends on changes introduced in:
Changes
bin/ncbo_ontology_indexandbin/ncbo_ontology_property_indexBoth scripts now support three workflows via new CLI flags:
-a -c NAME-a -c NAME -p--promote-only -c NAME-cis required with-a(full re-index must target an explicit collection name)-o(individual ontologies) continues to index directly into the live alias, unchangedGoo.promote_aliasinstead ofGoo.complete_reindex, preserving the old collection for rollbackbin/ncbo_search_aliases(new)Inspection script that reports the current state of
term_searchandprop_searchaliases and their backing collections.config/config.rb.sampleRemoved obsolete
search_index_all_urlandproperty_search_index_all_urlsettings, replaced with a comment explaining the new-cflag usage.Test plan
ncbo_ontology_index -handncbo_ontology_property_index -hdisplay updated help with examplesncbo_ontology_index -a -c term_search_testcreates collection, indexes, does not swap aliasncbo_ontology_index --promote-only -c term_search_testswaps alias without rebuildingncbo_ontology_index -a -c term_search_test -pcreates, indexes, and swaps alias on successncbo_ontology_index -o SNOMEDCTindexes into live alias (existing behavior)ncbo_search_aliasesreports correct alias → collection mappingsncbo_ontology_property_index