fix: align UDS parsing with the official Cisco v14 XML schemas - #40
Open
bandrel wants to merge 1 commit into
Open
fix: align UDS parsing with the official Cisco v14 XML schemas#40bandrel wants to merge 1 commit into
bandrel wants to merge 1 commit into
Conversation
Audited every /cucm-uds endpoint the tool touches against the official UDS v14 XSD bundle (developer.cisco.com, DevNet login required). No endpoint is version-gated and UDS has no version path segment, but four parsing defects turned up, two of them silently losing data. Closes #33: /users paging sent start=collected+1 while UDS indexes from 0, skipping one user at every page boundary. Paging is now driven by the <users> wrapper's own start/returnedCount attributes (both use="required" in users.get.xsd), so the next offset is start+returnedCount by construction and no index-base assumption is baked in. A server that clamps the page below the requested size (UserSearchLimit) is detected and logged rather than mistaken for the end of the directory. Servers that omit the attributes fall back to a 0-based collected offset. Closes #35: get_version parsed a <prefix> element that exists in no UDS release; only the _TEST_MODE stub ever populated it, so the operator output was permanently dead. Replaced with the wrapper's version= attribute, which is the UDS schema version and distinct from the CUCM version in <version>. Closes #36: get_servers_api looked for hostName/ipv4Address/ipv6Address/ serverType child elements and treated a plain-text <server> as a "15.x+" variant. servers.get.xsd declares <server> as type="xs:string" and permits no children, so the polarity was backwards and those four names are AXL vocabulary that UDS never returns. Parse is now text-only. UDS exposes neither addresses nor Publisher/Subscriber role, so the blank IPv4 column is gone from --show-db and the README no longer promises IPs. The ipv4/ipv6/server_type columns stay for compatibility with existing thief.db files and are documented as unpopulatable from UDS. Closes #37: parse and surface <upgradeInProgress>, a v14 capability flag absent from Cisco's prose docs. A cluster mid-upgrade can return partial directory data, which previously read as unexplained enumeration variance. Closes #39: parse_uds_directory used bare-tag regexes, so a field carrying a legal attribute silently became ''. users.get.xsd gives <directoryUri> an optional exist attribute, meaning the SIP/Jabber URI was dropped from every record on clusters that set it. Also removes the <link rel="next">/<next> HATEOAS probing and the <totalCount> element fallback: no UDS schema has ever defined them. #34 and #38 stay open by sequencing, not for lack of a live cluster. #38 must first establish whether the undocumented bulkSearch attribute on <users> is a request parameter, a mode, or a cursor — if it is a cursor it supersedes start/max arithmetic entirely, so implementing #34's UserSearchLimit probe first risks building offset paging twice. Test plan: 26 new tests pin each parser to the XSDs. Verified the pagination tests genuinely catch the regression by reintroducing the +1 in place: the boundary test then fails on the collected set (user064 dropped), not merely on a URL assertion. 274 passed, 2 skipped.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this is
An audit of every
/cucm-udsendpoint the tool touches, against the official Cisco UDS v14 XML schemas — plus fixes for what it found.Answering the original question first: no UDS endpoint is version-specific. All four we use (
/version,/users,/servers,/user/{id}) shipped in CUCM 9.1(2) via COP and natively in 10.0(1), and all still exist through 15. There is no version segment in the UDS path. The only version-dependent behaviors are the 8443→9443 Contact Search Authentication port shift and the pre-11.5(1) absence of<capabilities>, both of which were already handled correctly.The audit did, however, turn up four parsing defects — two of which silently lose data.
Why the schemas, not the docs
Cisco's prose UDS documentation is stale and self-contradictory: the changelog stops at 12.5(1), there is no version selector or per-release guide, and the
usersResourceAuthEnabledflag is spelled three different ways across the guide. It also documents element names our code was using that appear nowhere in any schema.The authoritative artifact is the UDS XML Schema for v14 XSD bundle at developer.cisco.com/site/user-data-services/downloads/schemas/ — newer than every prose page, and not linked from the
/docs/set. It requires a DevNet login (the older 11.0–12.5 bundles 403 on their direct CDN paths; only the v14 entry goes through the authenticated redirect). Every claim below is quoted from it.Fixes
/userspaging sentstart=collected+1, but UDS indexes from 0 — one user skipped at every page boundary<users>wrapper's ownstart/returnedCount(bothuse="required"inusers.get.xsd), so the next offset isstart + returnedCountby constructionget_version()parsed a<prefix>element that exists in no UDS release; only the_TEST_MODEstub populated it, so the operator output was permanently deadversion=attribute — the UDS schema version, distinct from the CUCM version in<version>get_servers_api()looked forhostName/ipv4Address/ipv6Address/serverTypechildren and treated plain text as a "15.x+" variantservers.get.xsddeclares<server>astype="xs:string"and permits no children. Text-only parse; the polarity was backwards and those four names are AXL vocabulary UDS never returns<upgradeInProgress>, a v14 capability flag absent from Cisco's prose docs entirelyparse_uds_directory()used bare-tag regexes, so a field carrying a legal attribute silently became''<directoryUri>has an optionalexistattribute per the schema — the SIP/Jabber URI was dropped from every record on clusters that set itThe two silent-data-loss bugs are #33 and #39. Both fail invisibly: enumeration reports success, and the missing users or blank
directory_uricolumn look like the cluster simply had nothing to give.Also removed: the
<link rel="next">/<next>HATEOAS probing and the<totalCount>element fallback. No UDS schema has ever defined any of them — offset arithmetic onstartis the only paging mechanism UDS offers.Operator-visible changes
--serversno longer prints an always-empty address/role suffix, and--show-dbdrops the permanently-blankIPv4column for cluster servers (replaced with the discovery timestamp). UDS exposes neither addresses nor Publisher/Subscriber role: resolving a hostname is a DNS lookup on our side, and role requires AXL. README corrected — it previously promised "hostnames + IPs".upgradeInProgress=true, so partial directory results read as "come back after the upgrade" rather than as a finding.The
ipv4/ipv6/server_typecolumns stay incluster_serversfor compatibility with existingthief.dbfiles, documented at the writer as unpopulatable from UDS. Dropping them would need a migration; they remain a plausible home for future DNS or AXL enrichment.Testing
274 passed, 2 skipped (baseline on
mainis 248 passed, 2 skipped — 26 new tests, no regressions).The new
tests/test_uds_schema_conformance.pypins each parser to the XSDs, with the schema shape quoted in the docstrings.One test-quality note worth recording, caught by a review pass: the page-boundary regression test initially returned pages by call count, ignoring the requested
start, so it passed against the buggy code on its data assertion and only caught the bug via a URL check. Its fake server now slices the directory by thestartit actually receives, as a real CUCM does. Verified by reintroducing the+1in place — the test then fails on the collected set withuser064dropped at index 64, which is the real symptom.Deliberately left open
#34 (send
maxup to the 500UserSearchLimit, ~7.7x fewer requests) and #38 (investigate the undocumentedbulkSearchattribute) stay open — by sequencing, not for lack of a live cluster. #34's probe is mockable and implementable today, but #38 must first establish whetherbulkSearchis a request parameter, a mode, or a cursor. If it is a cursor it supersedesstart/maxarithmetic entirely, and building the offset probe first means building paging twice.#34's clamp-detection item is included here, with coverage:
requestedCount > returnedCountis now logged rather than mistaken for the end of the directory.Not verified
CUCM 15. Cisco publishes no UDS artifact of any kind for it — no prose, no schema. Since the v14 schema permits no
<server>children and no extra<version>elements, the parsers are written to be correct regardless. Two things would close this properly, and both are cheap on the next engagement with a 15 cluster in scope: capture/cucm-uds/serversand/cucm-uds/version, and save the bodies as fixtures.Closes #33, #35, #36, #37, #39.