Skip to content

feat(mint): implement epoch-based proof of liabilities with compact merkle-sum trees#1046

Draft
a1denvalu3 wants to merge 41 commits into
mainfrom
pol-mint
Draft

feat(mint): implement epoch-based proof of liabilities with compact merkle-sum trees#1046
a1denvalu3 wants to merge 41 commits into
mainfrom
pol-mint

Conversation

@a1denvalu3

Copy link
Copy Markdown
Collaborator

Description

This PR implements the Mint and Wallet sides of the Proof of Liabilities (PoL) scheme using synchronized epoch-based Sparse Merkle Sum Trees (MS-SMT) and OpenTimestamps (OTS) attestations.

Key Features:

  1. Aggregated Single-OTS Attestations: Solves the keyset bloat problem by dynamically aggregating all unexpired keysets' tree roots into a single deterministic global digest. Only one single OpenTimestamps calendar request is made per epoch, drastically reducing server load.
  2. Compact Merkle Proofs: Implements a bitmask-based optimization that reduces proof size from ~20KB to < 1KB (95%+ bandwidth reduction) by omitting empty default nodes.
  3. Continuous Lifecycle Audits for Inactive Keysets: Because inactive keysets cannot mint new ecash but can still redeem/spent outstanding tokens, the background loop keeps publishing new epochs containing the frozen Issued Tree and the updated/growing Spent Tree until the keyset's final_expiry is reached.
  4. Wallet CLI Solvency Auditing: Added cashu pol manifest and cashu pol audit commands. The wallet can trustlessly audit all its unspent tokens against both the Spent Tree (proving non-inclusion/double-spend protection) and the Issued Tree (proving 100% mint liability backing).
  5. Testing & Hardening: Includes comprehensive unit and integration tests, complete with respx mock HTTP calendar failovers and automated DB-backed transition validations. Runs fully offline with optional mock OTS mode (MINT_POL_MOCK_OTS=TRUE).

@codecov

codecov Bot commented Jun 10, 2026

Copy link
Copy Markdown

❌ 1 Tests Failed:

Tests completed Failed Passed Skipped
709 1 708 88
View the top 1 failed test(s) by shortest run time
tests.wallet.test_wallet_v1_api::test_mint_and_split_and_state_and_restore_paths
Stack Traces | 0.006s run time
monkeypatch = <_pytest.monkeypatch.MonkeyPatch object at 0x7fc7ae3e9240>
api = <cashu.wallet.v1_api.LedgerAPI object at 0x7fc7ae3e8d30>

    @pytest.mark.asyncio
    async def test_mint_and_split_and_state_and_restore_paths(monkeypatch, api: LedgerAPI):
        output = BlindedMessage(id="kid", amount=1, B_="ab")
        proof = Proof(
            id="kid", amount=1, C=PrivateKey().public_key.format().hex(), secret="s1"
        )
        cast(Any, api).keysets = {"kid": object()}
    
        requests = []
    
        async def fake_request(self, method, path, **kwargs):
            requests.append((method, path, kwargs))
            if path == "mint/bolt11":
                return _response(
                    200,
                    {
                        "signatures": [
                            {
                                "id": "kid",
                                "amount": 1,
                                "C_": PrivateKey().public_key.format().hex(),
                            }
                        ]
                    },
                )
            if path == "swap":
                return _response(
                    200,
                    {
                        "signatures": [
                            {
                                "id": "kid",
                                "amount": 1,
                                "C_": PrivateKey().public_key.format().hex(),
                            }
                        ]
                    },
                )
            if path == "checkstate":
                return _response(
                    200,
                    {"states": [{"Y": proof.Y, "state": "UNSPENT"}]},
                )
            if path == "restore":
                return _response(
                    200,
                    {
                        "outputs": [{"id": "kid", "amount": 1, "B_": "ab"}],
                        "signatures": [
                            {
                                "id": "kid",
                                "amount": 1,
                                "C_": PrivateKey().public_key.format().hex(),
                            }
                        ],
                    },
                )
            if path == "mint":
                return _response(
                    200,
                    {
                        "signatures": [
                            {
                                "id": "kid",
                                "amount": 1,
                                "C_": PrivateKey().public_key.format().hex(),
                            }
                        ]
                    },
                )
            raise AssertionError(f"Unexpected path {path}")
    
        monkeypatch.setattr(
            "cashu.wallet.v1_api.httpx.AsyncClient", lambda **kwargs: object()
        )
        monkeypatch.setattr(api, "_request", MethodType(fake_request, api))
    
        promises = await api.mint(outputs=[output], quote="q", signature="sig")
        assert len(promises) == 1
    
        split_promises = await api.split([proof], [output])
>       assert len(split_promises) == 1
E       AssertionError: assert 2 == 1
E        +  where 2 = len(([BlindedSignature(id='kid', amount=1, C_='021d39455b2c7234e661ebc83eb0d80b97425a9e7e7139d0757b5864198e23bea3', dleq=None, pol_receipt=None)], None))

tests/wallet/test_wallet_v1_api.py:464: AssertionError

To view more test analytics, go to the Test Analytics Dashboard
📋 Got 3 mins? Take this short survey to help us improve Test Analytics.

@a1denvalu3

Copy link
Copy Markdown
Collaborator Author

The protocol specification has been drafted and submitted as Draft Pull Request cashubtc/nuts#388.

Updates Included:

  • Formally defines the Sparse Merkle Sum Tree (MS-SMT) leaf calculations and parent node hashing in plain-English prose.
  • Outlines the Synchronized OpenTimestamps Aggregation steps.
  • Establishes the Signed Transactional Proof of Liability Receipts scheme on an individual, order-preserving note basis (using keyset-amount private keys), allowing users to hold the mint cryptographically accountable for each unspent promise or spent input.
  • Standardizes the JSON schemas for responses and the cryptographic Fraud Challenge format.

@a1denvalu3

Copy link
Copy Markdown
Collaborator Author

⚡ Tree-Level Caching for Solvency Audits (PoL)

Hi team,

I have implemented an optimized Tree-Level Caching solution on the mint side to solve the performance and potential Denial of Service (DoS) vulnerability associated with on-the-fly tree reconstruction.

💡 The Problem with Endpoint-Level Caching

Normally, we cache HTTP responses at the route level using RedisCache. However, when a wallet queries inclusion proofs via /v1/pol/.../proofs/issued or /v1/pol/.../proofs/spent, the POST request body contains that individual wallet's unique active/spent secrets. Because each wallet's token payload is unique, HTTP-level caching yields a 0% cache hit rate, forcing the mint to run database queries and rebuild the entire $2^{256}$-leaf Sparse Merkle Sum Tree in-memory for every single request.

🚀 The Solution: Tree-Level Caching

Since historical epoch trees are completely static and immutable once finalized, we can cache the constructed tree objects themselves instead of the final HTTP responses.

We implemented a two-tier caching architecture directly inside the core tree builder:

  1. Tier 1 (Redis): If Redis caching is enabled (settings.mint_redis_cache_enabled), completed trees are serialized via high-performance pickle and saved under Redis keys pol:tree:issued:{keyset_id}:{epoch_index} and pol:tree:spent:{keyset_id}:{epoch_index}. This ensures horizontally scaled mint instances share a single cluster-wide cache.
  2. Tier 2 (In-Memory Fallback): If Redis is disabled, we fallback to a process-local, thread-safe memory dictionary (_FALLBACK_MEM_CACHE) capped at 20 tree pairs to prevent unbounded memory growth.
  3. Pre-Warming: Newly published epochs are automatically pre-warmed/cached as soon as the manifest is generated in update_pol_manifests, so the very first wallet to audit gets an instant response.

With this optimization, database queries and tree-construction CPU loops are run exactly once per epoch rather than once per wallet!

I have also added robust unit & integration tests covering both the Redis-primary and fallback in-memory caching layers (test_build_trees_caching in tests/mint/test_mint_pol.py), and formatted/resolved type-checking checks for repository stability.

Let me know if you have any feedback!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Backlog

Development

Successfully merging this pull request may close these issues.

1 participant