Enable debug logging of HTTP requests

[BigLep]

Done Criteria

It's possible to enable logging of all HTTP requests made by Synapse to help with debugging, specifically including:

• server
• path
• query string
• verb
• response code

There are docs on how to enable this logging for debugging.

Why Important

When debugging Curio PDP issues for example, it's really useful to know what did the client send to the Curio PDP SP. This is easy to get when Synapse is run within a browser context, but it would be nice to be able to get from the command line as well.

User/Customer

Maintainers attempting to debug the system.

Notes

Instances where this would have been useful:

• Investigation Ticket: PDP Piece Lookup Timeout on Calibration Network filecoin-services#380

[BigLep]

@rvagg @hugomrdias : I was wanting this today when working on FilOzone/filecoin-services#380. I slopped through this with #542 but I assume this isn't a good approach.

I'd like your input on whether there is already a good way to do this and if not, if this is worth doing...

[BigLep]

@SgtPooki did some work for RPC call tracing in Synapse in #541

That said, stepping back, this isn't a unique problem to us. Are there general ways the node community debugs http traffic like we get with web developer tools?

Quick searching:

1. Node HTTP_PROXY and related environment variables ?
2. NODE_DEBUG=http,https ?
3. Tools like https://garnik.medium.com/http-inspector-a-devtools-network-tab-for-node-js-process-5a008d17bf28

I assume there are some best practices here. I basically would want to make sure this is easier than installing wireshark but it's fine if there are some extra environment variables that need to be set. We should document our recommended approach because I assume this wilil continue to be useful in the heat of battle. I don't want each person spinning up at AI agent creating a gist each time...

[rvagg]

Normally you'd just fire up the dev console in Chrome and look at the network tab to see that. For Node.js we either need to intercept and wrap fetch globally (like we do for the telemetry thing) or there is NODE_DEBUG as you mention, but the fetch group for that is probably the cleanest here since http and https are more internal, they show connections but not URLs for fetches.

e.g. using example-storage-e2e.js using NODE_DEBUG=fetch:

--- Uploading ---
Uploading file to service provider...

FETCH 257985: sending request to POST https://warp.lotus.dedyn.io/pdp/piece/uploads
FETCH 257985: received response to POST https://warp.lotus.dedyn.io/pdp/piece/uploads - HTTP 201
FETCH 257985: trailers received from POST https://warp.lotus.dedyn.io/pdp/piece/uploads
FETCH 257985: connecting to warp.lotus.dedyn.io using https:undefined
  Upload progress: 16.5 KB / 16.5 KB (100.0%)
FETCH 257985: connected to warp.lotus.dedyn.io using https:h1
FETCH 257985: sending request to PUT https://warp.lotus.dedyn.io/pdp/piece/uploads/a2739f49-8650-49b3-8cc3-c488d018540b
FETCH 257985: received response to PUT https://warp.lotus.dedyn.io/pdp/piece/uploads/a2739f49-8650-49b3-8cc3-c488d018540b - HTTP 204
FETCH 257985: trailers received from PUT https://warp.lotus.dedyn.io/pdp/piece/uploads/a2739f49-8650-49b3-8cc3-c488d018540b
FETCH 257985: sending request to POST https://warp.lotus.dedyn.io/pdp/piece/uploads/a2739f49-8650-49b3-8cc3-c488d018540b
FETCH 257985: received response to POST https://warp.lotus.dedyn.io/pdp/piece/uploads/a2739f49-8650-49b3-8cc3-c488d018540b - HTTP 200
FETCH 257985: trailers received from POST https://warp.lotus.dedyn.io/pdp/piece/uploads/a2739f49-8650-49b3-8cc3-c488d018540b
FETCH 257985: sending request to GET https://warp.lotus.dedyn.io/pdp/piece?pieceCid=bafkzcibdqv5avhxs6k6n4edp2kdro2sf6x6v7qiopyj6qharrtq3uuywd5wmeoq2
FETCH 257985: received response to GET https://warp.lotus.dedyn.io/pdp/piece?pieceCid=bafkzcibdqv5avhxs6k6n4edp2kdro2sf6x6v7qiopyj6qharrtq3uuywd5wmeoq2 - HTTP 404
FETCH 257985: trailers received from GET https://warp.lotus.dedyn.io/pdp/piece?pieceCid=bafkzcibdqv5avhxs6k6n4edp2kdro2sf6x6v7qiopyj6qharrtq3uuywd5wmeoq2
FETCH 257985: sending request to GET https://warp.lotus.dedyn.io/pdp/piece?pieceCid=bafkzcibdqv5avhxs6k6n4edp2kdro2sf6x6v7qiopyj6qharrtq3uuywd5wmeoq2
FETCH 257985: received response to GET https://warp.lotus.dedyn.io/pdp/piece?pieceCid=bafkzcibdqv5avhxs6k6n4edp2kdro2sf6x6v7qiopyj6qharrtq3uuywd5wmeoq2 - HTTP 404
...

Showing me I'm stuck on pinging /pdp/piece/.

[BigLep]

NODE_DEBUG=fetch

Thanks @rvagg . I think that's good enough for now. I'm documenting that in filecion-pin with filecoin-project/filecoin-pin#300 . Even though that doesn't add docs within Synapse itself, I think the filecoin-pin docs are good enough. I'll resolve this issue.