Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
89 changes: 89 additions & 0 deletions DEVELOPMENT.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,89 @@
## Setup
```bash
# Clone and install
git clone https://github.com/filecoin-project/filecoin-pin
cd filecoin-pin
npm install
```

## Basic Execution
```bash
# Run the Pinning Server
npm run dev

# Run tests
npm test

# Compile TypeScript source
npm run build

# Run the cli
# This is the equivalent of running `filecoin-pin` if you had it installed globally (e.g., `npm install filecoin-pin -g`).
# It's like doing `npx filecoin-pin` that isn't stuck on that version until you `run npm install filecoin-pin -g` again.
node ./dist/cli.js
Comment on lines +20 to +23
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you can also run npx filecoin-pin or npx . to run the local version when you're in the package root. Note the foobar123 in the second npx filecoin-pin run when i'm in the package root.

    ~/code/work/filoz/filecoin-project 
 ✔ ❯ npx filecoin-pin
Usage: filecoin-pin [options] [command]

IPFS Pinning Service with Filecoin storage via Synapse
SDK

Options:
  -V, --version            output the version number
  -v, --verbose            verbose output
  --no-update-check        skip check for updates
  -h, --help               display help for command

Commands:
  server [options]         Start the IPFS Pinning Service API server
  payments                 Manage payment setup for Filecoin Onchain Cloud
  data-set|dataset         Inspect data sets managed through Filecoin Onchain Cloud
  import [options] <file>  Import an existing CAR file to Filecoin via Synapse
  add [options] <path>     Add a file or directory to Filecoin via Synapse (creates UnixFS CAR)
  rm [options]             Remove a Piece from a DataSet

    ~/code/work/filoz/filecoin-project 
 ✔ ❯ cd filecoin-pin
direnv: loading https://raw.githubusercontent.com/nix-community/nix-direnv/3.0.4/direnvrc (sha256-DzlYZ33mWF/Gs8DDeyjr8mnVmQGx7ASYqA5WlxwvBG4=)
direnv: export +PRIVATE_KEY_MAIN +PRIVATE_KEY_SUB +SHELL_DUMB +SHELL_INTERACTIVE +SHELL_LOGIN +SHELL_LOGIN2 +TEST_NET_PRIVATE_KEY +WALLET_ADDR

    ~/c/work/filo/file/filecoin-pin    303-validate…-descendants *3 !1 ?23 
 ✔ ❯ npx filecoin-pin
Usage: filecoin-pin [options] [command]

IPFS Pinning Service with Filecoin storage via Synapse
SDK foobar123

Options:
  -V, --version            output the version number
  -v, --verbose            verbose output
  --no-update-check        skip check for updates
  -h, --help               display help for command

Commands:
  server [options]         Start the IPFS Pinning Service API server
  payments                 Manage payment setup for Filecoin Onchain Cloud
  data-set|dataset         Inspect data sets managed through Filecoin Onchain Cloud
  import [options] <file>  Import an existing CAR file to Filecoin via Synapse
  add [options] <path>     Add a file or directory to Filecoin via Synapse (creates UnixFS CAR)
  rm [options]             Remove a Piece from a DataSet

```

## Testing

```bash
npm run test # All tests
npm run test:unit # Unit tests only
npm run test:integration # Integration tests
npm run test:browser # Browser tests
npm run lint:fix # Fix formatting
```

Below are various tips and suggestions for doing development with `filecoin-pin`.

## Tips
### Debug Logging
Prefix your `filecoin-pin` command with `LOG_LEVEL=debug`.

### HTTP Tracing on the CLI

If you want to see the HTTP calls and their response codes when running the `filecoin-pin` CLI, simply set `NODE_DEBUG=fetch`.

For example:

```bash
NODE_DEBUG=fetch filecoin-pin add $TMPFILE
```

This will yield log lines like:

```bash
FETCH 84357: connecting to calib2.ezpdpz.net using https:undefined
FETCH 84357: connected to calib2.ezpdpz.net using https:h1
FETCH 84357: sending request to POST https://calib2.ezpdpz.net//pdp/piece/uploads
FETCH 84357: received response to POST https://calib2.ezpdpz.net//pdp/piece/uploads - HTTP 201
FETCH 84357: trailers received from POST https://calib2.ezpdpz.net//pdp/piece/uploads
```

Note that this doesn't show headers or request/response payloads.

### Running CLI changes made to `filecoin-pin` only

If you want to quickly try out changes via CLI that were made in `filecoin-pin` only (i.e., not relying on unpublished changes in `synapse-sdk`):

```bash
npx tsx src/cli.ts $COMMAND
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this works for non-built versions, or you can use bun, or you can run npm run build && npx filecoin-pin in the package root

```

### Running CLI changes involving `synapse-sdk`

If you want within the `filecoin-pin` CLI to try out out `synapse-sdk` changes made locally that haven't been published:

```bash
# Commands for building Synapse
# Commands for adjusting filecoin-pin to use local Synapse
Comment on lines +77 to +78
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TODO: I need to fill this in

npx tsx src/cli.ts $COMMAND
```

### Quickly adding unique data

Because of content addressing, it's valuable to upload unique data to make sure there is no deduplication in the end-to-end store and retrieval flow. Various quick ways to do this:

1. Use temp file with the date/time. The file is small, it's very likely to be unique, and it's easy to verify in retrievals rather than random data.
```bash
TMPFILE=$(mktemp) && date > $TMPFILE && filecoin-pin add $TMPFILE
```
33 changes: 1 addition & 32 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -242,38 +242,7 @@ When `DATABASE_PATH` and `CAR_STORAGE_PATH` are not specified, data is stored in

## Development

Want to contribute to Filecoin Pin or run it locally?

```bash
# Clone and install
git clone https://github.com/filecoin-project/filecoin-pin
cd filecoin-pin
npm install

# Run the Pinning Server
npm run dev

# Run tests
npm test

# Compile TypeScript source
npm run build

# Run the cli
# This is the equivalent of running `filecoin-pin` if you had it installed globally (e.g., `npm install filecoin-pin -g`).
# It's like doing `npx filecoin-pin` that isn't stuck on that version until you `run npm install filecoin-pin -g` again.
node ./dist/cli.js
```

### Testing

```bash
npm run test # All tests
npm run test:unit # Unit tests only
npm run test:integration # Integration tests
npm run test:browser # Browser tests
npm run lint:fix # Fix formatting
```
See [DEVELOPMENT.md](./DEVELOPMENT.md).

## Community and Support

Expand Down