Skip to content

About

MogileFS client for nginx: asks a tracker where a key lives and serves it from the storage node that has it

Topics

Resources

Code of conduct

Contributing

Stars

0 stars

Watchers

1 watching

Forks

Latest commit

 

History

97 Commits

Folders and files

Repository files navigation

nginx-mogilefs-module

Build & Test Function Dynamic load FreeBSD A/UBSan Valgrind Reload CodeQL Lint

A MogileFS client for nginx. It asks a MogileFS tracker where a key lives and hands the answer to proxy_pass, so nginx serves the file straight from the storage node that has it. Writing and deleting through the tracker work as well.

Requirements

nginx 1.28 or newer. Older releases are not built against; 1.9.11 is the hard floor in the source, because that is where the module list moved into the cycle.

A MogileFS installation to talk to — a tracker, and storage nodes that serve over HTTP. For the test suite neither is needed: t/tracker.pl speaks the protocol, and nginx itself stands in for a storage node.

Building

Into the binary:

./configure --add-module=/path/to/nginx-mogilefs-module
make && make install

As a loadable module:

./configure --add-dynamic-module=/path/to/nginx-mogilefs-module
make && make install
load_module modules/ngx_http_mogilefs_module.so;

Quick start

location /download/ {
    mogilefs_tracker 192.168.2.2;
    mogilefs_domain  example_domain;

    mogilefs_pass {
        proxy_pass $mogilefs_path;
        proxy_hide_header Content-Type;
        proxy_buffering off;
    }
}

A request for /download/example.jpg asks the tracker for the key example.jpg and serves what comes back.

Writing and deleting belong in a location of their own, because they are not something to expose:

location /upload/ {
    allow 192.168.2.0/24;
    deny all;

    mogilefs_tracker 192.168.2.2;
    mogilefs_domain  example_domain;
    mogilefs_methods GET PUT DELETE;

    mogilefs_pass {
        proxy_pass $mogilefs_path;
        proxy_hide_header Content-Type;
    }
}

What the tracker's answers turn into

Tracker said nginx answers
unknown_key 404
domain_not_found 404
no_devices 503
no_key 400
unreg_class 400

Variables

$mogilefs_path holds the URL of the file on the storage node with the highest priority. $mogilefs_path1 to $mogilefs_path9 hold the remaining copies, in the order the tracker gave them, so proxy_next_upstream can walk them.

Directives

mogilefs_pass

Syntax mogilefs_pass [key] { ... }
Default none
Context server, location, limit_except

The key to ask the tracker for, and the block that fetches it.

Leave the key out and what remains of the request URI after the matched location prefix is used. The key may contain variables.

The block needs a proxy_pass $mogilefs_path. nginx reaches it through a hidden internal location that the module creates while reading the configuration.

mogilefs_tracker

Syntax mogilefs_tracker address
Default none
Context main, server, location

The tracker to ask: an address, an address with a port, or the name of an upstream block. May contain variables.

mogilefs_domain

Syntax mogilefs_domain name
Default default
Context main, server, location

The MogileFS domain to look in. May contain variables.

mogilefs_class

Syntax mogilefs_class class ...
Default none
Context main, server, location

The class to create a resource in. Every argument is evaluated and the first that is not empty is used, so a fallback can be written as a second argument. May contain variables.

mogilefs_methods

Syntax mogilefs_methods method ...
Default GET
Context main, server, location

Which methods reach MogileFS: GET fetches, PUT creates or replaces, DELETE removes. HEAD follows GET.

mogilefs_noverify

Syntax mogilefs_noverify on | off
Default off
Context main, server, location

Send noverify=1 to the tracker, which lets it answer without checking that the paths it hands out are reachable. Faster, and wrong more often.

mogilefs_connect_timeout

Syntax mogilefs_connect_timeout time
Default 60s
Context main, server, location

Cannot be longer than 75 seconds.

mogilefs_send_timeout

Syntax mogilefs_send_timeout time
Default 60s
Context main, server, location

mogilefs_read_timeout

Syntax mogilefs_read_timeout time
Default 60s
Context main, server, location

Testing

ci/build.sh 1.30.6 /tmp/nginx-test     # build it
ci/mogtest.sh 1.30.6                   # fetch, fail over, write, delete
ci/dso.sh 1.30.6                       # the same module, loaded dynamically

ci/mogtest.sh starts t/tracker.pl and an nginx that is both the client and the stand-in storage node, then walks the paths that matter: a key that exists, one that does not, each error the tracker can return, a key stored three times where only the third copy is actually there, and a write followed by a read and a delete. It also asks the tracker for malformed answers, because a tracker across a network is not a trusted source of well-formed lines, and counts the requests left open in the worker afterwards. It fails on an alert in the error log as well, because a worker that was restarted mid-run still answers every request.

License

BSD 3-Clause, Valery Kholodkov's original terms. See LICENSE.

About

MogileFS client for nginx: asks a tracker where a key lives and serves it from the storage node that has it

Topics

Resources

Code of conduct

Contributing

Stars

0 stars

Watchers

1 watching

Forks

Releases

Packages

Contributors

Languages