A MogileFS client for nginx. It asks a MogileFS tracker where a key lives
and hands the answer to proxy_pass, so nginx serves the file straight
from the storage node that has it. Writing and deleting through the
tracker work as well.
nginx 1.28 or newer. Older releases are not built against; 1.9.11 is the hard floor in the source, because that is where the module list moved into the cycle.
A MogileFS installation to talk to — a tracker, and storage nodes that
serve over HTTP. For the test suite neither is needed: t/tracker.pl
speaks the protocol, and nginx itself stands in for a storage node.
Into the binary:
./configure --add-module=/path/to/nginx-mogilefs-module
make && make installAs a loadable module:
./configure --add-dynamic-module=/path/to/nginx-mogilefs-module
make && make installload_module modules/ngx_http_mogilefs_module.so;location /download/ {
mogilefs_tracker 192.168.2.2;
mogilefs_domain example_domain;
mogilefs_pass {
proxy_pass $mogilefs_path;
proxy_hide_header Content-Type;
proxy_buffering off;
}
}A request for /download/example.jpg asks the tracker for the key
example.jpg and serves what comes back.
Writing and deleting belong in a location of their own, because they are not something to expose:
location /upload/ {
allow 192.168.2.0/24;
deny all;
mogilefs_tracker 192.168.2.2;
mogilefs_domain example_domain;
mogilefs_methods GET PUT DELETE;
mogilefs_pass {
proxy_pass $mogilefs_path;
proxy_hide_header Content-Type;
}
}| Tracker said | nginx answers |
|---|---|
unknown_key |
404 |
domain_not_found |
404 |
no_devices |
503 |
no_key |
400 |
unreg_class |
400 |
$mogilefs_path holds the URL of the file on the storage node with the
highest priority. $mogilefs_path1 to $mogilefs_path9 hold the
remaining copies, in the order the tracker gave them, so
proxy_next_upstream can walk them.
| Syntax | mogilefs_pass [key] { ... } |
|---|---|
| Default | none |
| Context | server, location, limit_except |
The key to ask the tracker for, and the block that fetches it.
Leave the key out and what remains of the request URI after the matched location prefix is used. The key may contain variables.
The block needs a proxy_pass $mogilefs_path. nginx reaches it through a
hidden internal location that the module creates while reading the
configuration.
| Syntax | mogilefs_tracker address |
|---|---|
| Default | none |
| Context | main, server, location |
The tracker to ask: an address, an address with a port, or the name of an
upstream block. May contain variables.
| Syntax | mogilefs_domain name |
|---|---|
| Default | default |
| Context | main, server, location |
The MogileFS domain to look in. May contain variables.
| Syntax | mogilefs_class class ... |
|---|---|
| Default | none |
| Context | main, server, location |
The class to create a resource in. Every argument is evaluated and the first that is not empty is used, so a fallback can be written as a second argument. May contain variables.
| Syntax | mogilefs_methods method ... |
|---|---|
| Default | GET |
| Context | main, server, location |
Which methods reach MogileFS: GET fetches, PUT creates or replaces,
DELETE removes. HEAD follows GET.
| Syntax | mogilefs_noverify on | off |
|---|---|
| Default | off |
| Context | main, server, location |
Send noverify=1 to the tracker, which lets it answer without checking
that the paths it hands out are reachable. Faster, and wrong more often.
| Syntax | mogilefs_connect_timeout time |
|---|---|
| Default | 60s |
| Context | main, server, location |
Cannot be longer than 75 seconds.
| Syntax | mogilefs_send_timeout time |
|---|---|
| Default | 60s |
| Context | main, server, location |
| Syntax | mogilefs_read_timeout time |
|---|---|
| Default | 60s |
| Context | main, server, location |
ci/build.sh 1.30.6 /tmp/nginx-test # build it
ci/mogtest.sh 1.30.6 # fetch, fail over, write, delete
ci/dso.sh 1.30.6 # the same module, loaded dynamicallyci/mogtest.sh starts t/tracker.pl and an nginx that is both the client
and the stand-in storage node, then walks the paths that matter: a key
that exists, one that does not, each error the tracker can return, a key
stored three times where only the third copy is actually there, and a
write followed by a read and a delete. It also asks the tracker for
malformed answers, because a tracker across a network is not a trusted
source of well-formed lines, and counts the requests left open in the
worker afterwards. It fails on an alert in the error log as well, because
a worker that was restarted mid-run still answers every request.
BSD 3-Clause, Valery Kholodkov's original terms. See LICENSE.