diff --git a/src/current/_data/v26.3/cockroach-commands.json b/src/current/_data/v26.3/cockroach-commands.json new file mode 100644 index 00000000000..f6150918b20 --- /dev/null +++ b/src/current/_data/v26.3/cockroach-commands.json @@ -0,0 +1,521 @@ +[ + { + "command_id": "cockroach_start", + "synopsis": { + "start_new_cluster": "cockroach start ", + "initialize_cluster": "cockroach init ", + "add_node": "cockroach start ", + "view_help": "cockroach start --help" + }, + "flags": [ + { + "flag": "--attrs", + "description": "Arbitrary strings, separated by colons, specifying node capability, which might include specialized hardware or number of cores, for example:

`--attrs=ram:64gb`

These can be used to influence the location of data replicas. See [Replication Controls](configure-replication-zones.html#replication-constraints) for full details." + }, + { + "flag": "--background", + "description": "Runs the node in the background. Control is returned to the shell only once the node is ready to accept requests, so this is recommended over appending `&` to the command. This flag is **not** available in Windows environments.

**Note:** `--background` is suitable for writing automated test suites or maintenance procedures that need a temporary server process running in the background. It is not intended to be used to start a long-running server, because it does not fully detach from the controlling terminal. Consider using a service manager or a tool like [daemon(8)](https://www.freebsd.org/cgi/man.cgi?query=daemon&sektion=8) instead. If you use `--background`, using `--pid-file` is also recommended. To gracefully stop the `cockroach` process, send the `SIGTERM` signal to the process ID in the PID file. To gracefully restart the process, send the `SIGHUP` signal." + }, + { + "flag": "--cache", + "description": "The total size for the [Pebble storage engine](architecture/storage-layer.html#pebble) block cache, shared evenly if there are multiple storage devices. This can be a percentage (notated as a decimal or with `%`) or any bytes-based unit, for example:

`--cache=.25`
`--cache=25%`
`--cache=1000000000 ----> 1000000000 bytes`
`--cache=1GB ----> 1000000000 bytes`
`--cache=1GiB ----> 1073741824 bytes`

Note: If you use the `%` notation, you might need to escape the `%` sign when configuring CockroachDB through `systemd` service files. For this reason, it's recommended to use the decimal notation instead.

**Note:** The sum of `--cache`, `--max-sql-memory`, and `--max-tsdb-memory` should not exceed 75% of the memory available to the `cockroach` process.

**Default:** `256 MiB`

The default cache size is reasonable for local development clusters. For production deployments, set this to 25% or higher. Increasing the cache size generally improves the node's read performance.

The block cache holds uncompressed blocks of persisted [key-value data](architecture/distribution-layer.html#overview) in memory. If a read misses within the block cache, the storage engine reads the file via the operating system's page cache, which may hold the relevant block in-memory in its compressed form. Otherwise, the read is served from the storage device. The block cache fills to the configured size and is then recycled using a least-recently-used (LRU) policy.

Refer to [Recommended Production Settings](recommended-production-settings.html#cache-and-sql-memory-size) for more details. Production systems should always configure this setting." + }, + { + "flag": "--clock-device", + "description": "Enable CockroachDB to use a [PTP hardware clock](https://www.kernel.org/doc/html/latest/driver-api/ptp.html) when querying the current time. The value is a string that specifies the clock device to use. For example: `--clock-device=/dev/ptp0`

**Note:** This is supported on Linux only and may be needed in cases where the host clock is unreliable or prone to large jumps (e.g., when using vMotion)." + }, + { + "flag": "--cluster-name", + "description": "A string that specifies a cluster name. This is used together with `--join` to ensure that all newly created nodes join the intended cluster when you are running multiple clusters.

**Note:** If this is set, [`cockroach init`](cockroach-init.html), [`cockroach node decommission`](cockroach-node.html), [`cockroach node recommission`](cockroach-node.html), and the `cockroach debug` commands must specify either `--cluster-name` or `--disable-cluster-name-verification` in order to work." + }, + { + "flag": "--disable-cluster-name-verification", + "description": "On clusters for which a cluster name has been set, this flag paired with `--cluster-name` disables the cluster name check for the command. This is necessary on existing clusters, when setting a cluster name or changing the cluster name: Perform a rolling restart of all nodes and include both the new `--cluster-name` value and `--disable-cluster-name-verification`, then a second rolling restart with `--cluster-name` and without `--disable-cluster-name-verification`." + }, + { + "flag": "--external-io-dir", + "description": "The path of the external IO directory with which the local file access paths are prefixed while performing backup and restore operations using local node directories or NFS drives. If set to `disabled`, backups and restores using local node directories and NFS drives, as well as [`cockroach nodelocal upload`](cockroach-nodelocal-upload.html), are disabled.

**Default:** `extern` subdirectory of the first configured [`store`](#store).

To set the `--external-io-dir` flag to the locations you want to use without needing to restart nodes, create symlinks to the desired locations from within the `extern` directory." + }, + { + "flag": "--listening-url-file", + "description": "The file to which the node's SQL connection URL will be written as soon as the node is ready to accept connections, in addition to being printed to the [standard output](#standard-output). When `--background` is used, this happens before the process detaches from the terminal.

This is particularly helpful in identifying the node's port when an unused port is assigned automatically (`--port=0`)." + }, + { + "flag": "--locality", + "description": "Arbitrary key-value pairs that describe the location of the node. Locality might include country, region, availability zone, etc. A `region` tier must be included in order to enable [multi-region capabilities](multiregion-overview.html). To specify locality in a file instead, refer to `--locality-file`. For more details, see [Locality](#locality) below." + }, + { + "flag": "--locality-file", + "description": "A file that contains arbitrary key-value pairs that describe the location of the node, as an alternative to the `--locality` flag." + }, + { + "flag": "--max-disk-temp-storage", + "description": "The maximum on-disk storage capacity available to store temporary data for SQL queries that exceed the memory budget (see `--max-sql-memory`). This ensures that JOINs, sorts, and other memory-intensive SQL operations are able to spill intermediate results to disk. This can be a percentage (notated as a decimal or with `%`) or any bytes-based unit (e.g., `.25`, `25%`, `500GB`, `1TB`, `1TiB`).

Note: If you use the `%` notation, you might need to escape the `%` sign, for instance, while configuring CockroachDB through `systemd` service files. For this reason, it's recommended to use the decimal notation instead. Also, if expressed as a percentage, this value is interpreted relative to the size of the first store. However, the temporary space usage is never counted towards any store usage; therefore, when setting this value, it's important to ensure that the size of this temporary storage plus the size of the first store doesn't exceed the capacity of the storage device.

The temporary files are located in the path specified by the `--temp-dir` flag, or in the subdirectory of the first store (see `--store`) by default.

**Default:** `32GiB`" + }, + { + "flag": "--max-go-memory", + "description": "The maximum soft memory limit for the Go runtime, which influences the behavior of Go's garbage collection. Defaults to `--max-sql-memory x 2.25`, but cannot exceed 90% of the node's available RAM. To disable the soft memory limit, set `--max-go-memory` to `0` (not recommended)." + }, + { + "flag": "--max-offset", + "description": "The maximum allowed clock offset for the cluster. If observed clock offsets exceed this limit, servers will crash to minimize the likelihood of reading inconsistent data. Increasing this value will increase the time to recovery of failures as well as the frequency of uncertainty-based read restarts.

Nodes can run with different values for `--max-offset`, but only for the purpose of updating the setting across the cluster using a rolling upgrade.

**Default:** `500ms`" + }, + { + "flag": "--max-sql-memory", + "description": "The maximum in-memory storage capacity available to store temporary data for SQL queries, including prepared queries and intermediate data rows during query execution. This can be a percentage (notated as a decimal or with `%`) or any bytes-based unit; for example:

`--max-sql-memory=.25`
`--max-sql-memory=25%`
`--max-sql-memory=10000000000 ----> 1000000000 bytes`
`--max-sql-memory=1GB ----> 1000000000 bytes`
`--max-sql-memory=1GiB ----> 1073741824 bytes`

The temporary files are located in the path specified by the `--temp-dir` flag, or in the subdirectory of the first store (see `--store`) by default.

**Note:** If you use the `%` notation, you might need to escape the `%` sign (for instance, while configuring CockroachDB through `systemd` service files). For this reason, it's recommended to use the decimal notation instead.

**Note:** The sum of `--cache`, `--max-sql-memory`, and `--max-tsdb-memory` should not exceed 75% of the memory available to the `cockroach` process.

**Default:** `25%`

The default SQL memory size is suitable for production deployments but can be raised to increase the number of simultaneous client connections the node allows as well as the node's capacity for in-memory processing of rows when using `ORDER BY`, `GROUP BY`, `DISTINCT`, joins, and window functions. For local development clusters with memory-intensive workloads, reduce this value to, for example, `128MiB` to prevent [out-of-memory errors](cluster-setup-troubleshooting.html#out-of-memory-oom-crash)." + }, + { + "flag": "--max-tsdb-memory", + "description": "Maximum memory capacity available to store temporary data for use by the time-series database to display metrics in the [DB Console](ui-overview.html). Consider raising this value if your cluster is comprised of a large number of nodes where individual nodes have very limited memory available (e.g., under `8 GiB`). Insufficient memory capacity for the time-series database can constrain the ability of the DB Console to process the time-series queries used to render metrics for the entire cluster. This capacity constraint does not affect SQL query execution. This flag accepts numbers interpreted as bytes, size suffixes (e.g., `1GB` and `1GiB`) or a percentage of physical memory (e.g., `0.01`).

**Note:** The sum of `--cache`, `--max-sql-memory`, and `--max-tsdb-memory` should not exceed 75% of the memory available to the `cockroach` process.

**Default:** `0.01` (i.e., 1%) of physical memory or `64 MiB`, whichever is greater." + }, + { + "flag": "--pid-file", + "description": "The file to which the node's process ID will be written as soon as the node is ready to accept connections. When `--background` is used, this happens before the process detaches from the terminal. When this flag is not set, the process ID is not written to file." + }, + { + "flag": "--store", + "short": "-s", + "description": "The file path to a storage device and, optionally, store attributes and maximum size. When using multiple storage devices for a node, this flag must be specified separately for each device, for example:

`--store=/mnt/ssd01 --store=/mnt/ssd02`

For more details, see [Store](#store) below." + }, + { + "flag": "--wal-failover", + "description": "Used to configure [WAL failover](#write-ahead-log-wal-failover) on [nodes](architecture/overview.html#node) with [multiple stores](#store). To enable WAL failover, pass `--wal-failover=among-stores`. To disable, pass `--wal-failover=disabled` on [node restart](node-shutdown.html#stop-and-restart-a-node)." + }, + { + "flag": "--spatial-libs", + "description": "The location on disk where CockroachDB looks for [spatial](spatial-data-overview.html) libraries.

**Defaults:**
  • `/usr/local/lib/cockroach`
  • A `lib` subdirectory of the CockroachDB binary's current directory.
" + }, + { + "flag": "--temp-dir", + "description": "The path of the node's temporary store directory. On node start up, the location for the temporary files is printed to the standard output.

**Default:** Subdirectory of the first [store](#store)" + }, + { + "flag": "--experimental-dns-srv", + "description": "When this flag is included, the node will first attempt to fetch SRV records from DNS for every name specified with `--join`. If a valid SRV record is found, that information is used instead of regular DNS A/AAAA lookups. This feature is experimental and may be removed or modified in a later version." + }, + { + "flag": "--listen-addr", + "description": "The IP address/hostname and port to listen on for connections from other nodes and clients. For IPv6, use the notation `[...]`, e.g., `[::1]` or `[fe80::f6f2:::]`.

This flag's effect depends on how it is used in combination with `--advertise-addr`. For example, the node will also advertise itself to other nodes using this value if `--advertise-addr` is not specified. For more details, see [Networking](recommended-production-settings.html#networking).

**Default:** Listen on all IP addresses on port `26257`; if `--advertise-addr` is not specified, also advertise the node's canonical hostname to other nodes" + }, + { + "flag": "--advertise-addr", + "description": "The IP address/hostname and port to tell other nodes to use. If using a hostname, it must be resolvable from all nodes. If using an IP address, it must be routable from all nodes; for IPv6, use the notation `[...]`, e.g., `[::1]` or `[fe80::f6f2:::]`.

This flag's effect depends on how it is used in combination with `--listen-addr`. For example, if the port number is different than the one used in `--listen-addr`, port forwarding is required. For more details, see [Networking](recommended-production-settings.html#networking).

**Default:** The value of `--listen-addr`; if `--listen-addr` is not specified, advertises the node's canonical hostname and port `26257`" + }, + { + "flag": "--advertise-http-addr", + "description": "The IP address/hostname and port to advertise for DB Console HTTP requests when [proxying connections to the DB Console](ui-overview.html#proxy-db-console). If omitted, the hostname is inherited from the operating system hostname or the hostname from `--advertise-addr`. If the port is omitted, it defaults to `8080` and is never inherited from `--advertise-addr`." + }, + { + "flag": "--http-addr", + "description": "The IP address/hostname and port on which to listen for DB Console HTTP requests. For IPv6, use the notation `[...]`, e.g., `[::1]:8080` or `[fe80::f6f2:::]:8080`.

**Default:** Listen on the address specified in `--listen-addr` and on port `8080`" + }, + { + "flag": "--locality-advertise-addr", + "description": "The IP address/hostname and port to tell other nodes in specific localities to use to connect to this node. This flag is useful when running a cluster across multiple networks, where nodes in a given network have access to a private or local interface while nodes outside the network do not. In this case, you can use `--locality-advertise-addr` to tell nodes within the same network to prefer the private or local address to improve performance and use `--advertise-addr` to tell nodes outside the network to use another address that is reachable from them. However, **do not** include addresses or hostnames that do not resolve to this node, because this will cause connection failures when other nodes attempt to connect to this node.

This flag relies on nodes being started with the [`--locality`](#locality) flag and uses the `locality@address` notation, for example:

`--locality-advertise-addr=region=us-west@10.0.0.0:26257`

For more details, refer to the [Start a multi-node cluster across private networks](#start-a-multi-node-cluster-across-private-networks) example." + }, + { + "flag": "--sql-addr", + "description": "The IP address/hostname and port on which to listen for SQL connections from clients. For IPv6, use the notation `[...]`, e.g., `[::1]` or `[fe80::f6f2:::]`.

This flag's effect depends on how it is used in combination with `--advertise-sql-addr`. For example, the node will also advertise itself to clients using this value if `--advertise-sql-addr` is not specified.

**Default:** The value of `--listen-addr`; if `--listen-addr` is not specified, advertises the node's canonical hostname and port `26257`

For an example, see [Start a cluster with separate RPC and SQL networks](#start-a-cluster-with-separate-rpc-and-sql-networks)" + }, + { + "flag": "--advertise-sql-addr", + "description": "The IP address/hostname and port to tell clients to use. If using a hostname, it must be resolvable from all nodes. If using an IP address, it must be routable from all nodes; for IPv6, use the notation `[...]`, e.g., `[::1]` or `[fe80::f6f2:::]`.

This flag's effect depends on how it is used in combination with `--sql-addr`. For example, if the port number is different than the one used in `--sql-addr`, port forwarding is required.

**Default:** The value of `--sql-addr`; if `--sql-addr` is not specified, advertises the value of `--listen-addr`" + }, + { + "flag": "--join", + "short": "-j", + "description": "The host addresses that connect nodes to the cluster and distribute the rest of the node addresses. These can be IP addresses or DNS aliases of nodes.

When starting a cluster in a single region, specify the addresses of 3-5 initial nodes. When starting a cluster in multiple regions, specify more than 1 address per region, and select nodes that are spread across failure domains. Then run the [`cockroach init`](cockroach-init.html) command against any of these nodes to complete cluster startup. See the [example](#start-a-multi-node-cluster) below for more details.

Use the same `--join` list for all nodes to ensure that the cluster can stabilize. Do not list every node in the cluster, because this increases the time for a new cluster to stabilize. Note that these are best practices; it is not required to restart an existing node to update its `--join` flag.

`cockroach start` must be run with the `--join` flag. To start a single-node cluster, use `cockroach start-single-node` instead." + }, + { + "flag": "--socket-dir", + "description": "The directory path on which to listen for [Unix domain socket](https://wikipedia.org/wiki/Unix_domain_socket) connections from clients installed on the same Unix-based machine. For an example, see [Connect to a cluster listening for Unix domain socket connections](cockroach-sql.html#connect-to-a-cluster-listening-for-unix-domain-socket-connections)." + }, + { + "flag": "--advertise-host", + "description": "Use `--advertise-addr` instead.", + "deprecated": true + }, + { + "flag": "--host", + "description": "Use `--listen-addr` instead.", + "deprecated": true + }, + { + "flag": "--port", + "short": "-p", + "description": "Specify port in `--advertise-addr` and/or `--listen-addr` instead.", + "deprecated": true + }, + { + "flag": "--http-host", + "description": "Use `--http-addr` instead.", + "deprecated": true + }, + { + "flag": "--http-port", + "description": "Specify port in `--http-addr` instead.", + "deprecated": true + }, + { + "flag": "--certs-dir", + "description": "The path to the [certificate directory](cockroach-cert.html). The directory must contain valid certificates if running in secure mode.

**Default:** `${HOME}/.cockroach-certs/`" + }, + { + "flag": "--insecure", + "description": "**Note:** The `--insecure` flag is intended for **non-production testing only**.

Run in insecure mode, skipping all TLS encryption and authentication. If this flag is not set, the `--certs-dir` flag must point to valid certificates.

**Note the following risks:** An insecure cluster is open to any client that can access any node's IP addresses; client connections must also be made insecurely; any user, even `root`, can log in without providing a password; any user, connecting as `root`, can read or write any data in your cluster; there is no network encryption or authentication, and thus no confidentiality.

**Default:** `false`" + }, + { + "flag": "--accept-sql-without-tls", + "description": "This flag (in [preview](cockroachdb-feature-availability.html)) allows you to connect to the cluster using a SQL user's password without [validating the client's certificate](authentication.html#client-authentication). When connecting using the built-in SQL client, [use the `--insecure` flag with the `cockroach sql` command](cockroach-sql.html#client-connection)." + }, + { + "flag": "--cert-principal-map", + "description": "A comma-separated list of `cert-principal:db-principal` mappings used to map the certificate principals to IP addresses, DNS names, and SQL users. This allows the use of certificates generated by Certificate Authorities that place restrictions on the contents of the `commonName` field. For usage information, see [Create Security Certificates using Openssl](create-security-certificates-openssl.html#examples)." + }, + { + "flag": "--enterprise-encryption", + "description": "This optional flag specifies the encryption options for one of the stores on the node. If multiple stores exist, the flag must be specified for each store.

This flag takes a number of options. For a complete list of options, and usage instructions, see [Encryption at Rest](encryption.html)." + }, + { + "flag": "--external-io-disable-http", + "description": "This optional flag disables external HTTP(S) access (as well as custom HTTP(S) endpoints) when performing bulk operations (e.g, [`BACKUP`](backup.html)). This can be used in environments where you cannot run a full proxy server.

If you want to run a proxy server, you can start CockroachDB while specifying the `HTTP(S)_PROXY` environment variable." + }, + { + "flag": "--external-io-disable-implicit-credentials", + "description": "This optional flag disables the use of implicit credentials when accessing external cloud storage services for bulk operations (e.g, [`BACKUP`](backup.html))." + }, + { + "flag": "--node-cert-distinguished-name", + "description": "A string with a comma separated list of distinguished name (DN) mappings in `{attribute-type}={attribute-value}` format in accordance with [RFC4514](https://www.rfc-editor.org/rfc/rfc4514) for the [`node` user](security-reference/authorization.html#node-user). If this flag is set, this needs to be an exact match with the DN subject in the client certificate provided for the `node` user. By exact match, we mean that the order of attributes in the argument to this flag must match the order of attributes in the DN subject in the certificate. For more information, see [Certificate-based authentication using multiple values from the X.509 Subject field](certificate-based-authentication-using-the-x509-subject-field.html)." + }, + { + "flag": "--root-cert-distinguished-name", + "description": "A string with a comma separated list of distinguished name (DN) mappings in `{attribute-type}={attribute-value}` format in accordance with [RFC4514](https://www.rfc-editor.org/rfc/rfc4514) for the [`root` user](security-reference/authorization.html#root-user). If this flag is set, this needs to be an exact match with the DN subject in the client certificate provided for the `root` user. By exact match, we mean that the order of attributes in the argument to this flag must match the order of attributes in the DN subject in the certificate. For more information, see [Certificate-based authentication using multiple values from the X.509 Subject field](certificate-based-authentication-using-the-x509-subject-field.html)." + }, + { + "flag": "--tls-cipher-suites", + "description": "A comma-separated list of TLS cipher suites to allow for SQL, RPC, and HTTP connections, limited to those [supported by CockroachDB](authentication.html#supported-cipher-suites). Connections using disallowed cipher suites will be rejected during the TLS handshake and logged to `cockroach.log`. Look for log messages containing: `presented cipher ... not in allowed cipher suite list`.
Example usage: `--tls-cipher-suites=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_AES_128_GCM_SHA256`." + } + ], + "standard_output": { + "fields": [ + { + "field": "build", + "description": "The version of CockroachDB you are running." + }, + { + "field": "webui", + "description": "The URL for accessing the DB Console." + }, + { + "field": "sql", + "description": "The connection URL for your client." + }, + { + "field": "sql (JDBC)", + "description": "The JDBC connection URL for your client." + }, + { + "field": "RPC client flags", + "description": "The flags to use when connecting to the node via cockroach client commands." + }, + { + "field": "logs", + "description": "The directory containing debug log data." + }, + { + "field": "temp dir", + "description": "The temporary store directory of the node." + }, + { + "field": "external I/O path", + "description": "The external IO directory for backup and restore operations using local node directories or NFS drives." + }, + { + "field": "attrs", + "description": "If node-level attributes were specified in the --attrs flag, they are listed in this field." + }, + { + "field": "locality", + "description": "If locality values were specified in the --locality field, they are listed in this field." + }, + { + "field": "store[n]", + "description": "The directory containing store data, where [n] is the index of the store. If store-level attributes were specified, they are listed as well." + }, + { + "field": "status", + "description": "Whether the node is the first in the cluster (initialized new cluster), joined an existing cluster for the first time (initialized new node, joined pre-existing cluster), or rejoined an existing cluster (restarted pre-existing node)." + }, + { + "field": "clusterID", + "description": "The ID of the cluster." + }, + { + "field": "nodeID", + "description": "The ID of the node." + } + ] + }, + "store": [ + { + "field": "type", + "description": "For in-memory storage, set this field to `mem`; otherwise, leave this field out. The `path` field must not be set when `type=mem`." + }, + { + "field": "path", + "description": "The file path to the storage device. When not setting `attr`, `size`, or `ballast-size`, the `path` field label can be left out:

`--store=/mnt/ssd01`

When either of those fields are set, however, the `path` field label must be used:

`--store=path=/mnt/ssd01,size=20GB`

**Default:** `cockroach-data`" + }, + { + "field": "attrs", + "description": "Arbitrary strings, separated by colons, specifying disk type or capability. These can be used to influence the location of data replicas. See [Replication Controls](configure-replication-zones.html#replication-constraints) for full details.

In most cases, node-level `--locality` or `--attrs` are preferable to store-level attributes, but this field can be used to match capabilities for storage of individual databases or tables. For example, an OLTP database would probably want to allocate space for its tables only on solid state devices, whereas append-only time series might prefer cheaper spinning drives. Typical attributes include whether the store is flash (`ssd`) or spinny disk (`hdd`), as well as speeds and other specs, for example:

`--store=path=/mnt/hda1,attrs=hdd:7200rpm`" + }, + { + "field": "size", + "description": "The maximum size allocated to the node. When this size is reached, CockroachDB attempts to rebalance data to other nodes with available capacity. When no other nodes have available capacity, this limit will be exceeded. Data may also be written to the node faster than the cluster can rebalance it away; as long as capacity is available elsewhere, CockroachDB will gradually rebalance data down to the store limit.

The `size` can be specified either in a bytes-based unit or as a percentage of hard drive space (notated as a decimal or with `%`), for example:

`--store=path=/mnt/ssd01,size=10000000000 ----> 10000000000 bytes`
`--store=path=/mnt/ssd01,size=20GB ----> 20000000000 bytes`
`--store=path=/mnt/ssd01,size=20GiB ----> 21474836480 bytes`
`--store=path=/mnt/ssd01,size=0.02TiB ----> 21474836480 bytes`
`--store=path=/mnt/ssd01,size=20% ----> 20% of available space`
`--store=path=/mnt/ssd01,size=0.2 ----> 20% of available space`
`--store=path=/mnt/ssd01,size=.2 ----> 20% of available space`

**Default:** 100%

For an in-memory store, the `size` field is required and must be set to the true maximum bytes or percentage of available memory, for example:

`--store=type=mem,size=20GB`
`--store=type=mem,size=90%`

Note: If you use the `%` notation, you might need to escape the `%` sign, for instance, while configuring CockroachDB through `systemd` service files. For this reason, it's recommended to use the decimal notation instead." + }, + { + "field": "ballast-size", + "description": "Configure the size of the automatically created emergency ballast file. Accepts the same value formats as the `size` field. For more details, see [Automatic ballast files](cluster-setup-troubleshooting.html#automatic-ballast-files).

To disable automatic ballast file creation, set the value to `0`:

`--store=path=/mnt/ssd01,ballast-size=0`" + }, + { + "field": "provisioned-rate", + "description": "A mapping of a store name to a bandwidth limit, expressed in bytes per second. This constrains the bandwidth used for [admission control](admission-control.html) for operations on the store. The disk name is separated from the bandwidth value by a colon (`:`). A value of `0` (the default) represents unlimited bandwidth. For example:

`--store=provisioned-rate=disk-name=/mnt/ssd01:200`

**Default:** 0

If the bandwidth value is omitted, bandwidth is limited to the value of the [`kv.store.admission.provisioned_bandwidth` cluster setting](cluster-settings.html#settings). Modify this setting only in consultation with your support team." + } + ] + }, + { + "command_id": "cockroach_init", + "synopsis": { + "initialize": "cockroach init ", + "view_help": "cockroach init --help" + }, + "flags": { + "client_connection": [ + { + "flag": "--url", + "description": "A connection URL to use instead of the other arguments. To convert a connection URL to the syntax that works with your client driver, run cockroach convert-url.

**Env Variable:** `COCKROACH_URL`
**Default:** no URL" + }, + { + "flag": "--host", + "description": "The server host and port number to connect to. This can be the address of any node in the cluster.

**Env Variable:** `COCKROACH_HOST`
**Default:** localhost:26257" + }, + { + "flag": "--port", + "description": "The server port to connect to. Note: The port number can also be specified via --host.

**Env Variable:** `COCKROACH_PORT`
**Default:** 26257", + "short": "-p" + }, + { + "flag": "--user", + "description": "The SQL user that will own the client session.

**Env Variable:** `COCKROACH_USER`
**Default:** root", + "short": "-u" + }, + { + "flag": "--insecure", + "description": "Use an insecure connection.

**Env Variable:** `COCKROACH_INSECURE`
**Default:** false" + }, + { + "flag": "--cert-principal-map", + "description": "A comma-separated list of : mappings. This allows mapping the principal in a cert to a DB principal such as node or root or any SQL user. This is intended for use in situations where the certificate management system places restrictions on the Subject.CommonName or SubjectAlternateName fields in the certificate (e.g., disallowing a CommonName like node or root). If multiple mappings are provided for the same , the last one specified in the list takes precedence. A principal not specified in the map is passed through as-is via the identity function. A cert is allowed to authenticate a DB principal if the DB principal name is contained in the mapped CommonName or DNS-type SubjectAlternateName fields." + }, + { + "flag": "--certs-dir", + "description": "The path to the certificate directory containing the CA and client certificates and client key.

**Env Variable:** `COCKROACH_CERTS_DIR`
**Default:** ${HOME}/.cockroach-certs/" + }, + { + "flag": "--cluster-name", + "description": "The cluster name to use to verify the cluster's identity. If the cluster has a cluster name, you must include this flag." + }, + { + "flag": "--disable-cluster-name-verification", + "description": "Disables the cluster name check for this command. This flag must be paired with --cluster-name." + } + ] + }, + "standard_output": { + "fields": [] + } + }, + { + "command_id": "cockroach_start_single_node", + "synopsis": { + "start": "cockroach start-single-node ", + "view_help": "cockroach start-single-node --help" + }, + "flags": { + "general": [ + { + "flag": "--attrs", + "description": "**Not relevant for single-node clusters.** Arbitrary strings, separated by colons, specifying node capability, which might include specialized hardware or number of cores. These can be used to influence the location of data replicas. See [Replication Controls](configure-replication-zones.html#replication-constraints) for full details.

**Example:** `--attrs=ram:64gb`" + }, + { + "flag": "--background", + "description": "Runs the node in the background. Control is returned to the shell only once the node is ready to accept requests, so this is recommended over appending & to the command. This flag is **not** available in Windows environments.

**Note:** --background is suitable for writing automated test suites or maintenance procedures that need a temporary server process running in the background. It is not intended to be used to start a long-running server, because it does not fully detach from the controlling terminal. Consider using a service manager or a tool like [daemon(8)](https://www.freebsd.org/cgi/man.cgi?query=daemon&sektion=8) instead. If you use --background, using --pid-file is also recommended. To gracefully stop the cockroach process, send the SIGTERM signal to the process ID in the PID file. To gracefully restart the process, send the SIGHUP signal." + }, + { + "flag": "--cache", + "description": "The total size for the [Pebble storage engine](architecture/storage-layer.html#pebble) block cache, shared evenly if there are multiple storage devices. This can be a percentage (notated as a decimal or with %) or any bytes-based unit. The sum of --cache, --max-sql-memory, and --max-tsdb-memory should not exceed 75% of the memory available to the cockroach process. The default cache size is reasonable for local development clusters. For production deployments, set this to 25% or higher. Increasing the cache size generally improves the node's read performance. The block cache holds uncompressed blocks of persisted [key-value data](architecture/distribution-layer.html#overview) in memory. If a read misses within the block cache, the storage engine reads the file via the operating system's page cache, which may hold the relevant block in-memory in its compressed form. Otherwise, the read is served from the storage device. The block cache fills to the configured size and is then recycled using a least-recently-used (LRU) policy. Refer to [Recommended Production Settings](recommended-production-settings.html#cache-and-sql-memory-size) for more details. Production systems should always configure this setting.
**Default:** 256 MiB

**Examples:**
`--cache=.25`
`--cache=25%`
`--cache=1000000000 ----> 1000000000 bytes`
`--cache=1GB ----> 1000000000 bytes`
`--cache=1GiB ----> 1073741824 bytes`

**Note:** If you use the % notation, you might need to escape the % sign when configuring CockroachDB through systemd service files. For this reason, it's recommended to use the decimal notation instead." + }, + { + "flag": "--external-io-dir", + "description": "The path of the external IO directory with which the local file access paths are prefixed while performing backup and restore operations using local node directories or NFS drives. If set to disabled, backups and restores using local node directories and NFS drives are disabled. To set the --external-io-dir flag to the locations you want to use without needing to restart nodes, create symlinks to the desired locations from within the extern directory.
**Default:** extern subdirectory of the first configured store" + }, + { + "flag": "--listening-url-file", + "description": "The file to which the node's SQL connection URL will be written on successful startup, in addition to being printed to the standard output. This is particularly helpful in identifying the node's port when an unused port is assigned automatically (--port=0)." + }, + { + "flag": "--locality", + "description": "**Not relevant for single-node clusters.** Arbitrary key-value pairs that describe the location of the node. Locality might include country, region, datacenter, rack, etc. For more details, see [Locality](cockroach-start.html#locality) below." + }, + { + "flag": "--locality-file", + "description": "A file that contains arbitrary key-value pairs that describe the location of the node, as an alternative to the --locality flag." + }, + { + "flag": "--max-disk-temp-storage", + "description": "The maximum on-disk storage capacity available to store temporary data for SQL queries that exceed the memory budget (see --max-sql-memory). This ensures that JOINs, sorts, and other memory-intensive SQL operations are able to spill intermediate results to disk. This can be a percentage (notated as a decimal or with %) or any bytes-based unit (e.g., .25, 25%, 500GB, 1TB, 1TiB). If expressed as a percentage, this value is interpreted relative to the size of the first store. However, the temporary space usage is never counted towards any store usage; therefore, when setting this value, it's important to ensure that the size of this temporary storage plus the size of the first store doesn't exceed the capacity of the storage device. The temporary files are located in the path specified by the --temp-dir flag, or in the subdirectory of the first store (see --store) by default.
**Default:** 32GiB

**Note:** If you use the % notation, you might need to escape the % sign, for instance, while configuring CockroachDB through systemd service files. For this reason, it's recommended to use the decimal notation instead." + }, + { + "flag": "--max-go-memory", + "description": "The maximum soft memory limit for the Go runtime, which influences the behavior of Go's garbage collection. Defaults to --max-sql-memory x 2.25, but cannot exceed 90% of the node's available RAM. To disable the soft memory limit, set --max-go-memory to 0 (not recommended)." + }, + { + "flag": "--max-sql-memory", + "description": "The maximum in-memory storage capacity available to store temporary data for SQL queries, including prepared queries and intermediate data rows during query execution. This can be a percentage (notated as a decimal or with %) or any bytes-based unit. The temporary files are located in the path specified by the --temp-dir flag, or in the subdirectory of the first store (see --store) by default. The sum of --cache, --max-sql-memory, and --max-tsdb-memory should not exceed 75% of the memory available to the cockroach process. The default SQL memory size is suitable for production deployments but can be raised to increase the number of simultaneous client connections the node allows as well as the node's capacity for in-memory processing of rows when using ORDER BY, GROUP BY, DISTINCT, joins, and window functions. For local development clusters with memory-intensive workloads, reduce this value to, for example, 128MiB to prevent [out-of-memory errors](cluster-setup-troubleshooting.html#out-of-memory-oom-crash).
**Default:** 25%

**Examples:**
`--max-sql-memory=.25`
`--max-sql-memory=25%`
`--max-sql-memory=10000000000 ----> 1000000000 bytes`
`--max-sql-memory=1GB ----> 1000000000 bytes`
`--max-sql-memory=1GiB ----> 1073741824 bytes`

**Note:** If you use the % notation, you might need to escape the % sign (for instance, while configuring CockroachDB through systemd service files). For this reason, it's recommended to use the decimal notation instead." + }, + { + "flag": "--max-tsdb-memory", + "description": "Maximum memory capacity available to store temporary data for use by the time-series database to display metrics in the [DB Console](ui-overview.html). Consider raising this value if your cluster is comprised of a large number of nodes where individual nodes have very limited memory available (e.g., under 8 GiB). Insufficient memory capacity for the time-series database can constrain the ability of the DB Console to process the time-series queries used to render metrics for the entire cluster. This capacity constraint does not affect SQL query execution. This flag accepts numbers interpreted as bytes, size suffixes (e.g., 1GB and 1GiB) or a percentage of physical memory (e.g., 0.01). The sum of --cache, --max-sql-memory, and --max-tsdb-memory should not exceed 75% of the memory available to the cockroach process.
**Default:** 0.01 (i.e., 1%) of physical memory or 64 MiB, whichever is greater" + }, + { + "flag": "--pid-file", + "description": "The file to which the node's process ID will be written on successful startup. When this flag is not set, the process ID is not written to file." + }, + { + "flag": "--store", + "description": "The file path to a storage device and, optionally, store attributes and maximum size. When using multiple storage devices for a node, this flag must be specified separately for each device. For more details, see Store below.

**Example:** `--store=/mnt/ssd01 --store=/mnt/ssd02`", + "short": "-s" + }, + { + "flag": "--temp-dir", + "description": "The path of the node's temporary store directory. On node start up, the location for the temporary files is printed to the standard output.
**Default:** Subdirectory of the first store" + } + ], + "networking": [ + { + "flag": "--listen-addr", + "description": "The IP address/hostname and port to listen on for connections from clients. For IPv6, use the notation [...], e.g., [::1] or [fe80::f6f2:::].
**Default:** Listen on all IP addresses on port 26257" + }, + { + "flag": "--http-addr", + "description": "The IP address/hostname and port to listen on for DB Console HTTP requests. For IPv6, use the notation [...], e.g., [::1]:8080 or [fe80::f6f2:::]:8080.
**Default:** Listen on the address part of --listen-addr on port 8080" + }, + { + "flag": "--socket-dir", + "description": "The directory path on which to listen for [Unix domain socket](https://wikipedia.org/wiki/Unix_domain_socket) connections from clients installed on the same Unix-based machine. For an example, see [Connect to a cluster listening for Unix domain socket connections](cockroach-sql.html#connect-to-a-cluster-listening-for-unix-domain-socket-connections)." + } + ], + "security": [ + { + "flag": "--certs-dir", + "description": "The path to the [certificate directory](cockroach-cert.html). The directory must contain valid certificates if running in secure mode.
**Default:** ${HOME}/.cockroach-certs/" + }, + { + "flag": "--insecure", + "description": "Run in insecure mode, skipping all TLS encryption and authentication. If this flag is not set, the --certs-dir flag must point to valid certificates. An insecure cluster is open to any client that can access any node's IP addresses; client connections must also be made insecurely; any user, even root, can log in without providing a password; any user, connecting as root, can read or write any data in your cluster; there is no network encryption or authentication, and thus no confidentiality.
**Default:** false

**Warning:** The --insecure flag is intended for **non-production testing only**." + }, + { + "flag": "--accept-sql-without-tls", + "description": "This flag (in [preview](cockroachdb-feature-availability.html)) allows you to connect to the cluster using a SQL user's password without [validating the client's certificate](authentication.html#client-authentication). When connecting using the built-in SQL client, [use the --insecure flag with the cockroach sql command](cockroach-sql.html#client-connection)." + }, + { + "flag": "--cert-principal-map", + "description": "A comma-separated list of cert-principal:db-principal mappings used to map the certificate principals to IP addresses, DNS names, and SQL users. This allows the use of certificates generated by Certificate Authorities that place restrictions on the contents of the commonName field. For usage information, see [Create Security Certificates using Openssl](create-security-certificates-openssl.html#examples)." + }, + { + "flag": "--enterprise-encryption", + "description": "This optional flag specifies the encryption options for one of the stores on the node. If multiple stores exist, the flag must be specified for each store. This flag takes a number of options. For a complete list of options, and usage instructions, see [Encryption at Rest](encryption.html)." + } + ], + "store": [ + { + "field": "type", + "description": "For in-memory storage, set this field to mem; otherwise, leave this field out. The path field must not be set when type=mem." + }, + { + "field": "path", + "description": "The file path to the storage device. When not setting attr, size, or ballast-size, the path field label can be left out. When either of those fields are set, however, the path field label must be used.
**Default:** cockroach-data

**Examples:**
`--store=/mnt/ssd01`
`--store=path=/mnt/ssd01,size=20GB`" + }, + { + "field": "attrs", + "description": "Arbitrary strings, separated by colons, specifying disk type or capability. These can be used to influence the location of data replicas. See [Replication Controls](configure-replication-zones.html#replication-constraints) for full details. In most cases, node-level --locality or --attrs are preferable to store-level attributes, but this field can be used to match capabilities for storage of individual databases or tables. For example, an OLTP database would probably want to allocate space for its tables only on solid state devices, whereas append-only time series might prefer cheaper spinning drives. Typical attributes include whether the store is flash (ssd) or spinny disk (hdd), as well as speeds and other specs.

**Example:** `--store=path=/mnt/hda1,attrs=hdd:7200rpm`" + }, + { + "field": "size", + "description": "The maximum size allocated to the node. When this size is reached, CockroachDB attempts to rebalance data to other nodes with available capacity. When no other nodes have available capacity, this limit will be exceeded. Data may also be written to the node faster than the cluster can rebalance it away; as long as capacity is available elsewhere, CockroachDB will gradually rebalance data down to the store limit. The size can be specified either in a bytes-based unit or as a percentage of hard drive space (notated as a decimal or with %). For an in-memory store, the size field is required and must be set to the true maximum bytes or percentage of available memory.
**Default:** 100%

**Examples:**
`--store=path=/mnt/ssd01,size=10000000000 ----> 10000000000 bytes`
`--store=path=/mnt/ssd01,size=20GB ----> 20000000000 bytes`
`--store=path=/mnt/ssd01,size=20GiB ----> 21474836480 bytes`
`--store=path=/mnt/ssd01,size=0.02TiB ----> 21474836480 bytes`
`--store=path=/mnt/ssd01,size=20% ----> 20% of available space`
`--store=path=/mnt/ssd01,size=0.2 ----> 20% of available space`
`--store=path=/mnt/ssd01,size=.2 ----> 20% of available space`
`--store=type=mem,size=20GB`
`--store=type=mem,size=90%`

**Note:** If you use the % notation, you might need to escape the % sign, for instance, while configuring CockroachDB through systemd service files. For this reason, it's recommended to use the decimal notation instead." + }, + { + "field": "ballast-size", + "description": "Configure the size of the automatically created emergency ballast file. Accepts the same value formats as the size field. For more details, see [Automatic ballast files](cluster-setup-troubleshooting.html#automatic-ballast-files). To disable automatic ballast file creation, set the value to 0.

**Example:** `--store=path=/mnt/ssd01,ballast-size=0`" + } + ] + }, + "standard_output": { + "fields": [ + { + "field": "build", + "description": "The version of CockroachDB you are running." + }, + { + "field": "webui", + "description": "The URL for accessing the DB Console." + }, + { + "field": "sql", + "description": "The connection URL for your client." + }, + { + "field": "RPC client flags", + "description": "The flags to use when connecting to the node via cockroach client commands." + }, + { + "field": "logs", + "description": "The directory containing debug log data." + }, + { + "field": "temp dir", + "description": "The temporary store directory of the node." + }, + { + "field": "external I/O path", + "description": "The external IO directory with which the local file access paths are prefixed while performing backup and restore operations using local node directories or NFS drives." + }, + { + "field": "attrs", + "description": "If node-level attributes were specified in the --attrs flag, they are listed in this field. These details are potentially useful for configuring replication zones." + }, + { + "field": "locality", + "description": "If values describing the locality of the node were specified in the --locality field, they are listed in this field. These details are potentially useful for configuring replication zones." + }, + { + "field": "store[n]", + "description": "The directory containing store data, where [n] is the index of the store, e.g., store[0] for the first store, store[1] for the second store. If store-level attributes were specified in the attrs field of the --store flag, they are listed in this field as well. These details are potentially useful for configuring replication zones." + }, + { + "field": "status", + "description": "Whether the node is the first in the cluster (initialized new cluster), joined an existing cluster for the first time (initialized new node, joined pre-existing cluster), or rejoined an existing cluster (restarted pre-existing node)." + }, + { + "field": "clusterID", + "description": "The ID of the cluster. When trying to join a node to an existing cluster, if this ID is different than the ID of the existing cluster, the node has started a new cluster. This may be due to conflicting information in the node's data directory. For additional guidance, see the [troubleshooting](common-errors.html#node-belongs-to-cluster-cluster-id-but-is-attempting-to-connect-to-a-gossip-network-for-cluster-another-cluster-id) docs." + }, + { + "field": "nodeID", + "description": "The ID of the node." + } + ] + } + } +] \ No newline at end of file diff --git a/src/current/_includes/v26.3/reference/fields-table.md b/src/current/_includes/v26.3/reference/fields-table.md new file mode 100644 index 00000000000..df0c60c1e14 --- /dev/null +++ b/src/current/_includes/v26.3/reference/fields-table.md @@ -0,0 +1,29 @@ +{% comment %} + This include generates a fields table (not flags) for cockroach commands. + Used for Store fields and Standard Output fields. + + Required parameters: + - fields: Array of field objects from the JSON data + + Each field object should have: + - field (required): The field name + - description (required): Full description including all metadata + - deprecated (optional): Boolean indicating if deprecated + + Sorting: Non-deprecated fields first (alphabetically), then deprecated fields (alphabetically) +{% endcomment %} + +{% assign non_deprecated_fields = include.fields | where_exp: "item", "item.deprecated != true" | sort: "field" %} +{% assign deprecated_fields = include.fields | where: "deprecated", true | sort: "field" %} +{% assign sorted_fields = non_deprecated_fields | concat: deprecated_fields %} + +Field | Description +-----|------------ +{% for field_item in sorted_fields -%} +{%- assign field_id = field_item.field | downcase | replace: ' ', '-' | replace: '[', '' | replace: ']', '' -%} +{% if field_item.deprecated -%} +`{{ field_item.field }}` | **Deprecated.** {{ field_item.description }} +{% else -%} +`{{ field_item.field }}` | {{ field_item.description }} +{% endif -%} +{% endfor %} \ No newline at end of file diff --git a/src/current/_includes/v26.3/reference/flags-table.md b/src/current/_includes/v26.3/reference/flags-table.md new file mode 100644 index 00000000000..80dd3745eb5 --- /dev/null +++ b/src/current/_includes/v26.3/reference/flags-table.md @@ -0,0 +1,37 @@ +{% comment %} + This include generates a flags table for cockroach commands. + + Required parameters: + - flags: Array of flag objects from the JSON data + + Each flag object should have: + - flag (required): The flag name (--flag) or field name (for store fields) + - short (optional): Short version of the flag (-f) + - description (required): Full description including all metadata + - deprecated (optional): Boolean indicating if deprecated + + Sorting: Non-deprecated flags first (alphabetically), then deprecated flags (alphabetically) +{% endcomment %} + +{% assign non_deprecated_flags = include.flags | where_exp: "item", "item.deprecated != true" | sort: "flag" %} +{% assign deprecated_flags = include.flags | where: "deprecated", true | sort: "flag" %} +{% assign sorted_flags = non_deprecated_flags | concat: deprecated_flags %} + +Flag | Description +-----|------------ +{% for flag_item in sorted_flags -%} +{%- assign flag_id = flag_item.flag | remove: '--' -%} +{% if flag_item.deprecated -%} +{% if flag_item.short -%} +`{{ flag_item.flag }}`
`{{ flag_item.short }}` | **Deprecated.** {{ flag_item.description }} +{% else -%} +`{{ flag_item.flag }}` | **Deprecated.** {{ flag_item.description }} +{% endif -%} +{% else -%} +{% if flag_item.short -%} +`{{ flag_item.flag }}`
`{{ flag_item.short }}` | {{ flag_item.description }} +{% else -%} +`{{ flag_item.flag }}` | {{ flag_item.description }} +{% endif -%} +{% endif -%} +{% endfor %} \ No newline at end of file diff --git a/src/current/v26.3/authentication.md b/src/current/v26.3/authentication.md index 202cd88867d..7e0a2f710bb 100644 --- a/src/current/v26.3/authentication.md +++ b/src/current/v26.3/authentication.md @@ -313,7 +313,7 @@ CockroachDB supports the [TLS 1.3 and TLS 1.2](https://wikipedia.org/wiki/Transp By default, CockroachDB negotiates TLS 1.3 for client connections when supported by the client. If the client does not support TLS 1.3, the connection will fall back to TLS 1.2. -To restrict your cluster to only allow connections using specific cipher suites, use the [`--tls-cipher-suites`]({% link {{ page.version.version }}/cockroach-start.md %}#tls-cipher-suites) startup flag with the `cockroach start` command. This setting applies to all incoming SQL, RPC, and HTTP connections. +To restrict your cluster to only allow connections using specific cipher suites, use the [`--tls-cipher-suites`]({% link {{ page.version.version }}/cockroach-start.md %}#flags-tls-cipher-suites) startup flag with the `cockroach start` command. This setting applies to all incoming SQL, RPC, and HTTP connections. The following cipher suites are enabled by default: diff --git a/src/current/v26.3/cockroach-init.md b/src/current/v26.3/cockroach-init.md index 6c88a0573e6..0e1c5e8f5c2 100644 --- a/src/current/v26.3/cockroach-init.md +++ b/src/current/v26.3/cockroach-init.md @@ -6,6 +6,9 @@ key: initialize-a-cluster.html docs_area: reference.cli --- +{% assign version = page.version.version | replace: ".", "" %} +{% assign init_cmd = site.data[version]["cockroach-commands"] | where: "command_id", "cockroach_init" | first %} + This page explains the `cockroach init` [command]({% link {{ page.version.version }}/cockroach-commands.md %}), which you use to perform a one-time initialization of a new multi-node cluster. For a full tutorial of the cluster startup and initialization process, see one of the [Manual Deployment]({% link {{ page.version.version }}/manual-deployment.md %}) tutorials. {{site.data.alerts.callout_info}} @@ -18,14 +21,14 @@ Perform a one-time initialization of a cluster: {% include_cached copy-clipboard.html %} ~~~ shell -$ cockroach init +{{ init_cmd.synopsis.initialize }} ~~~ View help: {% include_cached copy-clipboard.html %} ~~~ shell -$ cockroach init --help +{{ init_cmd.synopsis.view_help }} ~~~ ## Flags @@ -38,9 +41,7 @@ The `cockroach init` command supports the following [client connection](#client- ### Client connection -{% include {{ page.version.version }}/sql/connection-parameters.md %} -`--cluster-name` | The cluster name to use to verify the cluster's identity. If the cluster has a cluster name, you must include this flag. For more information, see [`cockroach start`]({% link {{ page.version.version }}/cockroach-start.md %}#general). -`--disable-cluster-name-verification` | Disables the cluster name check for this command. This flag must be paired with `--cluster-name`. For more information, see [`cockroach start`]({% link {{ page.version.version }}/cockroach-start.md %}#general). +{% include {{ page.version.version }}/reference/flags-table.md flags=init_cmd.flags.client_connection %} See [Client Connection Parameters]({% link {{ page.version.version }}/connection-parameters.md %}) for details. @@ -128,4 +129,4 @@ Usage of `cockroach init` assumes that nodes have already been started with [`co - [Orchestrated Deployment]({% link {{ page.version.version }}/kubernetes-overview.md %}) - [Local Deployment]({% link {{ page.version.version }}/start-a-local-cluster.md %}) - [`cockroach start`]({% link {{ page.version.version }}/cockroach-start.md %}) -- [`cockroach` Commands Overview]({% link {{ page.version.version }}/cockroach-commands.md %}) +- [`cockroach` Commands Overview]({% link {{ page.version.version }}/cockroach-commands.md %}) \ No newline at end of file diff --git a/src/current/v26.3/cockroach-init.md.original b/src/current/v26.3/cockroach-init.md.original new file mode 100644 index 00000000000..6c88a0573e6 --- /dev/null +++ b/src/current/v26.3/cockroach-init.md.original @@ -0,0 +1,131 @@ +--- +title: cockroach init +summary: Perform a one-time-only initialization of a CockroachDB cluster. +toc: true +key: initialize-a-cluster.html +docs_area: reference.cli +--- + +This page explains the `cockroach init` [command]({% link {{ page.version.version }}/cockroach-commands.md %}), which you use to perform a one-time initialization of a new multi-node cluster. For a full tutorial of the cluster startup and initialization process, see one of the [Manual Deployment]({% link {{ page.version.version }}/manual-deployment.md %}) tutorials. + +{{site.data.alerts.callout_info}} +When starting a single-node cluster with [`cockroach start-single-node`]({% link {{ page.version.version }}/cockroach-start-single-node.md %}), you do not need to use the `cockroach init` command. +{{site.data.alerts.end}} + +## Synopsis + +Perform a one-time initialization of a cluster: + +{% include_cached copy-clipboard.html %} +~~~ shell +$ cockroach init +~~~ + +View help: + +{% include_cached copy-clipboard.html %} +~~~ shell +$ cockroach init --help +~~~ + +## Flags + +The `cockroach init` command supports the following [client connection](#client-connection) and [logging](#logging) flags. + +{{site.data.alerts.callout_info}} +`cockroach init` must target one of the nodes that was listed with [`--join`]({% link {{ page.version.version }}/cockroach-start.md %}#networking) when starting the cluster. Otherwise, the command will not initialize the cluster correctly. +{{site.data.alerts.end}} + +### Client connection + +{% include {{ page.version.version }}/sql/connection-parameters.md %} +`--cluster-name` | The cluster name to use to verify the cluster's identity. If the cluster has a cluster name, you must include this flag. For more information, see [`cockroach start`]({% link {{ page.version.version }}/cockroach-start.md %}#general). +`--disable-cluster-name-verification` | Disables the cluster name check for this command. This flag must be paired with `--cluster-name`. For more information, see [`cockroach start`]({% link {{ page.version.version }}/cockroach-start.md %}#general). + +See [Client Connection Parameters]({% link {{ page.version.version }}/connection-parameters.md %}) for details. + +### Logging + +{% include {{ page.version.version }}/misc/logging-defaults.md %} + +## Examples + +Usage of `cockroach init` assumes that nodes have already been started with [`cockroach start`]({% link {{ page.version.version }}/cockroach-start.md %}) and are waiting to be initialized as a new cluster. For a more detailed tutorial, see one of the [Manual Deployment]({% link {{ page.version.version }}/manual-deployment.md %}) tutorials. + +### Initialize a Cluster on a Node's Machine + +
+ + +
+ +
+1. SSH to the machine where the node has been started. This must be a node that was listed with [`--join`]({% link {{ page.version.version }}/cockroach-start.md %}#networking) when starting the cluster. + +1. Make sure the `client.root.crt` and `client.root.key` files for the `root` user are on the machine. + +1. Run the `cockroach init` command with the `--certs-dir` flag set to the directory containing the `ca.crt` file and the files for the `root` user, and with the `--host` flag set to the address of the current node: + + {% include_cached copy-clipboard.html %} + ~~~ shell + $ cockroach init --certs-dir=certs --host=
+ ~~~ + + At this point, all the nodes complete startup and print helpful details to the [standard output]({% link {{ page.version.version }}/cockroach-start.md %}#standard-output), such as the CockroachDB version, the URL for the DB Console, and the SQL URL for clients. +
+ +
+1. SSH to the machine where the node has been started. This must be a node that was listed with [`--join`]({% link {{ page.version.version }}/cockroach-start.md %}#networking) when starting the cluster. + +1. Run the `cockroach init` command with the `--host` flag set to the address of the current node: + + {% include_cached copy-clipboard.html %} + ~~~ shell + $ cockroach init --insecure --host=
+ ~~~ + + At this point, all the nodes complete startup and print helpful details to the [standard output]({% link {{ page.version.version }}/cockroach-start.md %}#standard-output), such as the CockroachDB version, the URL for the DB Console, and the SQL URL for clients. +
+ +### Initialize a cluster from another machine + +
+ + +
+ +
+1. [Install the `cockroach` binary]({% link {{ page.version.version }}/install-cockroachdb.md %}) on a machine separate from the node. + +1. Create a `certs` directory and copy the CA certificate and the client certificate and key for the `root` user into the directory. + +1. Run the `cockroach init` command with the `--certs-dir` flag set to the directory containing the `ca.crt` file and the files for the `root` user, and with the `--host` flag set to the address of the node. This must be a node that was listed with [`--join`]({% link {{ page.version.version }}/cockroach-start.md %}#networking) when starting the cluster: + + {% include_cached copy-clipboard.html %} + ~~~ shell + $ cockroach init --certs-dir=certs --host=
+ ~~~ + + At this point, all the nodes complete startup and print helpful details to the [standard output]({% link {{ page.version.version }}/cockroach-start.md %}#standard-output), such as the CockroachDB version, the URL for the DB Console, and the SQL URL for clients. +
+ +
+1. [Install the `cockroach` binary]({% link {{ page.version.version }}/install-cockroachdb.md %}) on a machine separate from the node. + +1. Run the `cockroach init` command with the `--host` flag set to the address of the node. This must be a node that was listed with [`--join`]({% link {{ page.version.version }}/cockroach-start.md %}#networking) when starting the cluster: + + {% include_cached copy-clipboard.html %} + ~~~ shell + $ cockroach init --insecure --host=
+ ~~~ + + At this point, all the nodes complete startup and print helpful details to the [standard output]({% link {{ page.version.version }}/cockroach-start.md %}#standard-output), such as the CockroachDB version, the URL for the DB Console, and the SQL URL for clients. +
+ +## See also + +- [Manual Deployment]({% link {{ page.version.version }}/manual-deployment.md %}) +- [Orchestrated Deployment]({% link {{ page.version.version }}/kubernetes-overview.md %}) +- [Local Deployment]({% link {{ page.version.version }}/start-a-local-cluster.md %}) +- [`cockroach start`]({% link {{ page.version.version }}/cockroach-start.md %}) +- [`cockroach` Commands Overview]({% link {{ page.version.version }}/cockroach-commands.md %}) diff --git a/src/current/v26.3/cockroach-start-prose-to-move.md b/src/current/v26.3/cockroach-start-prose-to-move.md new file mode 100644 index 00000000000..659e4c0aec8 --- /dev/null +++ b/src/current/v26.3/cockroach-start-prose-to-move.md @@ -0,0 +1,83 @@ +# Content to Move from cockroach-start.md + +This document describes all of the conceptual/how-to documentation from the original `cockroach start` page that should be moved to other parts of the documentation. + +## From Flags Section + +### Introductory text about flags: + +The `cockroach start` command supports the following [general-use](#general), [networking](#networking), [security](#security), and [logging](#logging) flags. + +Many flags have useful defaults that can be overridden by specifying the flags explicitly. If you specify flags explicitly, however, be sure to do so each time the node is restarted, as they will not be remembered. The one exception is the `--join` flag, which is stored in a node's data directory. We still recommend specifying the `--join` flag every time, as this will allow nodes to rejoin the cluster even if their data directory was destroyed. + +## Locality Section + +The `--locality` flag accepts arbitrary key-value pairs that describe the location of the node. Locality should include a `region` key-value if you are using CockroachDB's [Multi-region SQL capabilities]({% link {{ page.version.version }}/multiregion-overview.md %}). + +Depending on your deployment you can also specify country, availability zone, etc. The key-value pairs should be ordered into _locality tiers_ from most inclusive to least inclusive (e.g., region before availability zone as in `region=eu-west-1,zone=eu-west-1a`), and the keys and order of key-value pairs must be the same on all nodes. It's typically better to include more pairs than fewer. + +- Specifying a region with a `region` tier is required in order to enable CockroachDB's [multi-region capabilities]({% link {{ page.version.version }}/multiregion-overview.md %}). + +- CockroachDB spreads the replicas of each piece of data across as diverse a set of localities as possible, with the order determining the priority. Locality can also be used to influence the location of data replicas in various ways using high-level [multi-region SQL capabilities]({% link {{ page.version.version }}/multiregion-overview.md %}) or low-level [replication zones]({% link {{ page.version.version }}/configure-replication-zones.md %}#replication-constraints). + +- When there is high latency between nodes (e.g., cross-availability zone deployments), CockroachDB uses locality to move range leases closer to the current workload, reducing network round trips and improving read performance, also known as ["follow-the-workload"]({% link {{ page.version.version }}/topology-follow-the-workload.md %}). In a deployment across more than 3 availability zones, however, to ensure that all data benefits from "follow-the-workload", you must increase your replication factor to match the total number of availability zones. + +- Locality is also a prerequisite for using the [Multi-region SQL abstractions]({% link {{ page.version.version }}/multiregion-overview.md %}), [table partitioning]({% link {{ page.version.version }}/partitioning.md %}), and [**Node Map**]({% link {{ page.version.version }}/enable-node-map.md %}) {{site.data.products.enterprise}} features. + +### Locality Example + +[Lines 84-145 containing the multi-region locality example with shell commands] + +### Load-based lease rebalancing in uneven latency deployments + +[Lines 147-159 containing information about lease rebalancing] + +## Storage Section Prose + +### Storage engine + +The `--storage-engine` flag is used to choose the storage engine used by the node. Note that this setting applies to all [stores](#store) on the node, including the [temp store](#flags-temp-dir). + +As of v21.1 and later, CockroachDB always uses the [Pebble storage engine]({% link {{ page.version.version }}/architecture/storage-layer.md %}#pebble). As such, `pebble` is the default and only option for the `--storage-engine` flag. + +### Store introductory text + +The `--store` flag allows you to specify details about a node's storage. + +To start a node with multiple disks or SSDs, provide a separate `--store` flag for each disk when starting the `cockroach` process on the node. For more details about stores, see [Start a Node]({% link {{ page.version.version }}/cockroach-start.md %}#store). + +{{site.data.alerts.callout_danger}} +If you start a node with multiple `--store` flags, it is not possible to scale back down to only using a single store on the node. Instead, you must decommission the node and start a new node with the updated `--store`. +{{site.data.alerts.end}} + +{{site.data.alerts.callout_info}} +In-memory storage is not suitable for production deployments at this time. +{{site.data.alerts.end}} + +{{site.data.alerts.callout_success}} +{% include {{ page.version.version }}/prod-deployment/prod-guidance-store-volume.md %} +{{site.data.alerts.end}} + +### Write Ahead Log (WAL) failover + +[Lines 195-233 containing all WAL failover documentation] + +## Logging Section + +By [default]({% link {{ page.version.version }}/configure-logs.md %}#default-logging-configuration), `cockroach start` writes all messages to log files, and prints nothing to `stderr`. This includes events with `INFO` [severity]({% link {{ page.version.version }}/logging.md %}#logging-levels-severities) and higher. However, you can [customize the logging behavior]({% link {{ page.version.version }}/configure-logs.md %}) of this command by using the `--log` flag: + +{% include {{ page.version.version }}/misc/logging-flags.md %} + +### Defaults + +See the [default logging configuration]({% link {{ page.version.version }}/configure-logs.md %}#default-logging-configuration). + +## Examples Section (All to be moved) + +[Lines 271-657 containing all examples] +- Start a multi-node cluster +- Start a multi-region cluster +- Start a multi-node cluster across private networks +- Add a node to a cluster +- Create a table with node locality information +- Start a cluster with separate RPC and SQL networks \ No newline at end of file diff --git a/src/current/v26.3/cockroach-start-single-node.md b/src/current/v26.3/cockroach-start-single-node.md index 58949bdc062..0dd07b33929 100644 --- a/src/current/v26.3/cockroach-start-single-node.md +++ b/src/current/v26.3/cockroach-start-single-node.md @@ -5,6 +5,9 @@ toc: true docs_area: reference.cli --- +{% assign version = page.version.version | replace: ".", "" %} +{% assign single_node_cmd = site.data[version]["cockroach-commands"] | where: "command_id", "cockroach_start_single_node" | first %} + This page explains the `cockroach start-single-node` [command]({% link {{ page.version.version }}/cockroach-commands.md %}), which you use to start a single-node cluster with replication disabled. A single-node cluster is appropriate for quick SQL testing or app development. {{site.data.alerts.callout_danger}} @@ -16,13 +19,13 @@ A single-node cluster is not appropriate for use in production or for performanc Start a single-node cluster: ~~~ shell -$ cockroach start-single-node +{{ single_node_cmd.synopsis.start }} ~~~ View help: ~~~ shell -$ cockroach start-single-node --help +{{ single_node_cmd.synopsis.view_help }} ~~~ ## Flags @@ -37,40 +40,15 @@ The `cockroach start-single-node` flags are identical to [`cockroach start`]({% ### General -Flag | Description ------|----------- -`--attrs` | **Not relevant for single-node clusters.** Arbitrary strings, separated by colons, specifying node capability, which might include specialized hardware or number of cores, for example:

`--attrs=ram:64gb`

These can be used to influence the location of data replicas. See [Replication Controls]({% link {{ page.version.version }}/configure-replication-zones.md %}#replication-constraints) for full details. -`--background` | Runs the node in the background. Control is returned to the shell only once the node is ready to accept requests, so this is recommended over appending `&` to the command. This flag is **not** available in Windows environments.

**Note:** `--background` is suitable for writing automated test suites or maintenance procedures that need a temporary server process running in the background. It is not intended to be used to start a long-running server, because it does not fully detach from the controlling terminal. Consider using a service manager or a tool like [daemon(8)](https://www.freebsd.org/cgi/man.cgi?query=daemon&sektion=8) instead. If you use `--background`, using `--pid-file` is also recommended. To gracefully stop the `cockroach` process, ssend the `SIGTERM` signal to the process ID in the PID file. To gracefully restart the process, send the `SIGHUP` signal. -`--cache` | The total size for the [Pebble storage engine]({% link {{ page.version.version }}/architecture/storage-layer.md %}#pebble) block cache, shared evenly if there are multiple storage devices. This can be a percentage (notated as a decimal or with `%`) or any bytes-based unit, for example:

`--cache=.25`
`--cache=25%`
`--cache=1000000000 ----> 1000000000 bytes`
`--cache=1GB ----> 1000000000 bytes`
`--cache=1GiB ----> 1073741824 bytes`

Note: If you use the `%` notation, you might need to escape the `%` sign when configuring CockroachDB through `systemd` service files. For this reason, it's recommended to use the decimal notation instead.

**Note:** The sum of `--cache`, `--max-sql-memory`, and `--max-tsdb-memory` should not exceed 75% of the memory available to the `cockroach` process.

**Default:** `256 MiB`

The default cache size is reasonable for local development clusters. For production deployments, set this to 25% or higher. Increasing the cache size generally improves the node's read performance.

The block cache holds uncompressed blocks of persisted [key-value data]({% link {{ page.version.version }}/architecture/distribution-layer.md %}#overview) in memory. If a read misses within the block cache, the storage engine reads the file via the operating system's page cache, which may hold the relevant block in-memory in its compressed form. Otherwise, the read is served from the storage device. The block cache fills to the configured size and is then recycled using a least-recently-used (LRU) policy.

Refer to [Recommended Production Settings]({% link {{ page.version.version }}/recommended-production-settings.md %}#cache-and-sql-memory-size) for more details. Production systems should always configure this setting. -`--external-io-dir` | The path of the external IO directory with which the local file access paths are prefixed while performing backup and restore operations using local node directories or NFS drives. If set to `disabled`, backups and restores using local node directories and NFS drives are disabled.

**Default:** `extern` subdirectory of the first configured [`store`](#store).

To set the `--external-io-dir` flag to the locations you want to use without needing to restart nodes, create symlinks to the desired locations from within the `extern` directory. -`--listening-url-file` | The file to which the node's SQL connection URL will be written on successful startup, in addition to being printed to the [standard output](#standard-output).

This is particularly helpful in identifying the node's port when an unused port is assigned automatically (`--port=0`). -`--locality` | **Not relevant for single-node clusters.** Arbitrary key-value pairs that describe the location of the node. Locality might include country, region, datacenter, rack, etc. For more details, see [Locality]({% link {{ page.version.version }}/cockroach-start.md %}#locality) below. -`--locality-file` | A file that contains arbitrary key-value pairs that describe the location of the node, as an alternative to the `--locality` flag. -`--max-disk-temp-storage` | The maximum on-disk storage capacity available to store temporary data for SQL queries that exceed the memory budget (see `--max-sql-memory`). This ensures that JOINs, sorts, and other memory-intensive SQL operations are able to spill intermediate results to disk. This can be a percentage (notated as a decimal or with `%`) or any bytes-based unit (e.g., `.25`, `25%`, `500GB`, `1TB`, `1TiB`).

Note: If you use the `%` notation, you might need to escape the `%` sign, for instance, while configuring CockroachDB through `systemd` service files. For this reason, it's recommended to use the decimal notation instead. Also, if expressed as a percentage, this value is interpreted relative to the size of the first store. However, the temporary space usage is never counted towards any store usage; therefore, when setting this value, it's important to ensure that the size of this temporary storage plus the size of the first store doesn't exceed the capacity of the storage device.

The temporary files are located in the path specified by the `--temp-dir` flag, or in the subdirectory of the first store (see `--store`) by default.

**Default:** `32GiB` -`--max-go-memory` | The maximum soft memory limit for the Go runtime, which influences the behavior of Go's garbage collection. Defaults to `--max-sql-memory x 2.25`, but cannot exceed 90% of the node's available RAM. To disable the soft memory limit, set `--max-go-memory` to `0` (not recommended). -`--max-sql-memory` | The maximum in-memory storage capacity available to store temporary data for SQL queries, including prepared queries and intermediate data rows during query execution. This can be a percentage (notated as a decimal or with `%`) or any bytes-based unit; for example:

`--max-sql-memory=.25`
`--max-sql-memory=25%`
`--max-sql-memory=10000000000 ----> 1000000000 bytes`
`--max-sql-memory=1GB ----> 1000000000 bytes`
`--max-sql-memory=1GiB ----> 1073741824 bytes`

The temporary files are located in the path specified by the `--temp-dir` flag, or in the subdirectory of the first store (see `--store`) by default.

**Note:** If you use the `%` notation, you might need to escape the `%` sign (for instance, while configuring CockroachDB through `systemd` service files). For this reason, it's recommended to use the decimal notation instead.

**Note:** The sum of `--cache`, `--max-sql-memory`, and `--max-tsdb-memory` should not exceed 75% of the memory available to the `cockroach` process.

**Default:** `25%`

The default SQL memory size is suitable for production deployments but can be raised to increase the number of simultaneous client connections the node allows as well as the node's capacity for in-memory processing of rows when using `ORDER BY`, `GROUP BY`, `DISTINCT`, joins, and window functions. For local development clusters with memory-intensive workloads, reduce this value to, for example, `128MiB` to prevent [out-of-memory errors]({% link {{ page.version.version }}/cluster-setup-troubleshooting.md %}#out-of-memory-oom-crash). -`--max-tsdb-memory` | Maximum memory capacity available to store temporary data for use by the time-series database to display metrics in the [DB Console]({% link {{ page.version.version }}/ui-overview.md %}). Consider raising this value if your cluster is comprised of a large number of nodes where individual nodes have very limited memory available (e.g., under `8 GiB`). Insufficient memory capacity for the time-series database can constrain the ability of the DB Console to process the time-series queries used to render metrics for the entire cluster. This capacity constraint does not affect SQL query execution. This flag accepts numbers interpreted as bytes, size suffixes (e.g., `1GB` and `1GiB`) or a percentage of physical memory (e.g., `0.01`).

**Note:** The sum of `--cache`, `--max-sql-memory`, and `--max-tsdb-memory` should not exceed 75% of the memory available to the `cockroach` process.

**Default:** `0.01` (i.e., 1%) of physical memory or `64 MiB`, whichever is greater. -`--pid-file` | The file to which the node's process ID will be written on successful startup. When this flag is not set, the process ID is not written to file. -`--store`
`-s` | The file path to a storage device and, optionally, store attributes and maximum size. When using multiple storage devices for a node, this flag must be specified separately for each device, for example:

`--store=/mnt/ssd01 --store=/mnt/ssd02`

For more details, see [Store](#store) below. -`--temp-dir` | The path of the node's temporary store directory. On node start up, the location for the temporary files is printed to the standard output.

**Default:** Subdirectory of the first [store](#store) +{% include {{ page.version.version }}/reference/flags-table.md flags=single_node_cmd.flags.general %} ### Networking -Flag | Description ------|----------- -`--listen-addr` | The IP address/hostname and port to listen on for connections from clients. For IPv6, use the notation `[...]`, e.g., `[::1]` or `[fe80::f6f2:::]`.

**Default:** Listen on all IP addresses on port `26257` -`--http-addr` | The IP address/hostname and port to listen on for DB Console HTTP requests. For IPv6, use the notation `[...]`, e.g., `[::1]:8080` or `[fe80::f6f2:::]:8080`.

**Default:** Listen on the address part of `--listen-addr` on port `8080` -`--socket-dir` | The directory path on which to listen for [Unix domain socket](https://wikipedia.org/wiki/Unix_domain_socket) connections from clients installed on the same Unix-based machine. For an example, see [Connect to a cluster listening for Unix domain socket connections]({% link {{ page.version.version }}/cockroach-sql.md %}#connect-to-a-cluster-listening-for-unix-domain-socket-connections). +{% include {{ page.version.version }}/reference/flags-table.md flags=single_node_cmd.flags.networking %} ### Security -Flag | Description ------|----------- -`--certs-dir` | The path to the [certificate directory]({% link {{ page.version.version }}/cockroach-cert.md %}). The directory must contain valid certificates if running in secure mode.

**Default:** `${HOME}/.cockroach-certs/` -`--insecure` | **Note:** The `--insecure` flag is intended for **non-production testing only**.

Run in insecure mode, skipping all TLS encryption and authentication. If this flag is not set, the `--certs-dir` flag must point to valid certificates.

**Note the following risks:** An insecure cluster is open to any client that can access any node's IP addresses; client connections must also be made insecurely; any user, even `root`, can log in without providing a password; any user, connecting as `root`, can read or write any data in your cluster; there is no network encryption or authentication, and thus no confidentiality.

**Default:** `false` -`--accept-sql-without-tls` | This flag (in [preview]({% link {{ page.version.version }}/cockroachdb-feature-availability.md %})) allows you to connect to the cluster using a SQL user's password without [validating the client's certificate]({% link {{ page.version.version }}/authentication.md %}#client-authentication). When connecting using the built-in SQL client, [use the `--insecure` flag with the `cockroach sql` command]({% link {{ page.version.version }}/cockroach-sql.md %}#client-connection). -`--cert-principal-map` | A comma-separated list of `cert-principal:db-principal` mappings used to map the certificate principals to IP addresses, DNS names, and SQL users. This allows the use of certificates generated by Certificate Authorities that place restrictions on the contents of the `commonName` field. For usage information, see [Create Security Certificates using Openssl]({% link {{ page.version.version }}/create-security-certificates-openssl.md %}#examples). -`--enterprise-encryption` | This optional flag specifies the encryption options for one of the stores on the node. If multiple stores exist, the flag must be specified for each store.

This flag takes a number of options. For a complete list of options, and usage instructions, see [Encryption at Rest]({% link {{ page.version.version }}/encryption.md %}). +{% include {{ page.version.version }}/reference/flags-table.md flags=single_node_cmd.flags.security %} ### Store @@ -80,13 +58,7 @@ The `--store` flag supports the following fields. Note that commas are used to s In-memory storage is not suitable for production deployments at this time. {{site.data.alerts.end}} -Field | Description -------|------------ -`type` | For in-memory storage, set this field to `mem`; otherwise, leave this field out. The `path` field must not be set when `type=mem`. -`path` | The file path to the storage device. When not setting `attr`, `size`, or `ballast-size`, the `path` field label can be left out:

`--store=/mnt/ssd01`

When either of those fields are set, however, the `path` field label must be used:

`--store=path=/mnt/ssd01,size=20GB`

**Default:** `cockroach-data` -`attrs` | Arbitrary strings, separated by colons, specifying disk type or capability. These can be used to influence the location of data replicas. See [Replication Controls]({% link {{ page.version.version }}/configure-replication-zones.md %}#replication-constraints) for full details.

In most cases, node-level `--locality` or `--attrs` are preferable to store-level attributes, but this field can be used to match capabilities for storage of individual databases or tables. For example, an OLTP database would probably want to allocate space for its tables only on solid state devices, whereas append-only time series might prefer cheaper spinning drives. Typical attributes include whether the store is flash (`ssd`) or spinny disk (`hdd`), as well as speeds and other specs, for example:

`--store=path=/mnt/hda1,attrs=hdd:7200rpm` - `size` | The maximum size allocated to the node. When this size is reached, CockroachDB attempts to rebalance data to other nodes with available capacity. When no other nodes have available capacity, this limit will be exceeded. Data may also be written to the node faster than the cluster can rebalance it away; as long as capacity is available elsewhere, CockroachDB will gradually rebalance data down to the store limit.

The `size` can be specified either in a bytes-based unit or as a percentage of hard drive space (notated as a decimal or with `%`), for example:

`--store=path=/mnt/ssd01,size=10000000000 ----> 10000000000 bytes`
`--store=path=/mnt/ssd01,size=20GB ----> 20000000000 bytes`
`--store=path=/mnt/ssd01,size=20GiB ----> 21474836480 bytes`
`--store=path=/mnt/ssd01,size=0.02TiB ----> 21474836480 bytes`
`--store=path=/mnt/ssd01,size=20% ----> 20% of available space`
`--store=path=/mnt/ssd01,size=0.2 ----> 20% of available space`
`--store=path=/mnt/ssd01,size=.2 ----> 20% of available space`

**Default:** 100%

For an in-memory store, the `size` field is required and must be set to the true maximum bytes or percentage of available memory, for example:

`--store=type=mem,size=20GB`
`--store=type=mem,size=90%`

Note: If you use the `%` notation, you might need to escape the `%` sign, for instance, while configuring CockroachDB through `systemd` service files. For this reason, it's recommended to use the decimal notation instead. - `ballast-size` | Configure the size of the automatically created emergency ballast file. Accepts the same value formats as the [`size` field](#store-size). For more details, see [Automatic ballast files]({% link {{ page.version.version }}/cluster-setup-troubleshooting.md %}#automatic-ballast-files).

To disable automatic ballast file creation, set the value to `0`:

`--store=path=/mnt/ssd01,ballast-size=0` +{% include {{ page.version.version }}/reference/fields-table.md fields=single_node_cmd.flags.store %} ### Logging @@ -126,21 +98,7 @@ nodeID: 1 These details are also written to the `INFO` log in the `/logs` directory. You can retrieve them with a command like `grep 'node starting' node1/logs/cockroach.log -A 11`. {{site.data.alerts.end}} -Field | Description -------|------------ -`build` | The version of CockroachDB you are running. -`webui` | The URL for accessing the DB Console. -`sql` | The connection URL for your client. -`RPC client flags` | The flags to use when connecting to the node via [`cockroach` client commands]({% link {{ page.version.version }}/cockroach-commands.md %}). -`logs` | The directory containing debug log data. -`temp dir` | The temporary store directory of the node. -`external I/O path` | The external IO directory with which the local file access paths are prefixed while performing [backup]({% link {{ page.version.version }}/backup.md %}) and [restore]({% link {{ page.version.version }}/restore.md %}) operations using local node directories or NFS drives. -`attrs` | If node-level attributes were specified in the `--attrs` flag, they are listed in this field. These details are potentially useful for [configuring replication zones]({% link {{ page.version.version }}/configure-replication-zones.md %}). -`locality` | If values describing the locality of the node were specified in the `--locality` field, they are listed in this field. These details are potentially useful for [configuring replication zones]({% link {{ page.version.version }}/configure-replication-zones.md %}). -`store[n]` | The directory containing store data, where `[n]` is the index of the store, e.g., `store[0]` for the first store, `store[1]` for the second store.

If store-level attributes were specified in the `attrs` field of the [`--store`](#store) flag, they are listed in this field as well. These details are potentially useful for [configuring replication zones]({% link {{ page.version.version }}/configure-replication-zones.md %}). -`status` | Whether the node is the first in the cluster (`initialized new cluster`), joined an existing cluster for the first time (`initialized new node, joined pre-existing cluster`), or rejoined an existing cluster (`restarted pre-existing node`). -`clusterID` | The ID of the cluster.

When trying to join a node to an existing cluster, if this ID is different than the ID of the existing cluster, the node has started a new cluster. This may be due to conflicting information in the node's data directory. For additional guidance, see the [troubleshooting]({% link {{ page.version.version }}/common-errors.md %}#node-belongs-to-cluster-cluster-id-but-is-attempting-to-connect-to-a-gossip-network-for-cluster-another-cluster-id) docs. -`nodeID` | The ID of the node. +{% include {{ page.version.version }}/reference/fields-table.md fields=single_node_cmd.standard_output.fields %} ## Examples diff --git a/src/current/v26.3/cockroach-start.md b/src/current/v26.3/cockroach-start.md index 89e6e23c4c0..f341617c634 100644 --- a/src/current/v26.3/cockroach-start.md +++ b/src/current/v26.3/cockroach-start.md @@ -6,6 +6,9 @@ key: start-a-node.html docs_area: reference.cli --- +{% assign version = page.version.version | replace: ".", "" %} +{% assign start_cmd = site.data[version]["cockroach-commands"] | where: "command_id", "cockroach_start" | first %} + This page explains the `cockroach start` [command]({% link {{ page.version.version }}/cockroach-commands.md %}), which you use to start a new multi-node cluster or add nodes to an existing cluster. {{site.data.alerts.callout_success}} @@ -23,703 +26,46 @@ Node-level settings are defined by [flags](#flags) passed to the `cockroach star Start a node to be part of a new multi-node cluster: ~~~ shell -$ cockroach start +{{ start_cmd.synopsis.start_new_cluster }} ~~~ Initialize a new multi-node cluster: ~~~ shell -$ cockroach init +{{ start_cmd.synopsis.initialize_cluster }} ~~~ Add a node to an existing cluster: ~~~ shell -$ cockroach start +{{ start_cmd.synopsis.add_node }} ~~~ View help: ~~~ shell -$ cockroach start --help +{{ start_cmd.synopsis.view_help }} ~~~ ## Flags -The `cockroach start` command supports the following [general-use](#general), [networking](#networking), [security](#security), and [logging](#logging) flags. - -Many flags have useful defaults that can be overridden by specifying the flags explicitly. If you specify flags explicitly, however, be sure to do so each time the node is restarted, as they will not be remembered. The one exception is the `--join` flag, which is stored in a node's data directory. We still recommend specifying the `--join` flag every time, as this will allow nodes to rejoin the cluster even if their data directory was destroyed. - -### General - -Flag | Description ------|----------- -`--attrs` | Arbitrary strings, separated by colons, specifying node capability, which might include specialized hardware or number of cores, for example:

`--attrs=ram:64gb`

These can be used to influence the location of data replicas. See [Replication Controls]({% link {{ page.version.version }}/configure-replication-zones.md %}#replication-constraints) for full details. -`--background` | Runs the node in the background. Control is returned to the shell only once the node is ready to accept requests, so this is recommended over appending `&` to the command. This flag is **not** available in Windows environments.

**Note:** `--background` is suitable for writing automated test suites or maintenance procedures that need a temporary server process running in the background. It is not intended to be used to start a long-running server, because it does not fully detach from the controlling terminal. Consider using a service manager or a tool like [daemon(8)](https://www.freebsd.org/cgi/man.cgi?query=daemon&sektion=8) instead. If you use `--background`, using `--pid-file` is also recommended. To gracefully stop the `cockroach` process, send the `SIGTERM` signal to the process ID in the PID file. To gracefully restart the process, send the `SIGHUP` signal. -`--cache` | The total size for the [Pebble storage engine]({% link {{ page.version.version }}/architecture/storage-layer.md %}#pebble) block cache, shared evenly if there are multiple storage devices. This can be a percentage (notated as a decimal or with `%`) or any bytes-based unit, for example:

`--cache=.25`
`--cache=25%`
`--cache=1000000000 ----> 1000000000 bytes`
`--cache=1GB ----> 1000000000 bytes`
`--cache=1GiB ----> 1073741824 bytes`

Note: If you use the `%` notation, you might need to escape the `%` sign when configuring CockroachDB through `systemd` service files. For this reason, it's recommended to use the decimal notation instead.

**Note:** The sum of `--cache`, `--max-sql-memory`, and `--max-tsdb-memory` should not exceed 75% of the memory available to the `cockroach` process.

**Default:** `256 MiB`

The default cache size is reasonable for local development clusters. For production deployments, set this to 25% or higher. Increasing the cache size generally improves the node's read performance.

The block cache holds uncompressed blocks of persisted [key-value data]({% link {{ page.version.version }}/architecture/distribution-layer.md %}#overview) in memory. If a read misses within the block cache, the storage engine reads the file via the operating system's page cache, which may hold the relevant block in-memory in its compressed form. Otherwise, the read is served from the storage device. The block cache fills to the configured size and is then recycled using a least-recently-used (LRU) policy.

Refer to [Recommended Production Settings]({% link {{ page.version.version }}/recommended-production-settings.md %}#cache-and-sql-memory-size) for more details. Production systems should always configure this setting. -`--clock-device` | Enable CockroachDB to use a [PTP hardware clock](https://www.kernel.org/doc/html/latest/driver-api/ptp.html) when querying the current time. The value is a string that specifies the clock device to use. For example: `--clock-device=/dev/ptp0`

**Note:** This is supported on Linux only and may be needed in cases where the host clock is unreliable or prone to large jumps (e.g., when using vMotion). -`--cluster-name` | A string that specifies a cluster name. This is used together with `--join` to ensure that all newly created nodes join the intended cluster when you are running multiple clusters.

**Note:** If this is set, [`cockroach init`]({% link {{ page.version.version }}/cockroach-init.md %}), [`cockroach node decommission`]({% link {{ page.version.version }}/cockroach-node.md %}), [`cockroach node recommission`]({% link {{ page.version.version }}/cockroach-node.md %}), and the `cockroach debug` commands must specify either `--cluster-name` or `--disable-cluster-name-verification` in order to work. -`--disable-cluster-name-verification` | On clusters for which a cluster name has been set, this flag paired with `--cluster-name` disables the cluster name check for the command. This is necessary on existing clusters, when setting a cluster name or changing the cluster name: Perform a rolling restart of all nodes and include both the new `--cluster-name` value and `--disable-cluster-name-verification`, then a second rolling restart with `--cluster-name` and without `--disable-cluster-name-verification`. -`--external-io-dir` | The path of the external IO directory with which the local file access paths are prefixed while performing backup and restore operations using local node directories or NFS drives. If set to `disabled`, backups and restores using local node directories and NFS drives, as well as [`cockroach nodelocal upload`]({% link {{ page.version.version }}/cockroach-nodelocal-upload.md %}), are disabled.

**Default:** `extern` subdirectory of the first configured [`store`](#store).

To set the `--external-io-dir` flag to the locations you want to use without needing to restart nodes, create symlinks to the desired locations from within the `extern` directory. -`--listening-url-file` | The file to which the node's SQL connection URL will be written as soon as the node is ready to accept connections, in addition to being printed to the [standard output](#standard-output). When `--background` is used, this happens before the process detaches from the terminal.

This is particularly helpful in identifying the node's port when an unused port is assigned automatically (`--port=0`). -`--locality` | Arbitrary key-value pairs that describe the location of the node. Locality might include country, region, availability zone, etc. A `region` tier must be included in order to enable [multi-region capabilities]({% link {{ page.version.version }}/multiregion-overview.md %}). To specify locality in a file instead, refer to `--locality-file`. For more details, see [Locality](#locality) below. -`--locality-file` | A file that contains arbitrary key-value pairs that describe the location of the node, as an alternative to the `--locality` flag. -`--max-disk-temp-storage` | The maximum on-disk storage capacity available to store temporary data for SQL queries that exceed the memory budget (see `--max-sql-memory`). This ensures that JOINs, sorts, and other memory-intensive SQL operations are able to spill intermediate results to disk. This can be a percentage (notated as a decimal or with `%`) or any bytes-based unit (e.g., `.25`, `25%`, `500GB`, `1TB`, `1TiB`).

Note: If you use the `%` notation, you might need to escape the `%` sign, for instance, while configuring CockroachDB through `systemd` service files. For this reason, it's recommended to use the decimal notation instead. Also, if expressed as a percentage, this value is interpreted relative to the size of the first store. However, the temporary space usage is never counted towards any store usage; therefore, when setting this value, it's important to ensure that the size of this temporary storage plus the size of the first store doesn't exceed the capacity of the storage device.

The temporary files are located in the path specified by the `--temp-dir` flag, or in the subdirectory of the first store (see `--store`) by default.

**Default:** `32GiB` -`--max-go-memory` | The maximum soft memory limit for the Go runtime, which influences the behavior of Go's garbage collection. Defaults to `--max-sql-memory x 2.25`, but cannot exceed 90% of the node's available RAM. To disable the soft memory limit, set `--max-go-memory` to `0` (not recommended). -`--max-offset` | The maximum allowed clock offset for the cluster. If observed clock offsets exceed this limit, servers will crash to minimize the likelihood of reading inconsistent data. Increasing this value will increase the time to recovery of failures as well as the frequency of uncertainty-based read restarts.

Nodes can run with different values for `--max-offset`, but only for the purpose of updating the setting across the cluster using a rolling upgrade.

**Default:** `500ms` -`--max-sql-memory` | The maximum in-memory storage capacity available to store temporary data for SQL queries, including prepared queries and intermediate data rows during query execution. This can be a percentage (notated as a decimal or with `%`) or any bytes-based unit; for example:

`--max-sql-memory=.25`
`--max-sql-memory=25%`
`--max-sql-memory=10000000000 ----> 1000000000 bytes`
`--max-sql-memory=1GB ----> 1000000000 bytes`
`--max-sql-memory=1GiB ----> 1073741824 bytes`

The temporary files are located in the path specified by the `--temp-dir` flag, or in the subdirectory of the first store (see `--store`) by default.

**Note:** If you use the `%` notation, you might need to escape the `%` sign (for instance, while configuring CockroachDB through `systemd` service files). For this reason, it's recommended to use the decimal notation instead.

**Note:** The sum of `--cache`, `--max-sql-memory`, and `--max-tsdb-memory` should not exceed 75% of the memory available to the `cockroach` process.

**Default:** `25%`

The default SQL memory size is suitable for production deployments but can be raised to increase the number of simultaneous client connections the node allows as well as the node's capacity for in-memory processing of rows when using `ORDER BY`, `GROUP BY`, `DISTINCT`, joins, and window functions. For local development clusters with memory-intensive workloads, reduce this value to, for example, `128MiB` to prevent [out-of-memory errors]({% link {{ page.version.version }}/cluster-setup-troubleshooting.md %}#out-of-memory-oom-crash). -`--max-tsdb-memory` | Maximum memory capacity available to store temporary data for use by the time-series database to display metrics in the [DB Console]({% link {{ page.version.version }}/ui-overview.md %}). Consider raising this value if your cluster is comprised of a large number of nodes where individual nodes have very limited memory available (e.g., under `8 GiB`). Insufficient memory capacity for the time-series database can constrain the ability of the DB Console to process the time-series queries used to render metrics for the entire cluster. This capacity constraint does not affect SQL query execution. This flag accepts numbers interpreted as bytes, size suffixes (e.g., `1GB` and `1GiB`) or a percentage of physical memory (e.g., `0.01`).

**Note:** The sum of `--cache`, `--max-sql-memory`, and `--max-tsdb-memory` should not exceed 75% of the memory available to the `cockroach` process.

**Default:** `0.01` (i.e., 1%) of physical memory or `64 MiB`, whichever is greater. -`--pid-file` | The file to which the node's process ID will be written as soon as the node is ready to accept connections. When `--background` is used, this happens before the process detaches from the terminal. When this flag is not set, the process ID is not written to file. - `--store`
`-s` | The file path to a storage device and, optionally, store attributes and maximum size. When using multiple storage devices for a node, this flag must be specified separately for each device, for example:

`--store=/mnt/ssd01 --store=/mnt/ssd02`

For more details, see [Store](#store) below. -`--wal-failover` | Used to configure [WAL failover](#write-ahead-log-wal-failover) on [nodes]({% link {{ page.version.version }}/architecture/overview.md %}#node) with [multiple stores](#store). To enable WAL failover, pass `--wal-failover=among-stores`. To disable, pass `--wal-failover=disabled` on [node restart]({% link {{ page.version.version }}/node-shutdown.md %}#stop-and-restart-a-node). -`--spatial-libs` | The location on disk where CockroachDB looks for [spatial]({% link {{ page.version.version }}/spatial-data-overview.md %}) libraries.

**Defaults:**
  • `/usr/local/lib/cockroach`
  • A `lib` subdirectory of the CockroachDB binary's current directory.
-`--temp-dir` | The path of the node's temporary store directory. On node start up, the location for the temporary files is printed to the standard output.

**Default:** Subdirectory of the first [store](#store) - -### Networking - -Flag | Description ------|----------- -`--experimental-dns-srv` | When this flag is included, the node will first attempt to fetch SRV records from DNS for every name specified with `--join`. If a valid SRV record is found, that information is used instead of regular DNS A/AAAA lookups. This feature is experimental and may be removed or modified in a later version. -`--listen-addr` | The IP address/hostname and port to listen on for connections from other nodes and clients. For IPv6, use the notation `[...]`, e.g., `[::1]` or `[fe80::f6f2:::]`.

This flag's effect depends on how it is used in combination with `--advertise-addr`. For example, the node will also advertise itself to other nodes using this value if `--advertise-addr` is not specified. For more details, see [Networking]({% link {{ page.version.version }}/recommended-production-settings.md %}#networking).

**Default:** Listen on all IP addresses on port `26257`; if `--advertise-addr` is not specified, also advertise the node's canonical hostname to other nodes -`--advertise-addr` | The IP address/hostname and port to tell other nodes to use. If using a hostname, it must be resolvable from all nodes. If using an IP address, it must be routable from all nodes; for IPv6, use the notation `[...]`, e.g., `[::1]` or `[fe80::f6f2:::]`.

This flag's effect depends on how it is used in combination with `--listen-addr`. For example, if the port number is different than the one used in `--listen-addr`, port forwarding is required. For more details, see [Networking]({% link {{ page.version.version }}/recommended-production-settings.md %}#networking).

**Default:** The value of `--listen-addr`; if `--listen-addr` is not specified, advertises the node's canonical hostname and port `26257` -`--advertise-http-addr` | The IP address/hostname and port to advertise for DB Console HTTP requests when [proxying connections to the DB Console]({% link {{ page.version.version }}/ui-overview.md %}#proxy-db-console). If omitted, the hostname is inherited from the operating system hostname or the hostname from `--advertise-addr`. If the port is omitted, it defaults to `8080` and is never inherited from `--advertise-addr`. -`--http-addr` | The IP address/hostname and port on which to listen for DB Console HTTP requests. For IPv6, use the notation `[...]`, e.g., `[::1]:8080` or `[fe80::f6f2:::]:8080`.

**Default:** Listen on the address specified in `--listen-addr` and on port `8080` -`--locality-advertise-addr` | The IP address/hostname and port to tell other nodes in specific localities to use to connect to this node. This flag is useful when running a cluster across multiple networks, where nodes in a given network have access to a private or local interface while nodes outside the network do not. In this case, you can use `--locality-advertise-addr` to tell nodes within the same network to prefer the private or local address to improve performance and use `--advertise-addr` to tell nodes outside the network to use another address that is reachable from them. However, **do not** include addresses or hostnames that do not resolve to this node, because this will cause connection failures when other nodes attempt to connect to this node.

This flag relies on nodes being started with the [`--locality`](#locality) flag and uses the `locality@address` notation, for example:

`--locality-advertise-addr=region=us-west@10.0.0.0:26257`

For more details, refer to the [Start a multi-node cluster across private networks](#start-a-multi-node-cluster-across-private-networks) example. -`--sql-addr` | The IP address/hostname and port on which to listen for SQL connections from clients. For IPv6, use the notation `[...]`, e.g., `[::1]` or `[fe80::f6f2:::]`.

This flag's effect depends on how it is used in combination with `--advertise-sql-addr`. For example, the node will also advertise itself to clients using this value if `--advertise-sql-addr` is not specified.

**Default:** The value of `--listen-addr`; if `--listen-addr` is not specified, advertises the node's canonical hostname and port `26257`

For an example, see [Start a cluster with separate RPC and SQL networks](#start-a-cluster-with-separate-rpc-and-sql-networks) -`--advertise-sql-addr` | The IP address/hostname and port to tell clients to use. If using a hostname, it must be resolvable from all nodes. If using an IP address, it must be routable from all nodes; for IPv6, use the notation `[...]`, e.g., `[::1]` or `[fe80::f6f2:::]`.

This flag's effect depends on how it is used in combination with `--sql-addr`. For example, if the port number is different than the one used in `--sql-addr`, port forwarding is required.

**Default:** The value of `--sql-addr`; if `--sql-addr` is not specified, advertises the value of `--listen-addr` -`--join`
`-j` | The host addresses that connect nodes to the cluster and distribute the rest of the node addresses. These can be IP addresses or DNS aliases of nodes.

When starting a cluster in a single region, specify the addresses of 3-5 initial nodes. When starting a cluster in multiple regions, specify more than 1 address per region, and select nodes that are spread across failure domains. Then run the [`cockroach init`]({% link {{ page.version.version }}/cockroach-init.md %}) command against any of these nodes to complete cluster startup. See the [example](#start-a-multi-node-cluster) below for more details.

Use the same `--join` list for all nodes to ensure that the cluster can stabilize. Do not list every node in the cluster, because this increases the time for a new cluster to stabilize. Note that these are best practices; it is not required to restart an existing node to update its `--join` flag.

`cockroach start` must be run with the `--join` flag. To start a single-node cluster, use `cockroach start-single-node` instead. -`--socket-dir` | The directory path on which to listen for [Unix domain socket](https://wikipedia.org/wiki/Unix_domain_socket) connections from clients installed on the same Unix-based machine. For an example, see [Connect to a cluster listening for Unix domain socket connections]({% link {{ page.version.version }}/cockroach-sql.md %}#connect-to-a-cluster-listening-for-unix-domain-socket-connections). -`--advertise-host` | **Deprecated.** Use `--advertise-addr` instead. -`--host` | **Deprecated.** Use `--listen-addr` instead. -`--port`
`-p` | **Deprecated.** Specify port in `--advertise-addr` and/or `--listen-addr` instead. -`--http-host` | **Deprecated.** Use `--http-addr` instead. -`--http-port` | **Deprecated.** Specify port in `--http-addr` instead. - -### Security - -Flag | Description ------|----------- -`--certs-dir` | The path to the [certificate directory]({% link {{ page.version.version }}/cockroach-cert.md %}). The directory must contain valid certificates if running in secure mode.

**Default:** `${HOME}/.cockroach-certs/` -`--insecure` | **Note:** The `--insecure` flag is intended for **non-production testing only**.

Run in insecure mode, skipping all TLS encryption and authentication. If this flag is not set, the `--certs-dir` flag must point to valid certificates.

**Note the following risks:** An insecure cluster is open to any client that can access any node's IP addresses; client connections must also be made insecurely; any user, even `root`, can log in without providing a password; any user, connecting as `root`, can read or write any data in your cluster; there is no network encryption or authentication, and thus no confidentiality.

**Default:** `false` -`--accept-sql-without-tls` | This flag (in [preview]({% link {{ page.version.version }}/cockroachdb-feature-availability.md %})) allows you to connect to the cluster using a SQL user's password without [validating the client's certificate]({% link {{ page.version.version }}/authentication.md %}#client-authentication). When connecting using the built-in SQL client, [use the `--insecure` flag with the `cockroach sql` command]({% link {{ page.version.version }}/cockroach-sql.md %}#client-connection). -`--cert-principal-map` | A comma-separated list of `cert-principal:db-principal` mappings used to map the certificate principals to IP addresses, DNS names, and SQL users. This allows the use of certificates generated by Certificate Authorities that place restrictions on the contents of the `commonName` field. For usage information, see [Create Security Certificates using Openssl]({% link {{ page.version.version }}/create-security-certificates-openssl.md %}#examples). -`--enterprise-encryption` | This optional flag specifies the encryption options for one of the stores on the node. If multiple stores exist, the flag must be specified for each store.

This flag takes a number of options. For a complete list of options, and usage instructions, see [Encryption at Rest]({% link {{ page.version.version }}/encryption.md %}). -`--external-io-disable-http` | This optional flag disables external HTTP(S) access (as well as custom HTTP(S) endpoints) when performing bulk operations (e.g, [`BACKUP`]({% link {{ page.version.version }}/backup.md %})). This can be used in environments where you cannot run a full proxy server.

If you want to run a proxy server, you can start CockroachDB while specifying the `HTTP(S)_PROXY` environment variable. -`--external-io-disable-implicit-credentials` | This optional flag disables the use of implicit credentials when accessing external cloud storage services for bulk operations (e.g, [`BACKUP`]({% link {{ page.version.version }}/backup.md %})). -`--node-cert-distinguished-name` | A string with a comma separated list of distinguished name (DN) mappings in `{attribute-type}={attribute-value}` format in accordance with [RFC4514](https://www.rfc-editor.org/rfc/rfc4514) for the [`node` user]({% link {{ page.version.version }}/security-reference/authorization.md %}#node-user). If this flag is set, this needs to be an exact match with the DN subject in the client certificate provided for the `node` user. By exact match, we mean that the order of attributes in the argument to this flag must match the order of attributes in the DN subject in the certificate. For more information, see [Certificate-based authentication using multiple values from the X.509 Subject field]({% link {{ page.version.version }}/certificate-based-authentication-using-the-x509-subject-field.md %}). -`--root-cert-distinguished-name` | A string with a comma separated list of distinguished name (DN) mappings in `{attribute-type}={attribute-value}` format in accordance with [RFC4514](https://www.rfc-editor.org/rfc/rfc4514) for the [`root` user]({% link {{ page.version.version }}/security-reference/authorization.md %}#root-user). If this flag is set, this needs to be an exact match with the DN subject in the client certificate provided for the `root` user. By exact match, we mean that the order of attributes in the argument to this flag must match the order of attributes in the DN subject in the certificate. For more information, see [Certificate-based authentication using multiple values from the X.509 Subject field]({% link {{ page.version.version }}/certificate-based-authentication-using-the-x509-subject-field.md %}). -`--tls-cipher-suites` | A comma-separated list of TLS cipher suites to allow for SQL, RPC, and HTTP connections, limited to those [supported by CockroachDB]({% link {{ page.version.version }}/authentication.md %}#supported-cipher-suites). Connections using disallowed cipher suites will be rejected during the TLS handshake and logged to `cockroach.log`. Look for log messages containing: `presented cipher ... not in allowed cipher suite list`.
Example usage: `--tls-cipher-suites=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_AES_128_GCM_SHA256`. - -### Locality - -The `--locality` flag accepts arbitrary key-value pairs that describe the location of the node. Locality should include a `region` key-value if you are using CockroachDB's [Multi-region SQL capabilities]({% link {{ page.version.version }}/multiregion-overview.md %}). - -Depending on your deployment you can also specify country, availability zone, etc. The key-value pairs should be ordered into _locality tiers_ from most inclusive to least inclusive (e.g., region before availability zone as in `region=eu-west-1,zone=eu-west-1a`), and the keys and order of key-value pairs must be the same on all nodes. It's typically better to include more pairs than fewer. - -- Specifying a region with a `region` tier is required in order to enable CockroachDB's [multi-region capabilities]({% link {{ page.version.version }}/multiregion-overview.md %}). - -- CockroachDB spreads the replicas of each piece of data across as diverse a set of localities as possible, with the order determining the priority. Locality can also be used to influence the location of data replicas in various ways using high-level [multi-region SQL capabilities]({% link {{ page.version.version }}/multiregion-overview.md %}) or low-level [replication zones]({% link {{ page.version.version }}/configure-replication-zones.md %}#replication-constraints). - -- When there is high latency between nodes (e.g., cross-availability zone deployments), CockroachDB uses locality to move range leases closer to the current workload, reducing network round trips and improving read performance, also known as ["follow-the-workload"]({% link {{ page.version.version }}/topology-follow-the-workload.md %}). In a deployment across more than 3 availability zones, however, to ensure that all data benefits from "follow-the-workload", you must increase your replication factor to match the total number of availability zones. - -- Locality is also a prerequisite for using the [Multi-region SQL abstractions]({% link {{ page.version.version }}/multiregion-overview.md %}), [table partitioning]({% link {{ page.version.version }}/partitioning.md %}), and [**Node Map**]({% link {{ page.version.version }}/enable-node-map.md %}) {{site.data.products.enterprise}} features. - - - -#### Example - -The following shell commands use the `--locality` flag to start 9 nodes to run across 3 regions: `us-east-1`, `us-west-1`, and `europe-west-1`. Each region's nodes are further spread across different availability zones within that region. - -{{site.data.alerts.callout_info}} -This example follows the conventions required to use CockroachDB's [multi-region capabilities]({% link {{ page.version.version }}/multiregion-overview.md %}). -{{site.data.alerts.end}} - -Nodes in `us-east-1`: - -{% include_cached copy-clipboard.html %} -~~~ shell -cockroach start --locality=region=us-east-1,zone=us-east-1a # ... other required flags go here -~~~ - -{% include_cached copy-clipboard.html %} -~~~ shell -cockroach start --locality=region=us-east-1,zone=us-east-1b # ... other required flags go here -~~~ - -{% include_cached copy-clipboard.html %} -~~~ shell -cockroach start --locality=region=us-east-1,zone=us-east-1c # ... other required flags go here -~~~ - -Nodes in `us-west-1`: - -{% include_cached copy-clipboard.html %} -~~~ shell -cockroach start --locality=region=us-west-1,zone=us-west-1a # ... other required flags go here -~~~ - -{% include_cached copy-clipboard.html %} -~~~ shell -cockroach start --locality=region=us-west-1,zone=us-west-1b # ... other required flags go here -~~~ - -{% include_cached copy-clipboard.html %} -~~~ shell -cockroach start --locality=region=us-west-1,zone=us-west-1c # ... other required flags go here -~~~ - -Nodes in `europe-west-1`: - -{% include_cached copy-clipboard.html %} -~~~ shell -cockroach start --locality=region=europe-west-1,zone=europe-west-1a # ... other required flags go here -~~~ - -{% include_cached copy-clipboard.html %} -~~~ shell -cockroach start --locality=region=europe-west-1,zone=europe-west-1b # ... other required flags go here -~~~ - -{% include_cached copy-clipboard.html %} -~~~ shell -cockroach start --locality=region=europe-west-1,zone=europe-west-1c # ... other required flags go here -~~~ - -For another multi-region example, see [Start a multi-region cluster](#start-a-multi-region-cluster). - -For more information about how to use CockroachDB's multi-region capabilities, see the [Multi-Region Capabilities Overview]({% link {{ page.version.version }}/multiregion-overview.md %}). - -#### Load-based lease rebalancing in uneven latency deployments - -When nodes are started with the `--locality` flag, CockroachDB attempts to place the replica lease holder (the replica that client requests are forwarded to) on the node closest to the source of the request. This means as client requests move geographically, so too does the replica lease holder. - -However, you might see increased latency caused by a consistently high rate of lease transfers between datacenters in the following case: - -- Your cluster runs in datacenters which are very different distances away from each other. -- Each node was started with a single tier of `--locality`, e.g., `--locality=datacenter=a`. -- Most client requests get sent to a single datacenter because that's where all your application traffic is. - -To detect if this is happening, open the [DB Console]({% link {{ page.version.version }}/ui-overview.md %}), select the **Queues** dashboard, hover over the **Replication Queue** graph, and check the **Leases Transferred / second** data point. If the value is consistently larger than 0, you should consider stopping and restarting each node with additional tiers of locality to improve request latency. - -For example, let's say that latency is 10ms from nodes in datacenter A to nodes in datacenter B but is 100ms from nodes in datacenter A to nodes in datacenter C. To ensure A's and B's relative proximity is factored into lease holder rebalancing, you could restart the nodes in datacenter A and B with a common region, `--locality=region=foo,datacenter=a` and `--locality=region=foo,datacenter=b`, while restarting nodes in datacenter C with a different region, `--locality=region=bar,datacenter=c`. - -### Storage +{% include {{ page.version.version }}/reference/flags-table.md flags=start_cmd.flags %} -- [Storage engine](#storage-engine) -- [Store](#store) -- [Write Ahead Log (WAL) Failover](#write-ahead-log-wal-failover) - -#### Storage engine - -The `--storage-engine` flag is used to choose the storage engine used by the node. Note that this setting applies to all [stores](#store) on the node, including the [temp store](#temp-dir). - - As of v21.1 and later, CockroachDB always uses the [Pebble storage engine]({% link {{ page.version.version }}/architecture/storage-layer.md %}#pebble). As such, `pebble` is the default and only option for the `--storage-engine` flag. - -#### Store - -The `--store` flag allows you to specify details about a node's storage. - -To start a node with multiple disks or SSDs, provide a separate `--store` flag for each disk when starting the `cockroach` process on the node. For more details about stores, see [Start a Node]({% link {{ page.version.version }}/cockroach-start.md %}#store). - - {{site.data.alerts.callout_danger}} - If you start a node with multiple `--store` flags, it is not possible to scale back down to only using a single store on the node. Instead, you must decommission the node and start a new node with the updated `--store`. - {{site.data.alerts.end}} +## Store The `--store` flag supports the following fields. Note that commas are used to separate fields, and so are forbidden in all field values. -{{site.data.alerts.callout_info}} -In-memory storage is not suitable for production deployments at this time. -{{site.data.alerts.end}} - -{{site.data.alerts.callout_success}} -{% include {{ page.version.version }}/prod-deployment/prod-guidance-store-volume.md %} -{{site.data.alerts.end}} - -Field | Description -------|------------ -`type` | For in-memory storage, set this field to `mem`; otherwise, leave this field out. The `path` field must not be set when `type=mem`. -`path` | The file path to the storage device. When not setting `attr`, `size`, or `ballast-size`, the `path` field label can be left out:

`--store=/mnt/ssd01`

When either of those fields are set, however, the `path` field label must be used:

`--store=path=/mnt/ssd01,size=20GB`

**Default:** `cockroach-data` -`attrs` | Arbitrary strings, separated by colons, specifying disk type or capability. These can be used to influence the location of data replicas. See [Replication Controls]({% link {{ page.version.version }}/configure-replication-zones.md %}#replication-constraints) for full details.

In most cases, node-level `--locality` or `--attrs` are preferable to store-level attributes, but this field can be used to match capabilities for storage of individual databases or tables. For example, an OLTP database would probably want to allocate space for its tables only on solid state devices, whereas append-only time series might prefer cheaper spinning drives. Typical attributes include whether the store is flash (`ssd`) or spinny disk (`hdd`), as well as speeds and other specs, for example:

`--store=path=/mnt/hda1,attrs=hdd:7200rpm` - `size` | The maximum size allocated to the node. When this size is reached, CockroachDB attempts to rebalance data to other nodes with available capacity. When no other nodes have available capacity, this limit will be exceeded. Data may also be written to the node faster than the cluster can rebalance it away; as long as capacity is available elsewhere, CockroachDB will gradually rebalance data down to the store limit.

The `size` can be specified either in a bytes-based unit or as a percentage of hard drive space (notated as a decimal or with `%`), for example:

`--store=path=/mnt/ssd01,size=10000000000 ----> 10000000000 bytes`
`--store=path=/mnt/ssd01,size=20GB ----> 20000000000 bytes`
`--store=path=/mnt/ssd01,size=20GiB ----> 21474836480 bytes`
`--store=path=/mnt/ssd01,size=0.02TiB ----> 21474836480 bytes`
`--store=path=/mnt/ssd01,size=20% ----> 20% of available space`
`--store=path=/mnt/ssd01,size=0.2 ----> 20% of available space`
`--store=path=/mnt/ssd01,size=.2 ----> 20% of available space`

**Default:** 100%

For an in-memory store, the `size` field is required and must be set to the true maximum bytes or percentage of available memory, for example:

`--store=type=mem,size=20GB`
`--store=type=mem,size=90%`

Note: If you use the `%` notation, you might need to escape the `%` sign, for instance, while configuring CockroachDB through `systemd` service files. For this reason, it's recommended to use the decimal notation instead. - `ballast-size` | Configure the size of the automatically created emergency ballast file. Accepts the same value formats as the [`size` field](#store-size). For more details, see [Automatic ballast files]({% link {{ page.version.version }}/cluster-setup-troubleshooting.md %}#automatic-ballast-files).

To disable automatic ballast file creation, set the value to `0`:

`--store=path=/mnt/ssd01,ballast-size=0` - `provisioned-rate` | A mapping of a store name to a bandwidth limit, expressed in bytes per second. This constrains the bandwidth used for [admission control]({% link {{ page.version.version }}/admission-control.md %}) for operations on the store. The disk name is separated from the bandwidth value by a colon (`:`). A value of `0` (the default) represents unlimited bandwidth. For example:

`--store=provisioned-rate=disk-name=/mnt/ssd01:200`

**Default:** 0

If the bandwidth value is omitted, bandwidth is limited to the value of the [`kv.store.admission.provisioned_bandwidth` cluster setting]({% link {{ page.version.version }}/cluster-settings.md %}#settings). Modify this setting only in consultation with your support team. - -#### Write Ahead Log (WAL) failover - -{% include {{ page.version.version }}/wal-failover-intro.md %} - -This page has basic instructions on how to enable WAL failover, disable WAL failover, and monitor WAL failover. - -For more detailed instructions showing how to use, test, and monitor WAL failover, as well as descriptions of how WAL failover works in multi-store configurations, see [WAL Failover]({% link {{ page.version.version }}/wal-failover.md %}). - -##### Enable WAL failover - -To enable WAL failover, you must take one of the following actions: - -- Pass [`--wal-failover=among-stores`](#flag-wal-failover) to `cockroach start`, or -- Set the environment variable `COCKROACH_WAL_FAILOVER=among-stores` before starting the node. - -[Writing log files to local disk]({% link {{ page.version.version }}/configure-logs.md %}#output-to-files) using the default configuration can lead to cluster instability in the event of a [disk stall]({% link {{ page.version.version }}/cluster-setup-troubleshooting.md %}#disk-stalls). It's not enough to failover your WAL writes to another disk: you must also write your log files in such a way that the forward progress of your cluster is not stalled due to disk unavailability. - -Therefore, if you enable WAL failover and log to local disks, you must also update your [logging]({% link {{page.version.version}}/logging-overview.md %}) configuration as follows: - -1. Disable [audit logging]({% link {{ page.version.version }}/sql-audit-logging.md %}). File-based audit logging cannot coexist with the WAL failover feature. File-based audit logging provides guarantees that every log message makes it to disk, or CockroachDB must be shut down. For this reason, resuming operations in the face of disk unavailability is not compatible with audit logging. -1. Enable asynchronous buffering of [`file-groups` log sinks]({% link {{ page.version.version }}/configure-logs.md %}#output-to-files) using the `buffering` configuration option. The `buffering` configuration can be applied to [`file-defaults`]({% link {{ page.version.version }}/configure-logs.md %}#configure-logging-defaults) or individual `file-groups` as needed. Note that enabling asynchronous buffering of `file-groups` log sinks is in [preview]({% link {{page.version.version}}/cockroachdb-feature-availability.md %}#features-in-preview). -1. Set `max-staleness: 1s` and `flush-trigger-size: 256KiB`. -1. When `buffering` is enabled, `buffered-writes` must be explicitly disabled as shown in the following example. This is necessary because `buffered-writes` does not provide true asynchronous disk access, but rather a small buffer. If the small buffer fills up, it can cause internal routines performing logging operations to hang. This will in turn cause internal routines doing other important work to hang, potentially affecting cluster stability. -1. The recommended logging configuration for using file-based logging with WAL failover is as follows: - - {% include {{ page.version.version }}/wal-failover-log-config.md %} - -As an alternative to logging to local disks, you can configure [remote log sinks]({% link {{page.version.version}}/logging-use-cases.md %}#network-logging) that are not correlated with the availability of your cluster's local disks. However, this will make troubleshooting using [`cockroach debug zip`]({% link {{ page.version.version}}/cockroach-debug-zip.md %}) more difficult, since the output of that command will not include the (remotely stored) log files. - -##### Disable WAL failover - -To disable WAL failover, you must [restart the node]({% link {{ page.version.version }}/node-shutdown.md %}#stop-and-restart-a-node) and either: - -- Pass the [`--wal-failover=disabled`](#flag-wal-failover) flag to `cockroach start`, or -- Set the environment variable `COCKROACH_WAL_FAILOVER=disabled` before restarting the node. - -##### Monitor WAL failover - -{% include {{ page.version.version }}/wal-failover-metrics.md %} - -### Logging - -By [default]({% link {{ page.version.version }}/configure-logs.md %}#default-logging-configuration), `cockroach start` writes all messages to log files, and prints nothing to `stderr`. This includes events with `INFO` [severity]({% link {{ page.version.version }}/logging.md %}#logging-levels-severities) and higher. However, you can [customize the logging behavior]({% link {{ page.version.version }}/configure-logs.md %}) of this command by using the `--log` flag: - -{% include {{ page.version.version }}/misc/logging-flags.md %} - -#### Defaults - -See the [default logging configuration]({% link {{ page.version.version }}/configure-logs.md %}#default-logging-configuration). +{% include {{ page.version.version }}/reference/fields-table.md fields=start_cmd.store %} ## Standard output When you run `cockroach start`, some helpful details are printed to the standard output: -~~~ shell -CockroachDB node starting at {{ now | date: "%Y-%m-%d %H:%M:%S.%6 +0000 UTC" }} -build: CCL {{page.release_info.version}} @ {{page.release_info.build_time}} (go1.12.6) -webui: http://localhost:8080 -sql: postgresql://root@localhost:26257?sslmode=disable -sql (JDBC): jdbc:postgresql://localhost:26257/defaultdb?sslmode=disable&user=root -RPC client flags: cockroach --host=localhost:26257 --insecure -logs: /Users//node1/logs -temp dir: /Users//node1/cockroach-temp242232154 -external I/O path: /Users//node1/extern -store[0]: path=/Users//node1 -status: initialized new cluster -clusterID: 8a681a16-9623-4fc1-a537-77e9255daafd -nodeID: 1 -~~~ - {{site.data.alerts.callout_success}} These details are also written to the `INFO` log in the `/logs` directory. You can retrieve them with a command like `grep 'node starting' node1/logs/cockroach.log -A 11`. {{site.data.alerts.end}} -Field | Description -------|------------ -`build` | The version of CockroachDB you are running. -`webui` | The URL for accessing the DB Console. -`sql` | The connection URL for your client. -`RPC client flags` | The flags to use when connecting to the node via [`cockroach` client commands]({% link {{ page.version.version }}/cockroach-commands.md %}). -`logs` | The directory containing debug log data. -`temp dir` | The temporary store directory of the node. -`external I/O path` | The external IO directory with which the local file access paths are prefixed while performing [backup]({% link {{ page.version.version }}/backup.md %}) and [restore]({% link {{ page.version.version }}/restore.md %}) operations using local node directories or NFS drives. -`attrs` | If node-level attributes were specified in the `--attrs` flag, they are listed in this field. These details are potentially useful for [configuring replication zones]({% link {{ page.version.version }}/configure-replication-zones.md %}). -`locality` | If values describing the locality of the node were specified in the `--locality` field, they are listed in this field. These details are potentially useful for [configuring replication zones]({% link {{ page.version.version }}/configure-replication-zones.md %}). -`store[n]` | The directory containing store data, where `[n]` is the index of the store, e.g., `store[0]` for the first store, `store[1]` for the second store.

If store-level attributes were specified in the `attrs` field of the [`--store`](#store) flag, they are listed in this field as well. These details are potentially useful for [configuring replication zones]({% link {{ page.version.version }}/configure-replication-zones.md %}). -`status` | Whether the node is the first in the cluster (`initialized new cluster`), joined an existing cluster for the first time (`initialized new node, joined pre-existing cluster`), or rejoined an existing cluster (`restarted pre-existing node`). -`clusterID` | The ID of the cluster.

When trying to join a node to an existing cluster, if this ID is different than the ID of the existing cluster, the node has started a new cluster. This may be due to conflicting information in the node's data directory. For additional guidance, see the [troubleshooting]({% link {{ page.version.version }}/common-errors.md %}#node-belongs-to-cluster-cluster-id-but-is-attempting-to-connect-to-a-gossip-network-for-cluster-another-cluster-id) docs. -`nodeID` | The ID of the node. - -## Examples - -### Start a multi-node cluster - -
- - -
- -To start a multi-node cluster, run the `cockroach start` command for each node, setting the `--join` flag to the addresses of the initial nodes. - -{% include {{ page.version.version }}/prod-deployment/join-flag-single-region.md %} - -{{site.data.alerts.callout_info}} -{% include {{ page.version.version }}/prod-deployment/join-flag-multi-region.md %} -{{site.data.alerts.end}} - -
- -{{site.data.alerts.callout_success}} -Before starting the cluster, use [`cockroach cert`]({% link {{ page.version.version }}/cockroach-cert.md %}) to generate node and client certificates for a secure cluster connection. -{{site.data.alerts.end}} - -{% include_cached copy-clipboard.html %} -~~~ shell -$ cockroach start \ ---certs-dir=certs \ ---advertise-addr= \ ---join=,, \ ---cache=.25 \ ---max-sql-memory=.25 -~~~ - -{% include_cached copy-clipboard.html %} -~~~ shell -$ cockroach start \ ---certs-dir=certs \ ---advertise-addr= \ ---join=,, \ ---cache=.25 \ ---max-sql-memory=.25 -~~~ - -{% include_cached copy-clipboard.html %} -~~~ shell -$ cockroach start \ ---certs-dir=certs \ ---advertise-addr= \ ---join=,, \ ---cache=.25 \ ---max-sql-memory=.25 -~~~ - -
- -
- -{% include_cached copy-clipboard.html %} -~~~ shell -$ cockroach start \ ---insecure \ ---advertise-addr= \ ---join=,, \ ---cache=.25 \ ---max-sql-memory=.25 -~~~ - -{% include_cached copy-clipboard.html %} -~~~ shell -$ cockroach start \ ---insecure \ ---advertise-addr= \ ---join=,, \ ---cache=.25 \ ---max-sql-memory=.25 -~~~ - -{% include_cached copy-clipboard.html %} -~~~ shell -$ cockroach start \ ---insecure \ ---advertise-addr= \ ---join=,, \ ---cache=.25 \ ---max-sql-memory=.25 -~~~ - -
- -Then run the [`cockroach init`]({% link {{ page.version.version }}/cockroach-init.md %}) command against any node to perform a one-time cluster initialization: - -
- -{% include_cached copy-clipboard.html %} -~~~ shell -$ cockroach init \ ---certs-dir=certs \ ---host=
-~~~ - -
- -
- -{% include_cached copy-clipboard.html %} -~~~ shell -$ cockroach init \ ---insecure \ ---host=
-~~~ - -
- -### Start a multi-region cluster - -In this example we will start a multi-node [local cluster]({% link {{ page.version.version }}/start-a-local-cluster.md %}) with a multi-region setup that uses the same regions (passed to the [`--locality`](#locality) flag) as the multi-region MovR demo application. - -1. Start a node in the `us-east1` region: - - {% include_cached copy-clipboard.html %} - ~~~ shell - cockroach start --locality=region=us-east1,zone=us-east-1a \ - --insecure --store=/tmp/node0 \ - --listen-addr=localhost:26257 \ - --http-port=8888 \ - --join=localhost:26257,localhost:26258,localhost:26259 - ~~~ - -1. Start a node in the `us-west1` region: - - {% include_cached copy-clipboard.html %} - ~~~ shell - cockroach start --locality=region=us-west1,zone=us-west-1a \ - --insecure \ - --store=/tmp/node2 \ - --listen-addr=localhost:26259 \ - --http-port=8890 \ - --join=localhost:26257,localhost:26258,localhost:26259 - ~~~ - -1. Start a node in the `europe-west1` region: - - {% include_cached copy-clipboard.html %} - ~~~ shell - cockroach start --locality=region=europe-west1,zone=europe-west-1a \ - --insecure \ - --store=/tmp/node1 \ - --listen-addr=localhost:26258 \ - --http-port=8889 \ - --join=localhost:26257,localhost:26258,localhost:26259 - ~~~ - -1. Initialize the cluster: - - {% include_cached copy-clipboard.html %} - ~~~ shell - cockroach init --insecure --host=localhost --port=26257 - ~~~ - -1. Connect to the cluster using [`cockroach sql`]({% link {{ page.version.version }}/cockroach-sql.md %}): - - {% include_cached copy-clipboard.html %} - ~~~ shell - cockroach sql --host=localhost --port=26257 --insecure - ~~~ - -1. Issue the [`SHOW REGIONS`]({% link {{ page.version.version }}/show-regions.md %}) statement to verify that the list of regions is expected: - - {% include_cached copy-clipboard.html %} - ~~~ sql - SHOW REGIONS; - ~~~ - - ~~~ - region | zones | database_names | primary_region_of - ---------------+-------+----------------+-------------------- - europe-west1 | {} | {} | {} - us-east1 | {} | {} | {} - us-west1 | {} | {} | {} - (3 rows) - ~~~ - -For more information about running CockroachDB multi-region, see the [Multi-region Capabilities Overview]({% link {{ page.version.version }}/multiregion-overview.md %}). - -{{site.data.alerts.callout_info}} -For more information about the `--locality` flag, see [Locality](#locality). -{{site.data.alerts.end}} - -### Start a multi-node cluster across private networks - -**Scenario:** - -- You have a cluster that spans GCE and AWS. -- The nodes on each cloud can reach each other on public addresses, but the private addresses aren't reachable from the other cloud. - -**Approach:** - -1. Start each node on GCE with `--locality` set to describe its location, `--locality-advertise-addr` set to advertise its private address to other nodes in on GCE, `--advertise-addr` set to advertise its public address to nodes on AWS, and `--join` set to the public addresses of 3-5 of the initial nodes: - - {% include_cached copy-clipboard.html %} - ~~~ shell - $ cockroach start \ - --certs-dir=certs \ - --locality=cloud=gce \ - --locality-advertise-addr=cloud=gce@ \ - --advertise-addr= \ - --join=,, \ - --cache=.25 \ - --max-sql-memory=.25 - ~~~ - -1. Start each node on AWS with `--locality` set to describe its location, `--locality-advertise-addr` set to advertise its private address to other nodes on AWS, `--advertise-addr` set to advertise its public address to nodes on GCE, and `--join` set to the public addresses of 3-5 of the initial nodes: - - {% include_cached copy-clipboard.html %} - ~~~ shell - $ cockroach start \ - --certs-dir=certs \ - --locality=cloud=aws \ - --locality-advertise-addr=cloud=aws@ \ - --advertise-addr= \ - --join=,, \ - --cache=.25 \ - --max-sql-memory=.25 - ~~~ - -1. Run the [`cockroach init`]({% link {{ page.version.version }}/cockroach-init.md %}) command against any node to perform a one-time cluster initialization: - - {% include_cached copy-clipboard.html %} - ~~~ shell - $ cockroach init \ - --certs-dir=certs \ - --host=
- ~~~ - -### Add a node to a cluster - -
- - -
- -To add a node to an existing cluster, run the `cockroach start` command, setting the `--join` flag to the same addresses you used when [starting the cluster](#start-a-multi-node-cluster): - -
- -{% include_cached copy-clipboard.html %} -~~~ shell -$ cockroach start \ ---certs-dir=certs \ ---advertise-addr= \ ---join=,, \ ---cache=.25 \ ---max-sql-memory=.25 -~~~ - -
- -
- -{% include_cached copy-clipboard.html %} -~~~ shell -$ cockroach start \ ---insecure \ ---advertise-addr= \ ---join=,, \ ---cache=.25 \ ---max-sql-memory=.25 -~~~ - -
- -### Create a table with node locality information - -Start a three-node cluster with locality information specified in the `cockroach start` commands: - -{% include_cached copy-clipboard.html %} -~~~ shell -$ cockroach start --insecure --port=26257 --http-port=26258 --store=cockroach-data/1 --cache=256MiB --locality=region=eu-west-1,cloud=aws,zone=eu-west-1a -~~~ - -{% include_cached copy-clipboard.html %} -~~~ shell -$ cockroach start --insecure --port=26259 --http-port=26260 --store=cockroach-data/2 --cache=256MiB --join=localhost:26257 --locality=region=eu-west-1,cloud=aws,zone=eu-west-1b -~~~ - -{% include_cached copy-clipboard.html %} -~~~ shell -$ cockroach start --insecure --port=26261 --http-port=26262 --store=cockroach-data/3 --cache=256MiB --join=localhost:26257 --locality=region=eu-west-1,cloud=aws,zone=eu-west-1c -~~~ - -You can use the [`crdb_internal.locality_value`]({% link {{ page.version.version }}/functions-and-operators.md %}#system-info-functions) built-in function to return the current node's locality information from inside a SQL shell. The example below uses the output of `crdb_internal.locality_value('zone')` as the `DEFAULT` value to use for the `zone` column of new rows. Other available locality keys for the running three-node cluster include `region` and `cloud`. - -{% include_cached copy-clipboard.html %} -~~~ shell -$ cockroach sql --insecure -~~~ - -{% include_cached copy-clipboard.html %} -~~~ sql -> CREATE TABLE charges ( - zone STRING NOT NULL DEFAULT crdb_internal.locality_value('zone'), - id INT PRIMARY KEY NOT NULL -); -~~~ - -{% include_cached copy-clipboard.html %} -~~~ sql -> INSERT INTO charges (id) VALUES (1); -~~~ - -{% include_cached copy-clipboard.html %} -~~~ sql -> SELECT * FROM charges WHERE id = 1; -~~~ - -~~~ - zone | id -+------------+----+ - eu-west-1a | 1 -(1 row) -~~~ - -The `zone ` column has the zone of the node on which the row was created. - -In a separate terminal window, open a SQL shell to a different node on the cluster: - -{% include_cached copy-clipboard.html %} -~~~ shell -$ cockroach sql --insecure --port 26259 -~~~ - -{% include_cached copy-clipboard.html %} -~~~ sql -> INSERT INTO charges (id) VALUES (2); -~~~ - -{% include_cached copy-clipboard.html %} -~~~ sql -> SELECT * FROM charges WHERE id = 2; -~~~ - -~~~ - zone | id -+------------+----+ - eu-west-1b | 2 -(1 row) -~~~ - -In a separate terminal window, open a SQL shell to the third node: - -{% include_cached copy-clipboard.html %} -~~~ shell -$ cockroach sql --insecure --port 26261 -~~~ - -{% include_cached copy-clipboard.html %} -~~~ sql -> INSERT INTO charges (id) VALUES (3); -~~~ - -{% include_cached copy-clipboard.html %} -~~~ sql -> SELECT * FROM charges WHERE id = 3; -~~~ - -~~~ - zone | id -+------------+----+ - eu-west-1c | 3 -(1 row) -~~~ - -### Start a cluster with separate RPC and SQL networks - -Separating the network addresses used for intra-cluster RPC traffic and application SQL connections can provide an additional level of protection against security issues as a form of [defense in depth](https://wikipedia.org/wiki/Defense_in_depth_(computing)). This separation is accomplished with a combination of the [`--sql-addr` flag](#networking) and firewall rules or other network-level access control (which must be maintained outside of CockroachDB). - -For example, suppose you want to use port `26257` for SQL connections and `26258` for intra-cluster traffic. Set up firewall rules so that the CockroachDB nodes can reach each other on port `26258`, but other machines cannot. Start the CockroachDB processes as follows: - -{% include_cached copy-clipboard.html %} -~~~ shell -$ cockroach start --sql-addr=:26257 --listen-addr=:26258 --join=node1:26258,node2:26258,node3:26258 --certs-dir=~/cockroach-certs -~~~ - -Note the use of port `26258` (the value for `listen-addr`, not `sql-addr`) in the `--join` flag. Also, if your environment requires the use of the `--advertise-addr` flag, you should probably also use the `--advertise-sql-addr` flag when using a separate SQL address. - -Clusters using this configuration with client certificate authentication may also wish to use [split client CA certificates]({% link {{ page.version.version }}/create-security-certificates-custom-ca.md %}#split-ca-certificates). +{% include {{ page.version.version }}/reference/fields-table.md fields=start_cmd.standard_output.fields %} ## See also @@ -727,4 +73,4 @@ Clusters using this configuration with client certificate authentication may als - [Manual Deployment]({% link {{ page.version.version }}/manual-deployment.md %}) - [Kubernetes Overview]({% link {{ page.version.version }}/kubernetes-overview.md %}) - [Local Deployment]({% link {{ page.version.version }}/start-a-local-cluster.md %}) -- [`cockroach` Commands Overview]({% link {{ page.version.version }}/cockroach-commands.md %}) +- [`cockroach` Commands Overview]({% link {{ page.version.version }}/cockroach-commands.md %}) \ No newline at end of file diff --git a/src/current/v26.3/set-up-physical-cluster-replication.md b/src/current/v26.3/set-up-physical-cluster-replication.md index c7e06ccd695..1a8e362b800 100644 --- a/src/current/v26.3/set-up-physical-cluster-replication.md +++ b/src/current/v26.3/set-up-physical-cluster-replication.md @@ -134,7 +134,7 @@ The standby cluster connects to the primary cluster's system virtual cluster usi cockroach workload init movr "postgresql://root@{node_advertise_address}:{node_advertise_port}?options=-ccluster=main&sslmode=verify-full&sslrootcert=certs/ca.crt&sslcert=certs/client.root.crt&sslkey=certs/client.root.key" ~~~ - Replace `{node_advertise_address}` and `{node_advertise_port}` with a node's [`--advertise-addr`]({% link {{ page.version.version }}/cockroach-start.md %}#flags-advert-addr) IP address or hostname and port. + Replace `{node_advertise_address}` and `{node_advertise_port}` with a node's [`--advertise-addr`]({% link {{ page.version.version }}/cockroach-start.md %}#flags-advertise-addr) IP address or hostname and port. {% include {{ page.version.version }}/connect/cockroach-workload-parameters.md %} As a result, for the example in this tutorial, you will need: - `options=-ccluster=main`