sfschunkserver.cfg(5)
=====================

== NAME

sfschunkserver.cfg - main configuration file for *sfschunkserver*

== DESCRIPTION

The file *sfschunkserver.cfg* contains configuration of SaunaFS chunkserver
process.

== SYNTAX

Syntax is:

'OPTION' *=* 'VALUE'::

Lines starting with *#* character are ignored.

== OPTIONS

Configuration options:

*DATA_PATH*:: where to store files with usage statistics and daemon lock file

*LABEL*:: the label of this chunkserver (for tiering)

*WORKING_USER*:: user to run daemon as

*WORKING_GROUP*:: group to run daemon as (optional - if empty then default user
group will be used)

*SYSLOG_IDENT*:: name of process to place in syslog messages (default is
sfschunkserver)

*LOCK_MEMORY*:: whether to perform mlockall() to avoid swapping out
sfschunkserver process (default is 0, i.e. no)

*LIMIT_GLIBC_MALLOC_ARENAS*:: Linux only: limit glibc malloc arenas to given
value - prevents from using huge amount of virtual memory. This can influence
performance by reducing memory fragmentation and improving cache locality, but
it may also lead to contention and reduced parallelism in multi-threaded
applications. Use it in constrained memory environments, recommended values are
4 or 8. (default is 0: disabled or let glibc decide)

*MALLOC_TRIM_INTERVAL*:: interval in seconds at which the chunkserver will attempt
to release unused memory back to the OS using malloc_trim(). This can help reduce
the process's memory footprint by returning free memory to the system more
frequently. Set to 0 to disable memory trimming entirely. (default is 0, 0 = disabled)

*NICE_LEVEL*:: nice level to run daemon with (default is -19 if possible; note:
process must be started as root to increase priority)

*MASTER_HOST*:: address of SaunaFS master host to connect with (default is
sfsmaster)

*MASTER_PORT*:: number of SaunaFS master port to connect with (default is 9420)

*CLUSTER_ID*:: Only connect to metadata servers with this CLUSTER_ID.
Prevents accidental connections to wrong metadata servers. (default is 'default')

*MASTER_RECONNECTION_DELAY*:: delay in seconds before trying to reconnect to
the master server after disconnection (default is 5)

*MASTER_TIMEOUT*:: timeout (in seconds) for the master server connection
(default is 60, minimum is 0.01)

*MASTER_NR_OF_WORKERS*:: number of threads that the connection to master may use
to process operations on chunks, like create, duplicate, replicate and truncate
(default is 10, minimum is 2)

*MASTER_REPLICATION_NR_OF_WORKERS*:: number of threads that the connection to
master may use to process replications operations on chunks.
(default is 5, minimum in 1)

*BIND_HOST*:: local address to use for connecting with the master server
(default is ***, i.e. default local address)

*CSSERV_LISTEN_HOST*:: IP address to listen on for client (mount) connections
(*** means any)

*CSSERV_LISTEN_PORT*:: port to listen on for client (mount) connections
(default is 9422)

*HDD_CONF_FILENAME*:: alternative name of *sfshdd.cfg* file

*HDD_LEAVE_SPACE_DEFAULT*:: free space threshold to set volume as 100% utisaued
when there is less than given amount of free space left (default is "4GiB").
This number is always added to the used disk space reported by chunkserver.

*HDD_TEST_FREQ*:: chunk test period in seconds (default is 10)

*HDD_CHECK_CRC_WHEN_READING*:: whether to check the CRC on every read operation
(default is 1)

*HDD_CHECK_CRC_WHEN_WRITING*:: whether to check the CRC on every write
operation. The safest and recommended value is 1, disable it only to test
maximum throughput.
(default is 1)

*HDD_ADVISE_NO_CACHE*:: whether to remove each chunk from page when closing it
to reduce cache pressure generated by chunkserver (default is 0, i.e. no)

*HDD_PUNCH_HOLES*:: if enabled then chunkserver detects zero values in chunk
data and frees corresponding file blocks (decreasing file system usage). This
option works only on Linux with file systems supporting punching holes (XFS,
ext4, Btrfs, tmpfs)

*ENABLE_LOAD_FACTOR*:: if enabled, chunkserver will send periodical reports of
its I/O load to master, which will be taken into consideration when picking
chunkservers for I/O operations.

*REPLICATION_BANDWIDTH_LIMIT_KBPS*:: limit how many kilobytes can be replicated
from other chunkservers to this chunkserver in every second (by default
undefined, i.e. no limits)

*NR_OF_NETWORK_WORKERS*:: number of threads which handle (in a round-robin
manner) connections with clients (default is 4); these threads are responsible
for reading from sockets and coping data from internal buffers to sockets

*NR_OF_HDD_WORKERS_PER_NETWORK_WORKER*:: number of threads that each network
worker may use to do disk operations like opening chunks, reading or writing
them (default is 16)

*BGJOBSCNT_PER_NETWORK_WORKER*:: maximum number of jobs that each network worker
may use for disk operations (default is 4000)

*POLL_TIMEOUT_MS*:: Maximum amount of time in milliseconds that the polling
operation will wait for events. In the chunkservers, the same value is applied
for the events loop and for the network worker threads. Smaller values could
reduce latency at the cost of CPU usage (default: 50)

*MAX_BLOCKS_PER_HDD_WRITE_JOB*:: maximum number of blocks written to the device
in a single write job. If possible, will try to make a single pwrite call
(default is 16, minimum is 1, maximum is 64)

*MAX_BLOCKS_PER_HDD_READ_JOB*:: maximum number of blocks read from the device
in a single read job. If possible, will try to make a single pread call
(default is 16, minimum is 1)

*MAX_PARALLEL_HDD_READ_JOBS_PER_CS_ENTRY*:: maximum number of read jobs processed
in parallel per client connection (default is 1)

*MAX_BUFFERS_POOL_SIZE_MB*:: maximum size of the buffers pool used for read, write
and replicate operations, in megabytes. Consider decreasing it if the system is
running out of memory, or increasing it to improve performance if there is enough
memory available. Unused buffers are regularly freed (default is 512)

*READ_AHEAD_KB*:: additional number of kilobytes which should be passed to
posix_fadvise(POSIX_FADV_WILLNEED) before reading data from a chunk (default is
0, i.e. use posix_fadvise only with the amount of data that is really needed;
the value is aligned down to 64 KiB)

*MAX_READ_BEHIND_KB*:: try to fix out-of-order read requests; the value tells
how much of skipped data to read if an offset of some read operation is greater
than the offset where the previous operation finished (default is 0, i.e. don't
read any skipped data; the value is aligned down to 64 KiB)

*PERFORM_FSYNC*:: call fsync() after a chunk is modified (default is 1, i.e.
enabled)

*STAT_CHUNKS_AT_DISK_SCAN*:: verify that chunk metadata and data parts exists at
scan time. If set to 1, the data part is also checked to be multiple of
SFSBLOCKSIZE. Disabling this option accelerates the scan process, but does not
check the chunk files. Warning: use it with caution, only in scenarios where a
fast chunkserver restart is needed.(default is 1, i.e. enabled)

*REPLICATION_TOTAL_TIMEOUT_MS*:: total timeout for single replication
operation. Replications that take longer than that are considered failed and
are immediately aborted (default: 60000)

*REPLICATION_CONNECTION_TIMEOUT_MS*:: connection for single replication
operation. If connecting to another chunkserver takes longer than this timeout,
the operation is considered failed and is immediately aborted (default: 1000)

*REPLICATION_WAVE_TIMEOUT_MS*:: *ADVANCED*:: timeout for single wave in
replication. After this timeout, next wave of read requests is sent to other
chunkservers (default: 500)

*PLUGINS_DIR*:: directory to search for Chunkserver plugins (absolute path).
When set, the plugin manager will search this directory first. If not set, then
it will search in usual plugin locations, like PLUGINS_PATH/chunkserver (PLUGINS_PATH is 
defined at build time), /usr/local/lib/saunafs/plugins/chunkserver, and
/usr/lib/saunafs/plugins/chunkserver (default is empty string, i.e. disabled)

*CHUNK_TRASH_ENABLED (EXPERIMENTAL)*:: enables or disables the chunk trash
feature. When enabled, deleted chunks are moved to a trash directory instead of
being immediately removed. (Default: 0)

*CHUNK_TRASH_EXPIRATION_SECONDS (EXPERIMENTAL)*:: specifies the timeout in
seconds for chunks to remain in the trash before being permanently deleted.
(Default: 259200)

*CHUNK_TRASH_FREE_SPACE_THRESHOLD_GB (EXPERIMENTAL)*:: sets the available space
threshold in gigabytes. If the available space on the disk falls below this
threshold, the system will start deleting older chunks from the trash to free
up space. A suggested value is about the 15% of the capacity of the smaller
configured disk. (Default: 0)

*CHUNK_TRASH_GC_BATCH_SIZE (EXPERIMENTAL)*:: defines the bulk size for the
garbage collector when processing chunks in the trash. This determines how many
files are processed in each garbage collection cycle. (Default: 1000)

*CHUNK_TRASH_GC_SPACE_RECOVERY_BATCH_SIZE (EXPERIMENTAL)*:: [ADVANCED] The
number of files to remove from the trash in a single GC cycle, in case the disk
is full and space needs to be recovered. (Default: 100)

*LOG_LEVEL*:: Setup logging. Uses the environment variable SAUNAFS_LOG_LEVEL or
config value LOG_LEVEL to determine logging level. Valid log levels are
- 'trace'
- 'debug'
- 'info'
- 'warn' or 'warning'
- 'err' or 'error'
- 'crit' or 'critical'
- 'off'

*IO_PRIORITY_MODE (EXPERIMENTAL)*:: Define IO jobs, i.e reads and writes from drives,
prioritization scheme. The available modes are:
- 'FIFO': prioritizes the IO jobs first enqueued.
- 'SWITCH': alternates read and write jobs (interleaving) to prevent one class of IO from
starving the other.
Not reloadable. (Default: FIFO)

*TLS_CERT_FILE (EXPERIMENTAL)*::
Path to the TLS certificate file the chunk server will use for TLS connections
(there is no default value).

*TLS_KEY_FILE (EXPERIMENTAL)*::
Path to the TLS private key file the chunk server will use for TLS connections
(there is no default value).

*TLS_CA_CERT_FILE (EXPERIMENTAL)*::
Path to the trusted CA certificate which is used to authenticate through TLS
to the master server (there is no default value).

== REPORTING BUGS

Report bugs to the GitHub repository <https://github.com/leil-io/saunafs> as an
issue.


== COPYRIGHT

Copyright 2008-2009 Gemius SA

Copyright 2013-2019 Skytechnology sp. z o.o.

Copyright 2023-2024 Leil Storage OÜ

SaunaFS is free software: you can redistribute it and/or modify it under the
terms of the GNU General Public License as published by the Free Software
Foundation, version 3.

SaunaFS is distributed in the hope that it will be useful, but WITHOUT ANY
WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with
SaunaFS. If not, see <http://www.gnu.org/licenses/>.

== SEE ALSO

sfschunkserver(8), sfshdd.cfg(5)
