lenskit.parallel.config

Functions

effective_cpu_count()	Return the effective CPU count using the best available data.
ensure_parallel_init()	Make sure LensKit parallelism is configured, and configure with defaults if it is not.
get_parallel_config()	Ensure that parallelism is configured and return the configuration.
initialize([config, processes, threads, ...])	Set up and configure LensKit parallelism.
subprocess_config()	Get a parallel configuration for a subprocess.

Classes

ParallelConfig(processes, threads, ...)

Parallel processing configuration.

class lenskit.parallel.config.ParallelConfig(processes, threads, backend_threads, child_threads)

Bases: object

Parallel processing configuration.

Parameters:

• processes (int)

• threads (int)

• backend_threads (int)

• child_threads (int)

property total_threads: int

Get the total number of threads for this worker (training threads times backend threads).

env_vars()

Get this parallel configuration as a set of environment variables. The set also includes OMP_NUM_THREADS to configure OMP early.

Return type:

dict[str, str]

lenskit.parallel.config.initialize(config=None, *, processes=None, threads=None, backend_threads=None, child_threads=None)

Set up and configure LensKit parallelism. This only needs to be called if you want to control when and how parallelism is set up; components using parallelism will call ensure_init(), which will call this function with its default arguments if it has not been called.

See also

Configuring Parallelism

Parameters:

• processes (int | None) – The number of processes to use for multiprocessing evaluations (see LK_NUM_PROCS)

• threads (int | None) – The number of threads to use for parallel model training and similar operations (see LK_NUM_THREADS).

• backend_threads (int | None) – The number of threads underlying computational engines should use (see LK_NUM_BACKEND_THREADS).

• child_threads (int | None) – The number of threads backends are allowed to use in the worker processes in multiprocessing operations (see LK_NUM_CHILD_THREADS).

• config (ParallelConfig | None)

lenskit.parallel.config.ensure_parallel_init()

Make sure LensKit parallelism is configured, and configure with defaults if it is not.

Components using parallelism or intensive computations should call this function before they begin training.

lenskit.parallel.config.get_parallel_config()

Ensure that parallelism is configured and return the configuration.

Return type:

ParallelConfig

lenskit.parallel.config.subprocess_config()

Get a parallel configuration for a subprocess.

Return type:

ParallelConfig

lenskit.parallel.config.effective_cpu_count()

Return the effective CPU count using the best available data. Tries the following in order:

1. os.process_cpu_count()

2. os.sched_getaffinity()

3. os.cpu_count()

Return type:

int