lenskit.parallel.config#

Functions

effective_cpu_count()

Return the effective CPU count using the best available data.

ensure_parallel_init()

Make sure LensKit parallelism is configured, and configure with defaults if it is not.

get_parallel_config()

Ensure that parallelism is configured and return the configuration.

initialize([config, processes, threads, ...])

Set up and configure LensKit parallelism.

subprocess_config()

Get a parallel configuration for a subprocess.

Classes

ParallelConfig(processes, threads, ...)

class lenskit.parallel.config.ParallelConfig(processes: 'int', threads: 'int', backend_threads: 'int', child_threads: 'int')#

Bases: object

Parameters:

  • processes (int)

  • threads (int)

  • backend_threads (int)

  • child_threads (int)

lenskit.parallel.config.initialize(config=None, *, processes=None, threads=None, backend_threads=None, child_threads=None)#

Set up and configure LensKit parallelism. This only needs to be called if you want to control when and how parallelism is set up; components using parallelism will call ensure_init(), which will call this function with its default arguments if it has not been called.

See also

Configuring Parallelism

Parameters:

  • processes (int | None) – The number of processes to use for multiprocessing evaluations (see LK_NUM_PROCS)

  • threads (int | None) – The number of threads to use for parallel model training and similar operations (see LK_NUM_THREADS).

  • backend_threads (int | None) – The number of threads underlying computational engines should use (see LK_NUM_BACKEND_THREADS).

  • child_threads (int | None) – The number of threads backends are allowed to use in the worker processes in multiprocessing operations (see LK_NUM_CHILD_THREADS).

  • config (ParallelConfig | None)

lenskit.parallel.config.ensure_parallel_init()#

Make sure LensKit parallelism is configured, and configure with defaults if it is not.

Components using parallelism or intensive computations should call this function before they begin training.

lenskit.parallel.config.get_parallel_config()#

Ensure that parallelism is configured and return the configuration.

Return type:

ParallelConfig

lenskit.parallel.config.subprocess_config()#

Get a parallel configuration for a subprocess.

Return type:

ParallelConfig

lenskit.parallel.config.effective_cpu_count()#

Return the effective CPU count using the best available data. Tries the following in order:

  1. os.process_cpu_count()

  2. os.sched_getaffinity()

  3. os.cpu_count()

Return type:

int