lenskit.parallel.ray#

Support for parallelism with Ray.

Functions

ensure_cluster()

inference_worker_cpus()

init_cluster(*[, num_cpus, proc_slots, ...])

Initialize or connect to a Ray cluster, with the LensKit options.

training_worker_cpus()

Classes

RayOpInvoker(model, func[, worker_parallel])

lenskit.parallel.ray.init_cluster(*, num_cpus=None, proc_slots=None, resources=None, worker_parallel=None, **kwargs)#

Initialize or connect to a Ray cluster, with the LensKit options.

Parameters:

  • num_cpus (int | None) – The total number of CPUs to allow. Defaults to :fun:`effective_cpu_count`.

  • proc_slots (int | None) – The number of “process slots” for LensKit parallel operations. Defaults to the LensKit process count. These slots are recorded as the lk_process resource on the Ray cluster.

  • resources (dict[str, float] | None) – Additional custom resources to register in the Ray cluster.

  • worker_parallel (ParallelConfig | None) – Parallel processing configuration for worker processes. If None, uses the default.

  • kwargs – Other options to pass to ray.init().

Stability:

Experimental

class lenskit.parallel.ray.RayOpInvoker(model, func, worker_parallel=None)#

Bases: ModelOpInvoker[A, R], Generic[M, A, R]

Parameters:

  • model (M)

  • func (InvokeOp[M, A, R])

  • worker_parallel (ParallelConfig | None)

map(tasks)#

Apply the configured function to the model and iterables. This is like map(), except it supplies the invoker’s model as the first object to func.

Parameters:

  • iterables – Iterables of arguments to provide to the function.

  • tasks (Iterable[A])

Returns:

An iterable of the results.

Return type:

iterable