lenskit.metrics.predict

Prediction accuracy metrics.

Functions

global_metric(predictions, *[, ...])	Compute a global prediction accuracy metric for a set of predictions.
mae(predictions, truth[, missing])	Compute MAE (mean absolute error).
rmse(predictions, truth[, missing])	Compute RMSE (root mean squared error).
user_metric(predictions, *[, score_column, ...])	Compute a mean per-user prediction accuracy metric for a set of predictions.

lenskit.metrics.predict.rmse(predictions, truth, missing='error')

Compute RMSE (root mean squared error). This is computed as:

\[\sum_{r_{ui} \in R} \left(r_{ui} - s(i|u)\right)^2\]

When used with user_metric(), or on series grouped by user, it computes a per-user RMSE; when applied to an entire prediction frame, it computes global RMSE. It does not do any fallbacks; if you want to compute RMSE with fallback predictions (e.g. usign a bias model when a collaborative filter cannot predict), generate predictions with lenskit.algorithms.basic.Fallback.

Parameters:

• predictions (pandas.Series) – the predictions

• truth (pandas.Series) – the ground truth ratings from data

• missing (string) – how to handle predictions without truth. Can be one of 'error' or 'ignore'.

Returns:

the root mean squared approximation error

Return type:

double

lenskit.metrics.predict.mae(predictions, truth, missing='error')

Compute MAE (mean absolute error). This is computed as:

\[\sum_{r_{ui} \in R} \left|r_{ui} - s(i|u)\right|\]

When used with user_metric(), or on series grouped by user, it computes a per-user MAE; when applied to an entire prediction frame, it computes global MAE. It does not do any fallbacks; if you want to compute MAE with fallback predictions (e.g. usign a bias model when a collaborative filter cannot predict), generate predictions with lenskit.algorithms.basic.Fallback.

Parameters:

• predictions (pandas.Series) – the predictions

• truth (pandas.Series) – the ground truth ratings from data

• missing (string) – how to handle predictions without truth. Can be one of 'error' or 'ignore'.

Returns:

the mean absolute approximation error

Return type:

double

lenskit.metrics.predict.global_metric(predictions, *, score_column='prediction', metric=<function rmse>, **kwargs)

Compute a global prediction accuracy metric for a set of predictions.

Parameters:

• predictions (pandas.DataFrame) – Data frame containing the predictions. Must have a column rating containing ground truth and a score column with rating predictions.

• score_column (str) – The name of the score column (defaults to 'prediction').

• metric (function) – A metric function of two parameters (prediction and truth). Defaults to rmse().

Returns:

The global metric value.

Return type:

float

lenskit.metrics.predict.user_metric(predictions, *, score_column='prediction', metric=<function rmse>, **kwargs)

Compute a mean per-user prediction accuracy metric for a set of predictions.

Parameters:

• predictions (pandas.DataFrame) – Data frame containing the predictions. Must have a column rating containing ground truth and a score column with rating predictions, along with a 'user' column with user IDs.

• score_column (str) – The name of the score column (defaults to 'prediction').

• metric (function) – A metric function of two parameters (prediction and truth). Defaults to rmse().

Returns:

The mean of the per-user value of the metric.

Return type:

float