flurs.evaluation.Evaluator

class flurs.evaluation.Evaluator(recommender, repeat=True, maxlen=None, debug=False)[source]

Wrap a streaming recommender and evaluate them in the positive-only feedback setting.

Parameters

  • recommender (RecommenderMixin) – Instance of a recommender which has already been initialized.

  • repeat (bool, default=True) – Whether the same item can be repeatedly interacted by the same user.

  • maxlen (int) – Size of an item buffer which stores most recently observed items.

  • debug (bool, default=False) – Verbose logging if True.

References

1

J. Vinagre et al. Fast Incremental Matrix Factorization for Recommendation with Positive-only Feedback. In Proc. of UMAP 2014, pp. 459-470, July 2014.

__init__(recommender, repeat=True, maxlen=None, debug=False)[source]

Methods

__init__(recommender[, repeat, maxlen, debug])

batch_evaluate(test_events)

Evaluate the current model in terms of Mean Percentile Rank (MPR), by using the given test events.

batch_update(train_events, test_events, n_epoch)

Update a model in the batch fashion.

evaluate(test_events)

Iterate recommend-then-update procedure and compute incremental recall.

fit(train_events, test_events[, n_epoch])

Train a model using the first 30% positive events to avoid cold-start.
batch_evaluate(test_events)[source]

Evaluate the current model in terms of Mean Percentile Rank (MPR), by using the given test events.

Parameters

test_events (list of Event) – Validation samples to evaluate the current model.

Returns

Mean Percentile Rank for the test set.

Return type

float

batch_update(train_events, test_events, n_epoch)[source]

Update a model in the batch fashion.

Parameters

  • train_events (list of Event) – Training samples.

  • test_events (list of Event) – Validation samples.

  • n_epoch (int) – Number of epochs for the batch training.

evaluate(test_events)[source]

Iterate recommend-then-update procedure and compute incremental recall.

Parameters

test_events (list of Event) – Positive test events.

Yields

tuple – (rank, recommend time, update time).

fit(train_events, test_events, n_epoch=1)[source]

Train a model using the first 30% positive events to avoid cold-start.

Evaluation of this batch training is done by using the next 20% positive events. After the batch SGD training, the models are incrementally updated by using the 20% test events.

Parameters

  • train_events (list of Event) – Positive training events (0-30%).

  • test_events (list of Event) – Test events (30-50%).

  • n_epoch (int, default=1) – Number of epochs for the batch training.