Skip to content

Cross-Framework Evaluation

WarpRec supports evaluating recommendation lists generated by external frameworks through the ProxyRecommender. This allows researchers and practitioners to leverage WarpRec's comprehensive evaluation suite -- 40+ GPU-accelerated metrics, statistical significance testing, and sampled evaluation -- on recommendations produced outside of WarpRec.

API Reference

For class signatures and parameters, see the Unpersonalized API Reference.

Use Case

When you have precomputed recommendation lists from another library (e.g., Surprise, LensKit, RecBole, or a custom production system), you can use the ProxyRecommender to evaluate them within WarpRec's standardized evaluation pipeline without retraining.

This is particularly useful for:

  • Benchmarking external models against WarpRec's built-in algorithms using identical evaluation protocols.
  • Reproducing results from published papers by feeding their recommendation files into WarpRec's evaluation engine.
  • Comparing models across different frameworks in a consistent and fair manner.

How It Works

The ProxyRecommender reads a precomputed recommendation file and presents it to WarpRec's evaluation system as if it were generated by an internal model. This means you get access to all of WarpRec's metrics, cutoffs, and reporting features.

The recommendation file should be formatted as a tabular file with user-item-score columns:

user_id,item_id,score
1,42,0.95
1,17,0.87
2,42,0.62
2,5,0.54
...

Configuration

models:
  ProxyRecommender:
    recommendation_file: /path/to/recommendations.tsv
    separator: "\t"
    header: True

The recommendation_file parameter points to the file containing the precomputed recommendation lists. WarpRec will load these recommendations and pass them through the configured evaluation pipeline, producing the same output artifacts (evaluation reports, statistical tests, etc.) as for any built-in model.