Fair Learning for Bias Mitigation and Quality Optimization in Paper Recommendation

📝 Paper Summary

Fairness in Recommender Systems Academic Peer Review

Fair-PaperRec optimizes academic paper acceptance by incorporating a fairness-aware penalty into a neural network, balancing demographic parity for race and country with paper quality metrics.

Core Problem

Double-blind reviews fail to eliminate systemic demographic biases (e.g., against racial minorities or developing countries), and standard recommendation algorithms trained on this data exacerbate these disparities by optimizing solely for accuracy.

Why it matters:

High-prestige institutions and certain demographics continue to receive favorable reviews despite anonymization, marginalizing underrepresented groups
Automated systems for grant distribution and paper selection risk perpetuating historical inequalities if they prioritize predictive accuracy over demographic justice

Concrete Example: A standard accuracy-focused model might reject high-quality papers from 'underdeveloped' countries because historical training data favors 'developed' nations. Fair-PaperRec adjusts the acceptance threshold to ensure statistical parity without significantly dropping quality metrics like h-index.

Key Novelty

Fair-PaperRec (Fairness-aware MultiLayer Perceptron)

Uses a specialized fairness loss function during training that penalizes deviations from demographic parity (equal acceptance rates across groups) for multiple attributes (race and country) simultaneously
Decouples protected attributes from input features during inference to prevent direct bias, while using them during training to guide the model toward equitable probability distributions

Architecture

The MLP-based neural network architecture for Fair-PaperRec

Evaluation Highlights

42.03% increase in participation of underrepresented groups compared to heuristic baselines
3.16% improvement in overall utility (measured by h-index), contradicting the assumption that fairness requires sacrificing academic rigor
Achieves optimal trade-off between race/country diversity and utility at fairness regularization parameter λ=2.5 to 3

Breakthrough Assessment

5/10

Applies established fairness concepts (demographic parity, MLP regularization) to a specific domain (academic review). Results are positive, but the method relies on standard architectural components.

⚙️ Technical Details

Problem Definition

Setting: Binary classification task predicting paper acceptance while minimizing statistical dependence between acceptance and protected attributes

Inputs: Paper features X_p (e.g., h-index, title features), excluding protected attributes

Outputs: Acceptance probability y_hat_p

Pipeline Flow

Preprocessing (One-hot encoding, Normalization)
Fairness-Aware Training (MLP with Fairness Loss)
Prediction Aggregation
Ranking & Selection

System Modules

Preprocessing

Prepare features and separate protected attributes

Model or implementation: Deterministic scripts

Fair-PaperRec Model

Predict acceptance probability while minimizing fairness loss

Model or implementation: Two-hidden-layer MLP (ReLU, Batch Norm, Sigmoid output)

Ranker

Select top papers to meet acceptance quota

Model or implementation: Sorting algorithm

Modeling

Base Model: Two-hidden-layer MLP

Training Method: Supervised learning with fairness regularization

Objective Functions:

Purpose: Accuracy.

Formally: Binary Cross-Entropy Loss L(f(X_p), y_p)
Purpose: Fairness (Demographic Parity).

Formally: Sum of absolute differences between acceptance probabilities of protected vs. non-protected groups for Race and Country
Purpose: Total Loss.

Formally: L_total = L_prediction + λ * L_fairness

Key Hyperparameters:

learning_rate: 0.001
epochs: 50
optimizer: Adam
+ 2 more
lambda: Tunable (1 to 10), optimal around 2.5-3
early_stopping_patience: 10 epochs

Compute: Two NVIDIA Quadro RTX 4000 GPUs

Comparison to Prior Work

vs. Heuristic Re-ranking: Fair-PaperRec incorporates fairness into the learning objective (in-processing) rather than just reordering outputs, avoiding local optima
vs. DeepFair: Specifically adapted for academic paper recommendation with intersectional attributes (race + country) rather than single-attribute commercial items

Limitations

Trade-off between utility and fairness persists at higher lambda values
Relies on binary gender classification and specific race categories, which may not capture full demographic complexity
Requires explicit protected attribute data which may not always be available in double-blind scenarios (though used here from metadata)

Reproducibility

Datasets are standard academic conference data (SIGCHI 2017, DIS 2017, IUI 2017). Code availability is not explicitly provided in the text. Implementation uses PyTorch.

📊 Experiments & Results

Evaluation Setup

Re-ranking 530 papers from SIGCHI, DIS, and IUI 2017 to simulating acceptance decisions

Benchmarks:

SIGCHI/DIS/IUI 2017 Dataset (Paper Recommendation / Binary Classification)

Metrics:

Macro Gain (diversity increase)
Micro Gain (author-level diversity)
Utility Gain (weighted h-index)
F-measure (balance of diversity and utility)
Statistical methodology: Experiments repeated 5 times, standard deviations reported (implied by text 'standard deviations provided').

Key Results

Benchmark	Metric	Baseline	This Paper	Δ
Ablation study on fairness weights and lambda parameter shows the trade-off between race/country diversity and utility.
Combined Conference Data	Race Macro Gain	0.0	30.51	+30.51
Combined Conference Data	Race Micro Gain	0.0	46.3	+46.3
Combined Conference Data	Utility Loss	0.0	-2.46	-2.46

Experiment Figures

Comparison of Utility Gain, Race Fairness, and Country Fairness across different lambda values

Main Takeaways

Fairness constraints (lambda=2.5 to 3) can significantly boost representation of underrepresented races and countries with minimal or positive impact on overall paper quality (h-index).
Race and Country require different regularization strengths (lambda); Race required a higher lambda (3) than Country (2.5) to address higher initial disparity ratios.
Macro diversity (paper level) is easier to improve stably than micro diversity (author level), which shows more variability.

📚 Prerequisite Knowledge

Prerequisites

Binary Cross-Entropy Loss
Demographic/Statistical Parity
Multi-Layer Perceptron (MLP) basics
h-index (bibliometrics)

Key Terms

Demographic Parity: A fairness criterion requiring that the probability of a positive outcome (acceptance) is equal across different demographic groups (e.g., protected vs. non-protected)

h-index: A metric measuring the productivity and citation impact of a researcher; used here as a proxy for paper/author quality

SIGCHI: Special Interest Group on Computer-Human Interaction; a major conference venue used as a data source

Macro Gain: The percentage increase in the diversity of each feature within selected papers compared to a baseline

Micro Gain: The percentage increase in the diversity of each feature among authors of selected papers

MLP: Multilayer Perceptron—a class of feedforward artificial neural network