Toward Reciprocity-Aware Distributed Learning in Referral Networks

摘要

Distributed learning in expert referral networks is an emerging challenge in the intersection of Active Learning and Multi-Agent Reinforcement Learning, where experts—humans or automated agents— have varying skills across different topics and can redirect difficult problem instances to connected colleagues with more appropriate expertise. The learning-to-refer challenge involves estimating colleagues' topic-conditioned skills for appropriate referrals. Prior research has investigated different reinforcement learning algorithms both with uninforma-tive priors and partially available (potentially noisy) priors. However, most human experts expect mutually-rewarding referrals, with return referrals on their expertise areas so that both (or all) parties benefit from networking, rather than one-sided referral flow. This paper analyzes the extent of referral reciprocity imbalance present in high-performance referral-learning algorithms, specifically multi-armed bandit (MAB) methods belonging to two broad categories - frequentist and Bayesian - and demonstrate that both algorithms suffer considerably from reciprocity imbalance. The paper proposes modifications to enable distributed learning methods to better balance referral reciprocity and thus make referral networks win-win for all parties. Extensive empirical evaluations demonstrate substantial improvement in mitigating reciprocity imbalance, while maintaining reasonably high overall solution performance.