Nihar Shah

I am an Associate Professor at Carnegie Mellon University in the Machine Learning and the Computer Science departments. I work in the areas of machine learning, statistics, information theory and game theory. My current takes principled and practical approaches to address various application challenges that require evaluations, such as in scientific peer review, university admissions, hiring, etc..

Distributed human evaluations are integral to various applications where a set of items is assessed by a group of individuals, each person evaluating only a subset of the items and each item being evaluated by only a handful of individuals. The decentralization of evaluations, however, often leads to a host of challenges including instances of fraud, subjectivity, miscalibration, breaches of privacy, prejudices, and operational inefficiencies. Our work addresses these challenges via algorithm design, theoretical (mathematical) proofs, experiments for policy design, and real-world deployments and evaluations.