Welcome! I am a first-year Ph.D. student in the Department of Industrial & Operations Engineering at the University of Michigan, Ann Arbor, fortunate to be advised by Professor Raed Al Kontar. My research focuses on integrating statistics and generative AI, with particular emphasis on uncertainty quantification in large language models and generative AI under heterogeneity. I am also deeply interested in exploring the theoretical foundations of these methods. Thanks for visiting!

Publications

Inv-Entropy: A Fully Probabilistic Framework for Uncertainty Quantification in Language Models Haoyi Song, Ruihan Ji, Naichen Shi, Fan Lai, Raed Al Kontar.Link

Large language models (LLMs) have transformed natural language processing, but their reliable deployment requires effective uncertainty quantification (UQ). Existing UQ methods are often heuristic and lack a fully probabilistic foundation. This paper begins by providing a theoretical justification for the role of perturbations in UQ for LLMs. We then introduce a dual random walk perspective, modeling input–output pairs as two Markov chains with transition probabilities defined by semantic similarity. Building on this, we propose a fully probabilistic framework based on an inverse model, which quantifies uncertainty by evaluating the diversity of the input space conditioned on a given output through systematic perturbations. Within this framework, we define a new uncertainty measure, Inv-Entropy. A key strength of our framework is its flexibility: it supports various definitions of uncertainty measures, embeddings, perturbation strategies, and similarity metrics. In addition, we introduce a new evaluation metric, Temperature Sensitivity of Uncertainty (TSU), which directly assesses uncertainty without relying on correctness as a proxy. Extensive experiments demonstrate that Inv-Entropy outperforms existing semantic UQ methods.

News

• September 2025: Our paper, “Inv-Entropy: A Fully Probabilistic Framework for Uncertainty Quantification in Language Models”, is selected as the finalist for the Data Mining best paper competition (student track) in INFORMS, 2025!