Lorenzo Pacchiardi

Research Associate, University of Cambridge

New paper: How to Catch an AI Liar: Lie Detection in Black-Box LLMs by Asking Unrelated Questions

I am a Research Associate at the Leverhulme Centre for the Future of Intelligence at the University of Cambridge, where I develop a framework for evaluating the cognitive capabilities of Large Language Models, together with Prof José Hernández-Orallo and Dr Lucy Cheke.

I previously worked on detecting lying in large language models with Dr Owain Evans and on technical standards for AI for the EU AI Act at the Future of Life Institute. I am deeply interested in AI policy (particularly at the EU level).

I obtained a PhD in Statistics and Machine Learning at Oxford, during which I worked on Bayesian simulation-based inference, generative models and probabilistic forecasting (with applications to meteorology). My supervisors were Prof. Ritabrata Dutta (Uni. Warwick) and Prof. Geoff Nicholls (Uni. Oxford).

Before my PhD studies, I obtained a Bachelor’s degree in Physical Engineering from Politecnico di Torino (Italy) and an MSc in Physics of Complex Systems from Politecnico di Torino and Université Paris-Sud, France. I carried out my MSc thesis at LightOn, a machine learning startup in Paris.

news

Aug 1, 2023 I am happy to communicate that I will join the Leverhulme Centre for the Future of Intelligence at the University of Cambridge in October 2023 :smile:
Jul 15, 2023 Today, I formally graduated from my PhD at Oxford! :smile:
Dec 20, 2022 I won a travel award at the International Conference for Statistics and Data Science by the Institute of Mathematical Statistics. I really enjoyed the opportunity to present my work there!
Nov 17, 2022 On the 9th of November, I successfully defended my PhD thesis :champagne:. A profound thanks to my examiners Chris Holmes and Chris Oates for their insightful comments and suggestions.
Jun 3, 2022 I have two preprints on arXiv exploring an alternative to adversarial training to train generative networks; the first applies it to probabilistic forecasting, while the second is concerned with Likelihood-Free Inference. I feel that is a promising overlooked approach. Happy to hear any feedback! :smile:

selected publications and preprints

  1. arXiv
    How to Catch an AI Liar: Lie Detection in Black-Box LLMs by Asking Unrelated Questions
    Pacchiardi, Lorenzo, Chan, Alex J., Mindermann, Soren, Moscovitz, Ilan, Pan, Alexa Y., Gal, Yarin, Evans, Owain, and Brauner, Jan
    2023
  2. arXiv
    Likelihood-Free Inference with Generative Neural Networks via Scoring Rule Minimization
    Pacchiardi, Lorenzo, and Dutta, Ritabrata
    arXiv preprint arXiv:2205.15784 2022
  3. arXiv
    Probabilistic Forecasting with Generative Networks via Scoring Rule Minimization
    Pacchiardi, Lorenzo, Adewoyin, Rilwan, Dueben, Peter, and Dutta, Ritabrata
    arXiv preprint arXiv:2112.08217 2022
  4. JMLR
    Score Matched Neural Exponential Families for Likelihood-Free Inference
    Pacchiardi, Lorenzo, and Dutta, Ritabrata
    Journal of Machine Learning Research 2022
  5. JSS
    ABCpy: A High-Performance Computing Perspective to Approximate Bayesian Computation
    Dutta, Ritabrata, Schoengens, Marcel, Pacchiardi, Lorenzo, Ummadisingu, Avinash, Widmer, Nicole, KĂĽnzli, Pierre, Onnela, Jukka-Pekka, and Mira, Antonietta
    Journal of Statistical Software 2021