Lorenzo Pacchiardi

prof_pic.jpg

Research Associate, University of Cambridge

New paper: How to Catch an AI Liar: Lie Detection in Black-Box LLMs by Asking Unrelated Questions

I am a Research Associate at the Leverhulme Centre for the Future of Intelligence at the University of Cambridge, where I develop a framework for evaluating the cognitive capabilities of Large Language Models, together with Prof José Hernández-Orallo and Dr Lucy Cheke.

I previously worked on detecting lying in large language models with Dr Owain Evans and on technical standards for AI for the EU AI Act at the Future of Life Institute. I am deeply interested in AI policy (particularly at the EU level).

I obtained a PhD in Statistics and Machine Learning at Oxford, during which I worked on Bayesian simulation-based inference, generative models and probabilistic forecasting (with applications to meteorology). My supervisors were Prof. Ritabrata Dutta (Uni. Warwick) and Prof. Geoff Nicholls (Uni. Oxford).

Before my PhD studies, I obtained a Bachelor’s degree in Physical Engineering from Politecnico di Torino (Italy) and an MSc in Physics of Complex Systems from Politecnico di Torino and Université Paris-Sud, France. I carried out my MSc thesis at LightOn, a machine learning startup in Paris.

news

Feb 01, 2024 I co-authored an op-ed on the OECD.AI policy website about how an exemption for “therapeutic purposes” in the EU AI Act could serve as a loophole.
Jan 16, 2024 Our paper How to Catch an AI Liar: Lie Detection in Black-Box LLMs by Asking Unrelated Questions has been accepted at ICLR 2024! :tada:
Aug 01, 2023 I am happy to communicate that I will join the Leverhulme Centre for the Future of Intelligence at the University of Cambridge in October 2023 :smile:
Jul 15, 2023 Today, I formally graduated from my PhD at Oxford! :smile:
Dec 20, 2022 I won a travel award at the International Conference for Statistics and Data Science by the Institute of Mathematical Statistics. I really enjoyed the opportunity to present my work there!

selected publications

  1. ICLR 2024
    How to Catch an AI Liar: Lie Detection in Black-Box LLMs by Asking Unrelated Questions
    Lorenzo Pacchiardi ,  Alex James Chan ,  Sören Mindermann ,  Ilan Moscovitz ,  Alexa Yue Pan ,  Yarin Gal ,  Owain Evans ,  and  Jan M. Brauner
    2024
  2. arXiv
    Likelihood-Free Inference with Generative Neural Networks via Scoring Rule Minimization
    Lorenzo Pacchiardi ,  and  Ritabrata Dutta
    arXiv preprint arXiv:2205.15784, 2022
  3. arXiv
    Probabilistic Forecasting with Generative Networks via Scoring Rule Minimization
    Lorenzo Pacchiardi ,  Rilwan Adewoyin ,  Peter Dueben ,  and  Ritabrata Dutta
    arXiv preprint arXiv:2112.08217, 2022
  4. JMLR
    Score Matched Neural Exponential Families for Likelihood-Free Inference
    Lorenzo Pacchiardi ,  and  Ritabrata Dutta
    Journal of Machine Learning Research, 2022
  5. JSS
    ABCpy: A High-Performance Computing Perspective to Approximate Bayesian Computation
    Ritabrata Dutta ,  Marcel Schoengens ,  Lorenzo Pacchiardi ,  Avinash Ummadisingu ,  Nicole Widmer ,  Pierre Künzli ,  Jukka-Pekka Onnela ,  and  Antonietta Mira
    Journal of Statistical Software, 2021