Lorenzo Pacchiardi

selected publications and preprints

1. arXiv

   How to Catch an AI Liar: Lie Detection in Black-Box LLMs by Asking Unrelated Questions

   Pacchiardi, Lorenzo, Chan, Alex J., Mindermann, Soren, Moscovitz, Ilan, Pan, Alexa Y., Gal, Yarin, Evans, Owain, and Brauner, Jan

   2023

   Abs Bib HTML Code

   Large language models (LLMs) can "lie", which we define as outputting false statements despite "knowing" the truth in a demonstrable sense. LLMs might "lie", for example, when instructed to output misinformation. Here, we develop a simple lie detector that requires neither access to the LLM’s activations (black-box) nor ground-truth knowledge of the fact in question. The detector works by asking a predefined set of unrelated follow-up questions after a suspected lie, and feeding the LLM’s yes/no answers into a logistic regression classifier. Despite its simplicity, this lie detector is highly accurate and surprisingly general. When trained on examples from a single setting – prompting GPT-3.5 to lie about factual questions – the detector generalises out-of-distribution to (1) other LLM architectures, (2) LLMs fine-tuned to lie, (3) sycophantic lies, and (4) lies emerging in real-life scenarios such as sales. These results indicate that LLMs have distinctive lie-related behavioural patterns, consistent across architectures and contexts, which could enable general-purpose lie detection.

   @misc{pacchiardi2023catch,
     abbr = {arXiv},
     bibtex_show = {true},
     title = {How to Catch an AI Liar: Lie Detection in Black-Box LLMs by Asking Unrelated Questions},
     author = {Pacchiardi, Lorenzo and Chan, Alex J. and Mindermann, Soren and Moscovitz, Ilan and Pan, Alexa Y. and Gal, Yarin and Evans, Owain and Brauner, Jan},
     journal = {arXiv preprint arXiv:2309.15840},
     year = {2023},
     eprint = {2309.15840},
     archiveprefix = {arXiv},
     primaryclass = {cs.CL},
     selected = {true},
     url = {https://arxiv.org/abs/2309.15840},
     html = {https://arxiv.org/abs/2309.15840},
     code = {https://github.com/lorypack/llm-liedetector},
     publisher = {arXiv},
     copyright = {arXiv.org perpetual, non-exclusive license}
   }

2. arXiv

   Likelihood-Free Inference with Generative Neural Networks via Scoring Rule Minimization

   Pacchiardi, Lorenzo, and Dutta, Ritabrata

   arXiv preprint arXiv:2205.15784 2022

   Abs Bib HTML Code

   Bayesian Likelihood-Free Inference methods yield posterior approximations for simulator models with intractable likelihood. Recently, many works trained neural networks to approximate either the intractable likelihood or the posterior directly. Most proposals use normalizing flows, namely neural networks parametrizing invertible maps used to transform samples from an underlying base measure; the probability density of the transformed samples is then accessible and the normalizing flow can be trained via maximum likelihood on simulated parameter-observation pairs. A recent work [Ramesh et al., 2022] approximated instead the posterior with generative networks, which drop the invertibility requirement and are thus a more flexible class of distributions scaling to high-dimensional and structured data. However, generative networks only allow sampling from the parametrized distribution; for this reason, Ramesh et al. [2022] follows the common solution of adversarial training, where the generative network plays a min-max game against a "critic" network. This procedure is unstable and can lead to a learned distribution underestimating the uncertainty - in extreme cases collapsing to a single point. Here, we propose to approximate the posterior with generative networks trained by Scoring Rule minimization, an overlooked adversarial-free method enabling smooth training and better uncertainty quantification. In simulation studies, the Scoring Rule approach yields better performances with shorter training time with respect to the adversarial framework.

   @article{pacchiardi2022likelihood,
     abbr = {arXiv},
     bibtex_show = {true},
     title = {Likelihood-Free Inference with Generative Neural Networks via Scoring Rule Minimization},
     author = {Pacchiardi, Lorenzo and Dutta, Ritabrata},
     journal = {arXiv preprint arXiv:2205.15784},
     year = {2022},
     eprint = {2205.15784},
     archiveprefix = {arXiv},
     url = {https://arxiv.org/abs/2205.15784},
     html = {https://arxiv.org/abs/2205.15784},
     doi = {10.48550/ARXIV.2205.15784},
     primaryclass = {stat.CO},
     keywords = {Computation (stat.CO), Machine Learning (cs.LG), Methodology (stat.ME), Machine Learning (stat.ML), FOS: Computer and information sciences, FOS: Computer and information sciences},
     publisher = {arXiv},
     copyright = {arXiv.org perpetual, non-exclusive license},
     selected = {true},
     code = {https://github.com/LoryPack/LFI_gen_networks_Scoring_Rules}
   }

3. arXiv

   Probabilistic Forecasting with Generative Networks via Scoring Rule Minimization

   Pacchiardi, Lorenzo, Adewoyin, Rilwan, Dueben, Peter, and Dutta, Ritabrata

   arXiv preprint arXiv:2112.08217 2022

   Abs Bib HTML Code

   Generative networks are often trained to minimize a statistical divergence between the reference distribution and the generative one in an adversarial setting. Some works trained instead generative networks to minimize Scoring Rules, functions assessing how well the generative distribution matches each training sample individually. We show how the Scoring Rule formulation easily extends to the so-called prequential (predictive-sequential) score, whose minimization allows performing probabilistic forecasting with generative networks. This objective leads to adversarial-free training, therefore easily avoiding uncertainty underestimation due to mode collapse, which is a common issue in the adversarial setting and undesirable for probabilistic forecasting. We provide consistency guarantees for the minimizer of the prequential score and employ that to perform probabilistic forecasting for two chaotic dynamical models and a benchmark dataset of global weather observations. For this last example, we define scoring rules for spatial data by drawing from the relevant literature, with which we obtain better uncertainty quantification with little hyperparameter tuning compared to adversarial training.

   @article{pacchiardi2021probabilistic,
     abbr = {arXiv},
     bibtex_show = {true},
     title = {Probabilistic Forecasting with Generative Networks via Scoring Rule Minimization},
     author = {Pacchiardi, Lorenzo and Adewoyin, Rilwan and Dueben, Peter and Dutta, Ritabrata},
     journal = {arXiv preprint arXiv:2112.08217},
     year = {2022},
     eprint = {2112.08217},
     archiveprefix = {arXiv},
     url = {https://arxiv.org/abs/2112.08217},
     html = {https://arxiv.org/abs/2112.08217},
     primaryclass = {stat.ML},
     selected = {true},
     code = {https://github.com/LoryPack/GenerativeNetworksScoringRulesProbabilisticForecasting}
   }

4. JMLR

   Score Matched Neural Exponential Families for Likelihood-Free Inference

   Pacchiardi, Lorenzo, and Dutta, Ritabrata

   Journal of Machine Learning Research 2022

   Abs Bib HTML Code

   Bayesian Likelihood-Free Inference (LFI) approaches allow to obtain posterior distributions for stochastic models with intractable likelihood, by relying on model simulations. In Approximate Bayesian Computation (ABC), a popular LFI method, summary statistics are used to reduce data dimensionality. ABC algorithms adaptively tailor simulations to the observation in order to sample from an approximate posterior, whose form depends on the chosen statistics. In this work, we introduce a new way to learn ABC statistics: we first generate parameter-simulation pairs from the model independently on the observation; then, we use Score Matching to train a neural conditional exponential family to approximate the likelihood. The exponential family is the largest class of distributions with fixed-size sufficient statistics; thus, we use them in ABC, which is intuitively appealing and has state-of-the-art performance. In parallel, we insert our likelihood approximation in an MCMC for doubly intractable distributions to draw posterior samples. We can repeat that for any number of observations with no additional model simulations, with performance comparable to related approaches. We validate our methods on toy models with known likelihood and a large-dimensional time-series model.

   @article{pacchiardi2020score,
     abbr = {JMLR},
     bibtex_show = {true},
     author = {Pacchiardi, Lorenzo and Dutta, Ritabrata},
     title = {Score Matched Neural Exponential Families for Likelihood-Free Inference},
     journal = {Journal of Machine Learning Research},
     year = {2022},
     volume = {23},
     number = {38},
     pages = {1-71},
     html = {http://jmlr.org/papers/v23/21-0061.html},
     url = {http://jmlr.org/papers/v23/21-0061.html},
     code = {http://github.com/LoryPack/SM-ExpFam-LFI},
     selected = {true}
   }

5. JSS

   ABCpy: A High-Performance Computing Perspective to Approximate Bayesian Computation

   Dutta, Ritabrata, Schoengens, Marcel, Pacchiardi, Lorenzo, Ummadisingu, Avinash, Widmer, Nicole, Künzli, Pierre, Onnela, Jukka-Pekka, and Mira, Antonietta

   Journal of Statistical Software 2021

   Abs Bib HTML Code

   ABCpy is a highly modular scientific library for approximate Bayesian computation (ABC) written in Python. The main contribution of this paper is to document a software engineering effort that enables domain scientists to easily apply ABC to their research without being ABC experts; using ABCpy they can easily run large parallel simulations without much knowledge about parallelization. Further, ABCpy enables ABC experts to easily develop new inference schemes and evaluate them in a standardized environment and to extend the library with new algorithms. These benefits come mainly from the modularity of ABCpy. We give an overview of the design of ABCpy and provide a performance evaluation concentrating on parallelization. This points us towards the inherent imbalance in some of the ABC algorithms. We develop a dynamic scheduling MPI implementation to mitigate this issue and evaluate the various ABC algorithms according to their adaptability towards high-performance computing.

   @article{JSSABCpy,
     abbr = {JSS},
     bibtex_show = {true},
     title = {ABCpy: A High-Performance Computing Perspective to Approximate Bayesian Computation},
     author = {Dutta, Ritabrata and Schoengens, Marcel and Pacchiardi, Lorenzo and Ummadisingu, Avinash and Widmer, Nicole and K{\"u}nzli, Pierre and Onnela, Jukka-Pekka and Mira, Antonietta},
     volume = {100},
     html = {https://www.jstatsoft.org/index.php/jss/article/view/v100i07},
     url = {https://www.jstatsoft.org/index.php/jss/article/view/v100i07},
     doi = {10.18637/jss.v100.i07},
     number = {7},
     journal = {Journal of Statistical Software},
     year = {2021},
     pages = {1--38},
     code = {https://github.com/eth-cscs/abcpy},
     selected = {true}
   }