Angelos Poulis

Center for Computing & Data Sciences

665 Commonwealth Ave

Boston, MA 02215

Hello I am Angelos, a PhD student at Boston University (BU), advised by Professors Evimaria Terzi and Mark Crovella.

My research focuses on mechanistic interpretability of large language models (LLMs). Currently, I study how models encode truth-related representations and the geometry of these representations in activation space.

Before BU, I completed my B.Sc. in Informatics and Telecommunications at the National and Kapodistrian University of Athens, Greece. During that time, I worked on evaluating the logical reasoning capabilities of LLMs over description logic ontologies, advised by Prof. Manolis Koubarakis.

I also did a research internship at the KDD Lab, NKUA, advised by Prof. Dimitrios Gunopulos, where I worked on fairness algorithms in machine learning.

news

Apr 04, 2026	New preprint “Testing the Limits of Truth Directions in LLMs”: Are truth directions in LLMs universal? We observe significant differences across probing layers, model instructions, and task difficulties.
Aug 22, 2025	Presenting a poster at the 2nd New England Mechanistic Interpretability (NEMI) workshop about representations of truth in LLMs.
Oct 09, 2024	Our paper Transformers in the Service of Description Logic-Based Contexts has been published at EKAW 2024. We evaluate transformer-based LMs on reasoning over expressive description logics.
Sep 01, 2024	Started my PhD in Computer Science at Boston University!
Mar 01, 2024	Graduated from the National and Kapodistrian University of Athens!

publications

Preprint

Testing the Limits of Truth Directions in LLMs

Angelos Poulis, Mark Crovella, and Evimaria Terzi

2026

Abs arXiv

Large language models (LLMs) have been shown to encode truth of statements in their activation space along a linear truth direction. Previous studies have argued that these directions are universal in certain aspects, while more recent work has questioned this conclusion drawing on limited generalization across some settings. In this work, we identify a number of limits of truth-direction universality that have not been previously understood. We first show that truth directions are highly layer-dependent, and that a full understanding of universality requires probing at many layers in the model. We then show that truth directions depend heavily on task type, emerging in earlier layers for factual and later layers for reasoning tasks; they also vary in performance across levels of task complexity. Finally, we show that model instructions dramatically affect truth directions; simple correctness evaluation instructions significantly affect the generalization ability of truth probes. Our findings indicate that universality claims for truth directions are more limited than previously known, with significant differences observable for various model layers, task difficulties, task types, and prompt templates.
EKAW

Transformers in the Service of Description Logic-Based Contexts

Angelos Poulis, Eleni Tsalapati, and Manolis Koubarakis

In Knowledge Engineering and Knowledge Management, 2025

Abs arXiv

Recent advancements in transformer-based models have initiated research interests in investigating their ability to learn to perform reasoning tasks. However, most of the contexts used for this purpose are in practice very simple: generated from short (fragments of) first-order logic sentences with only a few logical operators and quantifiers. In this work, we construct the natural language dataset, DELTA_D, using the description logic language ALCQ. DELTA_D contains 384K examples, and increases in two dimensions: i) reasoning depth, and ii) linguistic complexity. In this way, we systematically investigate the reasoning ability of a supervised fine-tuned DeBERTa-based model and of two large language models (GPT-3.5, GPT-4) with few-shot prompting. Our results demonstrate that the DeBERTa-based model can master the reasoning task and that the performance of GPTs can improve significantly even when a small number of samples is provided (9 shots). We open-source our code and datasets.