Master's Thesis - Attention - Based Hallucination Probability Estimation in Large Language Models

• Join our Team

About this opportunity:

Large Language Models produce fluent text that may contain fabricated claims, a failure mode known as hallucination. Estimating when a model is likely to hallucinate is critical for deployment in high-stakes domains.

As LLMs are increasingly integrated into telecommunications infrastructure for network optimisation, fault diagnosis, and autonomous control, ensuring the reliability and trustworthiness of their outputs becomes essential. A model that hallucinates while managing a mobile network could degrade service or mask real faults.

Recent work shows that attention patterns carry a measurable hallucination signal. Attention sinks-tokens that accumulate disproportionate attention from later tokens-correlate with hallucinated outputs. However, important questions remain. Hallucinations can be intrinsic, contradicting provided context, or extrinsic, fabricating absent information. Current attention-based methods handle intrinsic cases well but may struggle with extrinsic cases. The reasons why attention sinks form, and whether they are connected to training and memorisation, also remain unclear.

The goal of this thesis is to evaluate how reliably attention predicts hallucination across tasks and hallucination types, and to develop a risk-estimation framework based on this analysis. The work will use datasets with easily verifiable outputs, such as question-answering benchmarks, mathematical reasoning, and/or code generation.

The thesis will be conducted by one student, corresponds to 30 hp, and is expected to start in early January 2027. The location is Stockholm, Sweden. Supervisor: Konrad Nowosadko.

What you will do:

• Reproduce results from principal attention-based detection methods, such as SinkProbe and Head Entropy.
• Evaluate each method across task types and hallucination types, including intrinsic and extrinsic hallucinations.
• Incorporate the studied methods into a unified framework.
• Develop a hallucination-risk estimation method.
• Analyse the resulting framework by characterising failure modes, performance, and accuracy.
• Optionally, explore why attention sinks form during hallucination, including whether the memorisation status of queried knowledge influences sink formation.
• Deliver a reproducible experimental repository and present the results.

The skills you bring:

• You are a final-year Master's student in Computer Science, Machine Learning, Applied Mathematics, or a related programme.
• You have a solid foundation in machine learning, deep learning, and statistics.
• You have strong Python skills and experience with PyTorch.
• You are familiar with transformer architectures and attention mechanisms.
• You can work independently and communicate your findings clearly.
• You are curious and enthusiastic about exploratory research.

Experience with mechanistic interpretability or previous work with large language models is considered a plus.

Keywords: hallucination detection, attention, large language models, transformers, and mechanistic interpretability.

Why join Ericsson?At Ericsson, you´ll have an outstanding opportunity. The chance to use your skills and imagination to push the boundaries of what´s possible. To build solutions never seen before to some of the world's toughest problems. You´ll be challenged, but you won't be alone. You´ll be joining a team of diverse innovators, all driven to go beyond the status quo to craft what comes next.

Primary country and city: Sweden (SE) || Stockholm

Req ID: 791198