Samuel Adebayo

Research programme

Useful intelligence under imperfect information.

I work on intelligent systems that have to infer, decide and sometimes act when observations are incomplete, conditions shift and failure has practical consequences. The settings range from human-machine perception to physical assembly and deployed agentic systems.

QUB-PHEO multi-view human interaction footage
01 · interactionHuman cues over time
SLYKLatent gaze-estimation framework
02 · representationNon-invasive gaze estimation
ConPose detection and pose architecture
03 · embodimentPerception tied to action

The common thread

My PhD work focused on visual cues in collaborative assembly: gaze, hand and object motion, temporal behaviour and intention inference. The underlying question was how a system can make a useful judgement from signals that are partial and changing.

Core programme: reliable intelligent systems that learn from multimodal evidence and make useful decisions in real-world environments.

I now study that problem in additional settings. In embodied systems, perception has to remain meaningful as the physical state changes through interaction. In LLM and agentic systems, reliability depends on what the model is allowed to do, how its behaviour is evaluated and when control returns to a human.

Perception and interaction

This line of work began with the integration of gaze, hand motion and object interaction for intention inference. The Hand-Eye-Object framework combined synchronised visual cues with temporal learning, then later work removed the dependence on invasive eye-tracking and expanded into multi-view interaction data.

Hand-Eye-Object intention inference pipeline
Hand-Eye-Object intention inference: tracked visual cues are normalised and fused through a bidirectional LSTM model for intention classification.
QUB-PHEO multi-view footage showing collaborative assembly interaction
QUB-PHEO: multi-view visual data for human engagement and intention inference in collaborative assembly.

SLYKLatent

SLYKLatent developed the non-invasive gaze-estimation strand of this programme. It combines self-supervised representation learning with a patch-aware downstream network that uses full-face and eye-region information to estimate gaze under variation in appearance and capture conditions.

SLYKLatent framework showing self-supervised pretraining and downstream gaze estimation
SLYKLatent: self-supervised m-BYOL pretraining followed by patch-aware downstream fine-tuning for gaze estimation.

ARISE

Assembly and Robotics Innovation in Steel Building Erection is a US–Ireland research programme involving Queen's University Belfast, New York University, the University of Galway and the University of Texas at San Antonio, supported through linked DfE, NSF and Research Ireland awards.

As Research Fellow on the Queen's team, I lead machine-learning and computer-vision research for robotic structural assembly. I conceived and led ConPose from architecture and experimental design through implementation and manuscript, and I contribute to ISC-Perception and the wider perception stack.

ConPose detection and pose architecture
ConPose: a jointly trained detector and pose head operating directly from RGB regions of interest.
Open the ISC-Perception cover
ISC-Perception was selected for a Buildings cover. Open cover ↗

From perception to action

I am increasingly interested in representations that remain useful after simple visual localisation stops being enough. During physical interaction, visibility changes, contact introduces new evidence and the system must judge whether the state is still feasible. This brings together geometry, multimodal state estimation, temporal learning and bounded autonomous decision-making.

Compact ConPose architecture
The ConPose architecture is deliberately shown at a compact scale here: the broader research question is how perception can provide state information that remains useful for downstream action.

Reliable LLM and agentic systems

Production AI provides a second laboratory for reliability. At ISx4 I work on tool-using agents and LLM systems where evaluation, observability, policy constraints and escalation determine whether model capability becomes a dependable workflow.

I am especially interested in failure taxonomies, regression evaluation, trace-level diagnostics and evidence for operating boundaries. This connects production work on LLM observability, LLMOps and AI assurance with a broader question: how do we evaluate systems that do more than return a prediction?

Diagram showing retrieval, tools, policy controls, evaluation and human escalation around an agent
A conceptual reliability layer around a tool-using agent: grounded context, bounded actions, policy controls, evaluation and human escalation.

Methods

My work draws on deep learning and statistical machine learning, multimodal and temporal modelling, reinforcement learning, simulation, calibration and uncertainty-aware evaluation. In computer vision I work with detection, pose and geometric inference, camera calibration and synthetic-to-real validation. In deployed AI systems the emphasis expands to retrieval, tool use, observability, testing and assurance.