KDD 2026 Tutorial

Trustworthy LLM-Based Agents for Data-Centric AI

Sunday, August 9, 2026 · 1:00 PM–5:00 PM · PM09 · International Convention Center Jeju

When agents use tools, memory, and other agents, trust failures become real system failures.

View Program Why Attend? Meet the Organizers

Overview

LLM-based agents are moving beyond chatbots into systems that plan, reason, use tools, maintain memory, and coordinate with other agents. As they enter high-stakes domains such as healthcare, finance, science, engineering, and commerce, failure is no longer just an incorrect answer; it can become an unsafe tool call, a poisoned memory, brittle behavior under data shift, or an unreliable high-stakes decision.

This tutorial presents a data-centric framework for making these systems trustworthy. We connect robust learning under shifting, noisy, and adversarial data; security for tool-using and multi-agent workflows; domain-grounded requirements in science, engineering, medicine, and commerce; and multi-dimensional evaluation that exposes trade-offs instead of hiding them in one score.

01

Robustness & Generalizability

How agents behave under distribution shift, noisy data, adversarial conditions, changing tasks, and component-level interventions.

02

Agent Security

How indirect prompt injection, memory poisoning, unsafe tool use, supply-chain risks, and multi-agent propagation create new attack surfaces.

03

Domain-Grounded Trust

Why scientific, engineering, medical, and commercial agents require different constraints, validity checks, and failure criteria.

04

Evaluation & Trade-offs

How to evaluate robustness, security, fairness, reliability, and utility without collapsing trustworthiness into a single score.

Why Attend?

Beyond Prompt Attacks

We move from isolated prompt attacks to system-level risks: unsafe tool execution, poisoned memory, cross-agent propagation, and third-party component vulnerabilities.

Beyond Benchmark Scores

We show why trustworthiness cannot be reduced to one number, and how evaluation must expose trade-offs among robustness, security, fairness, utility, and oversight.

Beyond Static Surveys

Each topic starts from realistic deployment scenarios and asks what can go wrong, what failures are unacceptable, and how agents should be designed.

Built for KDD

The tutorial connects agent trustworthiness to data-centric AI: noisy data, shifting distributions, auditable workflows, benchmark design, and reliable deployment.

Program

Tentative 4-hour outline for KDD 2026 Tutorial PM09. Room information will be added when available.

Time Session Lead
1:00–1:15 PM Introduction: LLM agents, data-centric AI, and the tutorial trust framework All organizers
1:15–1:55 PM Generalizability: Distribution shift, noise, component interventions, and agent frameworks Jian Pei and Minxing Zhang
1:55–2:45 PM Security, Attacks, and Defense: Prompt injection, memory poisoning, unsafe tools, and multi-agent attack propagation Tianlong Chen
2:45–3:00 PM Break
3:00–3:50 PM Trustworthy LLM-Based Agents in Applications: Science, engineering, medicine, and commerce case studies Liang Zhao
3:50–4:35 PM Evaluation and Benchmarking: Trustworthiness dimensions, agent-specific metrics, and extensible benchmarks Jian Pei and Minxing Zhang
4:35–5:00 PM Summary, Discussion, and Future Directions: Recap, open problems, and KDD community directions All organizers

Organizers

Tianlong Chen

Tianlong Chen

UNC at Chapel Hill

Assistant Professor at UNC Chapel Hill and Chief AI Scientist at hireEZ. His work on reliable machine learning, LLM agent threat models, multi-agent attacks, and practical defenses anchors the tutorial's security pillar.

Jian Pei

Jian Pei

Duke University

Arthur S. Pearse Distinguished Professor at Duke University. His research in data mining, robust learning, data quality, and responsible AI grounds the tutorial's generalizability and evaluation perspectives.

Minxing Zhang

Minxing Zhang

Duke University

Ph.D. student at Duke University working on LLM, LLM-agent, and AI-model trustworthiness. His work on conversation generation evaluation and LLM judge quality evaluation supports the tutorial's benchmarking and empirical evaluation perspectives.

Liang Zhao

Liang Zhao

Emory University

Winship Distinguished Research Professor and Associate Professor at Emory University. His work on trustworthy, explainable, and knowledge-augmented AI informs domain-grounded applications.

Materials

Tutorial materials will be posted here before KDD 2026. The website will include slides and tutorial updates for evaluating and securing LLM-based agents.

Contact

Tianlong Chen: tianlong@cs.unc.edu · Jian Pei: j.pei@duke.edu · Minxing Zhang: minxing.zhang@duke.edu · Liang Zhao: liang.zhao@emory.edu