Development PracticesDevelopment Practices
Quickie15min
BEGINNER

Did AI really make your team faster? Six questions before you trust the number

This talk examines how to validate claims that AI doubled support-team productivity. Using synthetic data, it deconstructs headline metrics by ticket type, weighting, time window, baseline, and confounding changes, then proposes a smaller, size-weighted, more honest measure of performance improvement.

talk.summaryAiDisclaimer

Georgios Arampatzis
Georgios ArampatzisDeloitte

talkDetail.whenAndWhere

Friday, November 20, 14:40-14:55
Room 3 - Alexandros
talks.description
Your team got AI tools, and tickets closed per person nearly doubled. The team lead takes that number to the steering committee, and the CFO asks one question: are these the same kind of tickets? This talk starts from that question and adds the five that follow it: per what, weighted by what, in which window, against which baseline, and what else changed. Using a synthetic team built by one script, we take the headline number apart and rebuild the one that survives: smaller, size-weighted, and honest. This is not a talk about getting gains from AI. It is about how to check the claim that you did.
baselines
normalization
validation
measurement
talks.speakers
Georgios Arampatzis

Georgios Arampatzis

Deloitte

Greece

George is a Manager in Engineering, AI & Data at Deloitte Greece. He leads a data and BI team at a large Greek utility and builds AI accelerators that document, map and migrate data estates.