This talk is about the underlying large language models that underpin modern AI systems. We will present a technical overview of the core algorithms and models that make up these systems, namely the transformer and scaled dot-product attention, taking a few historical detours along the way. We will discuss how these incremental advances and scaling have led to emergent behaviours and systems with remarkable reasoning capabilities. Finally, we'll discuss limitations of current systems, risks and ongoing research initiatives at ANU and internationally.
Stephen Gould - Professor, ARC Future Fellow, and Intelligent Systems Lead, Australian National University (ANU)
Stephen Gould is a Professor of Computer Science at the Australian National University (ANU) and an expert in Artificial Intelligence (AI). He is internationally recognised for his contributions to computer vision, machine learning, and deep learning, with a particular focus on semantic, dynamic, and geometric understanding of images and video. Stephen's career spans academia, industry, and innovation. He has held roles as an Australian Research Council (ARC) Future Fellow, ARC Postdoctoral Fellow, Microsoft Faculty Fellow, Contributed Researcher at Data61, Principal Research Scientist at Amazon Inc, Director of the ARC Centre of Excellence in Robotic Vision, and Amazon Scholar. He also co-founded Sensory Networks, a startup later acquired by Intel. His academic journey includes degrees in mathematics, computer science, and electrical engineering from the University of Sydney, and a masters and a PhD from Stanford University. Stephen's research has led to advances in structured probabilistic models, declarative networks, and large-scale scene understanding. He has authored over 100 peer-reviewed publications and actively collaborates with industry to translate research into real-world impact.
#Oceania#InPerson#ITGoverance