I lead a team of applied scientists at Amazon focused on creating training data for large language models. My work centers on developing methods to improve LLMs’ knowledge and reasoning capabilities, with the goal of making them more useful and reliable tools for people.
More broadly, I’m interested in advancing the capabilities of foundation models through better data, evaluation, and understanding.
I previously worked on information extraction from natural language and semi-structured text. I hold a PhD in Computer Science & Engineering from the University of Washington, where I was advised by Hannaneh Hajishirzi and additionally supervised by Xin Luna Dong. Our work on the Ceres project was the subject of Luna's keynote address at the 2020 Conference on Information and Knowledge Management (CIKM).
My background also includes a BA in English from Harvard University and an MA in Interdisciplinary Computer Science from Mills College. Prior to my PhD, I worked with the Big Data Analytics & Machine Intelligence group at NASA Langley Research Center, taught at Peking University, and did some time in the business world.
Publications
Ritesh Sarkhel, Binxuan Huang, Colin Lockard, Prashant Shiralkar
in Proceedings of the VLDB Endowment (PVLDB), 2023
Yinghao Li, Colin Lockard, Prashant Shiralkar, Chao Zhang
in Proceedings of the Association for Computational Linguistics (ACL), 2023
Aidan San, Yuan Zhuang, Jan Bakus, Colin Lockard, David Ciemiewicz, Sandeep Atluri, Yangfeng Ji, Kevin Small, Heba Elfardy
in Proceedings of the Association for Computational Linguistics (ACL), 2023
Xiang Deng, Prashant Shiralkar, Colin Lockard, Binxuan Huang, Huan Sun
in CoRR, 2022
Daheng Wang, Prashant Shiralkar, Colin Lockard, Binxuan Huang, Xin Luna Dong, Meng Jiang
in Proceedings of the Web Conference (WWW), 2021
Colin Lockard, Prashant Shiralkar, Xin Luna Dong, Hannaneh Hajishirzi
in Proceedings of the Association for Computational Linguistics (ACL), 2020
[slides]
Colin Lockard, Prashant Shiralkar, Xin Luna Dong
in Proceedings of the North American Chapter of the Association for Computational Linguistics (NAACL), 2019
[slides] [Expanded SWDE dataset]
Dongxu Zhang, Subhabrata Mukherjee, Colin Lockard, Xin Luna Dong, Andrew McCallum
in Proceedings of the North American Chapter of the Association for Computational Linguistics (NAACL), 2019
Colin Lockard, Xin Luna Dong, Arash Einolghozati, Prashant Shiralkar
in Proceedings of the VLDB Endowment (PVLDB), 2018
[slides]
James Ferguson, Colin Lockard, Daniel S. Weld, Hannaneh Hajishirzi
in Proceedings of the North American Chapter of the Association for Computational Linguistics (NAACL), 2018
James Ferguson, Colin Lockard, Natalie Hawkins, Stephen Soderland, Hannaneh Hajishirzi, Daniel S. Weld
in Proceedings of TAC-KBP, 2017
Tutorials
Service
I regularly serve as a reviewer for conferences and journals in AI and NLP, including ACL, NAACL, EMNLP, KDD, AAAI, AKBC, and TKDE.