Avatar

Ben Bogin

Hi! I’m Ben, a Research Scientist at Google, and previously post-doc at AI2 and the University of Washington. I have pursued my PhD at Tel Aviv University, under the guidance of Prof. Jonathan Berant, where I studied compositional generalization in semantic parsing and question answering.

I’m always open to new collaborations, feel free to reach out!

Publications

DataDecide: How to Predict Best Pretraining Data with Small Experiments (ICML, 2025). PDF
Ian Magnusson*, Nguyen Tai*, Ben Bogin*, David Heineman, Jena D. Hwang, Luca Soldaini, Akshita Bhagia, Jiacheng Liu, Dirk Groeneveld, Oyvind Tafjord, Noah A. Smith, Pang Wei Koh, Jesse Dodge
SUPER: Evaluating Agents on Setting Up and Executing Tasks from Research Repositories (EMNLP, 2024). PDF Code
Outstanding Paper Award
Ben Bogin, Kejuan Yang, Shashank Gupta, Kyle Richardson, Erin Bransom, Peter Clark, Ashish Sabharwal, Tushar Khot
AssistantBench: Can Web Agents Solve Realistic and Time-Consuming Tasks? (EMNLP, 2024). PDF Code
Ori Yoran, Samuel Joseph Amouyal, Chaitanya Malaviya, Ben Bogin, Ofir Press, Jonathan Berant
Dolma: an Open Corpus of Three Trillion Tokens for Language Model Pretraining Research (ACL, 2024). PDF Code
Best Paper Award
Luca Soldaini, Rodney Kinney, Akshita Bhagia, Dustin Schwenk, David Atkinson, Russell Authur, Ben Bogin, Khyathi Chandu, Jennifer Dumas, Yanai Elazar, Valentin Hofmann, Ananya Harsh Jha, Sachin Kumar, Li Lucy, Xinxi Lyu, Nathan Lambert, Ian Magnusson, Jacob Morrison, Niklas Muennighoff, Aakanksha Naik, Crystal Nam, Matthew E. Peters, Abhilasha Ravichander, Kyle Richardson, Zejiang Shen, Emma Strubell, Nishant Subramani, Oyvind Tafjord, Pete Walsh, Luke Zettlemoyer, Noah A. Smith, Hannaneh Hajishirzi, Iz Beltagy, Dirk Groeneveld, Jesse Dodge, Kyle Lo
Leveraging Code to Improve In-context Learning for Semantic Parsing (NAACL, 2024). PDF Code
Ben Bogin*, Shivanshu Gupta*, Peter Clark, Ashish Sabharwal
Answering Questions by Meta-Reasoning over Multiple Chains of Thought (EMNLP, 2023). PDF Code
Ori Yoran*, Tomer Wolfson*, Ben Bogin, Uri Katz, Daniel Deutch, Jonathan Berant
Diverse Demonstrations Improve In-context Compositional Generalization (ACL, 2023). PDF Code
Itay Levy*, Ben Bogin*, Jonathan Berant
Unobserved Local Structures Make Compositional Generalization Hard (EMNLP, 2022). PDF Code
Ben Bogin, Shivanshu Gupta, Jonathan Berant
COVR: A Test-Bed for Visually Grounded Compositional Generalization with Real Images (EMNLP, 2021). PDF Code Project
Ben Bogin, Shivanshu Gupta, Matt Gardner, Jonathan Berant
Text-to-SQL in the Wild: A Naturally-Occurring Dataset Based on Stack Exchange Data (Workshop on Natural Language Processing for Programming @ ACL, 2021). PDF Code
Moshe Hazoom, Vibhor Malik, Ben Bogin
An autonomous debating system (Nature, 2021). PDF
Noam Slonim, Yonatan Bilu, Carlos Alzate, Roy Bar-Haim, Ben Bogin, Francesca Bonin, Leshem Choshen, Edo Cohen-Karlik, Lena Dankin, Lilach Edelstein, Liat Ein-Dor, Roni Friedman-Melamed, Assaf Gavron, Ariel Gera, Martin Gleize, Shai Gretz, Dan Gutfreund, Alon Halfon, Daniel Hershcovich, Ron Hoory, Yufang Hou, Shay Hummel, Michal Jacovi, Charles Jochim, Yoav Kantor, Yoav Katz, David Konopnicki, Zvi Kons, Lili Kotlerman, Dalia Krieger, Dan Lahav, Tamar Lavee, Ran Levy, Naftali Liberman, Yosi Mass, Amir Menczel, Shachar Mirkin, Guy Moshkowich, Shila Ofek-Koifman, Matan Orbach, Ella Rabinovich, Ruty Rinott, Slava Shechtman, Dafna Sheinwald, Eyal Shnarch, Ilya Shnayderman, Aya Soffer, Artem Spector, Benjamin Sznajder, Assaf Toledo, Orith Toledo-Ronen, Elad Venezian, Ranit Aharonov
Latent Compositional Representations Improve Systematic Generalization in Grounded Question Answering (TACL, 2020). PDF Code
Ben Bogin, Sanjay Subramanian, Matt Gardner, Jonathan Berant
Obtaining Faithful Interpretations from Compositional Neural Networks (ACL, 2020). PDF Code
Sanjay Subramanian*, Ben Bogin*, Nitish Gupta*, Tomer Wolfson, Sameer Singh, Jonathan Berant, Matt Gardner
Evaluating nlp models via contrast sets (Findings of EMNLP, 2020). PDF Code
Matt Gardner, Yoav Artzi, Victoria Basmova, Jonathan Berant, Ben Bogin, Sihao Chen, Pradeep Dasigi, Dheeru Dua, Yanai Elazar, Ananth Gottumukkala, Nitish Gupta, Hanna Hajishirzi, Gabriel Ilharco, Daniel Khashabi, Kevin Lin, Jiangming Liu, Nelson F Liu, Phoebe Mulcaire, Qiang Ning, Sameer Singh, Noah A Smith, Sanjay Subramanian, Reut Tsarfaty, Eric Wallace, Ally Zhang, Ben Zhou
MedICaT: A Dataset of Medical Images, Captions, and Textual References (Findings of ACL, 2020). PDF
Sanjay Subramanian, Lucy Lu Wang, Ben Bogin, Sachin Mehta, Madeleine van Zuylen, Sravanthi Parasa, Sameer Singh, Matt Gardner, Hannaneh Hajishirzi
Global Reasoning over Database Structures for Text-to-SQL Parsing (EMNLP, 2019). PDF Code
Ben Bogin, Matt Gardner, Jonathan Berant
Representing Schema Structure with Graph Neural Networks for Text-to-SQL Parsing (ACL, 2019). PDF Code
Ben Bogin, Matt Gardner, Jonathan Berant
Grammar-based Neural Text-to-SQL Generation (2019). PDF
Kevin Lin, Ben Bogin, Mark Neumann, Jonathan Berant, Matt Gardner
Emergence of Communication in an Interactive World with Consistent Speakers (Emergent Communication Workshop at NIPS, 2018). PDF Code
Ben Bogin, Mor Geva, Jonathan Berant
Towards an argumentative content search engine using weak supervision (COLING, 2018). PDF
Ran Levy*, Ben Bogin*, Shai Gretz*, Ranit Aharonov, Noam Slonim
Language Generation with Recurrent Generative Adversarial Networks without Pre-training (Subword and Character level models in NLP Workshop, EMNLP, 2017). PDF Code
Ofir Press*, Amir Bar*, Ben Bogin*, Jonathan Berant, Lior Wolf