The First Workshop on the Evaluation of Generative Foundation Models at CVPR 2024


The camera-ready paper has a restricted deadline of April 14, 2024, at 11:59 PM Pacific Daylight Time. Note, the camera-ready submission is **not** handled by CMT. Please carefully read the camera-ready submission instruction. A detailed instruction for the submission site can be found here.

The landscape of artificial intelligence is being transformed by the advent of Generative Foundation Models (GenFMs), such as Large Language Models (LLMs) and diffusion models. GenFMs offer unprecedented opportunities to enrich human lives and transform industries. However, they also pose significant challenges, including the generation of factually incorrect or biased information, which might be potentially harmful or misleading. With the emergence of multimodal GenFMs, which leverage and generate content in an increasing number of modalities, these challenges are set to become even more complex. This emphasizes the urgent need for rigorous and effective evaluation methodologies.

The 1st Workshop on Evaluation for Generative Foundation Models at CVPR 2024 aims to build a forum to discuss ongoing efforts in industry and academia, share best practices, and engage the community in working towards more reliable and scalable approaches for GenFMs evaluation. Full CFP can be found at CFP link .

Papers are limited to 6-8 pages, including figures and tables, in the CVPR style. Additional pages containing only cited references are allowed. Please download the CVPR 2024 Author Kit for detailed formatting instructions.

Workshop Time, June 18 2024, 1pm - 6:30pm PDT

Submission Timeline

CMT Site Open

Link Feb 9, 23:59 PT

Submission Deadline

Apr 4, 23:59 PST

Final Decisions

Apr 11, 23:59 PST

Camera Ready Deadline

Apr 14, 23:59 PST

Workshop Topics

  • Presentation of new datasets and benchmarks tailored for comprehensive GenFM evaluation.

  • Human-in-the-Loop methodologies that incorporate human judgment and preferences.

  • Quantitative methods that accurately reflect human judgments.

  • Pros and Cons analyses of existing evaluation techniques and benchmarks

  • Discussion of unique challenges of multimodal GenFM evaluation

  • Quantification of interplay between modalities and the emergence of new information within GenFMs

  • Exploration of techniques to leverage automated evaluation for model refinement

Workshop Tentative Schedule

Gathering + Posters Arrangement

Jun 18, 13:00 - 13:20 PDT

Opening Remarks

Jun 18, 13:20 - 13:30 PDT

Keynote 1

Jun 18, 13:30 - 14:00 PDT

Keynote 2

Jun 18, 14:00 - 14:30 PDT

Keynote 3

Jun 18, 14:30 - 15:00 PDT

Keynote 4

Jun 18, 15:00 - 15:25 PDT

Panel Discussion

Jun 18, 15:30 - 16:10 PDT

Keynote 5

Jun 18, 16:10 - 16:35 PDT

Keynote 6

Jun 18, 16:35 - 17:00 PDT

Keynote 7

Jun 18, 17:00 - 17:25 PDT

Keynote 8

Jun 18, 17:25 - 17:50 PDT

Poster Session

Jun 18, 17:50 - 18:30 PDT

Invited Speakers and Panelists

Tal Hassner

Applied Research Lead at Facebook AI

Bo Li

Associate Professor at University of Chicago

Ranjay Krishna

Associate Professor at University of Washington

Hanwang Zhang

Associate Professor at Nanyang Technological University

Leonid Karlinsky

Principal Research Scientist in the MIT-IBM lab

Jungo Kasai

Co-founder & CTO at Kotoba Technologies, Inc.

Sadeep Jayasumana

Staff Research Scientist at Google Research

Besmira Nushi

Researcher at Microsoft Research

Ece Kamar

Managing Director at Microsoft Research

Program Committee

Alex C Williams


Amir Tavanaei


Angel Martinez-Gonzalez


Bardiya Akhbari


Ebrahim Safadi


Fatemeh Ghezloo

University of Washington

Gauthier Guinet


Gokhan Kirlik


Gozde Sahin


Hammad Ayyubi

Columbia University

Hengameh M Dastjerdi


Hengkang Wang

University of Minnesota

Huitian Lei

Lyft INC

Jiawei Ma

Columbia University

Junyi Hu

UMass at Lowell

Kee Kiat Koo


Mahtab Bigverdi

University of Washington

Meng Han


Michael Opitz


Nadine Behrmann

Bosch Center for AI

Nicholas Dronen


Pradeep Yarlagadda


Prateek Singhi


Qi Li


Rahul Sharma


Rustin Soraki

University of Washington

Thomas Cilloni


Tristan McKinney


Wentai Zhang


Wenyi Wu


Wisdom Ikezogwo

University of Washington

Xiao Zhang


Xin Li


Yiming Qian


Yuguang Li

Zillow Group


Maria Zontak

Amazon (Primary Contact)

Xu Zhang


Mehmet Saygin Seyfioglu

University of Washington

Erran Li


Bahar Erar Hood


Suren Kumar


Karim Bouyarmane


© 2024 EVGENFM2024

We thank Jalpc for the jekyll template