The First Workshop on the Evaluation of Generative Foundation Models at CVPR 2024

EVGENFM2024

The camera-ready paper has a restricted deadline of April 14, 2024, at 11:59 PM Pacific Daylight Time. Note, the camera-ready submission is **not** handled by CMT. Please carefully read the camera-ready submission instruction. A detailed instruction for the submission site can be found here.

The landscape of artificial intelligence is being transformed by the advent of Generative Foundation Models (GenFMs), such as Large Language Models (LLMs) and diffusion models. GenFMs offer unprecedented opportunities to enrich human lives and transform industries. However, they also pose significant challenges, including the generation of factually incorrect or biased information, which might be potentially harmful or misleading. With the emergence of multimodal GenFMs, which leverage and generate content in an increasing number of modalities, these challenges are set to become even more complex. This emphasizes the urgent need for rigorous and effective evaluation methodologies.

The 1st Workshop on Evaluation for Generative Foundation Models at CVPR 2024 aims to build a forum to discuss ongoing efforts in industry and academia, share best practices, and engage the community in working towards more reliable and scalable approaches for GenFMs evaluation. Full CFP can be found at CFP link .

Papers are limited to 6-8 pages, including figures and tables, in the CVPR style. Additional pages containing only cited references are allowed. Please download the CVPR 2024 Author Kit for detailed formatting instructions.

Workshop Time, June 18 2024, 1pm - 6:30pm PDT

Submission Timeline

CMT Site Open

Link Feb 9, 23:59 PT

Submission Deadline

Apr 4, 23:59 PST

Final Decisions

Apr 11, 23:59 PST

Camera Ready Deadline

Apr 14, 23:59 PST


Workshop Topics

  • Presentation of new datasets and benchmarks tailored for comprehensive GenFM evaluation.

  • Human-in-the-Loop methodologies that incorporate human judgment and preferences.

  • Quantitative methods that accurately reflect human judgments.

  • Pros and Cons analyses of existing evaluation techniques and benchmarks

  • Discussion of unique challenges of multimodal GenFM evaluation

  • Quantification of interplay between modalities and the emergence of new information within GenFMs

  • Exploration of techniques to leverage automated evaluation for model refinement


Workshop Tentative Schedule

Gathering + Posters Arrangement

Jun 18, 13:00 - 13:20 PDT

Opening Remarks

Jun 18, 13:20 - 13:30 PDT

Keynote 1

Jun 18, 13:30 - 14:00 PDT

Keynote 2

Jun 18, 14:00 - 14:30 PDT

Keynote 3

Jun 18, 14:30 - 15:00 PDT

Keynote 4

Jun 18, 15:00 - 15:25 PDT

Panel Discussion

Jun 18, 15:30 - 16:10 PDT

Keynote 5

Jun 18, 16:10 - 16:35 PDT

Keynote 6

Jun 18, 16:35 - 17:00 PDT

Keynote 7

Jun 18, 17:00 - 17:25 PDT

Keynote 8

Jun 18, 17:25 - 17:50 PDT

Poster Session

Jun 18, 17:50 - 18:30 PDT


Invited Speakers and Panelists

Tal Hassner

Applied Research Lead at Facebook AI

Bo Li

Associate Professor at University of Chicago

Ranjay Krishna

Associate Professor at University of Washington

Hanwang Zhang

Associate Professor at Nanyang Technological University

Leonid Karlinsky

Principal Research Scientist in the MIT-IBM lab

Jungo Kasai

Co-founder & CTO at Kotoba Technologies, Inc.

Sadeep Jayasumana

Staff Research Scientist at Google Research

Besmira Nushi

Researcher at Microsoft Research

Ece Kamar

Managing Director at Microsoft Research


Program Committee

Alex C Williams

Amazon

Amir Tavanaei

Amazon

Angel Martinez-Gonzalez

Amazon

Bardiya Akhbari

Amazon

Ebrahim Safadi

Amazon

Fatemeh Ghezloo

University of Washington

Gauthier Guinet

Amazon

Gokhan Kirlik

Amazon

Gozde Sahin

Amazon

Hammad Ayyubi

Columbia University

Hengameh M Dastjerdi

Amazon

Hengkang Wang

University of Minnesota

Huitian Lei

Lyft INC

Jiawei Ma

Columbia University

Junyi Hu

UMass at Lowell

Kee Kiat Koo

Amazon

Mahtab Bigverdi

University of Washington

Meng Han

Amazon

Michael Opitz

Amazon

Nadine Behrmann

Bosch Center for AI

Nicholas Dronen

Amazon

Pradeep Yarlagadda

Amazon

Prateek Singhi

Amazon

Qi Li

Amazon

Rahul Sharma

Amazon

Rustin Soraki

University of Washington

Thomas Cilloni

Amazon

Tristan McKinney

Amazon

Wentai Zhang

Amazon

Wenyi Wu

Amazon

Wisdom Ikezogwo

University of Washington

Xiao Zhang

Amazon

Xin Li

Amazon

Yiming Qian

Amazon

Yuguang Li

Zillow Group


Organizers

Maria Zontak

Amazon (Primary Contact)

Xu Zhang

Amazon

Mehmet Saygin Seyfioglu

University of Washington

Erran Li

Amazon

Bahar Erar Hood

Amazon

Suren Kumar

Amazon

Karim Bouyarmane

Amazon

© 2024 EVGENFM2024

We thank Jalpc for the jekyll template