Workshop
The Second Workshop on Long-Context Foundation Models
Zexue He · Tianyu Gao · Amanda Bertsch · Howard Yen
West Meeting Room 202-204
Sat 19 Jul, 9:10 a.m. PDT
Foundation models have become a cornerstone in the advancement of artificial intelligence, enabling applications across a wide range of domains. Many complex tasks today require processing and synthesizing information over thousands to millions of individual pieces of data, from text and images to audio and genomic sequences. Recent progress in long-context models has made it possible to handle such extensive inputs, but significant challenges remain, particularly in terms of computational efficiency, data quality and quantity, and evaluation. This workshop will convene researchers to explore these challenges and foster developments in long-context foundation models. Key topics include new modeling architectures, training approaches, efficiency techniques, and comprehensive evaluation methods. Additionally, in this edition, special attention will be given to long-context reasoning, multimodal learning, and applications in scientific fields such as genomics, climate science, etc. By tackling these critical challenges, we aim to push the boundaries of long-context modeling and shape its future directions.
Schedule
Sat 9:10 a.m. - 9:15 a.m.
|
Introduction to Workshop
(
Opening Remarks
)
>
|
Howard Yen 🔗 |
Sat 9:15 a.m. - 9:45 a.m.
|
Invited Talk 1 - Tri Dao
(
Invited Talk
)
>
|
Tri Dao 🔗 |
Sat 9:45 a.m. - 10:00 a.m.
|
Oral Talk 1: Kinetics: Rethinking Test-Time Scaling Laws
(
Oral
)
>
|
Zhuoming Chen 🔗 |
Sat 10:00 a.m. - 10:30 a.m.
|
Invited Talk 2 - Pang Wei Koh
(
Invited Talk
)
>
|
Pang We Koh 🔗 |
Sat 10:30 a.m. - 10:45 a.m.
|
Break for Networking and Poster Setup
|
🔗 |
Sat 10:45 a.m. - 11:30 a.m.
|
Poster Session 1 (with US virtual posters together)
(
Poster Session
)
>
|
🔗 |
Sat 11:30 a.m. - 12:00 p.m.
|
Invited Talk 3 - Dima Rekesh
(
Invited Talk
)
>
|
Dima Rekesh 🔗 |
Sat 12:00 p.m. - 12:15 p.m.
|
Orals 2: PhantomWiki: On-Demand Datasets for Reasoning and Retrieval Evaluation
(
Oral
)
>
|
Albert Gong 🔗 |
Sat 12:15 p.m. - 12:30 p.m.
|
Orals 3: IceFormer: Accelerated Inference with Long-Sequence Transformers on CPUs
(
Oral
)
>
|
Yuzhen Mao 🔗 |
Sat 12:30 p.m. - 2:00 p.m.
|
Lunch Break
|
🔗 |
Sat 2:00 p.m. - 2:30 p.m.
|
Panel Discussion: How to evaluate long-context foundation models?
(
Penal
)
>
|
Yuandong Tian · Mohit Iyyer · Beidi Chen · Dima Rekesh 🔗 |
Sat 2:30 p.m. - 3:00 p.m.
|
Invited Talk 4 - Volodymyr Kuleshov
(
Invited Talk
)
>
|
Volodymyr Kuleshov 🔗 |
Sat 3:00 p.m. - 3:30 p.m.
|
Invited Talk 5 - Jiajun Wu
(
Invited Talk
)
>
|
Jiajun Wu 🔗 |
Sat 3:30 p.m. - 4:30 p.m.
|
Poster Session 2 (with US virtual posters together).
(
Poster Session
)
>
|
🔗 |
Sat 4:30 p.m. - 4:45 p.m.
|
Orals 4: e3: Learning to Explore Enables Extrapolation of Test-Time Compute for LLMs
(
Oral
)
>
|
Aviral Kumar 🔗 |
Sat 4:45 p.m. - 5:00 p.m.
|
Orals 5: OracleKV: Oracle Guidance for Question-Independent KV Cache Compression
(
Oral
)
>
|
Yuanbing Zhu 🔗 |
Sat 5:00 p.m. - 5:05 p.m.
|
Closing Remarks
(
Closing Remarks
)
>
|
🔗 |
Sat 6:00 p.m. - 7:00 p.m.
|
Beijing-Time Virtual Posters (9 AM in Beijing) ( Poster Session ) > link | 🔗 |