Challenge Tracks
Overview
Developing and integrating advanced image sensors with novel algorithms in camera systems is prevalent with the increasing demand for computational photography and imaging on mobile platforms. However, the lack of high-quality data for research and the rare opportunity for in-depth exchange of views from industry and academia constrain the development of mobile intelligent photography and imaging (MIPI).
The workshop's main focus is on MIPI, emphasizing the integration of novel image sensors and imaging algorithms. Together with the workshop, we organize a few exciting challenges and invite renowned researchers from both industry and academia to share their insights and recent work. Our challenge includes five tracks:
- RGB+ToF Depth Completion uses sparse, noisy ToF depth measurements with RGB images to obtain a complete depth map.
- Quad-Bayer Re-mosaic converts Quad-Bayer RAW data into Bayer format so that it can be processed with standard ISPs.
- RGBW Sensor Re-mosaic converts RGBW RAW data into Bayer format so that it can be processed with standard ISPs.
- RGBW Sensor Fusion fuses Bayer data and a monochrome channel data into Bayer format to increase SNR and spatial resolution.
- Under-display Camera Image Restoration improves the visual quality of image captured by a new imaging system equipped under-display camera.
Unlike previous workshops that focus on image or video manipulation, restoration and enhancement, or the efficient designs of AI models for mobile devices, the central theme of our workshop encompasses new sensors and imaging systems, which are the indispensable foundation for mobile intelligent photography and imaging. As the first workshop of this kind, MIPI aims to organize a dedicated workshop so that we can solicit relevant solutions and attract a focused group from both academia and industry for fruitful discussions.
Datasets and Submission
Datasets are available at the Codalab site of each challenge track. Submissions to all phases will be done through the CodaLab site. Please register to the site and refer to the instructions on how to download the datasets and submit your results. The evaluation metrics of each track will be introduced in the respective site.
Link to Codalab: RGB+ToF Depth Completion -- Quad-Bayer Re-mosaic -- RGBW Sensor Re-mosaic -- RGBW Sensor Fusion -- Under-display Camera Image Restoration
Awards and Prizes
The winner teams of each track will receive a certificate and be awarded a cash prize. Challenge participants with the most successful and innovative methods will be invited to present at the MIPI workshop. The cash prize for each track:
- First place: USD 1000
- Second place: USD 800
- Third place: USD 500
Terms and Conditions
Link to Terms and Conditions.
In the event of any inconsistency between the terms and details of this website and Codalab competition sites, the details on the Codalab sites shall prevail.
Contact
For more information, please contact us at mipi.challenge@gmail.com.
Call for Papers
Coupled with the workshop, besides the challenge, we also provide a fertile ground for researchers, scientists, and engineers from around the world to disseminate their research outcomes and push forward the frontiers of knowledge within novel image sensors and imaging systems-related areas. The MIPI workshop topics cover but are not limited to:
- ToF depth completion
- Quad-Bayer/RGBW sensor re-mosaic and fusion
- Under-display camera image restoration
- Event camera for image/video restoration
- Learnable ISPs
- Learning-based computational imaging systems: smart imaging and sensing systems, learned data acquisition, task-driven ISPs, etc
- Computationally efficient computational imaging systems and algorithms
Papers will be peer-reviewed and comply with the ECCV 2022 proceedings style, format and length. The deadline for camera ready submission is the same as the main conference. Papers accepted must be registered and presented to ensure their inclusion in the IEEE Xplore Library. Selected papers will be invited to present in the workshop. Some papers will also be invited to contribute to a book chapter or a special issue article in our future publication. The format for submission is the same as the ECCV 2022 requirements .
Paper submission and review site: Submission Site
Certificates for Best Paper Award and Best Paper Honorable Mention:
Best Paper Award:
Paper title: Event-based Image Deblurring with Dynamic Motion Awareness
Authors: Patricia Vitoria, Stamatios Georgoulis, Stepan Tulyakov, Alfredo Bochicchio, Julius Erbach, Yuanyou Li
Affiliation: Huawei
Prize: USD 1000
Best Paper Honorable Mention:
Paper title: Continuous Spectral Reconstruction from RGB Images via Implicit Neural Representation
Authors: Ruikang Xu, Mingde Yao, Chang Chen, Lizhi Wang, Zhiwei Xiong
Affiliations: University of Science and Technology of China, Huawei Noah's Ark Lab, Beijing Institute of Technology
Prize: USD 800
Important Dates
Event | Date (Always 11:59 PM Pacific Time) |
---|---|
Site online | Apr 08, 2022 |
Release of training data and validation data | May 15, 2022 |
Validation server online | May 15, 2022 |
Release of test data, test server online | Jul 13, 2022 |
Test results submission deadline, test server closed | Jul 20, 2022 |
Fact sheets submission deadline | Jul 20, 2022 |
Final test and rating results release to participants | Jul 30, 2022 |
Event | Date (Always 11:59 PM Pacific Time) |
---|---|
Site online | Apr 08, 2022 |
CMT online | Apr 25, 2022 |
Paper submission deadline | Aug 08, 2022 |
Supplementary materials deadline | Aug 08, 2022 |
Notification to authors | Aug 18, 2022 |
Camera ready deadline | Aug 22, 2022 |
Workshop date | Oct 23, 2022 |
Schedule
Time (Israel) |
Programme (Video Recording) |
---|---|
9:00 – 9:10
|
Opening and Welcome |
9:10 – 10:00
Keynote Speaker
|
Keynote: Denoising as a Building Block for Imaging, Inverse Problems, and Machine LearningDenoising is one of the oldest problems in imaging. In the last decade, the quality of denoising algorithms has reached phenomenal levels – almost as good as we can ever hope. There are thousands of papers on this topic, and their scope is vast and approaches so diverse that putting them in some order (as I will do) is both useful and challenging. I will describe what we can say about this general class of operators, and what makes them so special. I will argue that denoising is more important than ever; not simply as a process for removing noise, but especially now as a core engine and building block for much more complex tasks in imaging, inverse problems, and machine learning. Peyman MilanfarPrincipal Scientist/Director, Google Research |
10:00 – 10:15
Best Paper Award
|
Best Paper AwardWinner Paper: Event-based Image Deblurring with Dynamic Motion Awareness Presenter: Patricia Vitoria (Huawei Zurich Research Center) |
10:15 – 10:30
Best Paper Mention
|
Best Paper Honorable MentionWinner Paper: Continuous Spectral Reconstruction from RGB Images via Implicit Neural Representation Presenter: Ruikang Xu (University of Science and Technology of China) |
10:30 - 10:50
Invited Speaker
|
Invited Talk: Combining Internal and External Constraints for Unrolling Shutter in VideosPresenter: Eyal Naor (The Weizmann Institute of Science) |
10:50 - 11:10
Invited Speaker
|
Invited Talk: Synthesizing Light Field Video from Monocular VideoPresenter: Shrisudhan Govindarajan (Indian Institute of Technology Madras) |
11:10 - 11:20
|
☕ Tea Break |
11:20 - 11:35
Winning Solution
|
Challenge Winner: ToF Depth Completion TrackPresenter: Dewang Hou (Peking University) |
11:35-11:50
Winning Solution
|
Challenge Winner: Quad-Bayer Remosaic & RGBW Sensor Remosaic TracksPresenter: Lingchen Sun (OPPO Research Institute) |
11:50 - 12:10
Invited Speaker
|
Invited Talk: Animation from Motion Blur or Rolling Shutter DistortionPresenter: Zhihang Zhong (The University of Tokyo) |
12:10 - 12:30
Invited Speaker
|
Invited Talk: Event-Based Fusion for Motion Deblurring with Cross-Modal AttentionPresenter: Lei Sun (ETH Zurich) |
12:30 - 13:50
|
🍱 Break |
13:50 - 14:05
Winning Solution
|
Challenge Winner: RGBW Sensor Fusion TrackPresenter: DongJae Lee (Korea Advanced Institute of Science and Technology) |
14:05 - 14:20
Winning Solution
|
Challenge Runner-up: RGBW Sensor Remosaic TrackPresenter: Qiaosi Yi (East China Normal University) |
14:20 - 14:35
Winning Solution
|
Challenge Winner: UDC Image Restoration TrackPresenter: Yurui Zhu (University of Science and Technology of China) |
14:35 - 15:25
Keynote Speaker
|
Keynote: Recent Evolution on Image SensingThe current growth of the image sensor market is mainly driven by demands for the mobile imaging market. The driving force is further expected to continue more than several years from now. Thus, it is worthy to consider what kinds of demands come from the mobile imaging market to understand what kinds of imaging technologies are required. On the other hand, sensing applications, which utilize image sensor date not only for taking pictures but also for person identification, product inspection, obstacle avoidance and so on, are emerging. They seem to growth in industry and automotive markets at first but expected in mobile market soon. In sensing applications, non-RGB image sensors are utilized as well as RGB image sensors. In this talk, recent evolutions of image sensors for both RGB and other than RGB are introduced. First, RGB imaging technologies which have been implemented in recent image sensor products for mobile market are introduced. Second, some new-generation image sensors which are to acquire information other than RGB are introduced. Also, expectations that such new-generation image sensors are promising for mobile imaging applications in the future are explained by introducing some recent researches in vision-based imaging area. Tomoo MitsunagaManager, Sony Europe B.V. |
15:25 - 16:15
Keynote Speaker
|
Keynote: Robust Imaging Color and the Dual-Purpose ISPIn this talk, I will discuss recent work targeting imaging strategies to allow robust color imaging. In particular, I will overview capture-time algorithms that allows processed images to be unprocessed to correct potential errors that affect image color. I will also discuss the idea of a "dual-purpose" image signal processor (ISP) that is suitable for both photographic and scientific tasks. Michael S. BrownProfessor, York University |
16:15 - 17:05
Keynote Speaker
|
Keynote: Computer Vision, One Photon at a TimeSingle-photon avalanche diodes (SPADs) are an emerging sensor technology that promises single-photon sensitivity and the ability to time-tag photons with picosecond precision. In the last few years, we have witnessed a single-photon revolution, resulting for the first time in multi-megapixel SPAD arrays, which allow capturing sufficient spatial details needed for mainstream computer vision applications, including object detection and tracking, text recognition, pose recognition, and SLAM. This raises the following questions: Are SPADs ready to be used beyond niche scientific applications, as general-purpose computer vision sensors? Can they be deployed outside the lab in-the-wild? If so, what are their benefits over conventional cameras? Mohit GuptaAssociate Professor, University of Wisconsin-Madison |
17:05 - 17:20
|
Closing |
Keynote Speakers
Awards (Certificates)
Rank | Prize | Team Name | Affiliation | Team Members |
---|---|---|---|---|
1st | USD 1000 | ZoomNeXt | Peking University Tsinghua University |
Dewang Hou Kai Zhao |
2nd | USD 800 | GAMEON | The Chinese University of Hong Kong International Digital Economy Academy The Chinese University of Hong Kong SmartMore SmartMore The Chinese University of Hong Kong |
Liying Lu Yu Li Huaijia Lin Ruizheng Wu Jiangbo Lu Jiaya Jia |
3rd | USD 500 | Singer | BeiHang University BeiHang University BeiHang University BeiHang University BeiHang University BeiHang University |
Qiang Liu Haosong Yue Danyang Cao Lehang Yu Jiaxuan Quan Jixiang Liang |
Rank | Prize | Team Name | Affiliation | Team Members |
---|---|---|---|---|
1st | USD 1000 | op-summer-po | OPPO Research Institute OPPO Research Institute East China Normal University |
Rongyuan Wu Lingchen Sun Qiaosi Yi |
2nd | USD 800 | JHC-SJTU | Shanghai Jiao Tong University Shanghai Jiao Tong University Shanghai Jiao Tong University Shanghai Jiao Tong University Transsion Transsion |
Xiaohong Liu Jun Jia Hanchi Sun Guangtao Zhai Anlong Xiao Qihang Xu |
3rd | USD 500 | IMEC-IPI&NPU-MPI | IMEC research group in Ghent University Northwestern PolytechnicalUniversity Northwestern PolytechnicalUniversity IMEC & Ghent University IMEC & Ghent University IMEC & Ghent University IMEC & Ghent University |
Haijin Zeng Kai Feng Yongqiang Zhao Hiep Quang Luong Jan Aelterman Anh Minh Truong Wilfried Philips |
Rank | Prize | Team Name | Affiliation | Team Members |
---|---|---|---|---|
1st | USD 1000 | op-summer-po | OPPO Research Institute OPPO Research Institute East China Normal University |
Lingchen Sun Rongyuan Wu Qiaosi Yi |
2nd | USD 800 | Eating,
Drinking, and Playing |
East China Normal University OPPO Research Institute OPPO Research Institute East China Normal University |
Qiaosi Yi Rongyuan Wu Lingchen Sun Faming Fang |
3rd | USD 500 | HIT-IIL |
Harbin Institute of Technology Harbin Institute of Technology Harbin Institute of Technology Harbin Institute of Technology Harbin Institute of Technology Harbin Institute of Technology Harbin Institute of Technology |
Rongjian Xu Xiaohui Liu Zhilu Zhang Xiaohe Wu Ruohao Wang Junyi Li Wangmeng Zuo |
Rank | Prize | Team Name | Affiliation | Team Members |
---|---|---|---|---|
1st | USD 1000 | LLCKP | Korea Advanced Institute of Science & Technology | DongJae Lee |
2nd | USD 800 | MegNR |
Megvii Technology Megvii Technology Megvii Technology Megvii Technology Megvii Technology Megvii Technology Megvii Technology Megvii Technology |
Ting Jiang Qi Wu Chengzhi Jiang Mingyan Han Xinpeng Li Wenjie Lin Youwei Li Haoqiang Fan Shuaicheng Liu |
3rd | USD 500 | jzsherlock | Xiaomi | Zhuang Jia |
Rank | Prize | Team Name | Affiliation | Team Members |
---|---|---|---|---|
1st | USD 1000 | USTC_WXYZ |
University of Science and Technology of China University of Science and Technology of China University of Science and Technology of China Shanghai AI Laboratory |
Yurui Zhu Xi Wang Xueyang Fu Xiaowei Hu |
2nd | USD 800 | XPixel Group |
Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences University of Macau & Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences & Shanghai AI Laboratory Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences & Shanghai AI Laboratory |
Jinfan Hu Xina Liu Xiangyu Chen Chao Dong |
3rd | USD 500 | SRC-B |
Samsung Research China Samsung Research China Samsung Research China Samsung Research China Samsung Research China |
Dafeng Zhang Feiyu Huang Shizhuo Liu Xiaobing Wang Zhezhu Jin |