Information retrieval and multimedia content access has a long history of comparative evaluation and many of the advances in the area over the past decade can be attributed to the availability of open datasets that support comparative and repeatable experimentation. Sharing data and code to allow other researchers to replicate research results is needed in the multimedia modeling field and this will help to improve the performance of systems and the reproducibility of papers published.
In terms of existing state-of-the-art, there is one other related dataset track (at MMSys) and it has been heavily oversubscribed in recent years. Following discussions among members of the MMM Steering Committee at MMM2018, it was agreed that there is a clear need for a single index of (and venue for) datasets related to multimedia modeling. This special session proposal is a direct result of these discussions and we hope it will become a permanent track at the MMM conference series. Consequently, this proposal is for a special session, but associated with this, we will put in place a permanent and updatable archive of links to MMM datasets (mmdatasets.org) related to the MMM conference. This multimedia dataset track will be an opportunity for researchers and practitioners to make their work permanently available and citable in a single forum, as well as to increase the public awareness of their considerable efforts.
Researchers within the multimedia community will be encouraged to submit their datasets to this track. Together with the dataset, authors are asked to provide a paper describing its motivation, design, and usage, a brief summary of the experiments performed to date on the dataset, as well as discussing the way it can be useful to the community. The benefits for authors who successfully submit are:
-Accepted contributions will be included in the conference proceedings.
-Accepted contributions will be listed in a recognised index of multimedia datasets, thereby increasing their visibility.
-Authors of accepted contributions will be invited to present their dataset as part of the special session programme at MMM2020.
Regarding the submission of a dataset, the authors should make it available by providing a URL for download, as mentioned above, and agree to the link being maintained on an MMM datasets dedicated site. All datasets must be licensed in such a manner that it can be legally and freely used with all appropriate ethical and access approvals completed. Authors are encouraged to prepare appropriate and helpful documentation to accompany the dataset, including examples of how it can be used by the community, examples of successful usage and restrictions on usage.