The management and exchange of multimedia data is challenging due to the variety of formats, standards and intended applications. In addition, production of multimedia data is rapidly increasing due to the availability of off-the-shelf, modern digital devices that can be used by even inexperienced users. It is likely that this volume of information will only increase in the future. A key goal of the MUSCLE (Multimedia Understanding through Semantics, Computation and Learning) network is to develop tools, technologies and standards to facilitate the interoperability of multimedia content and support the exchange of such data. One approach for achieving this was the creation of a specific "E-Team", composed of the authors, to discuss core questions and practical issues based on the participant's individual work. In this paper, we present the relevant points of view with regards to sharing experiences and to extracting and integrating multimedia data and metadata from different modes (text, images, video). © 2007 IEEE.