Pan, J.-Y.Yang, H.-J.Faloutsos, C.Duygulu, Pınar2016-02-082016-02-082004-08http://hdl.handle.net/11693/27429Date of Conference: 22-25 August , 2004Conference name: KDD '04 Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data miningGiven an image (or video clip, or audio song), how do we automatically assign keywords to it? The general problem is to find correlations across the media in a collection of multimedia objects like video clips, with colors, and/or motion, and/or audio, and/or text scripts. We propose a novel, graph-based approach, "MMG", to discover such cross-modal correlations. Our "MMG" method requires no tuning, no clustering, no user-determined constants; it can be applied to any multi-media collection, as long as we have a similarity function for each medium; and it scales linearly with the database size. We report auto-captioning experiments on the "standard" Corel image database of 680 MB, where it outperforms domain specific, fine-tuned methods by up to 10 percentage points in captioning accuracy (50% relative improvement).EnglishAutomatic image captioningCross-modal correlationGraph-based modelApproximation theoryCorrelation methodsDatabase systemsGraph theoryImage analysisMathematical modelsMotion estimationProbabilityProblem solvingAutomatic image captioningCross-modal correlationGraph-based modelsVideo motionMultimedia systemsConference Paper10.1145/1014052.1014135