Title: Measuring similarity of classified advertisements using images and text, with applications to recommendation and search at finn.no
Abstract: A basic problem in large collections of documents is to find similar items, for basic search, recommendation, or other purposes. Traditionally techniques for this has relied only on the text in the document, since analysing images has been difficult and costly. However, recently, new methods for analysis of images known as 'deep learning' has made it possible to efficiently classify and compare images, which opens up for using images as additional information when comparing documents. In our case we will explore methods of comparing classified ads in the Norwegian online marketplace finn.no, utilizing both images and text in the ads, so called multi-modal comparison. We will evaluate two methods using both images and text for comparing classified ads, one based on extracted features from images and text, and one based on combining classifier output via a separately learned word-embedding model. We will compare these two methods with baseline comparison methods utilizing only text or only images.