Multi-modal recommendation system for e-commerce

Michal Daniluk

supervisor: Grzegorz Pastuszak

Artificial Intelligence-based recommender systems are present at almost every large e-commerce store and platform, spanning various product sectors from garments, through jewelry to food. It is usually impossible to adjust existing algorithms to include a new modality of data or a new type of attribute. A vast majority of existing recommender systems consider only a single type of interaction, e.g. clicks or purchases.

The goal of my PHD is to study a multi-modal context-aware recommendation system that can be fed with various types of data such as purchases, clicks, page visits, text, image, and other meta-data. The PHD is realized in cooperation with Synerise.

In the two years of my PHD, I proposed a multi-modal recommendation model– EMDE (Efficient Manifold Density Estimator). This work was presented during ICONIP in December 2021.

Furthermore, I also won three very prestigious Machine Learning competitions:

• Booking.com Data Challenge, which aims to make the best recommendation for the next destination of a user trip, based on a dataset with millions of real anonymized accommodation reservations.
• KDD CUP 2021, which goal was to predict the subject area of the given arXiv papers in a heterogeneous academic graph.
• RecSys Twitter Challenge, which focuses on a real-world task of tweet engagement prediction in a dynamic environment.