Automated Machine Learning for Recommendations:
Fundamentals and Advances

Keywords: Recommender Systems (RecSys), Deep Neural Networks (DNNs), Automated Machine Learning (AutoML)

Our Survey Paper: Automated Machine Learning for Deep Recommender Systems: A Survey

INTRO

Recommender systems have become increasingly important in our daily lives since they play an important role in mitigating the information overload problem, especially in many user-oriented online services. Recommender systems aim to identify a set of objects (i.e., items) that best match users’ explicit or implicit preferences, by utilizing the user and item interactions to improve the matching accuracy. With the fast advancement of deep neural networks (DNNs) in the past few decades, recommendation techniques have achieved promising performance. However, we face three inherent challenges to design deep recommender systems (DRS): 1) the majority of existing DRS are developed based on hand-crafted components, which requires ample expert knowledge of machine learning and recommender systems; 2) Human error and bias can lead to suboptimal components, which reduces the recommendation effectiveness; 3) Non-trivial time and engineering efforts are usually required to design the task-specific components in different recommendation scenarios.

In this tutorial, we aim to give a comprehensive survey on the recent progress of advanced Automated Machine Learning (AutoML) techniques for solving the above problems in deep recommender systems. In this way, we expect academic researchers and industrial practitioners in related fields can get deep understanding and accurate insight into the spaces, stimulate more ideas and discussions, and promote developments of technologies in recommendations.

Tutorial Syllabus [Slides] [Video]

The topics of this tutorial include (but are not limited to) the following:

  1. Introduction to Deep Recommender System (DRS)
  2. Preliminary of Automated Machine Learning (AutoML)
  3. AutoML for DRS Embedding Components
  4. AutoML for DRS Interaction Components
  5. AutoML for Whole DRS Architecture Search
  6. Conclusion and Future Direction

Presenters

Image

Xiangyu Zhao City University of Hong Kong

Image

Wenqi Fan Hong Kong Polytechnic University

Image

Huifeng Guo Huawei Noah’s Ark Lab

Image

Bo Chen Huawei Noah’s Ark Lab

Image

Yejing Wang City University of Hong Kong

Image

Ruiming Tang Huawei Noah’s Ark Lab

Presenters’ Biography

Dr. Xiangyu Zhao is an assistant professor of the school of data science at City University of Hong Kong (CityU). Prior to CityU, he completed his PhD (2021) at MSU, MS (2017) at USTC and BEng (2014) at UESTC. His current research interests include data mining and machine learning, especially (1) Information Retrieval and its applications in Personalization, Recommender System, Online Advertising and Search Engine; (2) AutoML, Reinforcement Learning, and Multimodal; (3) Urban Computing and Spatio-Temporal Data Analysis; and (4) AI for Social Computing, Finance, Education, Ecosystem, and Healthcare. He has published more than 20 papers in top conferences (e.g., KDD, WWW, AAAI, SIGIR, ICDE, CIKM, ICDM, WSDM, RecSys) and journals (e.g., SIGKDD, SIGWeb, EPL, APS). His research received Global Top 100 Chinese New Stars in Artificial Intelligence, CCF-Tencent Open Fund, Criteo Research Award, Bytedance Research Award and MSU Dissertation Fellowship. He serves as top data science conference (senior) program committee members and session chairs (e.g., KDD, AAAI, IJCAI, ICML, ICLR, CIKM), and journal reviewers (e.g., TKDE, TKDD, TOIS, CSUR). He serves as the organizers of DRL4KDD@KDD'19, DRL4IR@SIGIR'20, 2nd DRL4KD@WWW'21, 2nd DRL4IR@SIGIR'21. He also serves as the founding academic committee members of MLNLP, the largest AI community in China with 800,000 followers. The models and algorithms from his research have been launched in the online system of many companies, such as Amazon, Google, Facebook, Linkedin, Criteo, Lyft, JD.com, Kuaishou, Tencent, and Bytedance. Please find more information at https://zhaoxyai.github.io/. Dr. Zhao is a lead tutor at WWW'21 and IJCAI'21.

Dr. Wenqi Fan is currently a research assistant professor of the Department of Computing at The Hong Kong Polytechnic University (PolyU). He received his Ph.D. degree from the City University of Hong Kong (CityU) in 2020. From 2018 to 2020, he was a visiting research scholar at Michigan State University. His research interests are in the broad areas of machine learning and data mining, with a particular focus on Recommender Systems, Graph Neural Networks, and Adversarial Attacks. He has published innovative papers in top-tier journals and conferences such as IEEE TKDE, KDD, WWW, ICDE, IJCAI, AAAI, RecSys, WSDM, and SDM. He serves as top-tier conference program committee members (e.g., ICML, ICLR, NeurIPS, KDD, WWW, AAAI, IJCAI, CIKM, etc.), and journal reviewers (e.g., TKDE, TIST, TKDD, TOIS, TAI, etc.). He also serves as the lead tutor of tutorials in top-tier conferences (e.g., WWW 2021, IJCAI 2021, and ICAPS 2021). More information about him can be found at https://wenqifan03.github.io.

Dr. HuiFeng Guo is a senior researcher in Huawei Noah's Ark Lab since 2018. Before that, he got his Ph.D. degree from Harbin Institute of Technology in 2018 and his BEng from Lanzhou University in China in 2012. His research interests include deep learning, AutoML, graph learning, distributed training system and their applications in recommender systems. He has published more than 20 papers in top-tier journals and conferences such as IJCAI, KDD, SIGIR, CIKM, ICDM, WSDM, TKDE, TOIS and WWW.

Mr. Bo Chen is currently a researcher in Huawei Noah's Ark Lab since 2020. Before that, he got his MS in Software Engineering from Shanghai Jiao Tong University (SJTU) in 2020 and his BEng in Computer Science and Engineering from South China University of Technology (SCUT) in 2017. His research interests include recommender systems, computational advertising, deep learning, AutoML, Graph Neural Network. He has published several papers in his interested research areas (e.g., KDD, IJCAI, CIKM, TKDE, MLSys). Among which, some publications (e.g., KDD 2020/TKDE 2021) belong to the field of this tutorial, i.e., AutoML for recommendation. Besides, he also gave a keynote in DLP@KDD'21 on this research area.

Mr. Yejing Wang is currently a PhD student in City University of Hong Kong. Before that, he got his B.S. in Statistics from The University of Science and Technology of China (USTC). His research interests include AutoML, feature selection, recommender system.

Dr. Ruiming Tang is a principal researcher in Huawei Noah's Ark Lab since 2014. Before that, he got his PhD from National University of Singapore in 2014 and his BEng from Northeastern University in China in 2009. His research interests include deep learning, reinforcement learning, AutoML, graph learning and their applications in recommendation and search. He has published more than 30 papers in his interested research areas (e.g., IJCAI, AAAI, KDD, SIGIR, NeurIPS, CIKM, ICDM, WSDM, TKDE, TOIS, WWW). Specifically, he has achieved several high-quality publications (e.g., KDD 2020/SIGIR 2020/CIKM 2020/KDD 2021/TKDE) in the research area of this tutorial, namely, AutoML in recommendation. He is program committee members of top data science conferences (e.g., KDD, SIGIR, AAAI, IJCAI, WSDM, CIKM, WWW) and serves as journal reviewers for TKDE, IPM. He also serves as the organizers of DLP@KDD'21.