Mastering Summarization Techniques: A Practical Exploration with LLM

Jun 3, 2023

Speakers

About

In this talk, we would like to focus on the summarization of collections of feedback and describe all its challenges. We will focus on the state-of-the-art summarization models, such as GPT-3, open source GPT variants, Bart, and other transformers as well as some extractive approaches such as Gensim. We will show how they perform for summarization of different types of text such as conversations, reviews, long & short texts, etc. We will present what are the industry standard methods for the evaluation of summaries such as ROUGE, BLEU, BLANC, BERTscore, or Supert, and use them to evaluate the summarization models. We will show how we use these approaches in Productboard to automatically and without supervision evaluate the quality of thousands of summaries daily. We will talk about techniques to apply to summarization models to achieve significantly better summaries such as for example fine-tuning, ways how to query GPT models, text cleaning, etc. We will also focus on multi-document summarization. We will describe what are the state-of-the-art models for this task, how to evaluate the multi-document summary, and which techniques we use to preprocess the input documents when we need to summarize a collection comprising hundreds or thousands of texts into one paragraph (such as clustering, text relevancy or pre-summarization of single documents) In the last section of our talk, we will share our experience of implementing the summarization feature in Productboard, how we incorporate the user feedback into our summarization pipeline, how we connect summaries with other ML features and also which tech stack we use, and how we scale it to deploy an independent solution for thousands of companies (each with thousands of text/feedback).

Organizer

Categories

About Machine Learning Prague

Machines can learn. Incredibly fast. Faster than you. They are getting smarter and smarter every day. They are already changing your world, your business and your life. Artificial intelligence revolution is here. Come and learn how to turn this threat into your biggest opportunity. This is not another academic conference. Our goal is to foster discussion between machine learning practitioners and all people who are interested in applications of modern trends in artificial intelligence. You can look forward to inspiring people, algorithms, data, applications, workshops and a lot of fun during three days as well as at two great parties.

Like the format? Trust SlidesLive to capture your next event!

Professional recording and live streaming, delivered globally.

Sharing

Recommended Videos

Presentations on similar topic, category or speaker

Interested in talks like this? Follow Machine Learning Prague