Segment-Level Agentic Topic Modeling for Improved Data Exploration and Resource Efficiency
[Sztuczna inteligencja] Autorzy: Myeongjun Erik Jang, Antonios Georgiadis, Sae Young Moon i 1 więcej
Topic modeling is an effective technique for discovering hidden themes within documents and is widely used in text mining and data analysis across a variety of industry sectors. Recently, large language model (LLM)-based topic models have been emerged that prompt LLMs to generate topics then assign the topics to documents, producing more natural and human-readable topics than conventional topic modeling algorithms. However, the nature of topic assignment process causes certain drawbacks, such as the incapability to produce topic distributions over a document, too broad or narrow topics, and high resource consumption, which increases with the number and length of of documents being assigned topics. These issues are particularly critical for industrial applications, which require high-quality, in-depth analysis and the processing of large volumes of documents. In this context, this paper …
- Kategoria arxiv
- cs.AI
- Kategoria
- Sztuczna inteligencja