Persian Topic Modeling using BERTopic

Discovering meaningful topics from Persian text using transformer-based NLP.

Project Snapshot

Executive Summary

This project explored transformer-based topic modeling for Persian text using BERTopic. The objective was to discover meaningful semantic topics from large collections of unstructured Persian documents while preserving contextual relationships between words and sentences.

The project demonstrates the application of modern NLP techniques to a low-resource language where traditional topic modeling methods often struggle.

Business Problem

Large collections of Persian text are difficult to analyze manually. Organizations and researchers need automated approaches to identify hidden themes, organize documents, and better understand textual data.

Traditional probabilistic topic models often fail to capture semantic meaning, particularly for morphologically rich languages such as Persian.

Technical Solution

A topic modeling pipeline was developed using BERTopic, combining transformer-generated sentence embeddings with dimensionality reduction and density-based clustering.

The workflow automatically grouped semantically similar documents and generated interpretable topic representations for downstream analysis.

My Contributions

• Developed an end-to-end Persian topic modeling pipeline.
• Generated contextual text embeddings using transformer models.
• Applied UMAP for dimensionality reduction.
• Used HDBSCAN to discover semantic document clusters.
• Evaluated and interpreted generated topics.
• Explored topic quality through qualitative analysis.

Results

• Successfully identified coherent semantic topics from Persian documents.
• Demonstrated the effectiveness of transformer-based topic modeling for Persian NLP.
• Produced interpretable topic representations for exploratory text analysis.

Technology Stack

• Python
• BERTopic
• Sentence Transformers
• UMAP
• HDBSCAN
• Pandas
• NumPy

Engineering Decisions

Instead of using traditional probabilistic topic models such as LDA, I selected BERTopic because it leverages contextual sentence embeddings generated by transformer models.

This approach provides richer semantic representations and generally produces more coherent topics for complex natural language datasets.

Lessons Learned

This project expanded my understanding of modern topic modeling, semantic embeddings, and transformer-based NLP.

If I continue developing this project, I would evaluate multilingual embedding models, experiment with larger Persian datasets, and build an interactive visualization dashboard for topic exploration.