Semantic text segmentation based on topic recognition

Topics: Chunk Relevance, Document Classification, Indexing, LLMO / GEO, Microsoft

The Microsoft patent outlines a system and methodology for semantic text segmentation based on topic recognition, which aims to optimize how text content is indexed and searched within media, such as audio and video. By utilizing both text classification and segmentation techniques, the system improves the efficiency of identifying relevant segments in transcripts. It focuses on analyzing sentences to detect associated topics and refining these classifications through an iterative relationship between classification and segmentation models, thereby enhancing search capabilities.

General Information:

• Patent ID: US20250156642A1
• Assignee: Microsoft Technology Licensing LLC
• Countries: United States
• Last Publishing Date: November 13, 2023
• Inventors: Mattan Serry, Oron Nir
• Status: Pending

Background

The background of this patent addresses the limitations of current methods for indexing media content, particularly the challenges in making unstructured media, like audio and video, searchable. It discusses the inefficiencies of manual tagging and the need for an automated approach to classify and segment text in a structured manner, enabling users to locate relevant content based on identified topics seamlessly. This necessity arises from the increasing volume of media content, making traditional methods impractical and expensive. The proposed innovations respond to these issues by combining text classification and segmentation processes for enhanced accuracy and searchability.

Claims of the Patent

The patent describes systems and methods for semantic text segmentation based on topic recognition, which leverage text classification and segmentation models. The main claims include methods for:

1. Analyzing text files, such as transcripts, using a text classification model to identify topics associated with sentences.
2. Utilizing the output of the text classification model to inform a text segmentation model, which segments text into meaningful clusters like paragraphs.
3. Allowing the text classification model to refine its accuracy by analyzing segments rather than individual sentences, and vice versa.
4. Using the outputs (labels) from the segmentation process for indexing and searching media content efficiently.

• Semantic Analysis: Utilizes text classification models to analyze text and identify associated topics for each sentence.
• Text Segmentation: Segments text into meaningful units (e.g., paragraphs) based on classifications performed by the model.
• Feedback Loop: Outputs from the text classification model inform the text segmentation model and vice-versa, allowing for improved accuracy.
• Efficient Indexing: Labels generated from the classified segments enable effective indexing and searching in media archives.
• Applicability: While focused on transcripts, the methods can be adapted for any textual content.

Process

Step 1: Media Content Input

1. Media Content Reception: The methodology begins with receiving media content, which can be audio or video. This content is then transcribed into a text format using a transcriber (e.g., Automatic Speech Recognition (ASR) system).

Step 2: Transcript Generation

2. Transcript Creation: The transcriber generates a transcript that consists of individual sentences, each paired with a timestamp, indicating when each sentence occurs in the media content.

Step 3: Text Tokenization

3. Tokenization of Sentences: Each sentence in the transcript is tokenized to break it down into smaller components (tokens), such as words or phrases.

Step 4: Text Classification

4. Initial Topic Classification: A text classification model is employed to analyze each tokenized sentence. It computes the probability of each sentence belonging to predefined topic classes (e.g., SPORTS, HEALTH). This is done by:
   • Using algorithms like Transformer architectures (e.g., BERT, GPT) to assess the semantic content of sentences.
   • Generating a probability distribution for classes related to each tokenized sentence.

Step 5: Output of Classification Model

5. Generation of Classification Records: The outputs from the classification model, including the class probabilities for each sentence, are structured into classification records.

Step 6: Semantic Text Segmentation

6. Segmentation of Text: The classification records are then input into a text segmentation model. This model analyzes the relationships between consecutive sentences based on:
   • Temporal Proximity: How close the timestamps of two sentences are.
   • Semantic Relationship: How likely it is that two sentences pertain to the same topic based on the probabilities from the classification model.
   • The model determines if adjacent sentences should be grouped into a single segment (e.g., a paragraph).

Step 7: Segment Classification

7. Text Segment Classification: Once the text segments have been created, they are then classified by the text classification model again. The class probabilities are recalculated for the entire segment rather than individual sentences, allowing for more context-aware classification.

Step 8: Label Assignment

8. Labeling Segments: Based on the classification results from the text segments, labels corresponding to the identified topics are assigned to these segments. These labels facilitate searches and indexing of the media content.

Step 9: Storage and Indexing

9. Storage of Segments and Labels: The segments and associated labels, along with their corresponding timestamps, are stored in a media index. This index can be queried later to retrieve relevant portions of media content based on user searches for specific topics or labels.

Step 10: Feedback Loop for Model Improvement

10. Model Refinement: In some implementations, the outputs from the segmentation classification, which represent ground truth for segment labels, can be backpropagated to adjust and improve the parameters of the text classification model. This iterative feedback loop enhances the model’s accuracy for future analyses.