Skip to content

MDMS Data Extraction Methods

The MARS Data Mining Studio (MDMS) utilizes a versatile array of methods to accurately identify and extract specific data elements from diverse sources, including unstructured documents and complex data streams. These techniques form the core of MDMS's ability to transform raw content into structured, actionable information.

Key Extraction Techniques Employed:

  • Text Pattern Matching:
  • Regular Expressions (RegEx): Provides a powerful syntax for defining complex search patterns to locate and extract data that conforms to specific formats (e.g., identifying invoice numbers matching INV-\d{6}, extracting email addresses, parsing specific log entry formats).
  • Wildcard Patterns: Offers simpler pattern matching capabilities using common wildcard characters (*, ?) for less structured or variable text searches.
  • Positional Extraction (Zoning): Allows users to define precise rectangular coordinates (zones) on a document page template or within the known structure of a fixed-layout report. MDMS extracts the content found strictly within these predefined boundaries. This is highly effective for processing standardized forms.
  • Keyword/Anchor-Based Extraction: Identifies target data based on its location relative to known, stable text labels or keywords (anchors) within the document. For instance, extracting the value appearing on the same line after the text "Account Balance:".
  • Optical Character Recognition (OCR): Converts images containing text (found in scanned documents, image-only PDFs, or graphic files) into machine-readable character streams. This recognized text can then be subjected to further analysis using other extraction methods like pattern matching or keyword searches.
  • Intelligent Character Recognition (ICR): A specialized form of OCR often leveraging AI/ML techniques, specifically focused on improving the accuracy of recognizing handwritten text within scanned forms or documents.
  • Optical Mark Recognition (OMR): Detects the presence or absence of marks within predefined areas, commonly used for processing data from checkboxes or multiple-choice bubbles on scanned forms or surveys.
  • Barcode & QR Code Recognition: Automatically detects and decodes various standard 1D (linear) and 2D barcode symbologies (including QR codes) that are embedded within documents or images, extracting the encoded data string.
  • Signature Detection: Identifies regions within a document likely containing a handwritten signature, often used for verification flags or process triggers.
  • Table Extraction: Incorporates algorithms specifically designed to recognize data organized in tabular structures (rows and columns) within documents, enabling the extraction of cell values while maintaining their row/column relationships.
  • Rule-Based Logic & Scripting: Offers the capability to combine multiple extraction methods using conditional logic (e.g., "IF Zone A contains 'Invoice' THEN extract Zone B using RegEx pattern X ELSE use keyword Y"). Potential for custom scripting can further handle highly unique or complex extraction challenges.

By allowing the combination of these methods within extraction templates, MDMS provides the robustness needed to handle the variety and complexity of real-world enterprise data sources.