NeMo Retriever Library is a scalable microservice framework designed for extracting, structuring, and enriching content from documents to support downstream generative AI applications. It processes various document types by splitting them into components such as text, tables, charts, and images, and then applies OCR and contextual analysis to convert them into structured data formats. The system is built on NVIDIA NIM microservices, enabling high-performance parallel processing and efficient handling of large datasets. It supports multiple extraction strategies for different document formats, balancing accuracy and throughput depending on the use case. Additionally, it can generate embeddings for extracted content and integrate with vector databases like Milvus, making it well-suited for retrieval-augmented generation pipelines.
Features
- Extraction of text, tables, charts, and images from documents
- Parallelized document processing pipeline
- OCR-based contextualization and structured output generation
- Embedding creation and vector database integration
- Support for multiple file formats including PDF and media files
- Preprocessing and postprocessing pipelines for data transformation