Architecting a Low-Code RAG Pipeline: Implementing Semantic Search over MongoDB via KNIME and OpenAI Embeddings

In the modern data landscape, the ability to perform semantic queries on unstructured datasets is a significant competitive advantage. Traditional keyword-based searches (using regex or exact string matching) fail to capture the latent semantic meaning within conversational data, logs, or documentation. To bridge this gap, developers are increasingly turning to Retrieval-Augmented Generation (RAG).

While building RAG architectures typically requires a complex stack of Python libraries—such as LangChain, LlamaIndex, and specialized vector database drivers—it is possible to architect a production-grade, auditable, and repeatable pipeline using low-code orchestration via KNIME. This post details the technical implementation of a two-phase RAG system: an ingestion pipeline for vector store creation and a retrieval/inference pipeline for semantic querying.

The Core Architecture: Retrieval-Augmented Generation (RAG)

The objective is to move beyond "hard queries" (date ranges, specific IDs) toward "semantic queries." This requires transforming unstructured text into high-dimensional vectors through an embedding model. Once vectorized, these embeddings are stored in a specialized structure—a Vector Database—allowing for similarity searches based on cosine similarity or Euclidean distance.

Our implementation utilizes:

• Data Source: MongoDB (NoSQL document store).
• Orchestration Engine: KNIME (Node-based ETL and AI workflow editor).
• Embedding Model: OpenAI text-embedding-3-small.
• Vector Store Implementation: FAISS (Facebook AI Similarity Search) via the KNIME FAISS nodes.
• Inference LLM: OpenAI GPT-4o mini.

Phase 1: The Ingestion Pipeline (Vector Store Creation)

The first workflow focuses on the ETL (Extract, Transform, Load) process required to convert MongoDB documents into a persistent vector store.

1. Data Extraction and Transformation

The pipeline begins with a MongoDB Connector node, establishing a connection to the cluster using SRV records and authentication credentials. A MongoDB Reader node then executes a query—in this case, an empty set {} to fetch all documents from the target collection—and ingulating the data as JSON objects.

Because vectorization requires structured input, we implement a transformation layer:

1. JSON to Table Conversion: The raw JSON stream is parsed into a tabular format (rows and columns).
2. Data Cleaning: A Row Filter node is utilized to prune the dataset by removing entries where the clean_content field is null or empty, preventing errors during the embedding process.

2. Embedding and Vectorization

To transform text into numerical vectors, we integrate OpenAI’s API. For security and reproducibility, we avoid hardcoding API keys. Instead, we use the KNIME Hub Authenticator to retrieve credentials from a secure Secrets store within the KNKNIME profile.

Using the OpenAI Embedding Model Selector, we specify the text-embedding-3-small model. This model is passed into the FAISS Vector Store Creator node. The configuration involves:

• Target Column: Selecting the clean_content column for vectorization.
• Metadata Injection: Including all other available columns (e.g., author_username, channel, timestamp) as metadata. This is critical for downstream filtering during the retrieval phase.

3. Persistence

The resulting vector store exists in memory during execution. To ensure persistence, a ModelWriter node is used to serialize the FAISS index and its associated metadata to a local .model file on the filesystem.

Implemented Pipeline: Ingestion $\rightarrow$ Embedding $\rightarrow$ FAISS Indexing $\rightarrow$ Local Persistence

Phase 2: The Retrieval & Inference Pipeline (The Query App)

The second workflow functions as the user-facing application, capable of performing semantic lookups and generating natural language responses.

1. Model Loading and User Interface

The pipeline starts by loading the previously saved index via a Model Reader. To create an interactive experience, we implement a Component containing UI widgets:

• String Widget (Query): Captures the user's semantic question (e.g., "What are students saying about interviews?").
• String Widget (Username): Allows for targeted metadata filtering (e.g., searching only messages from a specific user).

These inputs are converted into a structured format using a Variable to Table Row node, allowing the query parameters to flow through the workflow as data attributes.

2. Semantic Retrieval and Metadata Filtering

The Vector Store Retriever node performs the heavy lifting. It takes the user's query string, vectorizes it using the same OpenAI embedding model used in Phase 1, and searches the FAISS index for the most relevant document chunks.

To enhance precision, we implement a Metadata Filter. By mapping the username variable to the author_username metadata field within the retriever node, we can restrict the search space to specific users, significantly reducing noise in large datasets.

3. Data Orchestration and Prompt Engineering

The retrieved data is often returned as an array of complex objects. To make this digestible for a Large Language Model (LLM), we perform several orchestration steps:

1. Ungrouping: Expanding the retrieved results into individual rows.
2. String Expression: A custom expression node constructs a standardized prompt format, such as: [User] in [Channel] said: [Content] on [Date]
3. Aggregation via Group By: To prevent overwhelming the LLM's context window with fragmented rows, we use a Group By node with Manual Aggregation. We concatenate all individual prompts into a single, large string using a pipe (|) delimiter.

4. The Inference Layer: Agent Chat View

The final step is the injection of this processed context into an LLM. We construct a "Final Prompt" that contains both the retrieved context and the original user query. This string is passed as a System Message to the Agent Chat View.

By utilizing GPT-4o mini, we leverage high-reasoning capabilities at a low token cost. The model receives the concatenated context (the "knowledge") and the user's question, allowing it to generate accurate, context-aware answers based solely on the provided MongoDB data.

Conclusion

This architecture demonstrates that complex AI workflows do not require massive codebases. By leveraging KNIME’s node-based orchestration, we have built a system that is:

• Auditable: Every transformation step from MongoDB to LLM is visible and traceable.
• Scalable: The pipeline can be extended to include more complex embedding models or different vector backends.
• Maintainable: Updates to the data schema or the prompt logic can be implemented via simple node reconfiguration rather than refactoring entire Python scripts.