Semantic query processing: an introduction

What is an SQPE?

A Semantic Query Processing Engine extends a declarative data interface with operations that interpret meaning using learned models. You describe a task—filter, match, extract, rank, or summarize—and the engine chooses an execution strategy. SQL, dataframe APIs, and declarative pipelines can all serve as the interface.

That is this site's working definition, not a formal industry standard. LOTUS, UQE, and Palimpzest illustrate different approaches. The common systems question is how to make expensive, imperfect model-backed operations composable and efficient.

Six useful operator concepts.

There is no universal operator vocabulary. These six concepts are an entry point; individual systems combine or divide them differently.

Logical and physical query plans

Consider a collection of product reviews. A conventional predicate can restrict the product and date range. A semantic predicate can then ask whether each remaining review describes a safety concern.

-- Illustrative pseudocode, not portable SQL
SELECT product_id, review_text
FROM reviews
WHERE review_date >= '2026-01-01'
  AND SEMANTIC_FILTER(
    review_text,
    'Describes a product safety concern'
  );

This example explains the operator boundary. Consult a system's documentation for executable syntax.

A naive plan sends every row to a large model. An optimizer may first apply inexpensive relational predicates, reuse results, batch requests, or route suitable records through a cheaper proxy. Each choice changes cost and may affect answer quality. PLOP studies semantic-operator placement; Fast semantic filtering studies adaptive filtering strategies.

From logical intent to physical choices

1. Parse the request. Separate ordinary relational operations from semantic tasks.
2. Construct a logical plan. Represent dependencies, inputs, output types, and task instructions.
3. Choose a physical plan. Select models, access paths, batching strategies, and execution order.
4. Execute and measure. Track latency, model calls, tokens, failures, and output quality.
5. Validate the result. Inspect evidence, assess errors, and retain provenance for repeated runs.

This is a conceptual architecture, not a claim that every listed system implements each layer. Kalypso explores how a serving layer can also exploit the query plan.

Optimization objectives and correctness

Traditional relational rewrites are judged by whether they preserve query semantics. With model-backed operators, changing the model, prompt, batch context, or candidate set can change the answer. “Faster” is meaningful only alongside an explicitly measured quality target.

Cost and latency are different

A smaller token bill does not necessarily produce a faster query. Concurrency, serving queues, network overhead, and cache behavior all matter. Report both end-to-end time and resource consumption, with model versions and configuration.

A proxy is not the oracle

A cheap classifier or embedding model can avoid some expensive calls. But pruning can miss relevant records. Distinguish guarantees of agreement with a reference model from correctness against human labels. SemViews illustrates this distinction in reuse and certification.

Context is part of the execution

Batching records together can reduce overhead while changing the information the model sees. Caching can save repeated work, but only under a valid key that accounts for the input, prompt, model, and relevant settings. NousDB studies token-aware execution; relational LLM optimization studies prefix reuse.

Evaluate the entire answer.

An accurate operator does not automatically yield an accurate final result. Errors may compound across filters and joins; aggregates can hide item-level disagreements. An evaluation should connect the operator-level metric to the user's actual task.

• Fix a documented workload, dataset version, model, prompt, and budget.
• Report precision and recall where appropriate, alongside latency and cost.
• Repeat runs and inspect answer variability, rather than reporting a single execution.
• Record timeouts, invalid outputs, retries, and omitted results.
• Use held-out human labels or a justified reference, and state its limitations.

Start with SemBench for end-to-end workloads, LROBench for a relational-operator view, UDA-Bench for document analytics against annotated relational views, and Same Query, Different Answer for reproducibility. These are complementary perspectives, not interchangeable scorecards.

A reading path through the field.

1. SUQL — understand the structured/unstructured boundary.
2. LOTUS — see semantic operations as a declarative abstraction.
3. Palimpzest and DocETL — examine pipeline optimization.
4. PLOP and LLM ORDER BY — explore physical planning choices.
5. Benchmarks and evaluation — decide what evidence a system comparison needs.