Semantic Search

Semantic search retrieves chunks by meaning rather than by shared words. Each chunk is turned into a vector embedding; at query time RCE Code embeds your query and returns the chunks whose vectors are nearest by cosine similarity. This finds “rate limiting” code when you search for “throttle requests”, even with no words in common.

Unlike lexical and grep, semantic search has a prerequisite: chunks must be embedded first. Until then it returns an empty list.

Two steps: embed, then search

Embedding is decoupled from indexing so you can index quickly and embed when you are ready. Call engine.embed() to embed every chunk that lacks a current embedding; it returns the number of chunks embedded. Embeddings are content-addressed, so re-running it after a small change only embeds what actually changed.

from rce.code import CodeEngine

engine = CodeEngine.open_or_create("/path/to/repo")
engine.index()                # build the chunk index
n = engine.embed()            # embed pending chunks; returns the count embedded
print(f"embedded {n} chunks")

for r in engine.search_semantic("throttle outgoing requests", top_k=10):
    print(round(r.score, 3), r.source, r.chunk.file_path,
          f"{r.chunk.start_line}-{r.chunk.end_line}")
engine.close()

How the query is answered

search_semantic(query, top_k=10) embeds the query string with the same embedder used at index time, then runs a K-nearest-neighbour search over the vector index and loads the matching chunks. The returned SearchResult list is ordered by descending cosine similarity, so .score is larger-is-better. Each result also carries .chunk and .source = SearchSource.SEMANTIC.

The default embedder is offline

An embedder is anything satisfying a small protocol: it exposes model_id, model_version, and dim, and an embed(texts) method returning a float array. By default RCE Code uses the HashingEmbedder, a deterministic, dependency-free stub that needs no network and no model download.

The hashing embedder is good enough to exercise and test the whole semantic and hybrid path locally, but it does not capture real semantic similarity the way a trained model does. For production-quality results, swap in a real embedder — an OpenAI-compatible endpoint, a sentence-transformers model, or your own — by passing it to CodeEngine.open_or_create(..., embedder=...).

CLI

Embed first, then run search-semantic. It prints one line per hit with the file, line span, and similarity score.

# 1) build embeddings for chunks that lack a current one
rce-code embed

# 2) run semantic search
rce-code search-semantic "throttle outgoing requests"

# point at another repo and cap results
rce-code search-semantic "parse config file" --repo /path/to/repo --top-k 5

src/net/limiter.py:18-44   (score 0.812)
src/net/client.py:90-121   (score 0.755)