AI Enterprise Search: Architecture and Grounded Answers

AI enterprise search is a permission-aware retrieval and answer system for company data. It connects to workplace sources, resolves the requesting user’s identity, retrieves only authorized evidence, ranks that evidence for the query, and uses a language model to produce an answer with citations.

That definition is deliberately stricter than “chat with your documents.” Enterprise data is distributed across documents, messages, tickets, CRM records, code, dashboards, and operational systems. Each source has its own identity model, access rules, update cadence, and definition of authority. An answer is useful only if it is relevant, current, supported, and safe for the specific person asking.

The short answer

AI enterprise search turns a natural-language question into a controlled evidence pipeline:

1. authenticate the user and resolve their current identities and groups;

2. classify the query and determine which sources and retrieval methods are appropriate;

3. retrieve lexical, semantic, structured, and graph candidates;

4. remove anything the user is not authorized to access;

5. rerank the surviving evidence for relevance, freshness, and authority;

6. assemble a bounded context packet for the model;

7. generate claims that cite supporting passages;

8. return no answer or expose conflict when evidence is missing, stale, or contradictory.

The language model is only the final reasoning and presentation layer. Connector fidelity, identity mapping, ACL propagation, retrieval quality, source governance, and evaluation determine whether the result can be trusted.

AI enterprise search architecture

An implementation may combine layers, but the contracts still exist. If a vendor cannot explain where permission filtering happens or how a citation maps to a passage, the architecture is not ready for sensitive work.

Step 1: connect company data without losing meaning

Enterprise connectors should ingest more than plain text. A useful document record contains a stable source ID, canonical URL, title, body, sections, comments, attachments, author, owner, timestamps, source type, workspace or project, relationships, and access-control data.

The connector also needs lifecycle behavior:

• initial full crawl;

• incremental content updates;

• permission-only updates;

• deletion and tombstone processing;

• retry and rate-limit handling;

• schema-change detection;

• connector health diagnostics.

Freshness is not one number. Measure new-content latency, edit latency, permission-revocation latency, and deletion latency separately. Revocation is usually the security-critical metric.

Some sources are better queried live. A CRM opportunity stage, incident state, or inventory count can change faster than an index sync. A mature system can combine indexed retrieval for durable knowledge with federated tools for operational state, while preserving the user’s source identity.

Step 2: resolve identity before retrieval

The same person can appear as an Entra ID user, Google Workspace account, Slack member, GitHub identity, Salesforce user, contractor guest, and renamed email address. AI enterprise search needs a current mapping between those identities and the groups or roles that grant access.

Identity resolution must handle:

• direct user grants;

• nested and dynamic groups;

• domain-wide or link-based sharing;

• guests and external collaborators;

• suspended and deprovisioned users;

• service accounts and application identities;

• aliases, mergers, and renamed accounts;

• source-specific exceptions.

Permissions should constrain retrieval before protected content enters the model context. Filtering only the final links is insufficient because titles, snippets, embeddings, cached passages, and generated claims may already disclose information.

Glean’s current connector documentation says connectors fetch both content and source permission maps. Its AI Answers documentation states that responses are generated from content the querying user is authorized to access. Microsoft’s synced Copilot connectors similarly ingest an ACL with each item, while federated connectors use the authenticated user’s identity and source permissions at request time.

These are product claims, not substitutes for testing. Build negative tests for users, groups, guests, revoked access, deleted documents, and conversation history.

Step 3: understand the query

Not every question should use the same retrieval plan. A query router can classify intent and extract constraints before recall.

Common query classes include:

• exact lookup: “INC-4821” or a customer ID;

• policy question: “How much can I expense for a hotel in Singapore?”;

• status question: “What is blocking the Atlas launch?”;

• expertise discovery: “Who has implemented SAML provisioning?”;

• comparison: “What changed between the April and July security policy?”;

• cross-source synthesis: “Summarize customer objections and related roadmap decisions”;

• action request: “Create follow-up tickets for unresolved issues.”

The router may identify source scope, time range, entity names, expected answer form, required freshness, and whether a read-only answer or an action workflow is being requested.

Query rewriting can expand acronyms, normalize project names, and produce multiple retrieval variants. It must not silently replace the user’s intent. Keep the original query and log every derived query for evaluation.

Step 4: retrieve with multiple signals

Lexical retrieval

Keyword or BM25-style retrieval remains best for identifiers, names, error strings, quoted language, and rare terms. It is deterministic and easy to debug.

Semantic retrieval

Vector retrieval helps when the question and evidence use different wording. “Parental leave” can match “family care policy,” and “customer cancellation” can match “churn reason.” Vector-only retrieval is risky for exact facts and can return conceptually related but operationally wrong content.

Structured retrieval

Filters and fielded queries use source, owner, status, product, geography, date, content type, or security label. For operational records, structured queries may be more reliable than embedding similarity.

Graph and organizational signals

Links between people, teams, documents, projects, tickets, and systems can improve relevance. Authorship, repeated collaboration, citations, and source relationships can help locate an owner or canonical decision. Behavioral signals should never broaden access.

Federated retrieval

Live source tools can fetch current state without copying it into a central index. They are useful for volatile systems but introduce source latency, tool availability, source-specific auth, and prompt-to-tool routing risk.

The typical production design is hybrid. It unions candidates from several methods, applies permission and policy constraints, deduplicates source objects, and sends a smaller set to a reranker.

Step 5: rerank for relevance, freshness, and authority

Recall finds plausible evidence. Reranking decides what the model actually sees.

A useful scoring model considers:

• query-passage relevance;

• exact entity and identifier matches;

• document freshness and effective date;

• verified or canonical status;

• source and content-type authority;

• ownership and review state;

• user, team, and project context;

• diversity across sources;

• duplication and supersession;

• retrieval confidence.

Personalization should improve usefulness without hiding organizational truth. A frequently opened team note should not outrank the current company policy merely because the user interacts with it often.

Source governance is therefore part of ranking. Verified pages, owners, effective dates, superseded-by relations, and review deadlines give the system signals that embeddings cannot infer reliably.

Step 6: build a bounded evidence packet

The context builder selects passages, not just whole documents. It should preserve the surrounding qualifier, source title, canonical URL, timestamps, and access provenance.

Context selection is a budget problem. Too little evidence omits qualifications; too much evidence dilutes the answer and increases cost. A good packet includes:

• the user’s question and normalized constraints;

• a small set of relevant passages;

• source identity and canonical link;

• date, owner, and authority signals;

• explicit conflict or staleness markers;

• instructions about allowed claim behavior.

Do not rely only on a global system prompt such as “answer from the context.” Validate the produced claims against the evidence after generation.

Step 7: generate claims with citations

Grounded generation should behave like evidence synthesis, not creative completion. Each material claim should map to one or more supporting passages. Citations should appear near the claim and open to a source the user can access.

Glean’s current citation documentation distinguishes inline citations and a source list, and notes that deep links may point to exact passages when supported by the source and admin settings. It also documents that citations can remain in conversation history after source permission is revoked, even though the user may no longer be able to open the item. That is exactly the kind of lifecycle detail a security review should test.

Evaluate citations at claim level:

• Does the source contain the claimed fact?

• Does the cited passage support the exact wording?

• Is the source current and authoritative?

• Can the user open it now?

• Did the answer preserve limitations and exceptions?

A citation is not automatically proof. Models can attach a relevant source to an unsupported claim.

Step 8: handle missing, stale, and conflicting evidence

AI enterprise search needs explicit answer states.

Supported

Current authorized sources agree and directly support the answer. Return the answer with claim-level citations.

Conflicting

Sources disagree or appear to describe different effective periods. Explain the conflict, identify owners and dates, and avoid selecting a winner without an authority rule.

Stale

The best evidence exceeds its review threshold or has been superseded. Label the answer and point to the owner or newer source.

Insufficient

Authorized evidence does not support an answer. State that clearly and offer a narrower query, an owner, or the sources that were searched.

Restricted

Do not reveal that a protected document exists. A safe response should not distinguish “no result” from “result exists but you cannot access it” unless policy explicitly permits that disclosure.

Search, answers, agents, and actions

These experiences share retrieval but have different risk.

Search results let users inspect ranked evidence. Errors are visible and usually reversible.

AI answers synthesize evidence. They add claim-grounding and citation risk.

Agents plan across multiple retrieval and reasoning steps. They add state, tool selection, and accumulated-context risk.

Actions change external systems. They add authorization, execution identity, approval, idempotency, and rollback requirements.

Do not grant write capability merely because read retrieval is permission-aware. Define which identity executes each action, what scopes it has, whether the user confirms, and how the operation is audited.

How to evaluate AI enterprise search

Build a judged question set

Collect 100 to 300 real questions from search logs, onboarding, support requests, and employee interviews. For each question, label the expected sources, required facts, forbidden sources, freshness threshold, acceptable no-answer outcome, and user roles.

Test retrieval and answers separately

Measure recall and ranking before judging prose. A fluent answer cannot repair missing evidence. Track recall at k, precision at k, ranking quality, answer correctness, citation precision, unsupported-claim rate, and abstention quality.

Mutate permissions and content

Grant access, revoke access, change group membership, delete a source, replace a policy, and suspend a user. Test new searches, existing conversations, cached previews, citations, exports, and agent memory.

Measure conflict behavior

Seed two plausible but contradictory documents. Verify that the system exposes the conflict or follows an explicit authority rule rather than selecting whichever passage ranks higher.

Include adversarial content

Insert instructions inside indexed content that attempt to redirect the model, reveal secrets, or ignore policy. Retrieval content is untrusted input. The system should preserve the evidence while preventing embedded instructions from controlling the assistant.

Require replayable evidence

For a sampled answer, an administrator should be able to inspect the authenticated user, original and rewritten queries, sources searched, candidates retrieved, security filters, reranker output, context passages, model and policy version, citations, latency, and feedback.

Metrics that matter

Avoid a single average “accuracy” score. Security, relevance, and abstention have different failure costs.

Common failure modes

ACLs are applied after generation

Protected content may already have influenced the answer. Filter candidates before context construction and test caches.

Vector search replaces every other retrieval method

Exact identifiers, structured state, dates, and source authority suffer. Use hybrid retrieval and query-specific routing.

The newest document is assumed to be authoritative

A recent draft can outrank an older approved policy. Model lifecycle and authority explicitly.

Citations point to documents, not supporting passages

Users cannot verify the claim. Store passage boundaries and test claim-to-passage support.

Personalization reinforces popularity

Frequently accessed but obsolete content continues to rank. Separate behavioral relevance from canonical authority.

No-answer is optimized away

The system produces plausible prose because product teams reward answer rate. Measure correct abstention and unsupported claims.

Conversation history outlives access

Permissions may change after an answer is generated. Define how cached text, previews, citations, exports, and shared conversations behave after revocation.

Retrieved content controls the model

Indexed pages can contain prompt-injection instructions. Treat retrieved content as evidence, never as privileged instructions.

Frequently asked questions

What is AI enterprise search?

AI enterprise search combines permission-aware company search with natural-language understanding and grounded answer generation. It retrieves authorized evidence across workplace systems and returns ranked results or cited answers.

How is AI enterprise search different from enterprise search?

Traditional enterprise search primarily returns ranked documents or passages. AI enterprise search adds query interpretation, evidence synthesis, conversational follow-up, and generated answers, which creates new grounding and lifecycle risks.

How is AI enterprise search different from RAG?

RAG is a method for retrieving context for a generative model. AI enterprise search is the broader product and operating system around it: connectors, identity, ACLs, ranking, citations, UX, analytics, governance, and evaluation.

Does AI enterprise search need a knowledge graph?

Not always. Graph signals can improve people, expertise, ownership, and relationship queries, but strong connectors, identity, hybrid retrieval, metadata, and governance are more fundamental.

Can two employees receive different answers?

Yes. They may have different permissions, work context, source interactions, or query history. The system should explain the evidence and never use personalization to bypass canonical authority.

Is a cited answer always grounded?

No. The citation may be relevant without supporting the exact claim. Grounding evaluation must compare claims with cited passages.

What should happen when permissions are revoked?

New retrieval should stop exposing the content within a defined latency. Organizations must also define behavior for cached snippets, generated answers, citations, shared chats, exports, and agent memory.

What is the safest first use case?

A read-only, well-governed knowledge domain with clear owners, current documents, representative permissions, and an acceptable no-answer path. Avoid write actions until retrieval and answer controls are proven.

Sources

Glean

• Glean AI Answers

• Glean citations

• How Glean accesses information

• Glean connector architecture

• Glean Assistant setup and controls

Microsoft, Google, and Elastic

• Microsoft 365 Copilot connectors

• Microsoft federated connectors

• Vertex AI Search overview

• Elastic connector security

Where Dokki fits

Dokki provides a shared knowledge and operating layer around grounded AI answers. Source documents, structured facts, decisions, review history, and agent output can stay connected, making it easier to test whether an answer used the right evidence and whether a person approved the next action.

_Last verified: July 21, 2026._

• What Is Enterprise Search? Architecture, Permissions, and Use Cases

• Best Enterprise Search Software: An Evaluation Framework

• Knowledge Base Template for People and AI Agents