About this collection

About this collection

AI Engineering Technology Landscape 2026

3 of 3 in this book · 17 min read

Version: 1.0
Updated: 2026-10-10
Audience: Software Architects, Engineering Managers, AI Platform Engineers, Technical Product Owners
Scope: Production-grade AI applications, RAG, agentic systems, and enterprise AI platforms.

Research note: This is a curated technology landscape, not a measured GitHub popularity ranking. Project maturity, license, features and production readiness must be verified against current official documentation before selection. Technologies listed together are not necessarily direct substitutes.

Executive summary

Building an AI application is no longer just calling a model API. A production system requires clear separation between the user experience, agent runtime, model access, context retrieval, persistent state, external integrations, execution environment, and cross-cutting operational controls. Not every application needs every layer: start with requirements and add complexity only when justified.

Architecture at a glance

flowchart TB
  UI[AI UX: Chat / Copilot / Generative UI] --> API[Application API / Streaming]
  API --> ORCH[Agent / Workflow Orchestration]
  ORCH --> GW[Model Gateway]
  GW --> MODELS[Hosted / Self-hosted Models]
  ORCH --> RAG[Retrieval and Context Engineering]
  RAG --> SEARCH[(Vector / Search / Graph)]
  ORCH --> MEMORY[(Agent Memory / State)]
  ORCH --> TOOLS[Tools / MCP / External APIs]
  ORCH --> EXEC[Isolated Code / Browser Sandbox]
  ORCH --> HUMAN[Human Review / Approval]
  OBS[Observability / Evaluation / Cost] -.-> ORCH
  SEC[Identity / ACL / Guardrails / Governance] -.-> ORCH
  CICD[Testing / Deployment / LLMOps] -.-> ORCH

1. AI UI and application experience

Capabilities: token streaming, tool-call visualization, structured response rendering, citations, multi-modal attachments, agent progress, interactive cards, user confirmation, error recovery, accessibility, and state synchronization.

Technology Main role Strengths Watch-outs
Vercel AI SDK TypeScript model/UI toolkit Streaming, structured output, React integration, provider abstraction Not a complete durable workflow platform
assistant-ui React AI chat components Customizable chat and tool UI Backend orchestration remains separate
CopilotKit Embedded AI copilots Agent interaction with app state/actions Carefully constrain allowed frontend actions
AG-UI Agent–UI event protocol Backend/frontend decoupling Requires compatible runtimes and event contracts
Streamlit Python data/AI app UI Rapid prototyping Complex enterprise UX may require custom frontend
Gradio Model demos Fast multimodal demos Less suitable for deeply customized product UX
Chainlit LLM conversational UI Quick agent/chat demos Evaluate customization and deployment constraints
Open WebUI Self-hosted model interface Ready-made chat UX Verify access controls and integration boundaries
MCP Apps Tool-provided interactive experiences Extensible agent/tool UI Check client support and trust boundaries

Practices: distinguish model text from executable UI actions; validate schema before rendering; allow cancellation; persist conversation and tool status; support retries without duplicate side effects; require explicit approval for destructive actions.

Use cases: enterprise ticket copilot, AI-powered form completion, interactive implementation-plan reviews, generated dashboards.

Common mistakes: rendering arbitrary AI-generated HTML; treating streamed partial JSON as final; allowing AI to trigger sensitive UI actions without server-side authorization; ignoring conversation restoration.

2. Agent frameworks and orchestration

Capabilities: planning, tool calling, agent handoffs, state, branching, parallel tasks, checkpointing, retries, timeout management, human-in-the-loop, and execution limits.

Technology Primary use Best fit Caution
LangGraph Stateful graph-based agent workflows Complex control flow, checkpoints Graph/state complexity
Mastra TypeScript AI agents and workflows TS-native platforms Verify current feature maturity
CrewAI Role-oriented agent teams Multi-agent experimentation Multi-agent overhead can exceed benefits
OpenAI Agents SDK Agents, handoffs, tools, tracing Lightweight agent workflows Check provider/runtime assumptions
Pydantic AI Type-safe Python agents Typed Python services Python-focused
Microsoft Agent Framework Microsoft-oriented agent workflows Enterprise Microsoft stacks Check version/status and migration guidance
VoltAgent TypeScript agent tooling Node/TS experimentation Verify ecosystem and maintenance
Temporal Durable general-purpose workflows Long-running, reliable business processes Operational complexity; not itself an LLM agent framework

Practices: choose deterministic workflows for predictable business steps; reserve agent autonomy for ambiguous steps; implement idempotency keys, execution budgets, step-level logs, state versioning, and approval gates.

Use cases: BRD generation → human review → implementation plan → ticket creation; multi-repository impact analysis; asynchronous document enrichment.

Common mistakes: assuming checkpointing guarantees exactly-once external side effects; recursive agent loops without budgets; adding multiple agents where a single workflow is sufficient; mixing authorization decisions with model instructions.

3. Model gateway and routing

Capabilities: provider abstraction, model routing, retries, fallback, quotas, tenant-level budgets, rate limiting, request logging, caching, structured output, and secret isolation.

Technologies: LiteLLM, Portkey, Kong AI Gateway, Vercel AI SDK provider integrations.

Practices: centralize credentials; use workload-specific model policies; version routing decisions; measure quality and latency before automatic failover; track prompt and completion tokens and per-tenant spend; define provider-specific feature compatibility.

Use cases: cheaper model for classification, stronger model for architecture review, specialized model for coding, fallback during provider outage.

Common mistakes: assuming model APIs have identical tool-calling semantics; failing to retest quality after fallback; retry storms; exposing raw provider keys in browsers.

4. Vector databases, graph databases, and search

Category Technologies Best for Limitations
Vector database Qdrant, Milvus, Pinecone, Weaviate, Chroma Semantic similarity and metadata filtering Similarity alone does not establish truth
Embedded vector FAISS, LanceDB Local retrieval and embedded deployments Distributed operations vary
SQL + vector PostgreSQL pgvector Unified transactional + vector data Tune indexing and scaling carefully
Graph database Neo4j, Memgraph, FalkorDB Entity relationships, traversals Requires entity/relation quality
Multi-model SurrealDB, ArangoDB Mixed document/graph models Evaluate query semantics and operational maturity
Search engine Elasticsearch, OpenSearch, Vespa Keyword, hybrid, ranking Search infrastructure complexity
Cache/state Redis, Valkey Session, cache, queues Not a substitute for durable source-of-truth design
Object storage S3, MinIO Source files and extracted assets Requires indexing and lifecycle management

Practices: model ACLs in metadata and enforce authorization during retrieval; benchmark hybrid search against vector-only; version embeddings; implement deletes across indexes; track provenance; test high-cardinality tenant filters.

Common mistakes: mixing data from different tenants; relying on nearest-neighbor scores as confidence; neglecting stale index cleanup; adopting graph DB without graph-specific queries.

5. RAG and context engineering

Capabilities: ingestion, parsing, OCR when necessary, chunking, indexing, embedding, metadata enrichment, query transformation, hybrid retrieval, reranking, graph traversal, context compression, citation, freshness, and feedback.

Technology Main capability
LlamaIndex Data-centric indexing and retrieval workflows
Haystack Modular RAG/search pipelines
LangChain Broad document/retrieval integration ecosystem
Microsoft GraphRAG Graph-based retrieval and community summaries
LightRAG Graph-enhanced retrieval
Pathway Live data processing and indexing
Docling Structured document extraction
Unstructured Document parsing and partitioning
ColBERT Late-interaction retrieval
BGE rerankers Reranking candidate passages
DSPy Programmatic optimization of LM pipelines

Reference retrieval flow: authorization context → query understanding → keyword/vector/graph retrieval → deduplication/fusion → reranking → context packing → answer with source references → evaluation.

Configuration surfaces: chunk size/overlap; embedding model and version; top-k; hybrid weights; reranker model; ACL filters; per-source freshness; context budget; domain-specific retrieval strategy; optional agent skills controlling retrieval workflow.

Use cases: Jira/Confluence enterprise assistant; cross-document impact analysis; API dependency search; design-document Q&A.

Common mistakes: treating chunking as one-size-fits-all; measuring only final-answer fluency; omitting permission-aware retrieval; embedding malformed tables; overloading context with low-relevance passages.

6. GraphRAG and knowledge graphs

Capabilities: entity resolution, relation extraction, ontology, multi-hop traversal, graph + vector retrieval, graph updates, temporal relationships, provenance.

Technologies: Microsoft GraphRAG, LightRAG, Neo4j, Graphiti, Memgraph, FalkorDB, SurrealDB (evaluate query model).

Use cases: trace requirements → features → tickets → code modules → APIs → incidents; answer dependency and blast-radius questions; identify duplicate entities across sources.

Practices: define identity keys, relation types, timestamps, evidence links, confidence; separately evaluate entity extraction and answer generation; support graph corrections and deletions.

Common mistakes: merging same-type nodes solely by label; treating inferred edges as verified facts; building expensive graphs without multi-hop business requirements.

7. Agent memory and persistent context

Technology Focus
Mem0 Persistent extracted memories
Zep Context and temporal knowledge
Graphiti Temporal knowledge graph
Letta Stateful agent memory
Cognee Semantic and graph-based memory
LangMem Memory management utilities

Memory types: working (current context), episodic (past interactions), semantic (stable knowledge), procedural (instructions and reusable skills).

Practices: separate user preferences from business facts; require provenance and update policies; scope by tenant/project/user; implement consent, retention, conflict resolution, and forgetting; evaluate stale-memory impact.

Common mistakes: persisting every conversation as a fact; no expiry; mixing personal and enterprise context; injecting memory without relevance checks.

8. Tools, MCP, interoperability and skills

Standard/concept Purpose Architectural boundary
MCP Tools, resources, prompts Agent ↔ external capabilities
A2A Agent-to-agent communication Independent agents
AG-UI Agent/UI events Runtime ↔ frontend
A2UI Declarative agent-driven UI patterns Agent ↔ UI representation
MCP Apps Interactive tool-linked UI Tool ↔ host experience
Agent Skills Reusable instructions, workflows and resources Agent behavior configuration

Practices: discover tools dynamically only within allowlists; version tool schemas; validate tool arguments; separate read/write permissions; propagate user identity to downstream services; enforce rate limits; log side effects; sandbox untrusted execution.

Use cases: Jira ticket discovery, Confluence search, Git operations, browser automation, reusable architecture-review skills.

Common mistakes: using MCP as a replacement for every ordinary application API; trusting tool output as instructions; letting a model bypass business approvals; confusing protocol support with guaranteed interoperability.

9. Evaluation frameworks and models

Technology Evaluation focus
DeepEval LLM application testing, agent/RAG metrics
Ragas RAG relevance, faithfulness and retrieval quality
Promptfoo Prompt regressions, model comparison, red teaming
Inspect AI Model/agent evaluation tasks
lm-evaluation-harness Model benchmarking
LightEval Evaluation pipelines
Braintrust Datasets, experiments and scoring
LangSmith Traces, datasets and agent evaluation
TruLens Feedback-based evaluation
PyRIT Security red teaming

Evaluation dimensions: retrieval recall/precision; groundedness; factual correctness; tool selection; trajectory quality; task success; instruction following; refusal correctness; latency; cost; robustness; security.

Evaluation approaches: deterministic assertions; reference-based scoring; reference-free LLM-as-judge; pairwise comparisons; human review; simulation; online feedback.

Practices: maintain representative golden datasets; include difficult negative cases; use blinded judge calibration; compare model/prompt/index versions; set acceptance thresholds per business workflow; report confidence intervals when sample sizes allow.

Common mistakes: using the same model for generation and judging without calibration; scoring only happy paths; interpreting a single aggregate score as production readiness; leaking expected answers into prompts.

10. Observability, tracing and prompt operations

Technologies: Langfuse, Arize Phoenix, LangSmith, Helicone, Comet Opik, W&B Weave, OpenTelemetry, OpenInference.

Track: trace ID; tenant/project; model/provider; prompt version; retrieved document IDs; tool calls; latency by step; token counts; costs; retries; guardrail decisions; evaluator scores; user feedback.

Practices: instrument spans consistently; redact PII and secrets; set retention policies; correlate agent traces with application logs; separate operational dashboards from quality dashboards.

Common mistakes: logging sensitive full prompts by default; tracing only model calls rather than retrieval and tool steps; lacking correlation IDs; having metrics without trustworthy source data.

11. Guardrails, security and governance

Technologies: NeMo Guardrails, Guardrails AI, LLM Guard, Presidio, Garak, PyRIT.

Controls: identity and SSO; RBAC/ABAC; least privilege; prompt injection defenses; tool-output validation; PII detection; secrets protection; tenant isolation; audit trails; model usage policies; human approvals.

Practices: enforce security in backend services and tool adapters, not in prompts; treat retrieved content as untrusted; use approval gates for irreversible actions; test indirect prompt injection and exfiltration paths.

Common mistakes: confusing guardrail prompts with authorization; assuming PII detectors are perfect; permitting an agent to retrieve data beyond the end-user's rights.

12. Sandboxed code and browser execution

Technologies: E2B, Daytona, Browser Use, Playwright, Firecracker, container isolation.

Capabilities: ephemeral environments, network egress policies, filesystem isolation, timeouts, resource quotas, audit logs, artifact capture, replayability.

Use cases: running generated scripts, browser-based QA, codebase inspection, reproducible test environments.

Common mistakes: executing arbitrary generated code on the application host; unrestricted internet access; mounting production credentials into sandboxes.

13. Inference serving and model optimization

Category Technologies
High-throughput serving vLLM, SGLang, Hugging Face TGI
Local inference Ollama, llama.cpp, LM Studio
Serving runtimes NVIDIA Triton, TensorRT-LLM, ONNX Runtime
Fine-tuning PEFT, LoRA, QLoRA, Axolotl, Unsloth
Optimization AWQ, GPTQ, quantization, speculative decoding, batching, prefix caching
Distributed training DeepSpeed, PyTorch FSDP, Ray Train

Practices: benchmark quality, throughput, p95 latency, memory footprint and operating cost on actual workloads; test quantization impact; plan concurrency and queueing.

Common mistakes: comparing only raw tokens/second; ignoring tool-call compatibility; selecting a model based solely on leaderboard performance.

14. Testing and red teaming

Test layers: unit tests for deterministic logic; contract tests for tools and model responses; retrieval tests; agent trajectory tests; end-to-end tests with real services; UI quality audits; adversarial security tests; performance/load tests; disaster recovery drills.

Techniques: synthetic test generation with human review; mutation testing; golden traces; adversarial prompt suites; metamorphic tests; shadow traffic; canary rollout.

Acceptance gates: no cross-tenant leakage; correct authorization; stable tool schemas; minimum task success; acceptable cost/latency; no critical security regressions; human-reviewed edge cases.

15. Human-in-the-loop and approval

Capabilities: pause/resume; reviewer assignment; diff views; evidence and citations; approval/rejection; edit-before-execute; escalation; immutable audit history.

Practices: distinguish review of content from authorization to execute; include idempotency; record approver identity and version; expire stale approvals; surface what changed since last review.

Common mistakes: auto-executing after content approval without separate permission; losing workflow state during handoff; reviewers receiving insufficient evidence.

16. Cost, FinOps and capacity

Capabilities: model spend by tenant/feature; token budgets; context caching; semantic caching; routing; rate limits; batching; latency-cost-quality optimization; forecast and anomaly alerts.

Practices: optimize total task cost rather than single-call cost; measure retries and tool overhead; distinguish inference cost from vector/search/storage cost; track unit economics per completed workflow.

Common mistakes: optimizing for cheap tokens while task success drops; not accounting for long context, reranking or evaluation costs.

17. Identity, permissions and data governance

Capabilities: SSO, OAuth/OIDC, RBAC/ABAC, project/tenant scopes, service identities, data lineage, retention, consent, data residency, audit.

Practices: propagate caller identity to retrieval and tool execution; filter data before it reaches the model; maintain source ACL sync; delete derived embeddings/memories when underlying data is deleted.

Common mistakes: trusting frontend role flags; stale permissions in vector metadata; sharing caches across tenants.

18. Deployment and LLMOps

Capabilities: model registry, prompt versioning, dataset versioning, CI evaluation, staged rollouts, feature flags, observability, rollback, disaster recovery, infrastructure-as-code.

Technologies: GitHub Actions, GitLab CI, Argo CD, Kubernetes, Helm, Terraform, MLflow, W&B, model gateways and observability platforms.

Practices: pin versions; promote model+prompt+retrieval configs together; use canary evaluation; keep rollback paths; monitor quality after release.

Common mistakes: treating prompt changes as non-code changes; deploying new embeddings without reindex strategy; no production rollback plan.

19. Data and feedback pipelines

Capabilities: source connectors, CDC, event streaming, extraction, enrichment, index refresh, feedback capture, dataset curation, lineage, data quality checks.

Technologies: Kafka, Debezium, Airbyte, Dagster, Prefect, Pathway, dbt, object storage, search/vector pipelines.

Practices: event-driven indexing where freshness matters; handle deletions and permission changes; deduplicate feedback; preserve source references and dataset versions.

Common mistakes: ingestion pipelines that never reconcile deletes; training/evaluation leakage; unlabeled or biased feedback.

20. Technology interoperability and standards

Key design question: which boundaries should be standardized versus implemented directly?

  • MCP for tools/resources/prompts when cross-client interoperability is valuable.
  • A2A for independently deployed agents needing cross-agent communication.
  • AG-UI for structured agent-to-frontend event streams.
  • OpenTelemetry/OpenInference for observability interoperability.
  • JSON Schema and typed contracts for model/tool outputs.
  • OpenAPI/GraphQL/REST for conventional application integrations.

Common mistakes: using agent protocols where normal APIs suffice; assuming standards solve identity, transactions, or business authorization; creating too many protocol adapters.

Technology selection playbook

A. Questions to answer before choosing a technology

  1. What business task and quality threshold must it satisfy?
  2. Is the workflow deterministic, agentic, or hybrid?
  3. Is the stack primarily TypeScript, Python, or mixed?
  4. Does the system need persistence, checkpointing, or durable execution?
  5. Are there tenant-level ACL and data residency constraints?
  6. What are p95 latency, throughput, cost, and availability targets?
  7. Is self-hosting required?
  8. Can the technology be replaced without rewriting business logic?
  9. What evaluation dataset demonstrates that it helps?
  10. What is the operational burden, license, maintenance and vendor lock-in?

B. Example decision matrix

Score candidates 1–5 on each criterion, with weights adjusted for the application:

Criterion Example weight
Functional fit 25%
Security / compliance 20%
Reliability / operations 15%
Integration / interoperability 15%
Developer productivity 10%
Total cost 10%
Community / maintenance 5%

Never use the weighted score as the sole decision criterion: security or compliance can be a hard veto.

C. Reference stack: TypeScript enterprise AI platform

Layer Candidate starting point Selection note
Frontend React + assistant-ui or CopilotKit Choose based on required embedded copilot UX
Agent/UI events AG-UI where supported Avoid protocol if simple streaming suffices
Agent runtime Mastra or LangGraph.js Prototype both against a real workflow
Model gateway LiteLLM Centralize providers, budgets and telemetry
Durable workflow Temporal if needed Only for long-running/reliable jobs
Knowledge LlamaIndex or custom RAG service Reuse existing vector service where possible
Vector store Qdrant or existing VectorDB Verify ACL, filter and latency requirements
Graph store Neo4j when graph queries justify it Not mandatory for basic RAG
Transactional data PostgreSQL Authoritative business records
Memory Mem0/Graphiti or custom Add only after concrete memory use cases
Integrations MCP plus direct REST APIs Security enforced in tool adapters
Evaluation DeepEval, Ragas, Promptfoo Offline regression + adversarial tests
Observability Langfuse + OpenTelemetry Trace retrieval, model and tool steps
Security RBAC/ABAC + guardrails Backend policy is authoritative
Sandbox E2B / isolated containers Only for untrusted execution

D. Phased adoption roadmap

Phase 1 — Foundation: model gateway, tracing, structured output, basic RAG, permission-aware retrieval, evaluation baseline.

Phase 2 — Reliable workflows: stateful agent runtime, tool contracts, approvals, regression testing, cost budgets.

Phase 3 — Advanced context: reranking, hybrid retrieval, GraphRAG where justified, persistent memory with lifecycle controls.

Phase 4 — Scale: model routing, self-hosted inference if economical, online evaluation, canary releases, red teaming, governance automation.

E. Suggested proof-of-concept experiments

Experiment Compare Success measures
AI chat UI assistant-ui vs CopilotKit Integration effort, streaming UX, action safety
Agent runtime Mastra vs LangGraph.js Recovery, maintainability, tool orchestration
RAG Vector-only vs hybrid vs graph-enhanced Recall, faithfulness, latency, cost
Evaluation DeepEval vs Ragas for shared dataset Metric relevance, judge consistency, runtime cost
Model gateway Direct APIs vs LiteLLM Routing, error handling, overhead, visibility
Memory No persistent memory vs Mem0/Graphiti Task success, stale memory, privacy risk
Database Qdrant + Neo4j vs multimodel DB Query complexity, correctness, operations

F. Practical production readiness checklist

  • User identity and authorization enforced server-side.
  • Source documents and retrieval results filtered by ACL.
  • Model and tool inputs/outputs validated against schemas.
  • Sensitive actions require appropriate approvals.
  • Model, prompt, index, and evaluation dataset versions recorded.
  • Agent loops, retries, timeouts and budgets bounded.
  • External side effects are idempotent or compensatable.
  • Retrieval, tools, models and workflow transitions traced.
  • PII/secrets redacted from telemetry.
  • Representative offline and adversarial evaluation suites run in CI.
  • Production quality, latency, spend and failures monitored.
  • Rollback and disaster recovery procedures tested.
  • Memory retention and deletion policies implemented where relevant.
  • License, vendor risk and maintenance posture reviewed.

Technology radar: initial research priorities

P0 — Study first: Vercel AI SDK, assistant-ui, CopilotKit/AG-UI, Mastra, LangGraph, LiteLLM, Langfuse, DeepEval, Ragas, Qdrant, Neo4j, LlamaIndex.

P1 — Study next: Promptfoo, Mem0, Graphiti, Temporal, DSPy, Docling, SurrealDB, E2B, vLLM, SGLang, NeMo Guardrails.

P2 — Explore as needed: A2UI/MCP Apps, multi-agent specialist frameworks, advanced fine-tuning, distributed inference optimization, specialized graph memory systems.

These are editorial learning priorities, not validated market-share, release-date, or star-growth rankings.

Primary references and ongoing research

Further research to maintain this document

For each shortlisted technology, periodically verify: repository activity, 6-month GitHub star delta, release frequency, contributor concentration, security advisories, license, API stability, reference architectures, production case studies, benchmarks on your own workloads, and migration/exit costs. Do not infer production readiness from star count alone.