Framework Architecture

Understanding LlamaIndex.TS monorepo structure, package organization, and design patterns

LlamaIndex.TS is built as a modular TypeScript framework organized as a pnpm monorepo. This architecture enables runtime-agnostic functionality, provider flexibility, and optimal bundle sizes.

Monorepo Structure

The framework is organized into three main package categories:

Core Packages

Abstract base classes and runtime-agnostic functionality

Main Package

Aggregates core functionality with concrete implementations

Provider Packages

Specific integrations for LLMs, embeddings, and vector stores

Core Packages

packages/core

@llamaindex/core       # Abstract base classes and interfaces
@llamaindex/env        # Runtime environment abstraction
@llamaindex/node-parser # Text chunking and parsing
@llamaindex/workflow   # Agent workflow orchestration

The @llamaindex/core package exports functionality as modular sub-modules for optimal tree-shaking:

import { BaseLLM } from "@llamaindex/core/llms";
import { BaseEmbedding } from "@llamaindex/core/embeddings";
import { BaseNode } from "@llamaindex/core/schema";
import { Settings } from "@llamaindex/core/global";
import { BaseVectorStore } from "@llamaindex/core/vector-store";
import { BaseRetriever } from "@llamaindex/core/retriever";

Each sub-module can be imported independently, reducing bundle size by only including what you use.

Main Package

The llamaindex package is the primary entry point that:

• Re-exports core abstractions from @llamaindex/core
• Provides concrete implementations (indices, engines, pipelines)
• Supports multiple runtime entry points
• Aggregates common functionality for convenience

package.json exports

{
  "exports": {
    ".": {
      "workerd": "./dist/index.workerd.js",      // Cloudflare Workers
      "edge-light": "./dist/index.edge.js",      // Vercel Edge
      "react-server": "./dist/index.react-server.js", // RSC
      "import": "./dist/index.js",                // Node.js ESM
      "require": "./dist/cjs/index.cjs"           // Node.js CJS
    },
    "./engines": "./engines/dist/index.js",
    "./indices": "./indices/dist/index.js",
    "./ingestion": "./ingestion/dist/index.js"
    // ... more sub-modules
  }
}

Provider Packages

Providers are separate packages under packages/providers/ for modularity:

LLM Providers

• @llamaindex/openai - OpenAI GPT models
• @llamaindex/anthropic - Anthropic Claude
• @llamaindex/ollama - Local Ollama models
• @llamaindex/google - Google Gemini
• @llamaindex/groq - Groq inference
• @llamaindex/mistral - Mistral AI
• And many more…

Vector Store Providers

• @llamaindex/pinecone - Pinecone vector database
• @llamaindex/qdrant - Qdrant vector store
• @llamaindex/chroma - ChromaDB
• @llamaindex/weaviate - Weaviate
• @llamaindex/postgres - PostgreSQL with pgvector

Reader Providers

• @llamaindex/notion - Notion workspace reader
• @llamaindex/discord - Discord message reader
• @llamaindex/assemblyai - Audio transcription

Abstract Base Classes and Provider Pattern

LlamaIndex.TS uses abstract base classes to define interfaces that providers implement:

LLM Provider Pattern

From @llamaindex/core/llms

abstract class BaseLLM {
  abstract chat(params: ChatParams): Promise<ChatResponse>;
  abstract complete(params: CompleteParams): Promise<CompletionResponse>;
  abstract metadata: LLMMetadata;
}

Providers extend these base classes:

Provider implementation

import { BaseLLM } from "@llamaindex/core/llms";

export class OpenAI extends BaseLLM {
  async chat(params: ChatParams): Promise<ChatResponse> {
    // OpenAI-specific implementation
  }
  
  async complete(params: CompleteParams): Promise<CompletionResponse> {
    // OpenAI-specific implementation
  }
  
  get metadata(): LLMMetadata {
    return {
      model: "gpt-4",
      temperature: 0.7,
      // ...
    };
  }
}

Embedding Provider Pattern

From @llamaindex/core/embeddings

abstract class BaseEmbedding {
  abstract getTextEmbedding(text: string): Promise<number[]>;
  abstract getQueryEmbedding(query: string): Promise<number[]>;
}

Vector Store Provider Pattern

From @llamaindex/core/vector-store

abstract class BaseVectorStore {
  abstract add(nodes: BaseNode[]): Promise<string[]>;
  abstract delete(refDocId: string): Promise<void>;
  abstract query(params: VectorStoreQueryParams): Promise<VectorStoreQueryResult>;
  
  storesText: boolean = false;
  embedModel?: BaseEmbedding;
}

This pattern allows you to swap providers without changing application code - just change the Settings configuration.

How Packages Work Together

Example: Full Stack

Real-world usage

import { Settings } from "llamaindex";
import { OpenAI } from "@llamaindex/openai";
import { OpenAIEmbedding } from "@llamaindex/openai";
import { PineconeVectorStore } from "@llamaindex/pinecone";
import { VectorStoreIndex } from "llamaindex/indices";

// Configure providers via Settings
Settings.llm = new OpenAI({ model: "gpt-4", apiKey: process.env.OPENAI_API_KEY });
Settings.embedModel = new OpenAIEmbedding({ apiKey: process.env.OPENAI_API_KEY });

// Use concrete implementations
const vectorStore = new PineconeVectorStore({
  apiKey: process.env.PINECONE_API_KEY,
  indexName: "my-index",
});

// Index uses Settings and vectorStore together
const index = await VectorStoreIndex.fromDocuments(documents, {
  vectorStores: { TEXT: vectorStore },
});

Package Dependencies

The dependency flow ensures runtime compatibility:

1. Core Package

   Depends only on @llamaindex/env for runtime abstraction

2. Main Package

   Depends on @llamaindex/core and @llamaindex/env

3. Provider Packages

   Depend on @llamaindex/core for base classes and types

Package dependency example

// @llamaindex/openai/package.json
{
  "dependencies": {
    "@llamaindex/core": "workspace:*",
    "@llamaindex/env": "workspace:*",
    "openai": "^4.0.0"
  }
}

Modular Design Benefits

Tree-Shaking

Only bundle what you import - sub-module exports enable optimal bundle sizes

Provider Flexibility

Swap LLMs, embeddings, or vector stores with minimal code changes

Runtime Agnostic

Core abstractions work across Node.js, Deno, Bun, and edge runtimes

Independent Releases

Provider packages can be updated independently from core

Build System

All packages use bunchee for building with dual CJS/ESM support:

Build output structure

packages/core/
├── agent/dist/
│   ├── index.js      # ESM
│   ├── index.cjs     # CommonJS
│   └── index.d.ts    # Types
├── llms/dist/
├── embeddings/dist/
└── ...

Always run pnpm build before testing, as tests depend on build artifacts.

Next Steps

Multi-Runtime Support

Learn how LlamaIndex.TS works across different JavaScript runtimes

Settings Configuration

Configure global LLM, embeddings, and other settings