Multi-Runtime Support

Use LlamaIndex.TS across Node.js, Deno, Bun, Cloudflare Workers, and Vercel Edge Runtime

LlamaIndex.TS is designed to work seamlessly across all major JavaScript runtimes. This is achieved through runtime-specific entry points and the @llamaindex/env abstraction layer.

Supported Runtimes

Node.js

Full-featured support with filesystem access

Deno

Native ESM with secure permissions

Bun

Fast runtime with Node.js compatibility

Cloudflare Workers

Edge runtime with Workers API

Vercel Edge

Vercel Edge Runtime for serverless

Browser

Limited support for client-side use cases

Runtime-Specific Entry Points

The llamaindex package provides different entry points that are automatically selected based on your runtime:

package.json conditional exports

{
  "exports": {
    ".": {
      "react-server": {
        "types": "./dist/type/index.react-server.d.ts",
        "default": "./dist/index.react-server.js"
      },
      "workerd": {
        "types": "./dist/type/index.workerd.d.ts",
        "default": "./dist/index.workerd.js"
      },
      "edge-light": {
        "types": "./dist/type/index.edge.d.ts",
        "default": "./dist/index.edge.js"
      },
      "import": {
        "types": "./dist/type/index.d.ts",
        "default": "./dist/index.js"
      },
      "require": {
        "types": "./dist/type/index.d.cts",
        "default": "./dist/cjs/index.cjs"
      }
    }
  }
}

The correct entry point is automatically selected by your bundler or runtime based on export conditions.

Entry Point Mapping

Runtime	Export Condition	Entry Point File
Node.js ESM	import	src/index.ts
Node.js CJS	require	src/index.ts (compiled to CJS)
Cloudflare Workers	workerd	src/index.workerd.ts
Vercel Edge	edge-light	src/index.edge.ts
React Server Components	react-server	src/index.react-server.ts
Deno/Bun	import	src/index.ts

Environment Abstraction Layer

The @llamaindex/env package provides a unified API across all runtimes:

@llamaindex/env exports

// Node.js runtime (index.ts)
export { fs, path, createWriteStream, randomUUID, createHash } from 'node:*';
export { AsyncLocalStorage } from './als/index.node.js';

// Cloudflare Workers runtime (index.workerd.ts)
export { AsyncLocalStorage } from './als/index.workerd.js';
export { getEnv } from './utils/index.js';
// No filesystem exports

// Edge Runtime (index.edge-light.ts)
export { AsyncLocalStorage } from './als/index.non-node.js';
export * from './node-polyfill.js';  // Polyfills for Node APIs

@llamaindex/env handles all runtime differences internally, so you can write runtime-agnostic code.

Runtime-Specific Examples

Node.js

Full Features

Node.js provides the complete LlamaIndex.TS feature set:

Node.js with filesystem

import { VectorStoreIndex, SimpleDirectoryReader, Settings } from "llamaindex";
import { OpenAI } from "@llamaindex/openai";

Settings.llm = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });

// File system access available
const reader = new SimpleDirectoryReader();
const documents = await reader.loadData({ directoryPath: "./docs" });

// Full index functionality
const index = await VectorStoreIndex.fromDocuments(documents);
const queryEngine = index.asQueryEngine();

const response = await queryEngine.query({
  query: "What is LlamaIndex?"
});

Runtime Check

Verify Node.js runtime

import { process } from "@llamaindex/env";

if (typeof process !== "undefined" && process.versions?.node) {
  console.log("Running in Node.js", process.versions.node);
}

Cloudflare Workers

Worker Implementation

src/index.ts

// Cloudflare Workers automatically use index.workerd.ts entry point
interface Env {
  OPENAI_API_KEY: string;
}

export default {
  async fetch(request: Request, env: Env, ctx: ExecutionContext): Promise<Response> {
    // Set environment variables from Workers env
    const { setEnvs } = await import("@llamaindex/env");
    setEnvs(env);
    
    // Dynamic imports for optimal bundle size
    const { OpenAI, OpenAIAgent } = await import("@llamaindex/openai");
    
    const text = await request.text();
    
    const agent = new OpenAIAgent({
      llm: new OpenAI({ apiKey: env.OPENAI_API_KEY }),
      tools: [],
    });
    
    const responseStream = await agent.chat({
      stream: true,
      message: text,
    });
    
    // Return streaming response
    const textEncoder = new TextEncoder();
    const stream = responseStream.pipeThrough(
      new TransformStream({
        transform: (chunk, controller) => {
          controller.enqueue(textEncoder.encode(chunk.delta));
        },
      })
    );
    
    return new Response(stream);
  },
};

wrangler.toml

Cloudflare Workers config

name = "llamaindex-agent"
main = "src/index.ts"
compatibility_date = "2024-04-23"
compatibility_flags = ["nodejs_compat"]

[vars]
# Non-sensitive environment variables

# For secrets, use: wrangler secret put OPENAI_API_KEY

Cloudflare Workers has no filesystem access. Use in-memory data or external storage services.

Vercel Edge Runtime

Next.js Route Handler

app/api/chat/route.ts

import { OpenAI } from "@llamaindex/openai";
import { Settings } from "llamaindex";

// Force Edge Runtime
export const runtime = "edge";

Settings.llm = new OpenAI({
  apiKey: process.env.OPENAI_API_KEY,
});

export async function POST(request: Request) {
  const { message } = await request.json();
  
  const response = await Settings.llm.chat({
    messages: [{ role: "user", content: message }],
  });
  
  return Response.json(response);
}

Runtime Detection

utils/llm.ts

import "llamaindex";

// Verify Edge Runtime
// @ts-expect-error EdgeRuntime is not defined
if (typeof EdgeRuntime !== "string") {
  throw new Error("Expected to run in EdgeRuntime");
}

next.config.mjs

next.config.mjs

import { withLlamaIndex } from "llamaindex/next";

/** @type {import('next').NextConfig} */
const nextConfig = {
  // Your Next.js config
};

export default withLlamaIndex(nextConfig);

Deno

Deno with import maps

import { VectorStoreIndex, Document, Settings } from "npm:llamaindex";
import { OpenAI } from "npm:@llamaindex/openai";

Settings.llm = new OpenAI({ apiKey: Deno.env.get("OPENAI_API_KEY") });

const documents = [
  new Document({ text: "LlamaIndex is a data framework for LLMs" }),
];

const index = await VectorStoreIndex.fromDocuments(documents);
const queryEngine = index.asQueryEngine();

const response = await queryEngine.query({
  query: "What is LlamaIndex?"
});

console.log(response.toString());

Bun

Bun runtime

import { VectorStoreIndex, Settings } from "llamaindex";
import { OpenAI } from "@llamaindex/openai";

Settings.llm = new OpenAI({ apiKey: Bun.env.OPENAI_API_KEY });

// Bun has fast file I/O
const file = Bun.file("./document.txt");
const text = await file.text();

const documents = [new Document({ text })];
const index = await VectorStoreIndex.fromDocuments(documents);

Runtime Capabilities

Feature	Node.js	Deno	Bun	CF Workers	Vercel Edge	Browser
File System	✅	✅	✅	❌	❌	❌
Vector Indices	✅	✅	✅	✅	✅	⚠️
LLM Chat	✅	✅	✅	✅	✅	✅
Streaming	✅	✅	✅	✅	✅	✅
Embeddings	✅	✅	✅	✅	✅	⚠️
Agents	✅	✅	✅	✅	✅	⚠️
Async Local Storage	✅	✅	✅	✅	✅	❌
Local Models	✅	✅	✅	❌	❌	⚠️

⚠️ = Limited support or requires specific configuration

Environment Variables

Node.js / Deno / Bun

.env

OPENAI_API_KEY=sk-...
PINECONE_API_KEY=...

Load with dotenv

import "dotenv/config";
import { Settings } from "llamaindex";
import { OpenAI } from "@llamaindex/openai";

Settings.llm = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });

Cloudflare Workers

Use setEnvs helper

interface Env {
  OPENAI_API_KEY: string;
}

export default {
  async fetch(request: Request, env: Env) {
    const { setEnvs } = await import("@llamaindex/env");
    setEnvs(env);  // Makes env.* available via getEnv()
    
    // Now use environment variables
    const { OpenAI } = await import("@llamaindex/openai");
    const llm = new OpenAI({ apiKey: env.OPENAI_API_KEY });
  }
};

Vercel Edge Runtime

Edge Runtime env

// Environment variables available via process.env
export const runtime = "edge";

Settings.llm = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });

Runtime Limitations

Cloudflare Workers

Limitations:

• No file system access (use KV, R2, or D1 for storage)
• No native Node.js modules
• 1MB script size limit (use dynamic imports)
• CPU time limits per request

Workarounds:

• Use vectorStores with external services (Pinecone, Qdrant)
• Dynamic imports to reduce initial bundle size
• Stream responses for long-running operations

Vercel Edge Runtime

Limitations:

• No file system access
• Limited Node.js API compatibility
• 1MB middleware size limit
• Cold start considerations

Best Practices:

• Use withLlamaIndex Next.js plugin
• Leverage Vercel KV or external vector stores
• Keep dependencies minimal

Browser

Limitations:

• No file system access
• CORS restrictions for API calls
• Bundle size concerns
• No native crypto APIs in older browsers

Use Cases:

• Client-side chat interfaces
• Browser-based document processing with FileReader API
• WebAssembly embeddings (limited)

AsyncLocalStorage

LlamaIndex.TS uses AsyncLocalStorage for context-aware Settings:

Settings context isolation

import { Settings } from "llamaindex";
import { OpenAI, Anthropic } from "@llamaindex/openai";

// Different LLMs in different contexts
Settings.withLLM(new OpenAI({ model: "gpt-4" }), async () => {
  // This code uses GPT-4
  const response = await Settings.llm.chat({ messages });
});

Settings.withLLM(new Anthropic({ model: "claude-3" }), async () => {
  // This code uses Claude-3
  const response = await Settings.llm.chat({ messages });
});

AsyncLocalStorage is polyfilled for runtimes that don’t have native support.

Best Practices

Use Dynamic Imports

In edge runtimes, use dynamic imports to reduce bundle size:

const { OpenAI } = await import("@llamaindex/openai");

Test Multiple Runtimes

Validate your code works across target runtimes in CI/CD

Handle Missing APIs

Check for API availability before using runtime-specific features

Optimize Bundle Size

Use sub-module imports and tree-shaking for edge deployments

Next Steps

Settings Configuration

Learn how to configure global settings for your runtime

Data Flow

Understand how data flows through the framework