MALV: A Better Way to Build AI Products

System Architecture

Building with MALV means working with three distinct architectural layers: the client layer providing user interfaces, the application layer containing domain-specific business logic, and the infrastructure layer providing shared services for security, storage, and deployment. This separation enables independent scaling and deployment of components while maintaining system-wide consistency through well-defined interfaces.

Layer Responsibilities

Client Layer

The client layer provides user interfaces for interacting with the system. Two primary implementations exist: a web-based client built with Vite and served as a static single-page application, and a command-line interface for terminal-based interaction. Both clients communicate exclusively with the orchestrator application and load application metadata from the Apps CDN.

Application Layer

Applications are independent Cloudflare Workers implementing specific business capabilities. Each application exposes a set of tools (discrete functions) that can be invoked by the orchestrator. Applications are stateless; all persistent state is managed through the storage service. The orchestrator application holds special status as the coordination point for AI-powered planning and execution.

Infrastructure Layer

Infrastructure services provide shared capabilities required by all applications. The token service handles cryptographic signing, the storage service enforces permissions and proxies R2 operations, the event service coordinates publish/subscribe messaging, the hub service manages deployment, and the Apps CDN distributes application assets.

Technology Stack

End-to-End Request Flow

This section illustrates a complete request lifecycle, from user query through semantic filtering, token-aware planning, tool execution, and response streaming. The flow demonstrates how the architecture's components coordinate to deliver sub-second response initiation while processing complex multi-step operations.

Performance Breakdown

Cryptographic Security Model

The security architecture employs Ed25519 public-key cryptography for token signing and verification. Applications authenticate requests using signed JWT tokens that embed permission grants. A three-phase validation system ensures that storage operations are authorized before execution. Automatic key rotation occurs every 30 days, maintaining a rolling window of valid keys for token verification.

Ed25519 Cryptographic Primitives

Ed25519 was selected for its performance characteristics and security properties. Signatures are generated in under 1ms on edge hardware, and verification completes in under 0.5ms. The algorithm provides 128-bit security with 32-byte public keys and 64-byte signatures, significantly smaller than RSA equivalents.

Three-Phase Permission Validation

Storage access requests undergo validation in three sequential phases, with each phase providing progressively granular control:

Phase 1: Application Secret Validation

Applications presenting a valid application secret are granted full access to their own storage namespace. This enables applications to manage their internal state without requiring per-operation token signing.

Phase 2: Embedded Token Permissions

Tokens may embed explicit permission grants as arrays of path patterns. The storage service checks whether the requested operation path matches any embedded permission. This phase enables cross-application access with explicit grants.

Phase 3: Declarative Schema Validation

If embedded permissions are not present, the storage service loads the token schema from the application's published configuration and validates the operation against declaratively specified permissions. This provides a fallback validation mechanism and enables permission updates without reissuing tokens.

Key Rotation Protocol

New Ed25519 keypairs are generated automatically every 30 days. The token service maintains four active private keys and publishes five public keys (current plus four historical). This overlap window ensures that tokens signed immediately before rotation remain valid during their lifetime. Old keys are archived to R2 but removed from active use.

Event-Driven Communication

The event system enables asynchronous, loosely-coupled communication between applications through a publish/subscribe pattern. Applications declare events they can emit, and other applications subscribe to receive those events. The event service manages subscription lifecycle, coordinates webhook setup and teardown, and delivers events to subscribers in parallel.

Event Handlers

Event source applications implement three handler functions for each event type:

start.ts

Invoked when the first subscriber registers interest in an event. Typically configures webhooks with external services and stores listener state. Receives a unique key identifying the subscription group and token payloads for authentication with external services.

handler.ts

Invoked by external webhooks when events occur. Processes the webhook payload, fetches additional data if needed, and calls the event service to deliver events to subscribers. May be invoked repeatedly for a single subscription.

stop.ts

Invoked when the last subscriber unsubscribes. Responsible for cleaning up webhooks and removing listener state. Receives the subscription key for identifying which listener to clean up.

Key-Based Multi-Tenancy

Subscriptions are grouped by deterministic keys generated from token payloads. For example, a Gmail event subscription might generate a key from the Google user ID in the authentication token. This ensures that each user's subscription is independent, enabling per-user webhook configuration and isolated event streams.

Idempotency Guarantees

The event service implements idempotent subscription operations. Multiple subscription requests with identical parameters result in a single stored subscription. The start handler is invoked exactly once for each unique key, even if multiple subscribers register simultaneously. This property simplifies subscription logic and prevents resource leaks.

Deployment Infrastructure

The hub service provides a centralized deployment gateway that eliminates the need for application developers to possess Cloudflare credentials or understand deployment mechanics. Applications are packaged as multipart form uploads containing compiled JavaScript, configuration files, and assets. The hub service processes these uploads, generates semantic embeddings for tool discovery, and orchestrates deployment to Cloudflare's edge network.

Multipart Upload Format

Applications are packaged as multipart/form-data with the following components:

• metadata: JSON object containing app ID, version, domain, and description
• worker_script: Compiled JavaScript bundle for Worker execution
• tools.json: Tool definitions with schemas and capability requirements
• tokens.json: Token type definitions with permission patterns
• events.json: Event definitions (optional)
• examples/*.json: Usage examples for semantic discovery (optional)
• Renderer modules: JavaScript modules for custom object visualization (optional)

Embedding Generation

The hub service generates 512-dimensional embeddings using OpenAI's text-embedding-3-small model for three categories of content:

1. Application Description: A single vector representing the application's overall purpose
2. Tool Descriptions: One vector per tool combining the tool name and description
3. Usage Examples: One vector per example combining the user query and execution plan

These embeddings enable the orchestrator to perform semantic similarity search during tool discovery, filtering irrelevant capabilities and reducing context size by approximately 90%.

Cloudflare API Integration

The hub service communicates with Cloudflare's Workers API to deploy applications. This includes uploading the compiled script, configuring environment bindings (R2 buckets, KV namespaces, secrets), and creating routes if custom domains are specified. The hub maintains Cloudflare API credentials, insulating application developers from infrastructure complexity.

AI Orchestration and Tool Discovery

The orchestrator application coordinates AI-powered planning and execution. When a user submits a query, the orchestrator performs semantic filtering to identify relevant tools, analyzes token requirements to determine tool availability, uses a two-phase decision process for efficient tool selection and input generation, executes the resulting plan, and streams updates to the client in real-time.

Semantic Tool Filtering

Tool discovery employs cosine similarity between the embedded user query and the embedded tool descriptions. Tools with similarity scores below 0.3 are excluded from the prompt. This filtering typically removes 80-90% of available tools, substantially reducing prompt size and associated API costs while maintaining high recall for relevant capabilities.

The filtering algorithm also considers application-level embeddings and usage example embeddings. If an application's description is semantically relevant, all of its tools receive a score boost. Similarly, if a usage example matches the query, the tools referenced in that example are prioritized.

Two-Phase Tool Decision

Rather than asking the AI to simultaneously select tools and generate their inputs, MALV separates these concerns into two phases for improved accuracy and cost efficiency:

Phase 1: Tool Selection

The AI receives a simplified prompt showing tool descriptions without detailed input schemas. This focused context allows the AI to match user intent to tool capabilities without distraction from schema complexity. The result is a list of selected tools (app + name pairs) and a "next step" observation providing strategic direction.

Phase 2: Input Generation

Selected tools are categorized by schema complexity. Tools with complex schemas ($defs, $ref, anyOf) receive individual AI calls executed in parallel, ensuring focused attention on intricate type structures. Simple tools are batched into a single call for efficiency. Full JSON schemas are provided only in this phase.

Complexity-Aware Batching

Schema complexity is scored based on structural features. Tools scoring above the threshold (e.g., those with recursive types or union schemas) are isolated to prevent malformed inputs:

Token-Aware Tool Presentation

Tools declare required tokens in their definitions. The orchestrator analyzes available tokens (provided by the client) and categorizes tools as available or unavailable. Available tools are presented normally in the AI prompt. Unavailable tools are presented with instructions on how to unlock them, typically by invoking an authentication tool. This enables the AI to automatically guide users through authentication flows when necessary.

Streaming Execution

Tool execution results stream to the client using Server-Sent Events (SSE). The orchestrator emits events for: tool decisions (when the AI selects tools to invoke), tool execution start/end (with streaming logs), token creation (when tools generate new authentication tokens), and final response generation. This provides real-time visibility into system behavior and improves perceived performance.

Cost Optimization

The combination of semantic filtering and two-phase planning compounds cost savings:

• Semantic filtering: 90% reduction in tools presented (20,000 → 2,000 tokens)
• Phase 1 simplification: No schemas in selection phase (~50% further reduction)
• Batched simple tools: Single call for multiple simple inputs
• Parallel complex calls: Latency reduction through concurrent execution

At $3 per million input tokens (Claude Sonnet 4.5 pricing), these optimizations reduce per-request cost from ~$0.06 (naive approach) to ~$0.002, representing a 30x improvement.

Object Rendering System

Unlike traditional AI approaches that limit outputs to text responses, MALV implements a sophisticated object rendering system for rich data visualization. Objects are persistent, renderable data entities created by tools and stored in R2. Each object can be visualized through custom renderers with full lifecycle management, enabling interactive dashboards, data tables, research boards, and other rich UI components.

Object Architecture

Objects follow a reference-based architecture where metadata stores pointers rather than actual data. When a tool creates a table object, it stores a tableId reference in the object metadata. The renderer then uses this reference to fetch the actual data through tool calls. This separation enables:

• Efficient Storage: Object metadata remains lightweight regardless of data size
• Live Data: Renderers always fetch current data, not stale snapshots
• Cross-App Access: Objects can reference data stored in other applications

Custom Renderers

Each object type can define custom renderers for different environments. Web renderers are ES modules loaded dynamically from the Apps CDN. CLI renderers produce formatted terminal output. Renderers receive structured parameters including object info, capabilities, and lifecycle hooks.

// objects/{type}/web.ts - Web Renderer Signature
export default async function web(
  info: { id: string; name: string; metadata: ObjectMetadata },
  capabilities: { callTool, storage, ai },
  lifecycle: { onDataUpdated, onUnmount }
): Promise<HTMLElement>

Lifecycle Hooks

Renderers can subscribe to lifecycle events for reactive updates:

onDataUpdated(callback)

Invoked when the underlying object data changes, enabling live updates without page refresh. The callback receives the new metadata, allowing renderers to re-fetch and re-render efficiently.

onUnmount(callback)

Invoked when the object tab is closed or the user navigates away. Enables cleanup of subscriptions, WebSocket connections, or other resources held by the renderer.

Cross-App Object Storage

Objects can be stored in applications other than the one that created them, enabling team-wide sharing. The storage configuration in objects.json specifies the target app and path template:

{
  "storage": {
    "inApp": "@malv/auth",
    "path": "/teams/<token.team>/objects/",
    "tokenType": "account",
    "tokenFromApp": "@malv/auth"
  }
}

This configuration stores objects under the team's namespace in the auth app, making them accessible to all team members regardless of which app created them.

Perceptions System

Traditional AI assistants react to explicit user requests. MALV's perception system enables proactive intelligence by defining contextual conditions that help the AI understand user state and suggest relevant actions. Apps declare "perceptions" that match token presence and storage values, surfacing suggested tasks when conditions are met.

Perception Definition

Perceptions are defined in perception/*.json files within each app. Each perception specifies conditions that must be met and the contextual insight to provide when matched:

{
  "tokens": {
    "exists": { "@malv/inventory": "warehouse" },
    "absent": { "@malv/inventory": "product" }
  },
  "storage": {
    "@malv/inventory": {
      "/teams/<warehouse.teamId>/warehouses/<warehouse.warehouseId>/config.json": {
        "lowStockThreshold": { "operator": "equals", "value": null }
      }
    }
  },
  "perception": "User has a warehouse configured but hasn't set the low stock threshold",
  "tasks": ["Help user define low stock threshold", "Suggest reorder policies"]
}

Condition Types

Two types of conditions enable precise state matching:

Token Conditions

Check for token presence (exists) or absence (absent). Token conditions are fast to evaluate as they only require checking the client-provided token list. Use these to gate perceptions by authentication state or resource selection.

Storage Conditions

Query actual data values in storage. Paths support template substitution using token payload fields (e.g., <warehouse.teamId>). Storage conditions enable state-aware perceptions like "warehouse exists but threshold is not set."

Storage Operators

Storage conditions support multiple comparison operators for flexible matching:

Proactive Suggestions

When perceptions match, their suggested tasks are presented to the AI as actionable next steps. This transforms the assistant from purely reactive to contextually proactive:

Semantic Storage Search

Beyond semantic tool discovery, MALV extends vector-based search to storage data itself. When applications write to storage, embeddings are automatically generated in the background, enabling AI-powered discovery of relevant data across all applications. This transforms storage from a simple key-value system into an intelligent data layer.

Automatic Embedding Generation

When apps configure storage paths for searchability, the infrastructure automatically processes writes through an embedding pipeline:

1. Write Interception: Storage service detects writes to searchable paths
2. Content Extraction: Relevant text content extracted from JSON data
3. Embedding Generation: HuggingFace model (bge-base-en-v1.5) creates 768-dimensional vectors
4. Index Storage: Embeddings stored with path and security metadata

This process runs asynchronously, ensuring write latency is not affected by embedding computation.

Security Key Filtering

Semantic search respects the same permission model as direct storage access. Each indexed item includes security keys derived from token payloads. Search requests specify which security keys the user holds, and results are filtered to include only accessible data:

// Search request with security keys
GET /search?q=project%20goals&types=storage&securityKeys=["account:user123","team:team456"]

// Only returns data where indexed securityKey matches one of the provided keys

Cross-App Data Discovery

Semantic storage search operates across application boundaries, enabling powerful cross-app queries. An inventory app can discover relevant data from an orders app, or a reporting tool can find metrics from multiple data sources:

Location-Based Filtering

Search queries can specify location constraints to scope results to specific apps or paths. This enables targeted searches within a project namespace or across a specific application's data:

// Search within a specific warehouse
GET /search?q=stock&types=storage&locations={"@malv/inventory":"/teams/team1/warehouses/wh1"}

// Search across all inventory data
GET /search?q=products&types=storage&locations={"@malv/inventory":"*"}

Search Response Format

Search results include the storage path, source application, similarity score, and a content preview. The AI or application can then fetch the full data using standard storage operations:

{
  "query": "low stock alerts",
  "types": ["storage"],
  "results": [
    {
      "type": "storage",
      "appName": "@malv/inventory",
      "path": "/teams/team1/warehouses/wh1/products/prod_001.json",
      "preview": "Widget A stock level below threshold, 12 units remaining...",
      "similarity": 0.89,
      "securityKey": "f8d2c9b1a5f3e7d9"
    }
  ]
}

// Storage path definition
path: ["teams", {teamId: {token: "account"}}, "warehouses", {warehouseId: {token: "current"}}]

// Actual path being written
/teams/team-123/warehouses/wh-456/data.json

// Extracted token requirements
[
  { app: "@malv/auth", tokenType: "account", fields: ["teamId"] },
  { app: "@malv/inventory", tokenType: "current", fields: ["warehouseId"] }
]

// Canonical key (sorted alphabetically)
"@malv/auth:account/team-123|@malv/inventory:current/wh-456"

// Final security key (16-char SHA-256 hash)
"f8d2c9b1a5f3e7d9"

// Concrete path from write operation
"/apps/@malv/inventory/private/teams/team-123/warehouses/west-coast/products/prod-1.json"

// Matched definition from storage.json
{
  "path": ["teams", {"teamId": {...}}, "warehouses", {"warehouseId": {...}}, "products", {"productId": {...}}, "data.json"],
  "description": "A product in the '' warehouse"
}

// Extracted path variables
{ teamId: "team-123", warehouseId: "west-coast", productId: "prod-1" }

// These variables are used for security key generation

// Input data
{
  "id": "row-1",
  "_metadata": { "created": "2024-01-01" },
  "name": "John Doe",
  "email": "john@example.com",
  "role": "Sales Manager"
}

// Extracted searchable text (with description prepended)
"A row in the 'customers' table John Doe john@example.com Sales Manager"

// Path definition
["teams", {team}, "warehouses", {id}, "products", "*"]

// Extract static (non-variable) segments
["teams", "warehouses", "products"]

// Create hash input
"@malv/inventory:teams/warehouses/products"

// Generate 4-character prefix hash
"a3Kx"

Message Summarization

Long conversations accumulate "behavioral gravity" - directive language, persuasive framing, and emotional tone that can bias AI responses when reflecting on history. MALV's background summarization system strips this gravity by converting messages into neutral, factual summaries that compress context while preserving essential information.

The Problem: Behavioral Gravity

When conversation history is passed to an AI, the language patterns in that history influence future responses. Phrases like "I really need," "this is critical," or "you should" create implicit pressure. Over long conversations, this accumulates, causing the AI to:

• Echo the user's emotional tone rather than remaining objective
• Over-commit to earlier suggestions rather than adapting to new information
• Treat opinions stated confidently as established facts
• Inherit biases from directive language in its own previous responses

Summarization Strategy

MALV applies different summarization strategies based on message role:

User Messages: Third-Person Past Tense

Converts user statements into observations about what the user wanted or did. This removes urgency and emotional framing while preserving intent. "I desperately need help with X" becomes "He wanted help with X."

Assistant Messages: First-Person Past Tense

Converts assistant statements into factual logs of actions taken. This removes persuasive language and hedging. "Want me to help you with X? I could also do Y..." becomes "I offered to help with X and mentioned Y as an option."

Background Processing

Summarization runs as a background job, queued after each response completes. This ensures summarization latency never impacts user-facing response time:

// Queued automatically after respond tool completes
await capabilities.tool.queue('@malv/orchestrator', 'summarize', {
  conversationId,
  assistantMessagePath
}, {
  maxBatchTimeout: 5000,  // Wait up to 5s for more items
  maxBatch: 5             // Process up to 5 messages together
});

Context Compression

Summaries enable efficient context compression for long conversations. When conversation history approaches token limits, the orchestrator can substitute summaries for older messages:

Storage Structure

Summaries are stored alongside original messages, enabling flexible retrieval:

/conversations/{id}/messages/
├── msg_001.json           # Original user message
├── msg_001-summary.json   # Third-person summary
├── msg_002.json           # Original assistant message
├── msg_002-summary.json   # First-person summary
└── ...

Technical Specifications

Performance Characteristics

Security Parameters

Scalability Limits

Embedding Configuration

Development Environment

TypeScript Version

5.x with strict mode enabled

Build Tools

Rollup 4.x for Workers, Vite 5.x for web clients

Runtime Environment

Node.js 20.x for development, V8 isolates for production

Package Manager

Yarn 4.x with workspace support

Testing Framework

Jest 29.x with TypeScript integration

Code Quality

ESLint 8.x, Prettier 3.x

Architectural Benefits

The MALV architecture provides significant technical benefits through its design choices. By handling infrastructure concerns at the architecture level, developers can focus on application logic while benefiting from optimizations that would otherwise require substantial engineering effort to implement.

Infrastructure Overhead Comparison

Key Technical Benefits

Architecture Comparison

The following table compares MALV's architectural choices against common alternatives: