Advanced Features

This page documents expert-level APIs for custom use cases. These features provide low-level access to vuer-rtc's internal data structures and operations.

RangeTree - B-tree Implementation

The RangeTree is a B-tree optimized for string CRDT operations. It stores items with O(log n) lookups by position and maintains character counts in each node.

When to Use

• Building custom text editors with specific performance requirements
• Implementing alternative text CRDT algorithms
• Debugging text CRDT behavior at the tree level
• Performance analysis and benchmarking

Structure

import { RangeTree, type Item, type ItemId } from '@vuer-ai/vuer-rtc';

const tree = new RangeTree();

// Item structure
interface Item {
  id: ItemId;           // Unique identifier { agent, seq }
  content: string;      // Text content
  isDeleted: boolean;   // Tombstone flag
  parentId: ItemId | null;  // Parent item for CRDT ordering
  seq: number;          // Sequence number
}

Key Operations

// Insert an item
tree.insert({
  id: { agent: 'alice', seq: 1 },
  content: 'Hello',
  isDeleted: false,
  parentId: null,
  seq: 1,
});

// Find by position (O(log n))
const result = tree.findByPosition(5);
// => { leaf: LeafNode, itemIndex: 2, offset: 1 }

// Find by ID (O(log m) where m = unique agents)
const item = tree.findById({ agent: 'alice', seq: 1 });

// Delete (tombstone)
tree.delete({ agent: 'alice', seq: 1 });

// Get full content
const text = tree.getContent();

// Get metadata
const length = tree.length;      // Total visible characters
const count = tree.itemCount;    // Total items (including deleted)

Pitfalls

• Manual count updates: After modifying items directly, you must call recalcAllCounts() to update character counts
• Memory overhead: Tombstones are never removed, only marked as deleted. Use compaction at the higher level.
• No automatic splitting: The tree splits leaf nodes automatically, but you cannot control the branching factor (fixed at 32)
• No parent pointers: Items don't maintain back-references, making parent-based traversal O(n)

createGraphFromServer() - Server Snapshot Initialization

Initialize a client graph store from a server-provided snapshot and journal history.

When to Use

• Client reconnection after disconnect
• Loading archived sessions
• Server-side rendering with hydration
• Multi-device sync on session join

Usage

import { createGraphFromServer } from '@vuer-ai/vuer-rtc';

const store = createGraphFromServer({
  sessionId: 'client-abc',
  snapshot: serverSnapshot,  // Server's current snapshot
  journal: serverJournal,    // Server's journal entries
  onSend: (msg) => websocket.send(JSON.stringify(msg)),
  onStateChange: (state) => console.log('State updated'),
});

// The store is now initialized with the server's state
// All journal entries are replayed in causal order

vs. Regular createGraph()

Pitfalls

• Clock skew: Server snapshot's vector clock must be consistent with the journal entries. Mismatched clocks cause duplicate operations.
• Journal ordering: Entries are sorted by (lamportTime, sessionId) before replay. Out-of-order server journals are handled automatically.
• Compaction loss: Journal entries older than the snapshot watermark are lost. Cannot undo operations that were compacted server-side.
• Memory spike: Replaying large journals (10k+ ops) can cause temporary memory pressure during initialization.

compactToWatermark() - Advanced Compaction

Compact the journal up to a specific vector clock watermark, rather than just compacting all acknowledged entries.

When to Use

• Selective compaction based on session visibility
• Multi-tenant systems where different clients see different subsets
• Memory-constrained environments needing fine-grained control
• Custom undo/redo windows (preserve recent history only)

Usage

import { VectorClockManager } from '@vuer-ai/vuer-rtc';

const clockManager = new VectorClockManager();

// Define a watermark (entries before this are compacted)
const watermark = {
  'alice': 10,  // Compact alice's ops up to seq 10
  'bob': 5,     // Compact bob's ops up to seq 5
};

store.compactToWatermark(watermark);

// After compaction:
// - Entries ≤ watermark are folded into snapshot
// - Entries > watermark remain in journal (can be undone)
// - Snapshot's vector clock is updated to watermark

vs. Regular compact()

Pitfalls

• Watermark must be monotonic: Compacting to a watermark older than the current snapshot is a no-op (silent)
• Partial compaction: If watermark only covers some sessions, others remain in journal (may grow unbounded)
• Lost causality: Operations referencing compacted entries (e.g., text CRDT parent IDs) must already be in the snapshot
• No validation: Invalid watermarks (e.g., { alice: -1 }) are accepted but have undefined behavior

initFromServer() - Server Initialization Patterns

Low-level action for initializing client state from server snapshot and journal. Used internally by createGraphFromServer().

When to Use

• Custom store initialization outside createGraph()
• Server-side state reconciliation
• Manual journal replay with custom filtering
• Testing and debugging CRDT behavior

Usage

import { initFromServer } from '@vuer-ai/vuer-rtc/client';

const state = initFromServer(
  'my-session-id',
  serverSnapshot,
  serverJournal
);

// Returns a ClientState with:
// - graph: rebuilt from snapshot + journal
// - journal: local copy of server journal
// - vectorClock: merged from all entries
// - snapshot: server snapshot as base

Internals

1. Clone snapshot graph: Deep-clones nodes, children arrays, and TextRope instances (avoids shared mutable state)
2. Sort journal: Orders entries by (lamportTime, sessionId) for causal replay
3. Rebuild graph: Applies each journal entry to the cloned graph
4. Merge clocks: Merges vector clocks from all journal entries
5. Return state: Returns a fresh ClientState object

Pitfalls

• No edit buffer: The returned state has an empty edit buffer. Pending local edits must be reapplied.
• Snapshot mutation: The input snapshot is cloned, but if you pass the same snapshot to multiple initFromServer() calls, ensure the server snapshot itself is immutable.
• Journal ownership: The returned state stores a copy of the journal array. Mutating the input array after initFromServer() won't affect the state.
• No deduplication: Duplicate journal entries (same id) are replayed multiple times. The server must deduplicate before sending.

EditBufferImpl - Low-Level Edit Buffer

The edit buffer accumulates uncommitted operations during gestures (e.g., dragging) and merges additive operations to reduce message size.

When to Use

• Custom gesture handling with operation merging
• Building higher-level transaction APIs
• Debugging edit buffer behavior
• Performance profiling of operation batching

Usage

import { EditBufferImpl, isAdditiveOp, mergeValues } from '@vuer-ai/vuer-rtc/client';

const buffer = new EditBufferImpl();

// Add operations (automatically merged if additive)
buffer.add({ otype: 'vector3.add', key: 'cube', path: 'position', value: [1, 0, 0] });
buffer.add({ otype: 'vector3.add', key: 'cube', path: 'position', value: [0, 1, 0] });
// => Merged to: { otype: 'vector3.add', key: 'cube', path: 'position', value: [1, 1, 0] }

// Get operations in order
const ops = buffer.getOps();

// Clear buffer
buffer.clear();

// Check status
const empty = buffer.isEmpty();
const count = buffer.size();

Operation Deduplication

Operations are deduplicated by a key computed from (otype, key, path, target):

// Property operations: key:path
'cube:position' // vector3.set on cube.position

// Node operations include target node
'scene:children:insert:cube-1'  // Insert cube-1 under scene
'scene:children:remove:cube-1'  // Remove cube-1 from scene

Additive Operations

These operations are merged (not replaced) when the same dedup key is used:

• vector3.add - Component-wise addition
• number.add - Numeric addition
• number.multiply - Numeric multiplication
• quaternion.multiply - Quaternion composition

Pitfalls

• LWW replacement: Non-additive ops (e.g., vector3.set) replace previous ops with the same dedup key, even if applied in different order
• No undo: The edit buffer itself has no undo. Use GraphStore.undo() which operates on the journal.
• Merge order matters: For quaternions, q1 * q2 ≠ q2 * q1. Merging preserves order, but concurrent quaternion ops may produce unexpected results.
• No validation: Invalid operations (e.g., vector3.add with a string value) are stored without validation

findItemById() - Text CRDT Item Lookup

Low-level API for looking up text CRDT items by ID in the TextRope structure.

When to Use

• Debugging text CRDT parent references
• Custom text CRDT algorithms
• Analyzing character-level causality
• Performance profiling of ID lookups

Usage

import { findItemById, type ItemId } from '@vuer-ai/vuer-rtc';

// Get the TextRope from a node property
const rope = graph.nodes['editor'].content;

// Find an item by ID
const item = rope.findById({ agent: 'alice', seq: 5 });

if (item) {
  console.log('Content:', item.content);
  console.log('Deleted:', item.isDeleted);
  console.log('Parent:', item.parentId);
}

Agent Index

TextRope maintains an agent-indexed structure for fast lookups:

• Span entries: Each agent's items are grouped into spans [startSeq, endSeq)
• Binary search: Finding an item by seq is O(log m) where m = number of spans for that agent
• Automatic updates: Inserting/deleting items updates the agent index

Pitfalls

• Deleted items: Deleted items (tombstones) are still returned. Check isDeleted before using.
• Parent references: parentId may reference an item that doesn't exist (out-of-order delivery). Handle null returns gracefully.
• No content lookup: There is no reverse lookup by content. Finding "where is the word 'hello'" requires iterating all items.
• Memory overhead: The agent index stores duplicate references to items. Large ropes with many agents consume more memory.

itemIdEquals() & itemIdCompare() - Item ID Utilities

Helper functions for comparing CRDT item IDs.

When to Use

• Custom CRDT implementations
• Sorting items by causal order
• Deduplicating item references
• Debugging parent-child relationships

Usage

import { itemIdEquals, itemIdCompare, type ItemId } from '@vuer-ai/vuer-rtc';

const id1: ItemId = { agent: 'alice', seq: 5 };
const id2: ItemId = { agent: 'alice', seq: 5 };
const id3: ItemId = { agent: 'bob', seq: 3 };

// Equality check
itemIdEquals(id1, id2);  // => true
itemIdEquals(id1, id3);  // => false
itemIdEquals(null, null); // => true
itemIdEquals(id1, null); // => false

// Comparison (for sorting)
itemIdCompare(id1, id2);  // => 0 (equal)
itemIdCompare(id1, id3);  // => -1 (alice < bob)
itemIdCompare(id3, id1);  // => 1 (bob > alice)

// Sort items by ID
items.sort((a, b) => itemIdCompare(a.id, b.id));

Ordering Rules

1. Agent first: Compare agent strings lexicographically
2. Sequence second: If agents match, compare seq numbers

This ensures deterministic ordering across clients regardless of arrival order.

Pitfalls

• No null in compare: itemIdCompare() does not handle null. Use itemIdEquals() for nullable IDs.
• String comparison: Agent IDs are compared as strings. 'alice10' < 'alice2' (lexicographic). Use UUIDs or zero-padded numbers.
• No causal ordering: This compares IDs, not causality. For causal order, use vector clocks.

VectorClockManager - Advanced Clock Operations

Manages vector clocks for causal ordering and conflict detection.

When to Use

• Custom conflict resolution strategies
• Implementing causal broadcast protocols
• Debugging concurrency issues
• Multi-session coordination logic

Usage

import { VectorClockManager, type VectorClock } from '@vuer-ai/vuer-rtc';

const manager = new VectorClockManager();

// Create a clock for a session
const clock1 = manager.create('alice');
// => { alice: 0 }

// Increment on local operation
const clock2 = manager.increment(clock1, 'alice');
// => { alice: 1 }

// Merge clocks (on receiving remote operation)
const remoteClock = { bob: 5 };
const merged = manager.merge(clock2, remoteClock);
// => { alice: 1, bob: 5 }

// Compare clocks
const cmp = manager.compare(clock1, clock2);
// => -1 (clock1 causally before clock2)

// Check concurrency
const concurrent = manager.areConcurrent(clock1, clock2);
// => false (clock1 < clock2)

Comparison Results

compare(clockA, clockB) returns:
  1  if clockA causally after clockB (A dominates)
 -1  if clockA causally before clockB (B dominates)
  0  if concurrent (neither dominates)

Causal Rules

• A < B: All of A's counters ≤ B's counters, and at least one is strictly less
• A > B: All of A's counters ≥ B's counters, and at least one is strictly greater
• A || B: (concurrent) Some of A's counters are greater, some of B's are greater

Pitfalls

• Immutable operations: All methods return new clocks. Don't mutate clocks in place.
• Missing sessions: Missing sessions are treated as 0. { alice: 1 } and { alice: 1, bob: 0 } are equivalent.
• No garbage collection: Old sessions remain in the clock forever. Large multi-session systems accumulate entries.
• No total order: Concurrent operations have no defined order. Use Lamport time for total ordering.

ConflictResolver - Custom Conflict Resolution

Merges concurrent property updates based on data type and operation semantics.

When to Use

• Implementing custom merge strategies
• Debugging conflict resolution behavior
• Building higher-level CRDT abstractions
• Analyzing concurrent edit patterns

Usage

import { ConflictResolver, type ValueWithMeta } from '@vuer-ai/vuer-rtc';

const resolver = new ConflictResolver();

// Merge a single property with concurrent updates
const values: ValueWithMeta[] = [
  { value: [1, 0, 0], lamportTime: 100, sessionId: 'alice' },
  { value: [0, 1, 0], lamportTime: 101, sessionId: 'bob' },
];

const merged = resolver.mergeProperty('vector3.add', values);
// => [1, 1, 0] (additive merge)

// Merge multiple properties
const updates = [
  {
    properties: { position: [1, 0, 0], visible: true },
    otypes: { position: 'vector3.add', visible: 'boolean.set' },
    lamportTime: 100,
    sessionId: 'alice',
  },
  {
    properties: { position: [0, 1, 0], color: '#ff0000' },
    otypes: { position: 'vector3.add', color: 'color.set' },
    lamportTime: 101,
    sessionId: 'bob',
  },
];

const result = resolver.mergeProperties(updates);
// => { position: [1, 1, 0], visible: true, color: '#ff0000' }

Supported Operations

See the DType System below for all supported data types and operations.

Pitfalls

• Otype required: Missing otype defaults to 'any.set' (LWW). Always specify explicit otypes.
• Unknown dtypes: Unrecognized dtypes fall back to LWW with a console warning.
• No schema validation: Values are not type-checked. Passing a string to vector3.add will cause runtime errors.
• Lamport time ties: When lamport times are equal, sessionId lexicographic order is used. Ensure Lamport clocks are properly incremented.

DType System - Data Type Abstraction Layer

The DType system defines merge behaviors for concurrent updates. Each data type supports multiple operations.

Structure

export const DType = {
  number: {
    set: (values) => /* LWW */,
    add: (values) => /* sum */,
    multiply: (values) => /* product */,
    min: (values) => /* minimum */,
    max: (values) => /* maximum */,
  },
  vector3: {
    set: (values) => /* LWW */,
    add: (values) => /* component-wise sum */,
    multiply: (values) => /* component-wise product */,
  },
  // ... more types
};

All Supported Types

Custom Merge Functions

import { DType, type ValueWithMeta, type MergeFn } from '@vuer-ai/vuer-rtc';

// Access built-in merge functions
const addFn: MergeFn<number> = DType.number.add;

const values: ValueWithMeta<number>[] = [
  { value: 10, lamportTime: 100, sessionId: 'alice' },
  { value: 20, lamportTime: 101, sessionId: 'bob' },
];

const sum = addFn(values);
// => 30

// Use for custom logic
function customMerge(values: ValueWithMeta<number>[]): number {
  const baseSum = DType.number.add(values);
  return Math.min(baseSum, 100); // Clamp at 100
}

When to Use

• Implementing custom data types
• Building domain-specific merge logic
• Testing CRDT merge behavior
• Analyzing conflict patterns

Pitfalls

• No type checking: DType.vector3.add accepts any array. Runtime errors if values aren't [number, number, number].
• Immutable is first-write-wins: Unlike LWW, immutable.set takes the earliest value, not latest.
• Color blend is averaging: color.blend averages RGB values, not perceptual blending. May produce muddy colors.
• Object.lwwPerKey is shallow: Only top-level keys are merged. Nested objects use LWW for the entire nested object.
• No custom dtypes: The DType registry is closed. To add custom types, fork the library or wrap in object.lwwPerKey.

Performance Considerations

RangeTree

• Best case: O(log n) position lookups, O(log m) ID lookups
• Worst case: O(n) when tree is degenerate (all items in one leaf)
• Memory: ~64 bytes per item + tree node overhead

TextRope

• Best case: O(log n) insert/delete, O(log m) ID lookup
• Worst case: O(n) when agent index has many small spans
• Memory: ~100 bytes per item (includes agent index, parent refs)

Compaction

• Time: O(j + n) where j = journal length, n = graph size
• Memory spike: 2x graph size during rebuild (old + new)
• Recommendation: Compact every 100-1000 operations, not on every ack

Vector Clocks

• Time: O(s) where s = number of sessions
• Memory: O(s) per message
• Growth: Unbounded as sessions join. Consider pruning old sessions.

Common Patterns

Custom Text Editor

import { RangeTree, type Item } from '@vuer-ai/vuer-rtc';

class CustomEditor {
  private tree = new RangeTree();

  insert(position: number, text: string, agentId: string, seq: number) {
    const result = this.tree.findByPosition(position);
    const parentId = result ? result.leaf.items[result.itemIndex].id : null;

    const item: Item = {
      id: { agent: agentId, seq },
      content: text,
      isDeleted: false,
      parentId,
      seq,
    };

    this.tree.insert(item);
  }

  getText(): string {
    return this.tree.getContent();
  }
}

Session Watermark Compaction

import { VectorClockManager } from '@vuer-ai/vuer-rtc';

function compactOldSessions(store: GraphStore, maxAge: number) {
  const state = store.getState();
  const now = Date.now();
  const watermark: VectorClock = {};

  // Build watermark from old entries
  for (const entry of state.journal) {
    const age = now - entry.msg.timestamp;
    if (age > maxAge) {
      const clock = entry.msg.clock[entry.msg.sessionId];
      watermark[entry.msg.sessionId] = Math.max(
        watermark[entry.msg.sessionId] || 0,
        clock
      );
    }
  }

  store.compactToWatermark(watermark);
}

Custom Conflict Resolver

import { ConflictResolver } from '@vuer-ai/vuer-rtc';

class GameConflictResolver extends ConflictResolver {
  mergeProperty(otype: string, values: ValueWithMeta[]): any {
    // Custom logic for game-specific types
    if (otype === 'health.damage') {
      // Damage is additive, but clamped at 0
      const total = values.reduce((sum, v) => sum + v.value, 0);
      return Math.max(0, total);
    }

    // Fall back to default
    return super.mergeProperty(otype, values);
  }
}

Further Reading

• Architecture - Understanding the CRDT graph structure
• Operations - All supported operation types
• Networking & Retry - Message acknowledgement and sync
• Rope (Text CRDT) - High-level text CRDT API