Keyboard shortcuts

Press or to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Holochain Testing

Four-Layer Testing Strategy

┌──────────────────────────────────────────────────────────────────┐
│  Layer 4 — Performance (Wind-Tunnel, load testing)               │
│  "How fast, scalable, and resilient is this under load?"         │
├──────────────────────────────────────────────────────────────────┤
│  Layer 3 — E2E UI (Playwright + real conductor)                  │
│  "Does the UI render real data and journeys work?"               │
├──────────────────────────────────────────────────────────────────┤
│  Layer 2 — Integration (Sweettest, cargo test)                   │
│  "Do zomes, DHT sync, and validation work?"                      │
├──────────────────────────────────────────────────────────────────┤
│  Layer 1 — Unit (Vitest, stores/services/mappers)                │
│  "Do computed values and business logic work?"                   │
└──────────────────────────────────────────────────────────────────┘
LayerToolOutputWhat it catches
UnitVitestpass/failStore logic, mappers, computed values
IntegrationSweettestpass/failZome logic, validation, DHT sync, auth
E2E UIPlaywright + @holochain/clientpass/failFull user journeys, real data display
PerformanceWind-Tunnelmetrics (latency/throughput)Regressions under load, DHT sync lag, soak issues

Gap: Browser-side signal handling (recv_remote_signal) is not well covered by any layer — it requires a running UI receiving WebSocket push events from a real conductor.


Framework Overview

  • Sweettest (holochain::sweettest) — Rust-native, in-process conductor. Official Holochain team recommendation. Run with cargo test.
  • Playwright + @holochain/client — Browser automation against a real conductor. No mocks.
  • Wind-Tunnel (holochain_wind_tunnel_runner) — Rust load testing. Separate repo. Measures latency, throughput, DHT sync lag. Used for Holochain core CI performance regression. See WindTunnel.md.

Note on Tryorama: Deprecated for HDK 0.7+ by the Holochain team. Use Sweettest for integration tests and Playwright for E2E UI tests.


When to Use Which

Use CaseSweettestPlaywrightWind-Tunnel
Zome logic, validation, CRUD✅ PreferredNoNo
DHT propagation, consistency✅ PreferredNoNo
Multi-agent scenarios✅ PreferredNoNo
Inline zomes (no WASM compile)✅ YesNoNo
Direct DHT database inspection✅ YesNoNo
Full UI user journeysNo✅ YesNo
Real data rendered in browserNo✅ YesNo
Latency / throughput metricsNoNo✅ Yes
DHT sync lag measurementNoNo✅ Yes
Soak / sustained load testingNoNo✅ Yes
LanguageRustTypeScriptRust

Sweettest (Rust-Native)

Setup (Cargo.toml)

[dev-dependencies]
holochain = { version = "=0.6.1", features = ["test_utils"] }
tokio = { version = "1", features = ["full"] }

Core Types

TypePurpose
SweetConductorSingle conductor instance
SweetConductorBatchMultiple conductors for multi-agent scenarios
SweetAppInstalled app with pre-built cells
SweetCellCell reference — access agent key, DNA hash, zome handles
SweetZome(CellId, ZomeName) handle passed to conductor.call()
SweetAgentsAgent key generation utilities
SweetDnaFileDNA construction helpers
SweetInlineZomesDefine zome functions directly in test code

Standard Two-Agent Test

#![allow(unused)]
fn main() {
use holochain::sweettest::*;
use std::path::Path;

#[tokio::test(flavor = "multi_thread")]
async fn two_agents_can_share_entries() {
    // 1. Create two conductors
    let mut conductors = SweetConductorBatch::from_config(
        2,
        SweetConductorConfig::standard(),
    ).await;

    // 2. Load DNA bundle
    let dna = SweetDnaFile::from_bundle(Path::new("workdir/my.dna")).await.unwrap();

    // 3. Install app on both conductors
    let apps = conductors.setup_app("my-app", &[dna]).await.unwrap();
    let ((alice_cell,), (bob_cell,)) = apps.into_tuples();

    // 4. Exchange peer info so conductors can gossip
    conductors.exchange_peer_info().await;

    // 5. Alice creates an entry
    let alice_zome = alice_cell.zome("my_coordinator");
    let hash: ActionHash = conductors[0]
        .call(&alice_zome, "create_my_entry", my_payload)
        .await;

    // 6. Wait for DHT consistency (replaces Tryorama's dhtSync)
    await_consistency(&[&alice_cell, &bob_cell]).await.unwrap();

    // 7. Bob reads the entry
    let bob_zome = bob_cell.zome("my_coordinator");
    let record: Option<Record> = conductors[1]
        .call(&bob_zome, "get_my_entry", hash)
        .await;

    assert!(record.is_some());
}
}

CRITICAL: await_consistency is MANDATORY Before Cross-Agent Reads

The Rust equivalent of Tryorama’s dhtSync:

#![allow(unused)]
fn main() {
// After any write, before cross-agent reads:
await_consistency(&[&alice_cell, &bob_cell]).await.unwrap();

// Custom timeout in seconds (default is 60s):
await_consistency_s(30, &[&alice_cell, &bob_cell]).await.unwrap();

// Instant non-waiting check:
check_consistency(&[&alice_cell, &bob_cell]).await.unwrap();
}

await_consistency polls every 500ms, comparing all peers’ DHT databases at the op level until every op is integrated across all nodes.

Calling Zome Functions

#![allow(unused)]
fn main() {
// Standard call — panics on error, uses authorship cap automatically:
let result: MyOutputType = conductor.call(&cell.zome("my_zome"), "fn_name", payload).await;

// Fallible call — returns ConductorApiResult:
let result = conductor.call_fallible(&cell.zome("my_zome"), "fn_name", payload).await?;

// Cross-agent call — simulate another agent calling with a cap secret:
let result: MyOutputType = conductor.call_from(
    &other_agent_key,
    Some(cap_secret),
    &cell.zome("my_zome"),
    "restricted_fn",
    payload,
).await;
}

Agent Key Generation

#![allow(unused)]
fn main() {
// Named deterministic keys (same every run — useful for debugging):
let (alice, bob) = SweetAgents::alice_and_bob();
let alice = SweetAgents::alice();

// Random keys:
let agent = SweetAgents::one(conductor.keystore()).await;
let (a, b, c) = SweetAgents::three(conductor.keystore()).await;
let agents: Vec<AgentPubKey> = SweetAgents::get(conductor.keystore(), 5).await;
}

Inline Zomes (Quick Isolated Tests, No WASM Compile)

#![allow(unused)]
fn main() {
let mut zomes = SweetInlineZomes::new();
zomes.function("create_thing", |api, input: MyInput| {
    let hash = api.create(CreateInput::new(
        EntryDefLocation::app(0, 0),
        EntryVisibility::Public,
        Entry::app(SerializedBytes::try_from(input)?)?,
        ChainTopOrdering::default(),
    ))?;
    Ok(hash)
});
let dna = SweetDnaFile::unique_from_inline_zomes(zomes).await.unwrap();
}

Single-Conductor Pattern (Validation and Unit Tests)

#![allow(unused)]
fn main() {
#[tokio::test(flavor = "multi_thread")]
async fn validate_entry_on_create() {
    let conductor = SweetConductor::from_config(SweetConductorConfig::standard()).await;
    let dna = SweetDnaFile::from_bundle(Path::new("workdir/my.dna")).await.unwrap();
    let app = conductor.setup_app("my-app", &[dna]).await.unwrap();
    let (cell,) = app.into_tuple();
    let zome = cell.zome("my_coordinator");

    // Test validation rejection
    let result = conductor.call_fallible(&zome, "create_my_entry", invalid_payload).await;
    assert!(result.is_err());
}
}

Common Sweettest Failures

SymptomRoot CauseFix
Bob can’t find Alice’s entryMissing await_consistencyAdd await_consistency(&[&alice_cell, &bob_cell]).await.unwrap()
Compilation error on call()Missing feature flagAdd features = ["test_utils"] to holochain dev-dep
Timeout in await_consistencyConductors not networkedCall conductors.exchange_peer_info().await after setup_app
Wrong type on call()Type annotation missingAdd explicit type: let result: MyType = conductor.call(...)
into_tuple() failsWrong number of cells destructuredMatch tuple arity to number of DNA roles

SweetConductorConfig — Network Tuning

Most tests use SweetConductorConfig::standard(). Override for stress tests or timing-sensitive scenarios:

#![allow(unused)]
fn main() {
let mut config = SweetConductorConfig::standard();

// Tune gossip frequency (default: 1000ms)
config.tune_network_config(|net| {
    net.gossip_initiate_interval_ms = 500;        // More frequent gossip
    net.gossip_round_timeout_ms = 20_000;          // Longer timeout
    net.gossip_min_initiate_interval_ms = 500;
    net.gossip_initiate_jitter_ms = 50;
});

// Tune validation and countersigning
config.tune_conductor(|tune| {
    tune.sys_validation_retry_delay = Some(Duration::from_secs(3));
    tune.countersigning_resolution_retry_delay = Some(Duration::from_secs(5));
    tune.countersigning_resolution_retry_limit = Some(10);
});

let conductor = SweetConductor::from_config_rendezvous(
    config,
    SweetLocalRendezvous::new().await,
).await;
}

Installation Patterns

Single Conductor, Multiple Agents

#![allow(unused)]
fn main() {
// Install same app for N generated agents (app IDs: "{prefix}0", "{prefix}1", ...)
let apps: SweetAppBatch = conductor
    .setup_apps("my-app", 3, &[dna_file])
    .await.unwrap();
let cells: Vec<SweetCell> = apps.cells_flattened();

// Install for a specific pre-generated agent
let agent = SweetAgents::one(conductor.keystore()).await;
let app: SweetApp = conductor
    .setup_app_for_agent("my-app", agent.clone(), &[dna_file])
    .await.unwrap();

// Install for multiple pre-generated agents
let agents = SweetAgents::get(conductor.keystore(), 3).await;
let apps: SweetAppBatch = conductor
    .setup_app_for_agents("my-app", &agents, &[dna_file])
    .await.unwrap();
}

Explicit DNA Role Binding

#![allow(unused)]
fn main() {
// Bind DNA to a named role (required when role name differs from DNA hash)
let dna_with_role: (RoleName, DnaFile) = ("my_role".into(), dna_file);
let app = conductor.setup_app("my-app", &[dna_with_role]).await.unwrap();
}

Multi-Cell App (Multiple DNA Roles)

#![allow(unused)]
fn main() {
let role_a = ("role_a", dna_a);
let role_b = ("role_b", dna_b);
let app = conductor.setup_app("my-app", &[role_a, role_b]).await.unwrap();

// Destructure cells by role order
let (cell_a, cell_b) = app.into_tuple();
}

SweetAppBatch Destructuring

#![allow(unused)]
fn main() {
// Two apps, one cell each
let ((alice,), (bob,)) = conductors
    .setup_app("my-app", &[dna_file])
    .await.unwrap()
    .into_tuples();

// Two apps, two cells each
let ((alice_a, alice_b), (bob_a, bob_b)) = conductors
    .setup_app("my-app", &[dna_a, dna_b])
    .await.unwrap()
    .into_tuples();
}

SweetConductorBatch — Advanced Patterns

#![allow(unused)]
fn main() {
// From custom config applied to all conductors
let conductors = SweetConductorBatch::from_config_rendezvous(
    3,
    SweetConductorConfig::standard(),
).await;

// From different configs per conductor
let configs = vec![config_a, config_b, config_c];
let conductors = SweetConductorBatch::from_configs_rendezvous(configs).await;

// Force peer visibility between two specific conductors (unidirectional)
conductors.reveal_peer_info(0, 1).await;  // conductor 0 sees conductor 1

// Persist databases for debugging
conductors[0].persist_dbs();  // must call BEFORE shutdown
}

App Lifecycle Management

#![allow(unused)]
fn main() {
// Disable then re-enable an app
conductor.disable_app("my-app".to_string(), DisabledAppReason::User).await.unwrap();
conductor.enable_app("my-app".to_string()).await.unwrap();

// Hot-reload coordinator zomes without restarting conductor
conductor.update_coordinators(
    cell.cell_id().clone(),
    updated_coordinator_zomes,
    vec![new_wasm],
).await.unwrap();

// Create a clone cell of an existing role
let cloned = conductor.create_clone_cell(
    &"my-app".to_string(),
    CreateCloneCellPayload {
        role_name: "clonable_role".into(),
        modifiers: DnaModifiersOpt::default().with_network_seed("clone-1"),
        membrane_proof: None,
        name: Some("My Clone".to_string()),
    },
).await.unwrap();

// Restart conductor
conductor.shutdown().await;
conductor.startup(false).await;
}

Database Access and Inspection

Use these for debugging or asserting internal state without going through zome calls:

#![allow(unused)]
fn main() {
// Access authored and DHT databases directly
let authored_db = cell.authored_db();
let dht_db = cell.dht_db();
let dht_db_from_conductor = conductor.get_dht_db(cell.dna_hash()).unwrap();

// Read the full source chain for an agent
let chain = conductor
    .get_agent_source_chain(&agent_key, cell.dna_hash())
    .await;

// Get invalid / rejected ops (validates your validation logic)
let invalid_ops = conductor.get_invalid_integrated_ops(&dht_db).await.unwrap();
assert!(invalid_ops.is_empty(), "Found invalid ops: {invalid_ops:?}");

// Persist databases to disk before shutdown (for debugging)
let path = conductor.persist_dbs();
println!("DB saved to: {}", path.display());
conductor.shutdown().await;
}

Network and Gossip Testing

#![allow(unused)]
fn main() {
// Wait for specific peers to become visible on this conductor
conductor.wait_for_peer_visible(
    vec![alice_pubkey.clone(), bob_pubkey.clone()],
    Some(cell.cell_id().clone()),
    Duration::from_secs(30),
).await.unwrap();

// Require at least N peers before gossip starts (avoids false positives)
conductor
    .require_initial_gossip_activity_for_cell(&cell, 2, Duration::from_secs(30))
    .await.unwrap();

// Declare this node holds the full DHT arc (affects peer routing)
conductor.declare_full_storage_arcs(cell.dna_hash()).await;

// Check consistency without blocking (instant snapshot)
check_consistency(&[&alice_cell, &bob_cell]).await.unwrap();

// Drop and restart signaling server (simulates network partition)
let rendezvous = SweetLocalRendezvous::new_raw().await;
rendezvous.drop_sig().await;   // kill signal channel
// ... test behavior during outage ...
rendezvous.start_sig().await;  // restore
}

Op Integration Verification

Assert that ops are fully integrated without using await_consistency:

#![allow(unused)]
fn main() {
// All ops in the DHT for this DNA are integrated
let integrated = conductor.all_ops_integrated(cell.dna_hash()).unwrap();
assert!(integrated, "Ops not yet integrated");

// All ops authored by a specific agent are integrated
let author_integrated = conductor
    .all_ops_of_author_integrated(cell.dna_hash(), cell.agent_pubkey())
    .unwrap();
}

Time-Based Testing (Scheduled Functions)

#![allow(unused)]
fn main() {
// Start scheduler with custom interval
conductor.start_scheduler(Duration::from_millis(100)).await.unwrap();

// Manually fire scheduled functions at a specific timestamp
let target_time = Timestamp::now() + Duration::from_secs(3600); // 1 hour in future
conductor.dispatch_scheduled_fns(target_time).await;

// Verify effects after scheduler fires
let result: Vec<Record> = conductor.call(&zome, "get_scheduled_entries", ()).await;
assert!(!result.is_empty());
}

SweetInlineZomes — Integrity and Coordinator Separation

The full pattern separates integrity (validation) from coordinator (business logic):

#![allow(unused)]
fn main() {
use holochain::sweettest::{SweetInlineZomes, SweetDnaFile};
use holochain_zome_types::{EntryDef, EntryVisibility};

let entry_def = EntryDef {
    id: "my_entry".into(),
    visibility: EntryVisibility::Public,
    required_validations: RequiredValidations::default(),
    cache_at_agent_activity: false,
    required_validation_type: Default::default(),
};

let zomes = SweetInlineZomes::new(vec![entry_def], /* num_link_types */ 0)
    // Integrity zome: validation callbacks
    .integrity_function("validate", |_api, _op: Op| {
        Ok(ValidateCallbackResult::Valid)
    })
    // Coordinator zome: zome functions
    .function("create_entry", |api, input: MyInput| {
        let hash = api.create(CreateInput::new(
            EntryDefLocation::app(0, 0),
            EntryVisibility::Public,
            Entry::app(SerializedBytes::try_from(input)?)?,
            ChainTopOrdering::default(),
        ))?;
        Ok(hash)
    })
    .function("get_entry", |api, hash: ActionHash| {
        api.get(vec![GetInput::new(hash.into(), GetOptions::default())])
            .map(|gets| gets.into_iter().next().flatten())
    });

let (dna, _, _) = SweetDnaFile::unique_from_inline_zomes(zomes).await;
}

Zome name constants: SweetInlineZomes::INTEGRITY = "integrity", SweetInlineZomes::COORDINATOR = "coordinator".


WebSocket Interface Testing

For tests that need to verify WebSocket behavior (signals, app interface):

#![allow(unused)]
fn main() {
// Get admin WebSocket client
let (admin_sender, _admin_recv) = conductor.admin_ws_client::<AdminResponse>().await;

// Get app WebSocket client (auto-authenticated)
let (app_sender, mut app_recv) = conductor
    .app_ws_client::<AppResponse>("my-app".to_string())
    .await;

// Or authenticate manually for custom setup
let (app_sender, _) = websocket_client_by_port(app_port).await.unwrap();
authenticate_app_ws_client(app_sender.clone(), admin_port, "my-app".to_string()).await;
}

Common Sweettest Failures (Extended)

SymptomRoot CauseFix
Bob can’t find Alice’s entryMissing await_consistencyAdd await_consistency(&[&alice_cell, &bob_cell]).await.unwrap()
Compilation error on call()Missing feature flagAdd features = ["test_utils"] to holochain dev-dep
Timeout in await_consistencyConductors not networkedCall conductors.exchange_peer_info().await after setup_app
Wrong type on call()Type annotation missingAdd explicit type: let result: MyType = conductor.call(...)
into_tuple() failsWrong number of cells destructuredMatch tuple arity to number of DNA roles
Invalid ops present unexpectedlyValidation logic accepting bad dataUse get_invalid_integrated_ops() to inspect rejected ops
reveal_peer_info / gossip never startsFull arc not declaredCall declare_full_storage_arcs() on test conductors
Scheduled fn never firesScheduler not startedCall start_scheduler() or dispatch_scheduled_fns(timestamp)
WebSocket auth fails in testUsing wrong portUse admin_ws_client() for admin, app_ws_client() for app calls

E2E UI Testing (Playwright + Real Conductor)

For full end-to-end tests that drive the UI against a real Holochain backend — no mocks. Use @holochain/client directly — Tryorama is deprecated.

Setup (package.json)

{
  "devDependencies": {
    "@playwright/test": "^1.40.0",
    "@holochain/client": "^0.18.0"
  }
}

Conductor Setup Pattern (globalSetup)

The critical pattern: use AdminWebsocket to install the app and get a proper auth token, then keep the conductor alive for all Playwright tests.

// tests/e2e/setup/global-setup.ts
import { AdminWebsocket, AppWebsocket } from '@holochain/client';
import { execSync, spawn } from 'child_process';

export default async function globalSetup() {
  // 1. Start conductor via hc sandbox
  const conductor = spawn('hc', ['sandbox', 'run', '--root', './test-workdir'], {
    stdio: ['ignore', 'pipe', 'pipe']
  });

  // 2. Wait for conductor ready signal in stdout (not polling)
  await new Promise<void>((resolve, reject) => {
    conductor.stdout?.on('data', (data: Buffer) => {
      if (data.toString().includes('Conductor ready')) resolve();
    });
    setTimeout(() => reject(new Error('Conductor startup timeout')), 30000);
  });

  // 3. Connect admin client (use admin port, not app port)
  const admin = await AdminWebsocket.connect({
    url: new URL('ws://localhost:8888')
  });

  // 4. Install and enable the happ properly
  const agentKey = await admin.generateAgentPubKey();
  await admin.installApp({
    installed_app_id: 'my_happ',
    agent_key: agentKey,
    path: './workdir/my_happ.happ',
  });
  await admin.enableApp({ installed_app_id: 'my_happ' });

  // 5. Open app interface on a free port
  const { port } = await admin.attachAppInterface({ port: 0 });

  // 6. Issue auth token
  const { token } = await admin.issueAppAuthenticationToken({
    installed_app_id: 'my_happ'
  });

  // 7. Connect app client and seed test data
  const client = await AppWebsocket.connect({
    url: new URL(`ws://localhost:${port}`),
    token,
  });

  await seedTestData(client);

  // 8. Store conductor process for teardown
  process.env.E2E_CONDUCTOR_PID = String(conductor.pid);
  process.env.E2E_APP_PORT = String(port);
}

Data Seeding

Seed data directly via AppWebsocket.callZome before Playwright opens the browser:

async function seedTestData(client: AppWebsocket) {
  // Seed in dependency order
  await client.callZome({
    role_name: 'my_dna',
    zome_name: 'my_coordinator',
    fn_name: 'create_service_type',
    payload: { name: 'Web Development', description: '...' },
  });

  await client.callZome({
    role_name: 'my_dna',
    zome_name: 'my_coordinator',
    fn_name: 'create_offer',
    payload: { title: 'Seed Offer', description: '...' },
  });
}

Playwright Test Pattern

// tests/e2e/specs/offers.spec.ts
import { test, expect } from '@playwright/test';

test('user sees seeded offers on load', async ({ page }) => {
  await page.goto('/offers');

  // Wait for Holochain connection (not a mock — real loading time)
  await expect(page.locator('[data-testid="offer-card"]'))
    .toHaveCount(1, { timeout: 15000 });

  await expect(page.locator('text=Seed Offer')).toBeVisible();
});

playwright.config.ts Key Settings

export default defineConfig({
  globalSetup: './tests/e2e/setup/global-setup.ts',
  globalTeardown: './tests/e2e/setup/global-teardown.ts',
  workers: 1,           // Single worker — one conductor, no conflicts
  fullyParallel: false, // Holochain state is shared across tests
  timeout: 60000,       // Holochain operations are slow
  use: {
    baseURL: 'http://localhost:5173',
  },
  webServer: {
    command: 'bun run dev',
    url: 'http://localhost:5173',
    reuseExistingServer: true,
  },
});

Common E2E Failures

SymptomRoot CauseFix
Conductor never readyPolling instead of stdoutListen for "Conductor ready" in stdout
callZome rejectedUsing AppWebsocket on admin portUse AdminWebsocket on admin port (8888), AppWebsocket on app port
Auth error on connectMissing tokenCall admin.issueAppAuthenticationToken() and pass token to AppWebsocket.connect
Tests interfere with each otherShared conductor stateRun with workers: 1, reset data in beforeEach if needed
UI shows no dataRace — browser loads before seedingSeed in globalSetup (runs before browser opens), not in beforeAll