Quick Start
# 1. Clone and install
git clone <repo-url> && cd minisworld
npm install
# 2. Set required env vars
cp .env.example .env
# Edit .env: OPENAI_API_KEY (required), SESSION_SECRET, JWT_SECRET
# 3. Start
npm run dev # development (file watching)
npm start # production
npm test # run 815+ unit tests
# 4. Open
open http://localhost:4040
Required Environment Variables
| Variable | Purpose |
|---|---|
OPENAI_API_KEY | OpenAI API key (chat, embeddings, DALL-E, realtime) |
SESSION_SECRET | Session cookie signing (change from default in prod) |
JWT_SECRET | HMAC-SHA256 token signing (change from default in prod) |
All other variables are optional — tools auto-disable when their credentials are missing.
What It Is
Mini's World is a full-stack AI automation platform built around Minis — 15 specialized AI employees, each with a distinct role, personality, knowledge base, and assigned tools. Minis live inside Companies (isolated workspaces) where they collaborate through Workflows, respond to Triggers, execute on Schedules, and use Tools (62 integrations).
Users interact with Minis through text chat (HTTP + SSE streaming, default), voice chat (WebSocket relay to OpenAI Realtime API), or phone calls (Twilio Media Streams + OpenAI Realtime, sub-second latency).
Core Primitives
| Primitive | Definition |
|---|---|
| Mini | An AI employee with a role, personality, voice, knowledge base, and assigned tools |
| Company | An isolated workspace with its own contacts, memories, workflows, and tool configurations |
| Workflow | A multi-step orchestrated task that passes through a state machine with human approval gates |
| Tool | An integration (email, social, CRM, etc.) executed by Minis — 62 total across 3 tiers |
| Trigger | An event listener that auto-creates workflows when matching events arrive |
| Schedule | A cron-based timer that executes workflows on a recurring basis |
High-Level Architecture
Connection types: HTTP (routes) · SSE (chat streaming) · WSS (voice/calls) · Webhooks (Twilio, Stripe, Teams, triggers)
Sequence: Text Chat
Sequence: Workflow Execution
Project Structure
agents/ — 15 AI agent configurations
agents/
├── founder/config.js # Co-Founder & Vision
├── developer/config.js # Lead Engineer
├── marketing/config.js # Head of Growth
├── designer/config.js # Head of Design
├── ops/config.js # Head of Operations
├── research/config.js # Head of Research
├── copywriter/config.js # Head of Copy
├── outreach/config.js # Head of Outreach
├── client/config.js # Client Success Lead
├── analyst/config.js # Head of Analytics
├── trading/config.js # Head of Trading
├── crypto/config.js # Head of Crypto Trading
├── studio/config.js # Podcast Producer
├── vision/config.js # Browser Automation
├── aria/config.js # Executive Assistant
└── <id>/knowledge/*.md # Per-agent knowledge bases
client/ — Vanilla JS browser app (no framework, no build)
client/
├── index.html # Entry point (33 CSS, loads main.js)
├── src/main.js # Bootstrap: auth, company sync, routing
├── src/ui/ # 70 UI components (vanilla JS classes)
│ ├── pages/ # CompanyHub, CompanyWorkspace, LoginPage, etc.
│ ├── chat/ # VoiceInterface, Transcript, CallPanel
│ ├── workflows/ # WorkflowLauncher, WorkflowPage, HumanReviewModal
│ ├── tools/ # ToolSettings, ToolSetupPanel
│ ├── features/ # CampaignManager, PodcastCard, WebsitePreview
│ ├── trading/ # TradingSettings, TradeCard
│ ├── admin/ # AuditDashboard, ImportPanel, ExportPanel
│ ├── common/ # MiniCard, Toast, ConfirmDialog, MiniAvatar
│ └── onboarding/ # OnboardingFRE, ApiKeyModal
├── src/store/ # 26 fetch wrappers
├── src/styles/ # 33 CSS files (glass morphism theme)
├── src/realtime/ # RealtimeClient (WS), EventBus
├── src/audio/ # AudioCapture (PCM16 mic), AudioPlayer
├── src/crypto/ # credentialVault.js (AES-256-GCM)
└── src/api/ # agentApi.js, sessionApi.js
server/ — Express + WebSocket server
server/
├── index.js # Boot: register agents, mount routes, start WS
├── config.js # Zod-validated env vars (80+ variables)
├── logger.js # Winston logger
├── agentRegistry.js # Agent Map + prompt builders
├── routes/ # 37 Express router files (195+ endpoints)
├── ai/ # openaiClient, completionLoop, guardrails, usageTracker
├── workflows/ # workflowEngine, miniOrchestrator, timeoutChecker
├── tools/ # toolDefinitions (62), local/ (15), api/ (13), toolConfigStore
├── ws/ # realtimeRelay (voice), callRelay (Twilio)
├── auth/ # 8 OAuth providers, userStore
├── middleware/ # security.js (Helmet, CORS, CSRF, rate limit)
├── memory/ # store.js (per-company embeddings + cosine search)
├── triggers/ # eventListener, gmailPoller, calendarPoller
├── schedules/ # scheduleStore, scheduleChecker, cronParser
├── mcp/ # mcpClient (3-tier routing), mcpServerManager
├── stores/ # BaseStore (PerUserStore, PerCompanyStore, GlobalStore)
├── studio/ # Podcast pipeline (HeyGen → DALL-E → FFmpeg)
├── companies/ # companyContextStore, knowledgeStore
├── contacts/ # contactStore (per-company)
├── trading/ # tradingStore, capitalRules, indicators
├── calls/ # callStore, callTelemetry
├── gallery/ # galleryStore, imageResolver
├── integrations/teams/ # Microsoft Teams bot
└── telemetry/ # telemetryStore, telemetryMiddleware
Core Systems
🧠 RAG & Prompt Engineering
7-layer prompt pipeline: agent personality → company context → training data → knowledge docs (RAG, cosine > 0.4) → contacts → memory → tool definitions. Embeddings: text-embedding-3-small (1536 dim).
⚙️ Workflow Engine
State machine: PENDING → RUNNING → AWAITING_HUMAN → APPROVED → EXECUTING → COMPLETED. Safety: 10-min timeout, max 2 revisions, 100 iteration loop cap, approval gating.
🔧 Tool System (3-Tier)
Local executors (15) → API tools (13) → MCP servers (34). Per-user defaults + per-company overrides. AES-256-GCM encrypted credentials with scrypt key derivation.
📞 Phone Calls
Twilio Media Streams + OpenAI Realtime API. Sub-second latency, barge-in support, hold tone during tool execution, live whisper injection, call transcripts.
🎙️ Podcast Studio
Pipeline: AI script → HeyGen avatar video → DALL-E cover art → FFmpeg assembly → YouTube auto-publish. SSE progress streaming throughout.
🌐 Website Builder
Natural language → HTML generation → live preview → one-click deploy (Vercel or self-hosted with Nginx subdomain routing).
🎯 Lead Generation
Apollo.io + Google Maps + Hunter.io for lead sourcing. Email verification, lead scoring, 4-email campaign sequences with reply tracking.
🖼️ Image Gallery
DALL-E 3 generation with prompt tracking. [[IMG-1035]] inline reference syntax in chat. Tags, favorites, search, archive.
📋 Meeting Bot
Recall.ai integration — join Zoom, Google Meet, or Teams meetings. Listen, take notes, provide transcripts.
💬 Microsoft Teams
Direct agent interaction in Teams. JWT-verified webhooks, Adaptive Cards for approvals, proactive notifications.
🗓️ Schedule System
Cron-based recurring workflows. Natural language parsing ("every Monday at 9 AM" → cron). One-time and trigger-now execution.
🔔 Trigger System
Event-driven automation. Types: email, call, message, form, payment, webhook. Modes: once (fire and complete) or continuous.
✦ Avatar (AI Self)
Persistent, always-on super-entity above all companies. Dual-model: Claude for deep reasoning, GPT-4o for speed. Cross-company monitoring (5-min cycles), morning briefings (chat/email/phone), opportunity research, draft company builder, data-grounded conversation.
📊 Eval System (LLM-as-Judge)
Production-grade evaluation scoring every AI interaction on 8 dimensions. Failure categorization (RAI refusal, tool error, hallucination, context gap). Admin dashboard with trace detail, filters, re-run. Clickable trace explorer.
Agent Roster
| ID | Name | Role | Voice | Key Tools |
|---|---|---|---|---|
founder | Founder Mini | Co-Founder & Vision | coral | linkedin, gmail, slack, calendar, notion, minisign |
developer | Dev Mini | Lead Engineer | echo | gmail, github, azure_devops, website_build |
marketing | Marketing Mini | Head of Growth | coral | linkedin, gmail, image_gen, twitter, canva, mailchimp |
designer | Design Mini | Head of Design | alloy | image_gen, figma, canva, notion |
ops | Ops Mini | Head of Operations | sage | gmail, whatsapp, slack, twilio, shopify, gorgias |
research | Research Mini | Head of Research | fable | web_search, apollo, brave, puppeteer, reddit |
copywriter | Copy Mini | Head of Copy | nova | gmail, image_gen, wordpress, notion, document |
outreach | Outreach Mini | Head of Outreach | onyx | gmail, linkedin, hubspot, salesforce, lead_gen |
client | Client Mini | Client Success Lead | juniper | gmail, stripe, shopify, gorgias, mailchimp |
analyst | Analyst Mini | Head of Analytics | ember | gmail, sheets, airtable, search_console, document |
trading | Trading Mini | Head of Trading | echo | alpaca_trading, web_search |
crypto | Crypto Mini | Head of Crypto | onyx | alpaca_crypto, web_search, brave_search |
studio | Studio Mini | Podcast Producer | alloy | heygen, youtube, image_gen, web_search |
vision | Vision Mini | Browser Automation | echo | vision_browse, web_search, document |
aria | Aria | Executive Assistant | sage | gmail, calendar, slack, twilio, meeting_bot |
Environment Variables
Core AI & Models
| Variable | Default | Required | Description |
|---|---|---|---|
OPENAI_API_KEY | — | Yes | OpenAI API key |
OPENAI_CHAT_MODEL | gpt-4.1 | No | Text chat & workflow model |
OPENAI_CHAT_MODEL_MINI | gpt-4.1-mini | No | Lighter model for specific tasks |
OPENAI_RT_MODEL | gpt-realtime-1.5 | No | Voice chat (realtime API) |
OPENAI_CALL_RT_MODEL | gpt-4o-mini-realtime-preview | No | Phone call relay model |
ANTHROPIC_API_KEY | — | No | Anthropic API key (Avatar deep reasoning via Claude) |
AVATAR_ENABLED | true | No | Enable Avatar background monitoring |
Server
| Variable | Default | Description |
|---|---|---|
PORT | 4040 | HTTP port |
NODE_ENV | development | development | production | test |
DATA_DIR | — | Persistent data directory (for ephemeral platforms) |
ALLOWED_ORIGINS | localhost:5173,4040 | CORS origins (CSV) |
LOG_LEVEL | info | error | warn | info | debug |
SESSION_SECRET | change-me… | Session cookie signing |
JWT_SECRET | change-me… | Token signing (HMAC-SHA256) |
ENCRYPTION_SECRET | — | Credential encryption (falls back to JWT_SECRET) |
MAX_CONCURRENT_SESSIONS | 50 | Max WebSocket sessions |
APP_BASE_URL | https://minisworld.ai | Public URL |
Rate Limiting & Security
| Variable | Default | Description |
|---|---|---|
RATE_LIMIT_SESSION_PER_MIN | 10 | Session endpoint rate limit |
RATE_LIMIT_WS_PER_MIN | 10000 | WebSocket messages/min |
LOGIN_ALERT_THRESHOLD | 5 | Failed logins before alert |
INVITE_ONLY | true | Require invite for signup |
ADMIN_EMAIL | — | Admin dashboard access |
MAX_HOURLY_COST | 2 (USD) | AI spend circuit breaker |
OAuth & Integrations (Google, Microsoft, LinkedIn, Twilio, etc.)
| Category | Variables |
|---|---|
| Google OAuth | GOOGLE_CLIENT_ID, GOOGLE_CLIENT_SECRET |
| Microsoft | MICROSOFT_CLIENT_ID, MICROSOFT_APP_ID, MICROSOFT_APP_SECRET, MICROSOFT_TENANT_ID |
LINKEDIN_CLIENT_ID/SECRET (personal), LINKEDIN_PAGES_CLIENT_ID/SECRET (pages) | |
WHATSAPP_PHONE_NUMBER_ID, WHATSAPP_ACCESS_TOKEN | |
| Twilio | TWILIO_ACCOUNT_SID, TWILIO_AUTH_TOKEN, TWILIO_FROM_NUMBER |
| Social & CRM | FACEBOOK_APP_*, SLACK_CLIENT_*, GITHUB_CLIENT_*, NOTION_OAUTH_*, HUBSPOT_CLIENT_* |
| Media | HEYGEN_API_KEY, CLOUDINARY_*, YOUTUBE_CLIENT_* |
| Trading | ALPACA_API_KEY, ALPACA_SECRET_KEY, ALPACA_PAPER_MODE |
| Lead Gen | APOLLO_API_KEY, GOOGLE_MAPS_API_KEY, HUNTER_API_KEY |
| Payments | STRIPE_SECRET_KEY, STRIPE_WEBHOOK_SECRET |
| Hosting | SELF_HOSTING, BASE_DOMAIN, VERCEL_API_TOKEN, VERCEL_TEAM_ID |
| Vision | VISION_MINI_ENABLED, VISION_MINI_MAX_STEPS, BROWSERLESS_API_KEY |
| Polling | POLL_INTERVAL_MS (5s), GMAIL_POLL_INTERVAL_MS (30s), CALENDAR_POLL_INTERVAL_MS (60s) |
| Analytics | POSTHOG_API_KEY, POSTHOG_HOST, RECALL_AI_API_KEY |
API Reference
Core Chat & Voice
| Method | Path | Description |
|---|---|---|
| POST | /api/chat/:companyId/:agentId/send | Text chat with SSE streaming + tool-call loop |
| POST | /api/chat/:companyId/:agentId/approve-action | Approve pending action (trades, emails) |
| POST | /api/voice/:companyId/:agentId/send | Voice chat (Whisper STT → GPT → TTS) |
| GET | /api/agents | List all agents |
| GET | /api/agents/:id | Get agent details |
| POST | /api/session | Get WebSocket session path |
Workflows
| Method | Path | Description |
|---|---|---|
| POST | /api/workflows | Create and start workflow |
| GET | /api/workflows | List workflows (?companyId=) |
| GET | /api/workflows/:id | Get single workflow |
| POST | /api/workflows/:id/human-decision | Submit approval/rejection |
| POST | /api/workflows/:id/rerun | Rerun completed workflow |
| POST | /api/workflows/:id/pause | Pause running workflow |
| POST | /api/workflows/:id/resume | Resume paused workflow |
| PUT | /api/workflows/:id | Update workflow fields |
| DEL | /api/workflows/:id | Delete workflow |
Tools & Credentials
| Method | Path | Description |
|---|---|---|
| GET | /api/tools/catalog | Tool catalog (?companyId= for override status) |
| GET | /api/tools/credentials/:toolId | Masked user-level credentials |
| PUT | /api/tools/credentials/:toolId | Save user-level credentials |
| POST | /api/tools/credentials/:toolId/validate | Test API connection |
| GET | /api/tools/:companyId/:agentId | Get Mini's assigned tools |
| PUT | /api/tools/:companyId/:agentId | Update tool assignments |
Companies, Contacts & Context
| Method | Path | Description |
|---|---|---|
| GET | /api/companies | List companies |
| PUT | /api/companies/:id | Upsert company |
| DEL | /api/companies/:id | Delete company (cascade) |
| GET | /api/contacts/:companyId | List/search contacts |
| POST | /api/contacts/:companyId | Create contact |
| POST | /api/contacts/:companyId/import | Import CSV/vCard |
| GET | /api/company-context/:companyId | Get company context |
| POST | /api/company-context/:companyId/knowledge | Upload knowledge document |
Triggers, Schedules & Webhooks
| Method | Path | Description |
|---|---|---|
| GET | /api/triggers | List triggers (?companyId=) |
| POST | /api/triggers | Create trigger |
| POST | /api/triggers/:id/toggle | Toggle active/paused |
| GET | /api/schedules | List schedules |
| POST | /api/schedules | Create schedule |
| POST | /api/schedules/:id/trigger-now | Execute immediately |
| POST | /api/schedules/parse-timing | Natural language → cron |
| POST | /wh/form/:companyId | Form submission webhook |
| POST | /wh/payment/:companyId | Payment webhook |
Trading, Gallery, Studio & Features
| Method | Path | Description |
|---|---|---|
| POST | /api/trading/:companyId/credentials | Set Alpaca keys |
| GET | /api/trading/:companyId/settings | Capital rules, watchlist |
| GET | /api/gallery | List images |
| POST | /api/gallery/upload | Upload image |
| GET | /api/studio/:companyId/episodes | List podcast episodes |
| POST | /api/vision/run | Start browser automation |
| GET | /api/campaigns | List campaigns |
| GET | /api/websites/:companyId | List web projects |
Admin (Admin Only)
| Method | Path | Description |
|---|---|---|
| GET | /api/admin/telemetry | Full metrics |
| GET | /api/admin/ai-usage | AI usage by period |
| GET | /api/admin/ai-usage/by-user | Per-user breakdown |
| POST | /api/admin/ai-usage/reset-halt | Reset cost breaker |
| GET | /api/admin/users | List all users |
| POST | /api/admin/invites/generate | Generate invite codes |
Avatar
| Method | Path | Description |
|---|---|---|
| GET | /api/avatar | Full avatar config (creates default if missing) |
| POST | /api/avatar/config | Update avatar config (partial) |
| POST | /api/avatar/chat | Chat with Avatar |
| POST | /api/avatar/chat/stream | SSE streaming chat |
| GET | /api/avatar/briefing | Get/generate briefing |
| POST | /api/avatar/briefing/deliver | Trigger briefing delivery |
| GET | /api/avatar/portfolio | Cross-company portfolio snapshot |
| GET | /api/avatar/contacts | Merged contacts across companies |
| GET | /api/avatar/pending | Pending approvals across companies |
| POST | /api/avatar/research | Run opportunity research pipeline |
| POST | /api/avatar/build-company | Build draft company from candidate |
| GET | /api/avatar/history | Conversation history |
| GET | /api/avatar/connected-channels | Check Gmail/Twilio connection status |
| POST | /api/avatar/action | Direct action execution |
Eval (Admin)
| Method | Path | Description |
|---|---|---|
| GET | /api/eval/traces | List traces (filters: company_id, failure_category) |
| GET | /api/eval/traces/:traceId | Get full trace detail (global search) |
| POST | /api/eval/traces/:traceId/run-eval | Re-run judge evaluation |
| GET | /api/eval/stats | Aggregate stats + failure category breakdown |
| GET | /api/eval/admin-companies | Admin's own companies |
Chat History & Import/Export
| Method | Path | Description |
|---|---|---|
| GET | /api/chat-history/:companyId/:agentId | Get history (paginated) |
| POST | /api/chat-history/:companyId/:agentId | Append message |
| DEL | /api/chat-history/:companyId/:agentId | Clear history |
| GET | /api/import-export/export/workflow/:id | Export workflow JSON |
| POST | /api/import-export/import/workflow | Import workflow |
Server Boot Sequence
- Load & validate config (Zod schema)
- Register all 15 agents + preload knowledge
- Ensure data directory exists
- Load all stores from disk (users, invites, workflows, triggers, schedules, trading, telemetry, etc.)
- Mount middleware: requestId → CSP nonce → Helmet → CORS → JSON → telemetry → sanitizeParams
- Mount pre-CSRF routes: Stripe webhook, Teams webhook, Twilio webhooks, trigger webhooks
- Mount CSRF protection
- Mount auth routes (public) + all protected /api/* routes
- Mount static file serving (client/)
- Create WebSocket servers: RealtimeRelay (/ws/session) + CallRelay (/media-stream)
- Start HTTP server on PORT
- Launch background tasks: timeoutChecker, scheduleChecker, gmailPoller, calendarPoller, reminderChecker, cleanupTimer
- Start Avatar monitor (5-min cycle for all users with Avatar enabled)
- Initialize MCP server connections (non-fatal)
- Cleanup orphaned triggers/schedules from deleted companies
Scripts & Dependencies
Commands
| Command | Description |
|---|---|
npm run dev | Start with --watch (auto-restart on changes) |
npm start | Production start |
npm test | Run all unit tests (Node.js built-in test runner) |
Key Dependencies
| Package | Purpose |
|---|---|
express | HTTP server + routing |
ws | WebSocket (voice + calls) |
googleapis | Google APIs (Gmail, Calendar, YouTube) |
helmet | Security headers |
cors | Cross-origin resource sharing |
express-rate-limit | Rate limiting |
winston | Logging |
zod | Config validation |
dotenv | Environment variables |
@modelcontextprotocol/sdk | MCP server integration |
@anthropic-ai/sdk | Anthropic Claude API (Avatar deep reasoning) |
Conventions
- ESM everywhere —
"type": "module",.jsextensions in all imports - No TypeScript, no JSX — plain JavaScript
- No build step — client served as static files
- UUIDs for workflow/memory/contact IDs
- Winston for server logging,
consoleon client - Zod for config validation
- No ORM — direct file I/O for persistence
- Atomic writes —
atomicWriteSync()for all disk operations - In-memory + disk — lazy load, save on every mutation
Security (31 Controls)
🔐 Authentication
HMAC-SHA256 tokens, httpOnly/Secure/SameSite cookies, OAuth 2.0 + PKCE, 8 providers
🛡️ CSRF
Double-submit cookie with timing-safe comparison
🔒 XSS Prevention
CSP with per-request nonce, Helmet security headers, HSTS
🗝️ Cryptography
AES-256-GCM (scrypt N=16384), per-credential salt+IV
🤖 AI Safety
Prompt injection detection, tool argument validation, daily rate limits, output filtering, cost circuit breaker
📦 Data Isolation
Per-user directories, per-company PerCompanyStore, encrypted credentials
See Security Reference for all 31 verified controls.