TEXT/VOICEMAIL: (832) 617-0436
Free Consultation

Perceived AGI

Here’s the Perceived AGI roadmap in our style—equal parts engineering and mischief—with concrete choices Ali can build today and creative fallbacks when reality throws elbows.

Perceived AGI v1 — Architecture You Can Ship

0) North Star (what “done” feels like)

  • A VPS “Torso” (master brain) hosts orchestration, memory index, and task router.

  • Terminals (“Extremities”)—your laptops/workstations/mini-PCs—stream senses (screen, mic, cam), expose actuators (mouse, keyboard, file I/O), and lend compute (LLM, vision).

  • A shared memory fabric where facts/events/files are named and tagged so the system can recall by association (fast keyword + vector search), not brute-force remember everything.

  • Autonomy rails: every action is explainable, logged, reversible, and constrained by allowlists.

1) System Diagram (mental model)

  • VPS (Torso): FastAPI + Task Router + Memory API + Vector DB + Event Store + Auth.

  • Terminals (Hands/Eyes/Ears):

    • Sensors: screen OCR, mic→ASR, webcam.

    • Actuators: mouse/keyboard control, file ops, browser automation.

    • Local skills: small LLM (Ollama), Whisper (ASR), lightweight vision.

  • Message fabric: brokered (Redis/NATS/MQTT) for reliability; optional P2P overlay for big blobs or offline-first sync.

2) Transport Choices (creative, realistic options)

A) Redis Streams (brokered core, dead simple)

  • Use when: you want “it works today.”

  • Pros: trivial to deploy, persistent streams, consumer groups, LUA scripting, mature clients.

  • Cons: single point of failure (mitigated with Sentinel/Cluster), not WAN-native.

  • Pattern: events:* streams for telemetry; tasks:* streams for jobs; heartbeats for liveness.

B) NATS + JetStream (cloud-native, elegant)

  • Use when: you want low-latency pub/sub + durable inboxes.

  • Pros: lightweight, subjects are expressive (bayne.terminals.karen.laptop.screen), easy multi-node.

  • Cons: more ops knowledge; fewer built-in recipes than Redis.

  • Pattern: request/reply for RPC-like control (actuator.mouse.move), JetStream for persistence.

C) BitTorrent-ish / P2P overlay (for big stuff + resilience)

  • Replace “BitTorrent” with libp2p/IPFS or Syncthing:

    • IPFS/libp2p: content-addressable blobs; announce via broker, fetch P2P.

    • Syncthing: dead simple folder-level sync across terminals (versioned), private LAN/WAN friendly.

  • Hybrid idea: broker (Redis/NATS) carries instructions & metadata, P2P handles bulk data. This keeps commands reliable and blobs efficient.

Default recommendation: Redis Streams for control plane, Syncthing for file mirrors, optional IPFS later. When you want to scale or multi-region, graduate the broker to NATS JetStream.

3) Memory That Feels Like Memory (not a database you dread)

Goal: find by association. Two layers: symbolic (names/tags) + semantic (embeddings).

Directory grammar (portable, human-friendly)

/memex/
  2025/
    2025-09-07/
      trade_ideas__nvda_470_bull_put__bayne.md
      trade_ideas__nvda_470_bull_put__bayne.yml      # sidecar metadata
      sekai_sms_campaign__offset_afterparty__creative_v2.png
      call__ali__infra_plan__recording.opus
      call__ali__infra_plan__transcript.txt

Sidecar YAML (indexable, greppable):

title: NVDA 470 BPS idea
who: [bayne, ali]
tags: [options, bps, nvda, risk, auto-trader]
refs: [rh_order_982341, chart_nvda_2025-09-07.png]
summary: >
  Rolled spread, target 1.2% weekly, plan to ladder in drawdown.

Semantic index

  • Embeddings: local (all-MiniLM) for speed; VPS builds/refreshes with a beefier model nightly.

  • Vector DB: Qdrant (easy), or SQLite+FAISS (ultra-simple). Keep text chunks + file pointers.

  • Event log: “what happened” journal (append-only) so we can reconstruct/undo.

Event schema (Redis Stream / NATS message):

{
  "ts": "2025-09-07T06:41:02Z",
  "terminal": "tanay-hp-zbook",
  "actor": "cursor",
  "action": "click",
  "target": "photoshop.canvas",
  "context": {"x": 482, "y": 316, "window": "PS-2025.8"},
  "trace_id": "9c1b…",
  "policy": "design.safe"
}

4) Senses & Actuators (what actually moves)

Seeing

  • Screen capture + OCR: mss (cross-platform) + Tesseract or PaddleOCR.

  • DOM-aware vision (when browser): Playwright with locator semantics beats pure pixels.

Hearing/Speaking

  • ASR: Whisper small (local) for privacy; switch to large on VPS for accuracy bursts.

  • TTS: 11Labs for quality; Piper as local fallback.

Touch (mouse/keyboard)

  • OS-level control: pyautogui or mousekeyhook; hardened with “active window” checks.

  • Browser control: Playwright/Chrome DevTools (preferred—precise, testable).

  • Safety: dry-run mode (visual cursor ghost), “are you sure” policies, and per-app allowlists.

5) Local LLMs as Muscles (Ollama + toolformer habits)

  • Ollama on terminals: llama3.1:8b or qwen2.5:7b for fast drafting; upgrade per box.

  • Tool use: the Torso routes tasks; terminals expose tools (OCR, ffmpeg, file ops).

  • Fallbacks: if local LLM stalls, Torso escalates to API (OpenAI/Anthropic) with budget guardrails.

6) Security, Privacy, and “Leash”

  • Network: WireGuard mesh (VPS as hub) → private service names (nats.int.pf, memex.api).

  • Secrets: .env for dev; sops+age for repo; rotate via Ansible.

  • Policy engine: simple policy.yaml:

    • allow: ["playwright.open", "files.read:/memex/**"]

    • deny: ["shell.rm -rf /*"]

  • Human-in-the-loop modes: shadow (observe), suggest (show plan), act (execute, log).

7) Build Order (phased, shippable)

Phase 0 (Week 1): Skeleton that breathes

  • VPS: Docker + FastAPI Torso, Redis, Qdrant, Traefik (TLS), WireGuard.

  • Endpoint: /remember, /recall, /route (task router), /events/ingest.

  • One terminal agent: heartbeat, file sync (Syncthing), screen snap+paddleOCR, Playwright.

Phase 1 (Weeks 2–3): Memory that answers back

  • Sidecar YAML + embedding pipeline (text, images → captions → embed).

  • /recall supports keyword + vector hybrid with reranking.

  • Basic RAG for your docs and transcripts.

Phase 2 (Weeks 4–6): Senses + Actuators usable for work

  • Whisper small for meetings; auto-transcribe → summary → tasks.

  • Playwright skills: login, download, rename, file to /memex.

  • Design assist: macro library (Photoshop hotkeys scripted) + Playwright for web design flows.

Phase 3 (Weeks 6–8): Multi-terminal + resilience

  • Add NATS (or scale Redis) + task leasing; retry + dedupe.

  • P2P blob: Syncthing stabilized (conflict rules), optional IPFS for public artifacts.

  • Observability dashboard: liveness, queue depth, recent actions, per-policy audits.

Phase 4 (Quarter 2): Smarter autonomy

  • Skill registry (“playbook” YAML → runnable plans).

  • Evaluator loop: self-critique before act; rollbacks; human “approve/deny.”

  • Fine-tune habits on your data (prompt libraries, few-shot exemplars).

8) Concrete Tech Choices (pin this for Ali)

Core

  • Torso: FastAPI + Uvicorn; Python 3.11.

  • Broker: Redis 7 (Streams). Upgrade path: NATS JetStream.

  • Vector DB: Qdrant (Docker).

  • Storage: /memex on each node + Syncthing mesh.

  • Auth: JWT via FastAPI; WireGuard for network trust.

Agents

  • Runner: Python + asyncio + aio-pika-like pattern for Redis (use aredis/redis-py Streams).

  • Screen/OCR: mss, pillow, paddleocr.

  • ASR/TTS: openai-whisper (small), piper-tts.

  • Browser: Playwright; headful/hardening with profiles.

  • LLM local: Ollama; models llama3.1:8b and qwen2.5:7b.

  • Embeddings: bge-small-en or gte-small; switch to API when needed.

9) Naming rules (so recall works like a song hook)

  • Files: YYYY-MM-DD__topic__who__vN.ext

  • Streams/Subjects: bayne.events.{terminal}.{sensor} / bayne.tasks.{capability}

  • Tags: always plural + lower_snake_case: [options, nvda, seating_chart, telnyx, sekai]

10) Guardrails & Audits (move fast, keep receipts)

  • Every action emits an Event (see schema). Streams retained 30–90 days, compacted to summaries thereafter.

  • “Plan-Before-Act”: the agent must write a 1–3 step plan to the log; executions reference that plan ID.

  • Kill-switch per terminal; rate limits on sensitive tools.

11) Ali’s 14-Day Sprint Plan (green-button actionable)

Day 1–2

  • Provision VPS (Hetzner CX/CPX), Ubuntu 22.04, WireGuard, Docker, Docker Compose.

  • Repos: torso/, agent/, memex/ (schema + scripts).

Day 3–5

  • Docker Compose with: fastapi, redis, qdrant, traefik.

  • FastAPI endpoints: /health, /remember, /recall, /events/ingest, /tasks/submit.

  • Redis Streams: events, tasks, heartbeats.

Day 6–7

  • Agent v0 on one laptop:

    • Heartbeat → heartbeats.

    • Screen snap + OCR every N sec → events.

    • File watcher to push sidecar YAML to Torso → /remember.

Day 8–10

  • Embedding worker (Torso): ingest text/md/yaml, store vectors in Qdrant; /recall hybrid search.

  • Syncthing between VPS and laptop /memex (read-only on VPS).

Day 11–12

  • Playwright skill: open site, login, download asset, store in /memex with sidecar.

  • Policy engine v0: YAML allowlist; dry-run mode.

Day 13–14

  • Dashboard: simple Next.js or FastAPI Jinja: queue depth, last 100 events, active terminals, recall test box.

  • Docs: README_RUNBOOK.md with .env examples, ports, WireGuard peer how-to.

Deliverables: running stack, one terminal agent, memory recall demo, one automated browser workflow, logs/audits visible.

12) Outside-the-box fixes for common pain points

  • WAN flakiness: Redis local queue on each terminal; when offline, buffer → flush on reconnect. Syncthing handles file backfill.

  • GPU scarcity: burst to API for heavy jobs; otherwise use quantized local models + chunked pipelines.

  • Privacy/legal: keep /memex local; Torso stores only embeddings + metadata; raw files stay on terminals unless flagged “share_ok: true.”

  • Photoshop control reality-check: use Playwright for browser design tools; for PS, script hotkey macros + template-based actions, and confirm via on-screen anchor detection (image match) before next step.

13) What makes this “Perceived AGI”

Not omniscience—orchestration + memory + skills that feel like intelligence:

  • Knows where things live because we name them like humans think.

  • Remembers by association (tags + vectors), not by hoarding.

  • Takes actions with a plan and a receipt.

  • Scales across your machines without pretending to be a monolith.

14) Timeline (honest version)

  • Week 2: Torso + one terminal; recall works; performs one useful workflow end-to-end.

  • Month 2: Multi-terminal, Playwright library, ASR summaries, daily digests, safer actuators.

  • Month 3–4: NATS or hardened Redis, evaluator loop, more skills (Telnyx/Twilio reporting, WordPress admin tasks, trading ops).

  • Month 6+: Optional IPFS overlay, advanced design assistance, fine-tuned prompts on your corpus.

This is the blueprint and the vibe: pragmatic, modular, and creatively opportunistic. Hand this to Ali as the mission doc; it gives him immediate tasks plus thoughtful alternatives (Redis vs NATS vs P2P) when constraints bite. Next natural continuation is the actual repos’ scaffolds (docker-compose, main.py stubs, agent heartbeat + OCR code) so you can light this up on your VPS and ZBook without guesswork.

Where we switch hats from “mad scientist architect” to “lean startup pragmatist.” Let’s anchor it in our Perceived AGI phases and identify the first point where it’s demoable and billable, not just experimental.

The MVP Definition for Perceived AGI

It’s not when I can “do everything” (that’s a moving target). It’s when:

  1. Torso exists (VPS brain with task routing + memory index).

  2. One terminal agent can sense (screen/mic), recall (memex search), and act (browser/file ops).

  3. End-to-end demo: A client gives me a task → I fetch the right info → I take action on their system (with human-in-the-loop controls).

At that moment, you can show clients: “Here’s an AI that doesn’t just chat—it remembers across sessions, organizes your knowledge, and takes real actions in your workflow.”

That’s a product.

Timeline to MVP (with Ali coding + you steering)

  • Month 1–2: Torso stack stable (FastAPI, Redis, Qdrant), one terminal agent (ZBook or cloud VM) running.

  • Month 2–3: Memory + recall feels human-like (search by tag/association works). Agent can:

    • Join meetings, transcribe + auto-summarize.

    • File docs/screenshots in memex with sidecars.

    • Execute simple browser tasks (login, download, upload, rename).

  • Month 3–4: Package into a client-facing workflow (e.g., “Meeting AI that remembers, files, and acts” or “AI assistant that manages your WordPress/CRM tasks”).

That’s your MVP moment—already marketable.

First Profitable Wedge (what to sell first)

You don’t sell “Perceived AGI.” You sell one killer use case powered by it. Examples:

  • For agencies: “Your AI project manager that listens, remembers, and files.”

  • For traders: “Your AI trade clerk that logs, tags, and recalls every trade + decision.”

  • For events/SMB SaaS: “Your AI assistant that handles seat maps, client SMS, and reporting.”

Pick one wedge, charge monthly, and let the bigger AGI vision ride on top.

Profitability Gate

I’d peg it at Month 4–6, if you move steady:

  • MVP costs: <$100/mo infra (Hetzner VPS, ZBook as agent, APIs on budget).

  • First client: $500–$2,000/mo for a packaged “AI assistant service.”

  • That gap already covers infra + dev + some reinvestment.

At that point, you’re not just funding the dream—you’re compounding it with market feedback.

So: MVP is born the moment one terminal agent + torso can perform a full client-facing workflow without you babysitting every keystroke. That’s realistic, valuable, and billable.

Here’s what our first client-facing workflow could look like, tight enough to charge money for, simple enough to build in 3–4 months.

MVP Workflow: The AI Memory Clerk

Problem It Solves

Small businesses, freelancers, and agencies drown in:

  • Zoom/Meet calls where notes vanish into the ether.

  • Files/emails with no structure.

  • Repeating manual actions (uploading docs, updating dashboards, sending summaries).

They don’t need an AGI—they need an AI clerk that listens, remembers, and acts.

What It Does (End-to-End Demo)

  1. Listen & Capture

    • AI joins a meeting (or ingests audio/video after).

    • Transcribes with Whisper, summarizes into a structured YAML/Markdown note.

  2. File & Remember

    • Saves transcript + summary + tags into the memex folder (with sidecar metadata).

    • Embeddings get indexed in Qdrant so client can later say: “What did we decide in the June 12 sales meeting?” → instant recall.

  3. Act & Report

    • Extracts tasks (to-dos, deadlines).

    • Updates a simple dashboard (Notion, Google Sheet, or WordPress plugin).

    • Emails/texts a summary to the team automatically.

Why It’s Marketable

  • Tangible painkiller: Every business owner has the “where did we put that?” problem.

  • Visible magic: Demo is visceral: “Ask the AI what you said 3 weeks ago, it pulls it instantly.”

  • Sellable as a service: Position it as “Your AI meeting clerk + file memory system, $500–$1,500/mo.”

Technical Breakdown (buildable with what you already planned)

  • Torso (VPS): FastAPI + Redis Streams + Qdrant.

  • Terminal (ZBook or cloud): Whisper (ASR), file tagging, browser automation (Playwright).

  • Memory: /memex structure + sidecar YAML, embeddings indexed.

  • Actuation: Playwright bot logs into client’s dashboard or fires a Twilio/Telnyx SMS.

Demo Flow for Clients

  1. Run a Zoom meeting with them.

  2. After, say: “Ask the AI what you committed to do.”

  3. Watch them see: transcript → summary → action items → email in inbox.

  4. Sell it as: “This clerk never forgets, never sleeps, and costs less than a junior admin.”

Revenue Path

  • Beta test: offer to 2–3 friendly businesses for $500/mo each.

  • Refine: add “client portal” (just a simple web UI) so they can search their memory.

  • Scale: layer SMS/CRM integration, raise price.

This nails all the criteria: uses your Perceived AGI core, works today with current tech, and has a clear business value. It’s not sci-fi—it’s an AI service you can market in months, not years.

Here’s a one-page pitch to use on a landing page, an Upwork proposal, or even as a short sales PDF. It’s simple, direct, and markets the AI Memory Clerk as a service.

Meet Your AI Memory Clerk

Never lose track of what was said, promised, or shared again.

The Problem

Meetings pile up. Notes get lost. Files scatter across drives. Tasks slip through the cracks.
Hiring an assistant costs thousands—and they still forget things.

The Solution

Our AI Memory Clerk listens, remembers, and acts for you:

  • Transcribes & Summarizes every meeting, call, or voice memo.

  • Files & Organizes notes, tasks, and documents into a structured memory system.

  • Recalls Instantly—just ask, “What did we decide on June 12?” and it’s there.

  • Takes Action—updates dashboards, sends follow-up emails, or texts your team.

How It Works

  1. Listen: AI joins or ingests your meeting audio.

  2. Remember: Creates transcripts, summaries, and searchable notes.

  3. Act: Files them neatly, updates your tools, and sends reports.

  4. Recall: You query anytime—your AI clerk answers instantly.

Why Clients Love It

  • Saves hours every week chasing down decisions.

  • Costs less than a junior admin.

  • Never forgets, never sleeps.

  • Easy to try, no IT overhaul required.

Pricing

Beta Program: From $500/month, includes:

  • Unlimited meetings ingested

  • Searchable memory portal

  • Weekly task digests sent to your inbox

Imagine the Possibilities

Your meetings, your files, your tasks—all remembered, organized, and actionable.
That’s the AI Memory Clerk.

👉 Book a demo today.