PYSHA v2

A modular, AI-powered virtual assistant with pluggable engines for speech, language, and automation.

PYSHA started in 2017 as a student project stitched together around Google STT, pyttsx, ChatterBot, and a hundred small HTML-scraping hacks. Version 2 is a ground-up rewrite that keeps the original spirit — a voice-first, pluggable, “do-stuff-for-me” assistant — but modernises every layer: async Python 3.11+, LLM-powered conversation, neural TTS, Whisper STT, a typed plugin system, a FastAPI web UI, a Rich terminal UI, and Docker packaging.

Highlights

LLM-powered — drop-in support for Anthropic Claude, OpenAI GPT, or local models via Ollama.
Modular engines — every STT, TTS, and LLM backend is a plugin.
Open core, closed extensions — third-party engines can ship under any license via Python entry-points. No fork required.
Skill-based commands — weather, Wikipedia, Wolfram Alpha, web search, news, system control, date/time. Add your own with ~30 lines of code.
Voice or text — speak to it, type to it, or call it from HTTP / WebSocket.
Persistent + short-term memory — async SQLite long-term store plus a 7-item sliding context window (a nod to Miller’s Law, carried over from v1).
First-class observability — structured logging (structlog) and an async event bus for UIs and telemetry.
Typed, tested, packaged — Pydantic settings, pyproject.toml, pytest/ruff/mypy, Docker, GitHub Actions.

Architecture

┌────────────────┐   text / voice   ┌────────────────────────────────────┐
│  CLI / Web UI  │ ────────────────▶│            Assistant               │
└────────────────┘                  │  (orchestrator + EventBus + memory)│
                                    └──────┬──────────────┬──────────────┘
                                           │              │
                 ┌─────────────────────────┤              ├─────────────────┐
                 ▼                         ▼              ▼                 ▼
          ┌────────────┐           ┌──────────────┐ ┌───────────┐   ┌───────────────┐
          │ STT Engine │           │  TTS Engine  │ │ LLM Engine│   │    Skills     │
          │ (protocol) │           │  (protocol)  │ │ (protocol)│   │  (protocol)   │
          └────────────┘           └──────────────┘ └───────────┘   └───────────────┘
          whisper · google           edge · pyttsx   anthropic        weather · news
          *your-engine*              *your-engine*   openai · ollama  wikipedia · …
                                                     *your-engine*    *your-skill*

Everything below the protocol line is a plugin discovered at runtime via entry-point groups. The assistant itself never imports a specific engine.

Quickstart

1. Install

# Clone, create a venv, install the open-source defaults + a Claude backend:
git clone https://github.com/shafaypro/pysha && cd pysha
python -m venv .venv && source .venv/bin/activate
pip install -e ".[llm-anthropic,tts-edge,skills-web,skills-wikipedia,skills-news]"

2. Configure

cp .env.example .env
# edit .env — set PYSHA_ANTHROPIC_API_KEY at minimum.

3. Run

pysha chat          # text chat in your terminal
pysha chat --speak  # also speaks replies aloud
pysha listen        # voice conversation via microphone
pysha web           # FastAPI + WebSocket UI at http://127.0.0.1:8000
pysha plugins       # show every discovered engine / skill

Configuration

All settings are environment variables with the PYSHA_ prefix, or in a .env file in the working directory.

Key	Default	Description
`PYSHA_STT_ENGINE`	`google`	STT plugin name
`PYSHA_TTS_ENGINE`	`edge`	TTS plugin name
`PYSHA_LLM_ENGINE`	`anthropic`	LLM plugin name
`PYSHA_ANTHROPIC_API_KEY`	—	Claude API key
`PYSHA_OPENAI_API_KEY`	—	OpenAI API key
`PYSHA_OLLAMA_BASE_URL`	`http://localhost:11434`	Local Ollama host
`PYSHA_TTS_VOICE`	`en-US-AriaNeural`	Edge TTS voice
`PYSHA_WOLFRAM_APP_ID`	—	Wolfram Alpha key
`PYSHA_WEB_HOST` / `PYSHA_WEB_PORT`	`127.0.0.1:8000`	Web UI bind

See src/pysha/config.py for the full list.

Writing a plugin (open or closed source)

PYSHA’s core is MIT-licensed. Your plugin is not. You can ship it under any license you like — GPL, Apache, proprietary, commercial — and PYSHA will discover it automatically if you declare an entry-point.

Example: a custom STT engine

# my_package/stt.py
from pysha.core.engine import STTEngine, Transcript

class MyCompanySTT:
    name = "my-company-stt"

    async def start(self):  ...
    async def stop(self):   ...

    async def transcribe(self, audio_bytes, *, language="en") -> Transcript:
        # call your proprietary service here
        return Transcript(text="hello world")

    async def stream(self, *, language="en"):
        yield Transcript(text="streamed chunk")

# my_package/pyproject.toml
[project.entry-points."pysha.engines.stt"]
my-company-stt = "my_package.stt:MyCompanySTT"

Then:

pip install my-package                 # your proprietary wheel
PYSHA_STT_ENGINE=my-company-stt pysha chat

The same pattern works for pysha.engines.tts, pysha.engines.llm, and pysha.skills. See docs/PLUGINS.md for a full authoring guide.

Skills

Built-in skills (all optional, installed with extras):

Skill	Trigger examples	Extra
`datetime_skill`	“what’s the time?” · “today’s date”	—
`weather`	“weather in Paris” · “temperature in Tokyo”	— (uses Open-Meteo)
`wikipedia`	“tell me about quantum physics”	`skills-wikipedia`
`web_search`	“search for site reliability engineering”	`skills-web`
`news`	“what’s in the news?” · “headlines”	`skills-news`
`wolfram`	“compute integral of sin(x) dx”	`skills-wolfram`
`system_control`	“open calculator” · “close chrome”	`skills-system`

Unmatched utterances fall through to the configured LLM.

Docker

docker build -t pysha .
docker run --rm -it --env-file .env -p 8000:8000 pysha
# or
docker compose up

Development

pip install -e ".[dev]"
ruff check src tests
pytest -q

src/pysha/
├── __main__.py          # CLI entry
├── app.py               # Assistant orchestrator
├── config.py            # Pydantic settings
├── core/                # Protocols, plugin loader, event bus, memory
├── engines/
│   ├── stt/             # whisper, google, (your-engine)
│   ├── tts/             # edge, pyttsx, (your-engine)
│   └── llm/             # anthropic, openai, ollama, (your-engine)
├── skills/              # datetime, weather, wikipedia, wolfram, …
├── ui/                  # cli.py (Rich), web.py (FastAPI)
└── utils/

Migrating from v1

The v1 monolith (2123-line __PYSHA.py et al.) has been preserved under legacy/ so old behaviours remain inspectable and reusable. Features you relied on in v1 map to v2 as follows:

v1 module	v2 equivalent
`__Voice.py` + `pyttsx`	`engines.tts.edge` / `engines.tts.pyttsx`
`speech_recognition`	`engines.stt.google` / `engines.stt.whisper`
`ChatterBot` corpus	LLM (Anthropic / OpenAI / Ollama)
`_WolFrameAlphaClass.py`	`skills.wolfram`
`_YouTube.py`, `__github.py`, …	`skills.web_search` (or custom skill)
`__shorttermmemory.py` (7 ± 2)	`core.memory.ConversationMemory`
`_dbdata.py` (SQLite)	`core.memory.MemoryStore` (async)
Tkinter `__init__.py`	`ui.cli` (Rich) + `ui.web` (FastAPI)

Licensing

Core (this repository): MIT — free for open-source and commercial use.
Plugins: author’s choice. Proprietary, closed-source engines are a first-class citizen: install the wheel, set the env var, run PYSHA.

See LICENSE for the full dual-licensing notice.

Credits

Originally created in 2017 by Shafay as an undergraduate project. v2.0 is a modernising rewrite — new stack, new architecture, same spirit.