Autonomous AI Microservice (v3.5.0)

ML Backend Engine

High-performance FastAPI service running local GGUF Large Language Models (Qwen 2.5 3B Instruct via llama.cpp) alongside agile cloud LLMs (DeepSeek, Google Gemini). Powers the conversational AI financial advisor (/api/v1/chat) with live financial history aggregation.

FastAPI 0.110+ Conversational AI Llama.cpp / DeepSeek Live Telemetry
Model Setup Guide Hardware Specs
Port 9050 /api/v1/chat GGUF Quantized 100% On-Premise
Local Inference & Financial Chat

FastAPI Worker + llama-server Daemon + Dynamic LLM Router

Why Local LLMs Beat Regex Rules

Traditional regex parsers fail as formats change. A localized LLM understands human syntax, slang, and evolving banking formats.

Brittle Regex Parsers
  • Crash when Safaricom alters spacing, words, or adds new disclosures.
  • Incapable of parsing new financial institutions (e.g. SACCOs, micro-lenders).
  • Zero context awareness — cannot distinguish transaction notes from amounts.
  • High maintenance burden with dozens of error-prone regular expressions.
Semantic GGUF LLM Engine
  • Infers semantic intent across English, Swahili, and Sheng terminology.
  • Adapts seamlessly to new templates and unexpected transaction types.
  • Extracts structured JSON: counterparty, amount, fees, balance, and timestamps.
  • 100% private and on-premise — zero API keys or per-token fees.

Two-Stage Processing Architecture

Maximizing throughput by pairing lightning-fast heuristics with deep semantic reasoning.

1. Raw Loot Batch

Encrypted SMS batches are ingested from the Android queue into tbl_Loot.

2. Pre-Classification

Verified financial senders (MPESA, banks) are identified instantly in sub-milliseconds.

3. GGUF Extraction

The local Qwen model extracts structured fields into standard JSON under strict temperature controls.

4. Normalized Ledger

Transactions are committed into tbl_Transactions with full audit logs and job metrics.

Enterprise Control & Telemetry

SuperAdmins manage models, tune prompts, and monitor container diagnostics in real time.

Live Telemetry Endpoint (/admin/telemetry)

Exposes real-time container vitals for high-availability production monitoring:

  • Process RSS Headroom: Tracks exact memory footprints of FastAPI and llama-server.
  • Active Model Metrics: Displays model parameters, quantization level, and context size.
  • Health & Uptime: Instant heartbeat alerts for process crashes or memory pressure.
Hugging Face Model Presets

Manage models effortlessly directly through the WebApp dashboard:

  • One-Click Presets: Pre-configured downloads for Qwen 2.5 1.5B/3B, Llama 3.2, and Mistral.
  • Hardware Safeguards: Verifies disk space and RAM headroom before starting downloads.
  • Hot-Switching: Switch active GGUF models on the fly with graceful daemon restarts.
Dynamic Prompt Versioning

Tune system prompts and output schemas without modifying application code:

  • Versioned History: Every prompt edit creates a traceable snapshot in MySQL.
  • Fallback Safeguard: Automatically falls back to verified defaults if no active DB prompt is set.
  • Prompt Optimization: Adjust instruction framing for specific East African banking formats.
Batch Rescan & Full Reset

Reprocess historical data safely with interactive modal dialogs:

  • Incremental Rescan: Analyzes newly synced or unclassified SMS batches.
  • Full Reset: Safely clears previous inferences and re-runs current models across all records.
  • Live Progress Polling: Real-time progress bars prevent timeouts during heavy multi-thousand SMS batches.
\n