Offline AI Chat App
A native Android application running on-device large language models completely offline. Delivers private, zero-latency generative AI interactions without internet connectivity or cloud API costs.
Overview
Halo is a fully offline conversational AI mobile application built with Flutter. By running Qwen3.5-2B (quantized Q4_K_M) through llama.cpp (llamadart) and integrating sherpa-onnx with a Zipformer model for offline Vietnamese speech recognition, it generates answers completely on-device without servers, user accounts, or internet permissions.
Product Demo & Walkthrough
Demo videos coming soon — please check back or contact me for a live demo.
Key Features
- 100% on-device local LLM execution using llama.cpp and Qwen3.5-2B (no network permissions requested)
- Offline Vietnamese voice-to-text input powered by sherpa-onnx and Zipformer speech model
- Optimized inference speed: cached system prompt state and 4 prefill threads reducing time-to-first-token to 0.93s
- In-app local model file import with streaming SHA-256 verification and atomic installation
- Deterministic pre-processing in Dart for calendar and arithmetic accuracy
- Local chat history persistence with SQLite and state management with Riverpod
Best For
Privacy-Conscious Users
100% on-device AI inference with zero network traffic, ensuring complete conversation privacy.
Offline Learning & Productivity
Knowledge Q&A, writing assistance, and concept explanations functioning in airplane mode.
Internal Assistants Without Cloud Inference
Zero recurring third-party API subscription costs while keeping internal notes confidential.
Organizations With Limited Connectivity
Field operations in remote regions, ships, construction sites, and network-isolated zones.
Technology Stack
Technology Stack
Supported Platforms
Customization & Delivery Possibilities
- Integration of domain-specific GGUF quantized models (Qwen, Gemma, Phi)
- Custom system prompts and specialized field-inspection assistant behaviors
- Offline voice model customization for specific vocabularies and acoustic conditions
- White-label branding and adaptation for mission-critical offline environments
Interested in Offline AI Chat App?
Reach out to request customization, ask questions about integration, or receive a live walkthrough.
Contact About This Product