APEX Overview

APEX is an open-source, notebook-first AI model runtime platform. It deploys open-source machine learning models using Google Colab or local compute resources and exposes them through an accessible web interface and an OpenAI-compatible REST API.

What is APEX?

Running modern open-source language models typically requires configuring inference engines, managing GPU memory allocations, securing tunnels, generating credentials, and adapting API client contracts.

APEX simplifies this entire operational pipeline into a single, clean Jupyter notebook interface. Users select an open-source model, allow automatic hardware detection to determine optimal precision, and launch a complete production-grade runtime with one click.

How It Works

APEX operates directly on user-controlled hardware. The architecture follows a strict, layered dataflow:

Google Colab / Local Runtime
          │
          ▼
      APEX Runtime
          │
          ▼
   Loaded AI Model
          │
          ▼
   Public API / Web Chat
          │
          ▼
   User Applications

Once booted, APEX establishes a secure Cloudflare Quick Tunnel, providing a verified public HTTPS URL without requiring manual port forwarding, DNS configuration, or account registrations.

Verified Capabilities

• Notebook-First Control: A clean two-section notebook interface (Setup & Run APEX and Shutdown APEX) eliminates terminal clutter and exposes zero internal classes.
• Hugging Face Integration: Download, verify, and cache open-source weights using model repository IDs or direct Hugging Face URLs.
• OpenAI-Compatible REST API: Native /v1/chat/completions and /v1/models endpoints allowing immediate integration with Continue, Cline, Roo Code, Aider, Open WebUI, and standard OpenAI SDKs.
• Streaming Inference: Real-time token streaming via Server-Sent Events (SSE) with standard data: [DONE] termination.
• Web Chat Interface: A browser-accessible chat application available at /chat with live markdown rendering and attachment previews.
• Document Ingestion: Ingest text, code, PDF, Word documents, spreadsheets, and structured data via /v1/files for conversational context.
• Automated Hardware Detection: Real-time inspection of CUDA accelerators, Apple Silicon MPS, system RAM, and GPU VRAM with precision tier recommendations.
• Graceful Lifecycle Management: Single-click resource release, VRAM cache eviction, and safe process shutdown.