Getting Started

This guide walks you through launching APEX in Google Colab, starting a model, and accessing both the interactive web chat interface and the OpenAI-compatible REST API.

1. Launch the Notebook

Open the official APEX notebook in Google Colab:

▶ Open APEX in Google Colab

In Google Colab, verify that a GPU runtime is selected: Runtime → Change runtime type → Hardware accelerator → T4 GPU (or A100/V100 if subscribed to Colab Pro).

2. Standard User Workflow

The notebook contains exactly two main sections:

Step A: Run Section 1 (Setup & Run APEX)

Execute the code cell in Section 1 — Setup & Run APEX. This automatically sets up the environment, verifies dependencies, and renders the APEX Launcher card.

Step B: Review Detected Hardware

The launcher inspects your compute environment and displays:

• Runtime Environment: Google Colab or Local Environment
• Accelerator: e.g. NVIDIA Tesla T4 (CUDA)
• GPU VRAM: Total available video memory (e.g. 15.0 GB)
• System RAM: Available host memory
• Compute Device: cuda:0, mps, or cpu

Step C: Select Model & Precision

• Preset Dropdown: Select from curated models like Qwen/Qwen2.5-0.5B, Qwen/Qwen2.5-1.5B, deepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B, or choose Custom Hugging Face Model or URL...
• Precision: Set to Auto (recommended) or explicitly choose float16, bfloat16, or float32.
• Hugging Face Token: Optional. Required only if you select gated models (e.g. Meta LLaMA 3.2).

Step D: Click START APEX

Click the green START APEX button. The launcher executes the 10-stage deployment pipeline in the background:

1. Environment Initialization: Prepares workspace and cache paths.
2. Dependency Verification: Checks core packages (PyTorch, Transformers, FastAPI).
3. Hardware Detection: Verifies accelerator availability and compatibility.
4. Model Resolution: Resolves Hugging Face repository snapshot.
5. Model Download: Downloads weights with cached resume support.
6. Model Loading: Allocates model weights into GPU VRAM.
7. Engine Initialization: Warms up tokenizer and generation pipeline.
8. API Startup: Binds Uvicorn server to internal port.
9. Health Verification: Checks local /health endpoint for readiness.
10. Tunnel Verification: Acquires and verifies the Cloudflare Quick Tunnel URL.

3. Connecting to Your Deployment

When the status badge shows ● APEX is running, the Connection Panel renders:

• Web Chat Interface: Click the green Open APEX Chat ↗ button to open the browser-based chat app in a new tab.
• OpenAI API Base URL: Copy the public endpoint (e.g. https://<subdomain>.trycloudflare.com/v1).
• Active API Key: Click the reveal/hide toggle to display your secure auto-generated API key or copy it directly.
• Connect Your Application: Expand the code snippet accordion to copy a pre-populated Python script using the official openai SDK.

4. Hardware & GPU Realities

Model loading depends strictly on available video memory (VRAM). Before selecting a model, review the memory benchmarks below:

5. Graceful Teardown

When you are finished with your session, scroll down to Section 2 — Shutdown APEX and click STOP APEX.

APEX closes the Cloudflare tunnel, terminates the Uvicorn server, unloads model weights from GPU memory, triggers CUDA cache evacuation, and confirms: "APEX has been safely shut down. All resources released."