Configuration Reference
This reference describes all active configuration options supported by the APEX runtime. Settings can be configured interactively through the notebook Launcher or programmatically via configuration files.
Model Configuration
Hugging Face Model ID or URL
APEX downloads model weights directly from the Hugging Face Hub. The input field supports multiple formats:
- Standard Repo ID:
Qwen/Qwen2.5-1.5B,google/gemma-2-2b-it - Full Repository URL:
https://huggingface.co/Qwen/Qwen2.5-1.5B-Instruct - URL with Branch Tree:
https://huggingface.co/meta-llama/Llama-3.2-1B/tree/main - Single-Word Models:
gpt2
APEX automatically normalizes these inputs, stripping trailing slashes, .git suffixes, and branch paths.
Hugging Face Access Token
Required only when deploying gated models (such as Meta LLaMA 3.2 or Mistral) or private organization repositories. Enter your token (starts with hf_...). In the notebook UI, this input is password-masked to protect your credentials.
Precision & VRAM Allocation
The precision setting directly dictates how model weight tensors are cast in memory:
| Precision | Bytes / Weight | Memory Impact | Recommended Use |
|---|---|---|---|
Auto (Default) |
16-bit / 32-bit | Optimized | Selects float16 when a CUDA accelerator is detected; float32 on CPU. |
float16 |
2 bytes | -50% vs fp32 | Standard on NVIDIA Volta, Turing, Ampere (Colab T4, V100). |
bfloat16 |
2 bytes | -50% vs fp32 | Preferred on Ampere/Hopper (A100, H100, RTX 3000+) for numerical stability. |
float32 |
4 bytes | Baseline (High) | Required only when running on CPU without half-precision support. |
Advanced Settings
Advanced options are accessible through the collapsible Advanced Settings accordion in the notebook Launcher:
| Setting | Default | Description |
|---|---|---|
| API Port | 8000 |
Internal port for the local Uvicorn/FastAPI server (allowed range: 1024–65535). |
| Enable Authentication | True |
Enforces Bearer token verification on all protected inference and file endpoints. |
| Custom API Key | Auto-generated | Leave blank to allow APEX to generate a cryptographically secure token (APEX_...), or enter your own secret string. |
| Transport | cloudflare |
cloudflare initializes a public Cloudflare Quick Tunnel. local restricts binding to 127.0.0.1. |
| Max Tokens | 512 |
Upper limit on new tokens generated per completion request (slider range: 64–4096). |
| Temperature | 0.70 |
Sampling temperature controlling randomness (slider range: 0.00–2.00). |
Model Caching & Storage Architecture
APEX caches model files locally within the runtime filesystem (e.g. /content/cache on Colab, or ./cache locally):
- Fast Local SSD Caching: Weights downloaded from Hugging Face are stored in local high-speed storage. Re-running the model in the same runtime session verifies cache integrity instantly without re-downloading.
- Google Drive vs. Local Storage: Storing large multi-gigabyte models directly on Google Drive via Colab's FUSE mount creates severe network I/O bottlenecks and frequent disconnects. APEX uses fast local runtime storage for models by default, and reserves Google Drive strictly for lightweight conversation workspaces.