Connecting Custom & Local AI Providers to The Job Log Cloud
Want to use open-source models, self-hosted networks, or cost-efficient cloud providers? Our Custom Provider option dynamically adapts to any OpenAI-compatible API format. Follow this guide to connect either your local machine (Ollama) or a third-party cloud provider (OpenRouter, Together AI, Groq).
Option A: Local & Self-Hosted
Expose a local model running on your own computer (Ollama, vLLM, LocalAI) using secure reverse tunnels for 100% free, private, and unlimited generations.
Option B: Cloud Provider
Connect directly to a third-party cloud service (OpenRouter, Together AI, Groq, Anyscale) that implements the standard OpenAI API structure.
Option A: Exposing Local Ollama / vLLM
Since The Job Log operates in the cloud, our servers cannot communicate directly with a literal http://localhost:11434 on your physical computer. To bridge the connection securely, we establish a temporary HTTPS public tunnel.
1. Start Ollama and Pull a Model
Ensure you have Ollama running, and pull your target model from your command line:
ollama pull llama3.12. Configure CORS Origins
Ollama blocks cross-origin requests by default. You must configure it to allow inbound connections:
Close Ollama from system tray, run in PowerShell:
[System.Environment]::SetEnvironmentVariable('OLLAMA_ORIGINS', '*', 'User')Then, relaunch the Ollama app.
Stop any active services, then run in your terminal:
OLLAMA_ORIGINS="*" ollama serve3. Expose Your Local Port Securely
Use a public tunneling tool to create a secure bridge to your port 11434.
If using ngrok (free, requires account):
ngrok http 11434If using localtunnel (requires Node.js, zero setup):
npx localtunnel --port 114344. Input Parameters in The Job Log
Navigate to your AI Settings Panel and enter:
- AI Provider:
🔌 Custom (OpenAI-Compatible) - Base URL: Your public tunnel HTTPS address with
/v1appended (e.g.https://8b4a-12-34-56-78.ngrok-free.app/v1) - Model ID: The exact model name you pulled (e.g.
llama3.1) - Custom API Key: Leave blank or enter any mock text (we default to a safe value internally).
Option B: Connecting a Cloud OpenAI-Compatible Provider
If you do not want to run local models on your own GPU, you can connect directly to cloud aggregators like OpenRouter, Together AI, or Groq. These cloud services connect straight to our cloud backend natively—no ngrok tunnels required!
1. Acquire Credentials from Your Provider
Sign up for your preferred provider and generate an API key:
- OpenRouter: Direct access to Claude 3.5 Sonnet, GPT-4o, Gemini Pro, and Llama 3.
- Together AI: Blazing fast inference speeds on standard open-source weights.
- Groq: Sub-second token generation times for active tailoring.
2. Input Configuration in Settings
Go to the Settings panel and configure the Custom connection:
- AI Provider: Custom
- Base URL:
https://openrouter.ai/api/v1 - Model ID:
meta-llama/llama-3.1-8b-instruct - API Key: Your private OpenRouter key
- AI Provider: Custom
- Base URL:
https://api.together.xyz/v1 - Model ID:
meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo - API Key: Your private Together AI key
3. Encryption & double-envelope Vaults
When saving cloud API credentials, they are encrypted in your browser using standard secure keys and saved securely in your Supabase vault. If you select "Save purely in browser local storage", your credentials remain 100% private in your client cache—they never touch our database.