Level 1 Tool: Prompt Lab — Fundamentals Playground
Our first learning tool is live. Prompt Lab lets you test prompts across multiple models and parameters in real time.
What It Does
1. Prompt Comparison
Write a prompt, run it across 4+ models simultaneously:
- GPT-4o
- Claude 3.5 Sonnet
- Gemini 1.5 Flash
- Llama 3.1 70B
See how each model interprets the same prompt differently.
2. Parameter Playground
Adjust parameters and see immediate effects:
- Temperature: 0.0 (deterministic) to 2.0 (creative)
- Top-p: 0.0 (narrow) to 1.0 (broad)
- Top-k: Vocabulary restriction
- Max tokens: Output length limit
3. Token Analyzer
Before you run:
- Token count for your prompt
- Estimated cost per model
- Context window usage
- Tokenization visualization
4. Output Comparison
Side-by-side comparison with:
- Response time
- Token count
- Cost
- Quality score (length, structure, relevance)
How to Use It
Quick Test
- Enter a prompt
- Select models to compare
- Click “Run”
- Analyze results
Parameter Sweep
- Set parameter range
- Run multiple iterations
- Compare output diversity
- Find optimal settings
Cost Estimation
- Enter your prompt
- See token count and cost per model
- Estimate monthly usage
- Find cheapest model for your quality bar
Implementation
Built with:
- Astro for the static site
- React for the interactive playground
- OpenRouter API for model access
- Tokenizers for accurate token counting
Open source — contributions welcome.
What’s Next
Level 2: Prompt Engineer Tool
- Visual chain builder
- Multi-step prompt sequencing
- Automatic output evaluation
- Prompt version control
Level 3: RAG Builder
- Document upload and chunking
- Embedding model comparison
- Vector database integration
- RAG evaluation metrics
The Learning Philosophy
Each level follows the same pattern:
- Learn the concept — Read the blog post
- Build the tool — Implement it yourself
- Test with real models — Use our deployed version
- Share your findings — Contribute back to the community
The goal isn’t just to read about AI — it’s to build intuition through hands-on experimentation.
Detailed Analysis
Prompt Comparison
CoreRun the same prompt across multiple models simultaneously. See how GPT-4, Claude, Gemini, and Llama respond differently.
Parameter Playground
CoreAdjust temperature, top-p, top-k in real-time. See how randomness and sampling affect outputs.
Token Analyzer
CoreCount tokens, estimate costs, and understand how your prompt is tokenized across different models.
Output Comparison
EvaluationSide-by-side comparison of outputs with quality scoring and automatic metrics.
Prompt Crafting
Learn to write effective prompts by testing variations
Model Intuition
Build intuition for how different models behave
Parameter Sensitivity
Understand how parameters affect outputs
Cost Awareness
Estimate costs before running large experiments
OpenRouter
Free tier availableAccess 200+ models through one API. Powers the Prompt Lab.
Best for testing across many models
Local Models
Free (local compute)Run models locally via Ollama or LM Studio.
Best for privacy and unlimited testing
- +Free to use — no signup required
- +Test across multiple models instantly
- +Visual parameter tuning
- +Token counting and cost estimation
- +Side-by-side output comparison
- −Requires OpenRouter API key for cloud models
- −Limited free tier without API key
- −Local models require powerful hardware
- −No persistence — refresh loses work
Prompt Lab is completely free. Uses OpenRouter's free tier for API access. Local models are unlimited.