# PromptLab – AI Prompt‑Engineering Experiment Tracker

## 1️⃣ Overview & Quick Start  
> 💡 Tip: Duplicate the entire page hierarchy into your own workspace before adding any data – this preserves all relations and view configurations.  

PromptLab is a ready‑to‑duplicate Notion workspace that lets you:

* Organize **Projects → Experiments → Prompt Versions** in a clean hierarchy.  
* Auto‑capture every OpenAI model run (tokens, cost, latency) via a simple webhook or Zapier integration.  
* Roll‑up key metrics (average quality, total cost, latency) at every level.  
* Visualize a **Version‑vs‑Quality matrix** to spot the most efficient prompts.  
* Export a one‑click PDF report for stakeholders.

---

## 2️⃣ Projects Database  

| Project Name | Owner | Status | Start Date | End Date | Budget ($) | Experiments (Rollup) | Avg. Quality (Rollup) |
|--------------|-------|--------|------------|----------|------------|----------------------|-----------------------|
| Customer Support Bot | Alice Chen | 🚀 In‑Progress | 2024‑04‑01 | 2024‑09‑30 | 12,000 | 3 | 84 |
| Content Generation Suite | Marco Ruiz | 🟡 Planning | 2024‑05‑15 | 2024‑12‑15 | 8,500 | 0 | – |
| Data‑Extraction API | Priya Singh | ✅ Completed | 2023‑11‑01 | 2024‑02‑28 | 5,200 | 5 | 91 |

> 💡 Tip: Use the **Status** select options (Planning, In‑Progress, Completed, On Hold) to filter the Projects board view automatically.

### Views (pre‑configured)

* **Board – By Status** – Kanban view for quick status tracking.  
* **Table – Financials** – Shows Budget, Total Cost (rollup from Prompt Versions), and Cost per Experiment.  
* **Timeline – Roadmap** – Visual timeline of start/end dates.

---

## 3️⃣ Experiments Database  

| Experiment Title | Project (Relation) | Goal | Status | Start Date | End Date | Prompt Versions (Rollup) | Avg. Quality (Rollup) |
|------------------|--------------------|------|--------|------------|----------|--------------------------|-----------------------|
| FAQ Retrieval v1 | Customer Support Bot | Reduce false positives in FAQ matching | 🚀 In‑Progress | 2024‑04‑05 | 2024‑05‑20 | 2 | 78 |
| Blog Outline Generator | Content Generation Suite | Generate SEO‑friendly outlines in <30 s | 🟡 Planning | 2024‑06‑01 | 2024‑07‑15 | 0 | – |
| Invoice Parsing v2 | Data‑Extraction API | Improve accuracy on multi‑page PDFs | ✅ Completed | 2023‑11‑10 | 2024‑01‑30 | 5 | 94 |

> 💡 Tip: Link each Experiment to its parent Project via the **Project** relation – this enables automatic roll‑ups of experiment counts and average quality on the Project page.

### Views (pre‑configured)

* **Gallery – Experiments** – Card view with status badge and key metrics.  
* **Table – Metrics** – Shows roll‑up of total tokens, cost, and latency per experiment.  
* **Calendar – Deadlines** – Highlights upcoming end dates.

---

## 4️⃣ Prompt Versions Database  

| Version ID | Experiment (Relation) | Prompt Text (Preview) | Model | Temperature | Tokens Used | Cost ($) | Latency (ms) | Quality Score (0‑100) | Run Date |
|------------|-----------------------|-----------------------|-------|-------------|-------------|----------|--------------|-----------------------|----------|
| PV‑001 | FAQ Retrieval v1 | “Given a user query, retrieve the most relevant FAQ from …” | gpt‑4o | 0.2 | 1,240 | 0.018 | 420 | 81 | 2024‑04‑12 |
| PV‑002 | FAQ Retrieval v1 | “Summarize the top‑3 matching FAQs for …” | gpt‑4o | 0.0 | 1,080 | 0.016 | 380 | 75 | 2024‑04‑18 |
| PV‑003 | Invoice Parsing v2 | “Extract line‑item tables from multi‑page PDF invoices …” | gpt‑4‑turbo | 0.1 | 2,560 | 0.037 | 620 | 96 | 2024‑01‑22 |
| PV‑004 | Invoice Parsing v2 | “Identify vendor name, date, total amount from PDF …” | gpt‑4‑turbo | 0.0 | 2,300 | 0.033 | 560 | 92 | 2024‑01‑25 |
| PV‑005 | Invoice Parsing v2 | “Classify each line‑item into expense categories …” | gpt‑4‑turbo | 0.3 | 2,800 | 0.041 | 710 | 94 | 2024‑01‑28 |

> 💡 Tip: Set up an **Automation** (Zapier, Make, or Notion API) that posts each OpenAI API response to this database, filling the Tokens, Cost, Latency, and Quality Score fields automatically.

### Views (pre‑configured)

* **Table – All Versions** – Full list with filters for Model, Temperature, or Quality Score.  
* **Board – By Model** – Kanban columns for each model (gpt‑4o, gpt‑4‑turbo, etc.).  
* **Scatter Plot – Quality vs. Cost** – Visual matrix (requires Notion’s built‑in chart block or an embedded Google Sheet).

---

## 5️⃣ Dashboard & Visual Insights  

### 5.1 Version‑vs‑Quality Matrix (Embedded Chart)  
> 💡 Tip: Use the **Scatter Plot** view on the Prompt Versions table, set **X‑axis** = Cost, **Y‑axis** = Quality Score, and color‑code by Model. This instantly shows which prompts give the best ROI.

### 5.2 Project‑Level KPI Summary (Linked Database)  

| Project | Total Cost ($) | Avg. Latency (ms) | Avg. Quality | Experiments |
|---------|----------------|-------------------|--------------|-------------|
| Customer Support Bot | 0.034 | 400 | 78 | 3 |
| Content Generation Suite | – | – | – | 0 |
| Data‑Extraction API | 0.131 | 630 | 94 | 5 |

> 💡 Tip: Pin this KPI table to the top of the **Workspace Home** page for executive visibility.

### 5.3 Stakeholder Reporting Page  
A ready‑to‑print page that pulls in:

* Project overview (linked Project page).  
* Experiment summary table.  
* Version‑vs‑Quality chart (exported as PNG).  
* **Export → PDF** button (via Notion’s built‑in “Export” command – one click from the page menu).

---

## 6️⃣ One‑Click PDF Export Setup  

1. Add a **Button** block on each Project page titled **“Export Report PDF”**.  
2. Set the button action to **“Export → PDF”** (available in Notion v2.15+).  
3. In the button’s **Advanced Settings**, enable **Include sub‑pages** so Experiments and Prompt Versions are bundled.  
4. Optionally, use a **Template Button** to duplicate a pre‑formatted “Stakeholder Report” page before exporting.

> 💡 Tip: Name the exported file using the pattern `ProjectName_YYYYMMDD.pdf` – you can automate this with a small Notion API script if you need batch exports.

---

## 7️⃣ How to import into Notion  

1. **Create a new page** in your Notion workspace where you want PromptLab.  
2. Click **“Import” → “Markdown & CSV”**, then **upload this .md file**.  
3. Notion will create a hierarchy of pages matching the H2 headings.  
4. For each table labeled as a **Database** (Projects, Experiments, Prompt Versions, KPI Summary), click the three‑dot menu on the table → **“Turn into → Database”**.  
5. Open each newly created database, verify that **Relation** and **Rollup** properties are correctly linked (e.g., Experiments → Project, Prompt Versions → Experiment). Adjust property types if needed.  
6. Duplicate the **Dashboard** page into your workspace’s sidebar for quick access.  
7. (Optional) Set up your **OpenAI webhook / Zapier** integration to auto‑populate Prompt Versions. Follow the integration guide in the “Automation” section of each database.  

Your PromptLab workspace is now live – start adding Projects, spin up Experiments, and watch the metrics roll up automatically!