You can not select more than 25 topics
Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
167 lines
9.0 KiB
167 lines
9.0 KiB
---
|
|
title: "feat: Category domain decomposition for Overton report"
|
|
type: feat
|
|
status: active
|
|
date: 2026-06-15
|
|
---
|
|
|
|
# feat: Category domain decomposition for Overton report
|
|
|
|
## Summary
|
|
|
|
Add 3 new Plotly charts and a narrative section to `reports/overton_window/overton_window.qmd` that decompose the Overton shift by policy category (asiel/vreemdelingen, landbouw/natuur, energie/klimaat, etc.), making visible which domains drove the shift, which resisted it, and how each category's centrist support evolved over time. The yearly CS chart gains category filtering via Plotly dropdown menu.
|
|
|
|
---
|
|
|
|
## Problem Frame
|
|
|
|
The current Overton report treats all right-wing motions as a single aggregate. The synthesis mentions migration vs non-migration, but with 10 categories now populated across all 3,030 motions, we can show a richer picture: energie/klimaat had the second-largest CS surge (+0.107), landbouw/natuur actually declined (−0.063), and onderwijs/wetenschap barely moved (+0.053) despite being the highest-consensus domain. These stories are invisible in the current charts.
|
|
|
|
---
|
|
|
|
## Requirements
|
|
|
|
- R1. Add a horizontal bar chart showing pre/post centrist support delta per category, sorted by magnitude
|
|
- R2. Add category filtering to the yearly CS timeline (Chart 1) via Plotly dropdown menu, so users can view a single category or "All right-wing"
|
|
- R3. Add domain trajectories to the quarterly chart showing 4-5 key categories as separate lines
|
|
- R4. Add a "Domain Decomposition" narrative section with the category delta table and interpretive prose
|
|
- R5. All new charts must use the existing Plotly styling conventions (colors, template, height)
|
|
|
|
---
|
|
|
|
## Scope Boundaries
|
|
|
|
- No changes to the synthesis markdown or other analysis scripts
|
|
- No changes to the DB or data pipeline
|
|
- No interactive filtering beyond Plotly's built-in updatemenu (no Streamlit/JS)
|
|
|
|
---
|
|
|
|
## Key Technical Decisions
|
|
|
|
- **Plotly updatemenu for filtering:** Uses Plotly's built-in `updatemenus` with `buttons` to toggle between "All right-wing" (current line) and individual categories. No external JS or Dash needed.
|
|
- **Color scheme for categories:** Use a qualitative 10-color palette (Plotly `alphabet` or `set2`), not PARTY_COLOURS, to avoid confusion with party lines.
|
|
- **New chart cells inserted after existing Chart 1:** Category delta bar chart goes after the yearly CS chart (Chart 1) and its narrative. Domain trajectories go in the quarterly section (Chart 6). A new "Domain Decomposition" section links them.
|
|
|
|
---
|
|
|
|
## Open Questions
|
|
|
|
### Resolved During Planning
|
|
|
|
- **Standalone HTML filtering:** Plotly updatemenu works in standalone HTML (confirmed via existing usage in the repo — the Overton report is a standalone HTML file with embedded Plotly).
|
|
|
|
### Deferred to Implementation
|
|
|
|
- **Whether to show all 10 categories or a curated subset:** The quarterly trajectories chart should show at most 5-6 lines to avoid visual clutter. The delta bar chart can show all 10.
|
|
- **Exact Plotly color assignment to each category:** Will match categories to a qualitative palette at implementation time.
|
|
|
|
---
|
|
|
|
## Implementation Units
|
|
|
|
- U1. **[Add category delta bar chart and dropdown-filtered yearly CS]**
|
|
|
|
**Goal:** Two linked chart additions: (1) a category delta horizontal bar chart, (2) convert Chart 1's yearly CS line chart to support category filtering via Plotly updatemenu.
|
|
|
|
**Requirements:** R1, R2, R5
|
|
|
|
**Dependencies:** None
|
|
|
|
**Files:**
|
|
- Modify: `reports/overton_window/overton_window.qmd`
|
|
|
|
**Approach:**
|
|
- Compute yearly CS per category via SQL: `SELECT year, category, AVG(centrist_support_strict) as cs, COUNT(*) as n FROM right_wing_motions WHERE classified = TRUE GROUP BY year, category`
|
|
- For the delta chart (new cell): compute pre/post CS per category, then use a horizontal bar (go.Bar with orientation='h'), sorted by delta descending. Color bars: green for positive delta, red for negative.
|
|
- For Chart 1 modification: extend the existing yearly SQL to GROUP BY year, category. In Python, pivot to get per-category columns. Create a go.Figure with all categories as traces plus an aggregate "All right-wing" trace. Add an updatemenu dropdown with buttons: "All right-wing" (restores all visible with only the aggregate line shown) and each category name (shows only that category's trace).
|
|
- Keep the existing pre/post mean lines and the break-year vertical line intact. The dropdown only controls which category trace is visible.
|
|
- Use a 10-color qualitative palette from plotly.express.colors.qualitative.
|
|
|
|
**Patterns to follow:**
|
|
- Existing Chart 1 for SQL pattern, figure layout, annotation style
|
|
- Existing Chart 6 for quarterly trajectory styling
|
|
|
|
**Test scenarios:**
|
|
- N/A — this is a Quarto document rendering change. Verify by rendering the QMD and checking that (a) the delta chart shows all 10 categories, (b) the dropdown in Chart 1 cycles through categories correctly, (c) pre/post mean lines remain visible in all views.
|
|
|
|
**Verification:**
|
|
- `uv run quarto render reports/overton_window/overton_window.qmd` succeeds
|
|
- `overton_report.html` contains the new delta chart section and Chart 1 responds to dropdown interaction
|
|
|
|
---
|
|
|
|
- U2. **[Add domain trajectories to quarterly chart]**
|
|
|
|
**Goal:** Enhance the existing quarterly chart (Chart 6) by overlaying 4-5 key category lines alongside the aggregate.
|
|
|
|
**Requirements:** R3, R5
|
|
|
|
**Dependencies:** U1 (the SQL for per-category yearly CS shares the same data approach; the quarterly chart needs its own SQL)
|
|
|
|
**Files:**
|
|
- Modify: `reports/overton_window/overton_window.qmd`
|
|
|
|
**Approach:**
|
|
- Select 5 categories to show as individual lines: asiel/vreemdelingen, energie/klimaat, buitenland/europa, landbouw/natuur, economie. These are the categories with largest deltas, largest volumes, or most story value.
|
|
- Compute quarterly CS per category: `SELECT EXTRACT(YEAR FROM m.date) AS y, CEIL(EXTRACT(MONTH FROM m.date) / 3.0) AS q, r.category, AVG(r.centrist_support_strict) AS cs, COUNT(*) AS n FROM right_wing_motions r JOIN motions m ON r.motion_id = m.id WHERE r.classified = TRUE AND m.date IS NOT NULL AND r.category IN (...) GROUP BY y, q, r.category`
|
|
- Add each category as a separate go.Scatter trace with distinct colors and dashed lines (to distinguish from the aggregate solid line).
|
|
- Keep the existing aggregate line in solid bold. Use the same inflection/peak annotations.
|
|
- Add a legend entry for each category.
|
|
|
|
**Patterns to follow:**
|
|
- Existing Chart 6 for quarterly SQL, figure layout, inflection/peak annotations
|
|
|
|
**Test scenarios:**
|
|
- N/A — verify by rendering and checking that 5 category traces appear with distinct colors and dashed styles alongside the aggregate.
|
|
|
|
**Verification:**
|
|
- `uv run quarto render reports/overton_window/overton_window.qmd` succeeds
|
|
- The quarterly chart shows 5 category lines with a legend
|
|
|
|
---
|
|
|
|
- U3. **[Add Domain Decomposition narrative section]**
|
|
|
|
**Goal:** Add a heading-2 section "Domain Decomposition" between Indicator 1 and Indicator 2, with a markdown table of category deltas and 2-3 paragraphs of interpretive prose.
|
|
|
|
**Requirements:** R4
|
|
|
|
**Dependencies:** U1 (the delta data is used; the table can be hardcoded from the pre-computed values or computed via inline SQL)
|
|
|
|
**Files:**
|
|
- Modify: `reports/overton_window/overton_window.qmd`
|
|
|
|
**Approach:**
|
|
- Insert the section after the existing Indicator 1 narrative text and before the "## Indicator 2: Spatial Divergence" heading.
|
|
- Include a markdown table with columns: Category, Pre-2024 CS, Post-2024 CS, Delta, Volume, Extremity gap (M−S). Use the known data from the DB queries.
|
|
- Write 3 paragraphs:
|
|
1. Overview: which categories drove the shift (migration, energy, foreign affairs) and which resisted (agriculture, healthcare, infrastructure).
|
|
2. The polarization paradox: landbouw/natuur and zorg/gezondheid as domains where centrist support declined despite content moderation.
|
|
3. The consensus domains: onderwijs/wetenschap and economie as stable high-CS categories where the window didn't need to shift.
|
|
- Reference the category delta chart (U1) and domain trajectories chart (U2) by their cell labels.
|
|
|
|
**Patterns to follow:**
|
|
- Existing prose style in Indicator 1 and Indicator 2 sections
|
|
|
|
**Test scenarios:**
|
|
- N/A — prose section. Review for factual accuracy against the DB data.
|
|
|
|
**Verification:**
|
|
- Section renders in the HTML output with correct numbers and coherent prose
|
|
|
|
---
|
|
|
|
## System-Wide Impact
|
|
|
|
- **Interaction graph:** The QMD is the only file. No analysis scripts, DB schemas, or other artifacts are affected.
|
|
- **Unchanged invariants:** All existing charts, narrative sections, and data remain intact. New cells are added after existing ones, and the new section is inserted between existing sections.
|
|
|
|
---
|
|
|
|
## Risks & Dependencies
|
|
|
|
| Risk | Mitigation |
|
|
|------|------------|
|
|
| Plotly updatemenu may not fully render in Quarto HTML output | Test with a minimal prototype in an isolated QMD cell first; fallback is a faceted chart showing all categories as subplots |
|
|
| 5 category lines on the quarterly chart may be visually noisy | Use dashed lines for category traces and a solid bold line for aggregate. If too noisy, reduce to 3 categories |
|
|
|