9.0 KiB
| title | type | status | date |
|---|---|---|---|
| feat: Category domain decomposition for Overton report | feat | active | 2026-06-15 |
feat: Category domain decomposition for Overton report
Summary
Add 3 new Plotly charts and a narrative section to reports/overton_window/overton_window.qmd that decompose the Overton shift by policy category (asiel/vreemdelingen, landbouw/natuur, energie/klimaat, etc.), making visible which domains drove the shift, which resisted it, and how each category's centrist support evolved over time. The yearly CS chart gains category filtering via Plotly dropdown menu.
Problem Frame
The current Overton report treats all right-wing motions as a single aggregate. The synthesis mentions migration vs non-migration, but with 10 categories now populated across all 3,030 motions, we can show a richer picture: energie/klimaat had the second-largest CS surge (+0.107), landbouw/natuur actually declined (−0.063), and onderwijs/wetenschap barely moved (+0.053) despite being the highest-consensus domain. These stories are invisible in the current charts.
Requirements
- R1. Add a horizontal bar chart showing pre/post centrist support delta per category, sorted by magnitude
- R2. Add category filtering to the yearly CS timeline (Chart 1) via Plotly dropdown menu, so users can view a single category or "All right-wing"
- R3. Add domain trajectories to the quarterly chart showing 4-5 key categories as separate lines
- R4. Add a "Domain Decomposition" narrative section with the category delta table and interpretive prose
- R5. All new charts must use the existing Plotly styling conventions (colors, template, height)
Scope Boundaries
- No changes to the synthesis markdown or other analysis scripts
- No changes to the DB or data pipeline
- No interactive filtering beyond Plotly's built-in updatemenu (no Streamlit/JS)
Key Technical Decisions
- Plotly updatemenu for filtering: Uses Plotly's built-in
updatemenuswithbuttonsto toggle between "All right-wing" (current line) and individual categories. No external JS or Dash needed. - Color scheme for categories: Use a qualitative 10-color palette (Plotly
alphabetorset2), not PARTY_COLOURS, to avoid confusion with party lines. - New chart cells inserted after existing Chart 1: Category delta bar chart goes after the yearly CS chart (Chart 1) and its narrative. Domain trajectories go in the quarterly section (Chart 6). A new "Domain Decomposition" section links them.
Open Questions
Resolved During Planning
- Standalone HTML filtering: Plotly updatemenu works in standalone HTML (confirmed via existing usage in the repo — the Overton report is a standalone HTML file with embedded Plotly).
Deferred to Implementation
- Whether to show all 10 categories or a curated subset: The quarterly trajectories chart should show at most 5-6 lines to avoid visual clutter. The delta bar chart can show all 10.
- Exact Plotly color assignment to each category: Will match categories to a qualitative palette at implementation time.
Implementation Units
- U1. [Add category delta bar chart and dropdown-filtered yearly CS]
Goal: Two linked chart additions: (1) a category delta horizontal bar chart, (2) convert Chart 1's yearly CS line chart to support category filtering via Plotly updatemenu.
Requirements: R1, R2, R5
Dependencies: None
Files:
- Modify:
reports/overton_window/overton_window.qmd
Approach:
- Compute yearly CS per category via SQL:
SELECT year, category, AVG(centrist_support_strict) as cs, COUNT(*) as n FROM right_wing_motions WHERE classified = TRUE GROUP BY year, category - For the delta chart (new cell): compute pre/post CS per category, then use a horizontal bar (go.Bar with orientation='h'), sorted by delta descending. Color bars: green for positive delta, red for negative.
- For Chart 1 modification: extend the existing yearly SQL to GROUP BY year, category. In Python, pivot to get per-category columns. Create a go.Figure with all categories as traces plus an aggregate "All right-wing" trace. Add an updatemenu dropdown with buttons: "All right-wing" (restores all visible with only the aggregate line shown) and each category name (shows only that category's trace).
- Keep the existing pre/post mean lines and the break-year vertical line intact. The dropdown only controls which category trace is visible.
- Use a 10-color qualitative palette from plotly.express.colors.qualitative.
Patterns to follow:
- Existing Chart 1 for SQL pattern, figure layout, annotation style
- Existing Chart 6 for quarterly trajectory styling
Test scenarios:
- N/A — this is a Quarto document rendering change. Verify by rendering the QMD and checking that (a) the delta chart shows all 10 categories, (b) the dropdown in Chart 1 cycles through categories correctly, (c) pre/post mean lines remain visible in all views.
Verification:
uv run quarto render reports/overton_window/overton_window.qmdsucceedsoverton_report.htmlcontains the new delta chart section and Chart 1 responds to dropdown interaction
- U2. [Add domain trajectories to quarterly chart]
Goal: Enhance the existing quarterly chart (Chart 6) by overlaying 4-5 key category lines alongside the aggregate.
Requirements: R3, R5
Dependencies: U1 (the SQL for per-category yearly CS shares the same data approach; the quarterly chart needs its own SQL)
Files:
- Modify:
reports/overton_window/overton_window.qmd
Approach:
- Select 5 categories to show as individual lines: asiel/vreemdelingen, energie/klimaat, buitenland/europa, landbouw/natuur, economie. These are the categories with largest deltas, largest volumes, or most story value.
- Compute quarterly CS per category:
SELECT EXTRACT(YEAR FROM m.date) AS y, CEIL(EXTRACT(MONTH FROM m.date) / 3.0) AS q, r.category, AVG(r.centrist_support_strict) AS cs, COUNT(*) AS n FROM right_wing_motions r JOIN motions m ON r.motion_id = m.id WHERE r.classified = TRUE AND m.date IS NOT NULL AND r.category IN (...) GROUP BY y, q, r.category - Add each category as a separate go.Scatter trace with distinct colors and dashed lines (to distinguish from the aggregate solid line).
- Keep the existing aggregate line in solid bold. Use the same inflection/peak annotations.
- Add a legend entry for each category.
Patterns to follow:
- Existing Chart 6 for quarterly SQL, figure layout, inflection/peak annotations
Test scenarios:
- N/A — verify by rendering and checking that 5 category traces appear with distinct colors and dashed styles alongside the aggregate.
Verification:
uv run quarto render reports/overton_window/overton_window.qmdsucceeds- The quarterly chart shows 5 category lines with a legend
- U3. [Add Domain Decomposition narrative section]
Goal: Add a heading-2 section "Domain Decomposition" between Indicator 1 and Indicator 2, with a markdown table of category deltas and 2-3 paragraphs of interpretive prose.
Requirements: R4
Dependencies: U1 (the delta data is used; the table can be hardcoded from the pre-computed values or computed via inline SQL)
Files:
- Modify:
reports/overton_window/overton_window.qmd
Approach:
- Insert the section after the existing Indicator 1 narrative text and before the "## Indicator 2: Spatial Divergence" heading.
- Include a markdown table with columns: Category, Pre-2024 CS, Post-2024 CS, Delta, Volume, Extremity gap (M−S). Use the known data from the DB queries.
- Write 3 paragraphs:
- Overview: which categories drove the shift (migration, energy, foreign affairs) and which resisted (agriculture, healthcare, infrastructure).
- The polarization paradox: landbouw/natuur and zorg/gezondheid as domains where centrist support declined despite content moderation.
- The consensus domains: onderwijs/wetenschap and economie as stable high-CS categories where the window didn't need to shift.
- Reference the category delta chart (U1) and domain trajectories chart (U2) by their cell labels.
Patterns to follow:
- Existing prose style in Indicator 1 and Indicator 2 sections
Test scenarios:
- N/A — prose section. Review for factual accuracy against the DB data.
Verification:
- Section renders in the HTML output with correct numbers and coherent prose
System-Wide Impact
- Interaction graph: The QMD is the only file. No analysis scripts, DB schemas, or other artifacts are affected.
- Unchanged invariants: All existing charts, narrative sections, and data remain intact. New cells are added after existing ones, and the new section is inserted between existing sections.
Risks & Dependencies
| Risk | Mitigation |
|---|---|
| Plotly updatemenu may not fully render in Quarto HTML output | Test with a minimal prototype in an isolated QMD cell first; fallback is a faceted chart showing all categories as subplots |
| 5 category lines on the quarterly chart may be visually noisy | Use dashed lines for category traces and a solid bold line for aggregate. If too noisy, reduce to 3 categories |