Implement article search with PostgreSQL full-text search #41
Reference in New Issue
Block a user
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Summary
The wireframe includes a search box on the article list view (line 337-338 of
wireframe.html), but search was never implemented. We need to add article search functionality to the site, including replacing the "Subscribe" CTA in the desktop/mobile navigation with a search box.Current State
wagtail.searchinINSTALLED_APPS)ArticlePage.search_fieldsindexestitle(inherited from Page) +summary— but no frontend search existsWAGTAILSEARCH_BACKENDSsetting is configured (defaults to Wagtail DB backend)#newsletter— this can be replaced with a search inputProposed Strategy: PostgreSQL Full-Text Search via Wagtail
Use Wagtail's built-in
wagtail.search.backends.database(PostgreSQL) backend rather than adding Elasticsearch/Meilisearch. This is the right fit because:pg_trgmgives us fuzzy/partial matching;tsvectorgives us ranked full-text searchdatabasebackend integrates withsearch_fieldsalready defined onArticlePageScope: Article Search (not site-wide)
This is article search only — querying
ArticlePageobjects. We are not searching across all page types (AboutPage, LegalPages, etc.). If cross-page-model search is needed later, we can add a unified query strategy as a follow-up.Database Indexing
Wagtail's
databasesearch backend creates its own index tables (wagtailsearch_indexentry) that use PostgreSQL FTS internally. We should:django.contrib.postgrestoINSTALLED_APPS(required for full Postgres search features per Wagtail docs)WAGTAILSEARCH_BACKENDSto use thedatabasebackend explicitly withenglishsearch configArticlePage.search_fieldsto includebody(StreamField text content, excluding code blocks) and tags viaRelatedFieldsAutocompleteFieldontitlefor future type-ahead supportFilterFieldoncategoryandpublished_datefor filtered search./manage.py update_indexto build the search index (automated in deploy, not manual one-off)Implementation Plan
1. Backend: Configure Wagtail PostgreSQL Search
django.contrib.postgrestoINSTALLED_APPSinconfig/settings/base.pyWAGTAILSEARCH_BACKENDStoconfig/settings/base.py:ArticlePage.search_fieldsusing correct patterns for related data:index.RelatedFieldsis the correct pattern for tag/category name matching —index.SearchField("tags")would not work properly.2. StreamField Body Indexing: Exclude Code Blocks
body(StreamField) will by default include all block text including code blocks, which produces noisy results (matching random code tokens like variable names).get_search_text()method (or use Wagtail'ssearch_index.get_searchable_content()override) to extract only prose text from the StreamField, skippingcodeblocks.3. Backend: Search View
apps/blog/views.pythat:?q=query parameterArticlePage.objects.live().search(query)(Wagtail search API).search()(Wagtail docs require this ordering)qon pagination links (e.g.,?q=llm&page=2)path("search/", search_view, name="search")inconfig/urls.pypath("", include(wagtail_urls))or Wagtail's catch-all will swallow it4. Frontend: Navigation Search Box (Desktop)
<a>button intemplates/components/nav.htmlwith a search form:5. Frontend: Navigation Search Box (Mobile)
nav.html, above the newsletter form6. Frontend: Article Index Page Search
templates/blog/article_index_page.htmlalongside the category/tag filters/search/or filter inline with a?q=param on the article index7. Search Results Page
templates/blog/search_results.html:article_card.htmlcomponent for results?q=across pages)8. Reindex Lifecycle
./manage.py update_indexmust be automated as part of deploy, not a one-off manual stepsearch_fieldsadditions are always reflected9. Tests (including existing test updates)
New tests:
?q=preserved)search_fieldsindexes the expected fieldsExisting tests that need updating (due to Subscribe → Search nav change):
apps/blog/tests/test_views.py::test_newsletter_forms_render_in_nav_and_footer— currently assertshref="#newsletter"in nav; needs updating to expect search form insteade2e/test_home.py::test_nav_subscribe_cta_present— currently assertsSubscribelink visible in nav; needs updating to expect search inpute2e/test_newsletter.py) — these test the sidebar newsletter form, not the nav CTA, so should be unaffectede2e/test_article_detail.py::test_article_newsletter_aside_present— tests sidebar, should be unaffectedDesign Notes
w-48on desktop) to not crowd the navtype="search"androle="search"for accessibilityOut of Scope (for now)
Thanks for writing this up — the overall direction (Wagtail DB backend on Postgres, no extra search infra) looks right for this project size. I’d support this with a few important adjustments to avoid subtle bugs.
If helpful, I can split this into a concrete implementation checklist (model/search_fields, URL/view, templates, and tests) to reduce risk during build.
Correcting formatting from my previous comment (the CLI ate inline code formatting). Reposting cleanly:
Thanks for writing this up — the overall direction (Wagtail DB backend on Postgres, no extra search infra) looks right for this project size. I’d support this with a few important adjustments to avoid subtle bugs.
django.contrib.postgresinINSTALLED_APPSwhen using the Postgres backend (per Wagtail docs), otherwise Postgres search features can be incomplete.search_fieldsshape for related dataindex.SearchField("tags")is risky/not the right pattern for tag text matching.index.RelatedFields("tags", [index.SearchField("name")])so searches match tag names.path("search/", ...)must be added beforepath("", include(wagtail_urls)), or Wagtail’s catch-all will swallow it.ArticlePage.objects...). That’s fine, but then we should call it article search.q, handle empty query explicitly (don’t run full search on blank/whitespace).qon pagination links..search()(Wagtail docs note this ordering).bodyis useful, but this will likely include code block text too. That may produce noisy results (e.g., matching random code tokens).update_indexshould be part of rollout/backfill, not just a one-off manual step, so existing content is guaranteed indexed after deploy.If helpful, I can split this into a concrete implementation checklist (model/search_fields, URL/view, templates, and tests) to reduce risk during build.
Implement site-wide search with PostgreSQL full-text searchto Implement article search with PostgreSQL full-text searchAll 9 points incorporated into the updated issue description. Here's what changed:
django.contrib.postgres— added to INSTALLED_APPS stepRelatedFieldsfor tags — replacedSearchField("tags")withRelatedFields("tags", [SearchField("name")])+ added explicit note about whypath("search/", ...)must come beforewagtail_urlscatch-allqon pagination links.search()per Wagtail docsupdate_indexautomated in deploy/entrypoint, not a manual one-offtest_newsletter_forms_render_in_nav_and_footer(assertshref="#newsletter") andtest_nav_subscribe_cta_present(asserts Subscribe link). Confirmed newsletter E2E tests are sidebar-only so unaffected.