feat(feed): broaden search to a weighted document (title + keywords + queries + description)
Makes the local search behave more like YouTube's — finding videos by the uploader's own keywords or the query that surfaced them, not only words in the title. A DB-generated, weighted search_vector (migration 0032) replaces the title-only FTS index: - keywords: the creator's snippet.tags (free — already in the snippet we fetch), stored on enrich. - search_terms: distinct live-search queries that surfaced the video (across all users), appended by the search route — folds YouTube's relevance into local search (a video YT returned for a query becomes findable by it even without a title match), the user's own idea. - description (truncated) for broad recall on the existing catalog. Weighted title(A) > keywords+queries(B) > description(C) so ts_rank keeps title hits on top. A plain GIN index on the generated column guarantees index use (no expression/param matching). Verified on localdev: recall 146->213 for one query; 7 'eurovision' hits via the document but not the title; index scan confirmed.
This commit is contained in:
parent
20bcdf5ecb
commit
4395afc210
5 changed files with 115 additions and 11 deletions
|
|
@ -221,22 +221,19 @@ def _filtered_query(
|
|||
published_before, datetime.min.time(), tzinfo=timezone.utc
|
||||
) + timedelta(days=1)
|
||||
query = query.where(Video.published_at < end)
|
||||
# Full-text relevance search on the title: YouTube-like "fuzzy" matching (word-order-
|
||||
# independent, multi-word AND, prefix on the word being typed) + accent-insensitive via
|
||||
# the unaccent_simple config. The channel name still matches as a substring so channel
|
||||
# searches work. `rank_expr` (ts_rank) drives the optional "relevance" sort. The
|
||||
# to_tsvector expression must match the GIN index (migration 0031) verbatim to be used.
|
||||
# Full-text relevance search over the weighted search document (title > creator keywords +
|
||||
# the queries that surfaced the video > description; see Video.search_vector). YouTube-like
|
||||
# "fuzzy" matching: word-order-independent, multi-word AND, prefix on the word being typed,
|
||||
# accent-insensitive (unaccent_simple). The channel name still matches as a substring so
|
||||
# channel searches work. `rank_expr` (weight-aware ts_rank) drives the "relevance" sort.
|
||||
rank_expr = None
|
||||
if q:
|
||||
ts_str = _to_tsquery_str(q)
|
||||
if ts_str:
|
||||
tsq = func.to_tsquery("public.unaccent_simple", ts_str)
|
||||
title_vec = func.to_tsvector(
|
||||
"public.unaccent_simple", func.coalesce(Video.title, "")
|
||||
)
|
||||
query = query.where(
|
||||
or_(
|
||||
title_vec.op("@@")(tsq),
|
||||
Video.search_vector.op("@@")(tsq),
|
||||
func.unaccent(Channel.title).ilike(func.unaccent(f"%{q}%")),
|
||||
)
|
||||
)
|
||||
|
|
@ -244,7 +241,7 @@ def _filtered_query(
|
|||
# EXACTLY — a raw float4 vs the float8 cursor value mismatches on round-trip and
|
||||
# breaks paging (the same page repeats). 1e6 granularity is ample; ties break on
|
||||
# published_at then id.
|
||||
rank_expr = cast(func.ts_rank(title_vec, tsq) * 1000000, BigInteger)
|
||||
rank_expr = cast(func.ts_rank(Video.search_vector, tsq) * 1000000, BigInteger)
|
||||
else:
|
||||
# No usable tokens (e.g. only punctuation): fall back to a plain substring match.
|
||||
like = func.unaccent(f"%{q}%")
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue