The Idea in One Paragraph
Arabic shoppers type short queries that rarely match how products are described online. Searching ثلاجة fails when the catalog says براد (dialect synonym) or ثلاجه (orthographic variant).
This thesis designs, builds, and rigorously evaluates an e-commerce search pipeline combining:
- (a) Hybrid retrieval architecture — MySQL as transactional source of truth; Elasticsearch as a searchable projection supporting lexical BM25 and dense vector search, with Arabic-aware normalization.
- (b) LLM-based query expansion — large language models generate synonyms, dialectal variants, and full rewrites of the query before retrieval.
We measure how much each layer improves retrieval quality (NDCG / MRR / Recall) and what latency and cost it adds — comparing several open Arabic-centric LLMs against commercial ones.
Alternative titles (for you to steer)
— most descriptive; names the architecture and the technique.
— more “IR-research” flavored.
— most “scientific study” flavored.
Why Arabic Product Search Is Hard
| Challenge | Example | Who handles it today |
|---|---|---|
| Orthographic variability | ثلاجة / ثلاجه · أرنب / ارنب | Elastic normalization ✓ |
| Diacritics & tatweel | مُدرَّسة vs مدرسة · ووووصلات | Elastic normalization ✓ |
| Morphological richness | والثلاجاتِ = و+ال+ثلاجة+ات | Light stemming ~partial |
| Dialect ↔ MSA vocabulary gap | براد vs ثلاجة · جوال / موبايل / هاتف | Nobody ✗ |
| Brand transliterations | سامسونج / سامسونغ · Samsung · ايفون / آيفون · iPhone | Nobody ✗ |
| Mixed-script queries | ايفون 15 pro max (AR+EN+digits) | Nobody ✗ |
The crux: keyword engines fix rows 1–3 but cannot handle semantics (rows 4–6). Dense embeddings handle paraphrase but fail exact SKUs, codes, rare brands. Industrial evidence (Amazon, Taobao, Walmart) shows LLM query rewriting closes exactly this gap in English — no rigorous study exists for Arabic product search.
The Gap This Thesis Fills
- No public benchmark for Arabic e-commerce product-search relevance. Amazon’s ESCI covers EN/JA/ES only. Arabic IR benchmarks (Mr. TyDi-ar, MIRACL-ar) exist but target Wikipedia-style QA search — not catalogs.
- No systematic evaluation of LLM query-expansion strategies (synonyms vs. rewrite vs. HyDE-style hypothetical documents) in Arabic. English equivalents: Google QE-prompting 2023, Taobao BEQUE 2024, Walmart 2024.
- No reference architecture integrating relational data (MySQL) + hybrid lexical/dense retrieval (Elasticsearch) + LLM expansion under realistic latency constraints for Arabic.
Positioning sentence used in the thesis “Arabic retrieval benchmarks exist for open-domain search (Mr. TyDi, MIRACL), but none targets e-commerce product search with catalog semantics — we fill that gap.”
- An Arabic e-commerce search benchmark (annotated artifact, publishable on its own)
- Open-source end-to-end reference implementation (MySQL→ES sync, Arabic analyzers, hybrid retriever, pluggable LLM-expansion service)
- First systematic comparison of LLMs for Arabic query expansion across strategies × models × query types
- Deployment guidelines given latency/cost budgets
Research Questions & Hypotheses
How much does LLM query expansion improve NDCG@10 / MRR@10 / Recall@100 over (a) plain BM25 and (b) BM25 + rule-based Arabic normalization/synonyms?
Which expansion strategy works best for Arabic: synonym generation, full rewrite, or HyDE-style hypothetical documents? And which LLMs — open Arabic-centric (ALLaM, Jais, AceGPT, Qwen) vs commercial (GPT/Claude/Gemini)?
Does BM25 + dense-vector retrieval fused via RRF beat each individual retriever for Arabic product search, consistent with English findings?
What latency/cost does the expansion step add, and can caching + small local models make it production-viable?
Do gains differ between MSA and dialectal queries?
Statistical testing planned: paired t-test / Friedman on per-query metrics.
Data Strategy — Assessed This Month
Track A — Existing Arabic benchmark usable with caveats
prestoai/arabic-ecom-search-bench (Hugging Face, Apr 2026) — audited by me on 2026‑08‑22:
| Format | Proper TREC-style benchmark (queries.jsonl + corpus.jsonl + qrels.tsv) — plugs into standard IR evaluation directly |
| Scale | 29,014 queries · 107,041 products · 262,599 judgments (graded 0–3 + hard negatives) |
| Published baseline | Meilisearch run included: nDCG@10 = 0.624, Recall@10 = 0.483 — an immediate number to beat |
| Dialect coverage | MSA + Libyan dialect explicitly tagged |
⚠ Red flags: judgment provenance undisclosed (human? click logs? LLM?) · no license file · corpus has product titles only · Libyan-dialect skew · brand new, zero community validation.
Action taken: will email the authors asking about provenance + redistribution permission; their answer feeds the dataset chapter either way.
Track B — Our own annotated slice primary artifact
- 300–500 realistic queries written by us: MSA head terms, long-tail, Egyptian/Gulf/Levantine dialect variants (deliberately beyond the Libyan-only bench), brand transliterations, mixed AR/EN.
- 2–3 annotators using translated WANDS annotation guidelines; inter-annotator agreement Cohen’s κ ≥ 0.7.
- Judgments pooled from multiple retrieval runs so no single system biases candidates.
Supporting layers
| Resource | Role in thesis |
|---|---|
| Mr. TyDi-ar · MIRACL-ar · mMARCO-ar | Open-domain Arabic retrieval benchmarks — sanity-check embedding models & analyzer settings before product experiments (papers already collected) |
| Amazon ESCI (130K queries, 2.6M judgments) | Methodological template for labels & protocol (no Arabic split) |
| Wayfair WANDS (233K judgments, MIT) | Annotation-guideline template — includes public guidelines PDF we adapt |
| milistu/AMAZON-Products-2023-Arabic (~117K products) | Bulk real catalog for scaling index/load tests |
Proposed Architecture
Data layer: MySQL = source of truth (products, categories) → one-command sync projects into Elasticsearch for search. Everything runs locally via Docker Compose; fully reproducible.
Models & Tools (shortlist)
Embedding models (dense side)
| Model | Why |
|---|---|
| GATE Arabic-Triplet-Matryoshka-V2 (135M) | SOTA Arabic semantic similarity on MTEB |
| MARBERT (163M) | dialect coverage baseline |
| multilingual-e5-large (560M) | strong multilingual fallback |
| Commercial APIs | upper-bound reference points |
LLMs for expansion (the experiment grid)
| Model | Access | Note |
|---|---|---|
| ALLaM-7B | open, Apache-2.0 | Saudi national model, strong Arabic scores |
| Jais 13B/70B | open, Apache-2.0 | UAE, good dialect handling |
| AceGPT, Qwen2.5 | open | culturally aligned / strong multilingual |
| GPT-4o-mini · Claude Haiku · Gemini Flash | API | commercial upper bounds + cost/latency reference |
Tooling: Elasticsearch free tier (BM25 + kNN + RRF) · CAMeL Tools & Farasa (Arabic processing) · ranx/pytrec_eval (metrics) · Label Studio (annotation UI) · Ollama/vLLM (local LLM serving) · Locust/k6 (load tests). All free/open except commercial API calls.
12-Month Plan
| Months | Milestone |
|---|---|
| 1–2 | Literature deep-read · finalize RQs · proposal defense ← we are here |
| 2–3 | Dataset acquisition + annotation guidelines (adapt WANDS/ESCI methodology) |
| 3–4 | Baselines: MySQL-LIKE → BM25 → BM25 + tuned Arabic analyzers |
| 4–6 | Hybrid retrieval: embeddings selection, kNN, RRF fusion |
| 6–8 | LLM expansion service · model × strategy experiment grid |
| 8–9 | Full evaluation matrix · significance testing · ablations |
| 9–10 | Latency/cost experiments · caching design |
| 10–12 | Writing · error analysis · polish · defense |
Reading Done So Far
28 papers collected as PDFs, organized and annotated in the repository (research/papers/ + docs/02-literature-review.md):
(AraBERT, MARBERT, GATE, stemming, dialect normalization, Arabic LLMs)
(ESCI, WANDS, Amazon/Taobao/Walmart query rewriting)
(Google QE-prompting, CSQE, LLM-QE, best practices)
(RRF, SPLADE, score fusion)
(Mr. TyDi, MIRACL, mMARCO)
Every paper has an annotation entry (findings + relevance to our work); sources tracked with download status in research/SOURCES.md. Closest prior art identified: SHEINfer (Arabic product category inference, 2026) and an Arabic e-commerce search challenges survey (2022) — neither does query expansion nor relevance benchmarking.
Decision Points — Where I Need Your Guidance ⭐
These are the choices I can defend either way, but would rather align with you early. Space is left under each for your notes.