Relevance without negative labels
The training data tells us which products match a Turkish search query, but supplies no explicit negative pairs. The task therefore starts before model fitting: deciding how to create useful negatives without leaking information from test candidates.
I built an end-to-end retrieval and ranking pipeline for 966,444 products and 3,359,679 candidate pairs in the TEKNOFEST 2026 Trendyol competition.
How the pipeline works
- Retrieve and mine negatives. Deterministic catalog sampling combines uniform, hard, and semi-hard examples.
- Represent the match. Turkish-aware lexical features capture token overlap, brands, categories, and product attributes. Transformer bi-encoders add semantic similarity.
- Rank the candidates. LightGBM and CatBoost combine lexical and category signals. Within-query ranks make outputs from different model families comparable.
- Calibrate the decision. A positive-rate decision rule turns the blended ranking into the competition's binary output.
I used term-grouped validation so that a search term stays in one fold, keeping the evaluation focused on unseen queries.
Results
The recorded public Macro-F1 increased from 0.760 for the lexical baseline to 0.870 for the final rank blend. Intermediate runs made the effect of semantic blending and positive-rate calibration visible. The final recorded placement was 116 of 377 participants.
The useful lesson
Better ranking does not automatically mean a better binary decision. Macro-F1 was sensitive to the positive rate, and lexical and semantic models made different mistakes. The strongest improvement came from making those models work together and validating the decision rule with the same care as the model.