A comprehensive, interactive guide to how X decides what appears in your For You feed — from raw candidates to ranked results.
The For You feed blends posts from accounts you follow (in-network) with posts discovered through ML (out-of-network). Everything is ranked by Phoenix — a Grok-based transformer model that predicts how likely you are to engage with each post.
An in-memory post store that tracks recent posts from all users in real time. It serves posts from accounts you follow in sub-millisecond lookups via Kafka-ingested events.
Rust · In-MemoryA two-tower ML model that encodes you and all posts into embedding vectors, then retrieves the most relevant out-of-network posts via approximate nearest-neighbour search.
JAX · Two-TowerA Grok-based transformer that reads your engagement history and scores every candidate post across 19 engagement types. The weighted sum becomes each post's final rank.
Transformer · Multi-actionThe glue layer written in Rust that wires together all pipeline stages: query hydration, candidate sourcing, enrichment, filtering, scoring, and final selection.
Rust · gRPCAn AI pipeline that classifies every new post for spam, safety violations, and topic category using Grok-powered vision-language models before posts enter the ranking pool.
Python · Grok VLMA reusable Rust framework defining composable traits (Source, Filter, Scorer, Hydrator…) that run in parallel where possible with built-in observability and error handling.
Rust · AsyncEvery feed request runs through these stages in sequence. Click any stage to expand details.
All query hydrators run in parallel. Their results are merged back into the query object before the next stage begins.
Sources run in parallel and their results are pooled together into a single candidate list for the next stage.
Hydrators fetch additional data and write it back to each candidate. They run in parallel since they don't depend on each other.
Filters run one after another. Each partitions candidates into "kept" and "removed." Removed candidates are discarded (or tracked for logging) and never scored.
Removes: duplicates, posts older than the age threshold, your own posts, posts from blocked/muted accounts, previously seen posts, paywalled content you can't access, and muted keywords.
Scorers run in order, each updating candidates with new fields. The full scoring chain is:
The TopKScoreSelector sorts all surviving candidates by their final score (descending) and takes the top K. Non-selected candidates are passed to side effects for logging and caching.
VF Filter runs a visibility-filtering check and drops anything deleted, spam, violent, or gore since the pre-scoring stage. DedupConversation removes duplicate branches of the same thread.
Side effects run async in the background: caching scored posts in Redis, publishing served candidate IDs to Kafka, updating impression history, logging for A/B experiments.
Four major subsystems, each with a distinct responsibility and technology stack.
Written in Rust. Exposes a gRPC ScoredPostsService endpoint. Wires together all pipeline stages and owns the final response format.
Consumes post create/delete Kafka events in real time. Maintains per-user stores for original posts, replies/reposts, and video posts. Auto-trims old posts.
Two JAX models: a two-tower retrieval model (user + candidate towers) and a Grok-based transformer ranker. Ported from Grok-1, adapted for RecSys.
Python pipeline that classifies new posts for spam, safety violations, and topics using Grok's VLM. Powers embeddings and policy enforcement at ingest time.
A Rust crate defining trait-based abstractions for building recommendation pipelines. Sources, Hydrators, Filters, Scorers, Selectors, and SideEffects.
New in 2026. Blends ads into the organic feed at appropriate positions. Tracks brand-safety signals so ads don't appear adjacent to sensitive content.
The Phoenix transformer predicts probabilities for 19 engagement types. These are combined via a weighted sum — with negative weights for actions you'd rather not see.
Adjust engagement probabilities to see how the algorithm would score a post
Filters run at two points: before scoring (to avoid wasting ML compute on ineligible posts) and after selection (for final safety checks).
Phoenix is a two-stage ML system: a two-tower retrieval model to narrow millions of posts to thousands, and a Grok-based transformer ranker to score each one with full context.
Encodes you and every post into a shared embedding space. Finds the top-K posts most similar to you via dot-product ANN search.
Input: [user token] + [engagement history sequence] + [candidate posts]. Candidates attend to user + history but not to each other.
Candidates can only attend to the user token and engagement history — never to each other. This means each post's score is independent of what else is in the batch, making scores cacheable and consistent across requests.
Can attend Blocked Self only
Neither the ranking nor retrieval model uses hand-crafted feature IDs. Instead, both users and posts are embedded via multiple independent hash functions:
Multiple hash embeddings are summed (reduced) before entering the transformer, providing collision resistance and graceful handling of unseen IDs.
Every post goes through Grox before it can be ranked. Grox uses Grok's vision-language model to understand text and images together — enabling nuanced classification that pure text models miss.
Uses Grok VLM to classify posts as spam. Has a dedicated path for low-follower accounts (SpamEapiLowFollowerClassifier) that applies stricter standards to new or small accounts.
Two-pass safety system: a fast initial screen (BangerInitialScreen) followed by the full PostSafetyScreenDeluxe that evaluates PTOS (Policy, Terms of Service) categories.
Classifies posts into categories to power topic-based feeds and filtering experiments. Supports post-based filtering at 90%, 75%, and 50% confidence thresholds.
TopicsGenerates dense embedding vectors for posts using both text and image content (v2 and v5 embedders). Used as features for the Phoenix retrieval model's candidate tower.
MLGenerates natural-language summaries of posts, used as additional input features to the Phoenix embedding pipeline for richer content understanding.
NLPA DAG-based task scheduler (grox/engine.py) that orchestrates classifiers, embedders, and publishers. Tasks declare dependencies and the engine resolves execution order.
Five things worth knowing about how the algorithm actually works — and what they imply for users and creators.
The system has eliminated manual feature engineering entirely. The Grok transformer learns what matters directly from your engagement history. This means the algorithm adapts continuously to new content types and user behaviours without engineer intervention.
Block, mute, "not interested," and report all carry negative weights in the scoring formula. Using them actively trains the algorithm away from similar content. The scoring formula explicitly penalises posts you're predicted to dislike.
The author diversity scorer applies an exponential decay to repeated authors sorted by score. Even if one account dominates your highest scores, later posts from that account get progressively smaller multipliers — ensuring your feed isn't flooded by a single creator.
After the diversity step, out-of-network posts get score × OON_WEIGHT_FACTOR (less than 1). However, new users get a higher OON factor to help them discover content before building a follow graph, and topic feeds get their own OON factor.
Candidate isolation in the transformer means a post's score doesn't change based on what else is in the request. This makes scored posts cacheable in Redis — so if you've already paid the ML cost for a post, its score can be reused in future requests.
The new ads blending system includes brand safety hydrators that track safety labels on organic content. Ads are not injected adjacent to content that violates brand safety thresholds, and the injection positions are validated against the organic post layout.
The May 15, 2026 open-source release is a major update. Here's what changed.
A new phoenix/run_pipeline.py replaces separate retrieval and ranking scripts. It runs the full retrieval → ranking chain from a single command using exported checkpoints, matching how the two stages are composed in production.
A ~3 GB pre-trained mini Phoenix model (256-dim embeddings, 4 attention heads, 2 transformer layers) is packaged via Git LFS. You can run inference immediately without training a model from scratch, using a 537K sports-post demo corpus.
The new grox/ directory adds classifiers, embedders, and a task engine for content understanding workloads: spam detection, topic classification, multimodal embeddings, and PTOS policy enforcement — all powered by Grok's VLM.
The home-mixer/ads/ module handles ad injection and positioning within the organic feed, including brand-safety tracking that prevents ads from appearing adjacent to sensitive content.
New query hydrators populate followed topics, starter packs, impression bloom filters, IP addresses, mutual follow graphs, and served history — giving the ranker substantially richer user context.
Additional hydrators for engagement counts, brand safety signals, language codes, media detection, quote post expansion, and mutual follow Jaccard scores enrich each candidate before scoring.
New sources for ads, Who-to-Follow recommendations, Phoenix MoE (Mixture-of-Experts), Phoenix Topics, and prompts are now included alongside the core Thunder and Phoenix retrieval sources.