Traffic Splitting

Understanding the technical details of how Fly Super routes visitors to variants, assigns them consistently, and ensures accurate weight distribution.

How Traffic Splitting Works

Overview

When a visitor clicks your link:

1. Request arrives at edge network
2. Link lookup in Redis (< 10ms)
3. Variant assignment via algorithm
4. Cookie set for persistence
5. Redirect to variant URL

Total time: ~10-50ms globally

Variant Assignment Algorithm

First-Time Visitors

Step 1: Generate Hash

Input: IP Address + User Agent + Link ID
Algorithm: SHA-256
Output: Deterministic hash (same input = same hash)

Step 2: Convert to Number

Hash: abc123def456...
Take first 8 characters: abc123de
Convert to decimal: 2882400222

Step 3: Modulo 100

2882400222 % 100 = 22

Step 4: Map to Variant

Weights: [50, 50]
Ranges: [0-49], [50-99]
Result (22): Falls in range [0-49] → Variant A

Step 5: Set Cookie

Cookie: fs_variant_[linkId] = variantA_id
Max-Age: 2592000 seconds (30 days)

Returning Visitors

Fast Path:

1. Check for cookie: fs_variant_[linkId]
2. If exists: Route to stored variant
3. If missing: Run assignment algorithm (user cleared cookies)

Result: Consistent experience across sessions.

Weight Distribution

How Weights Map to Ranges

Weights divide the 0-99 number space:

Example 1: 50/50 Split

Variant A: Weight 50
Variant B: Weight 50

Ranges:
Variant A: 0-49 (50 numbers)
Variant B: 50-99 (50 numbers)

Hash % 100 result:
0-49 → Variant A
50-99 → Variant B

Example 2: 70/30 Split

Variant A: Weight 70
Variant B: Weight 30

Ranges:
Variant A: 0-69 (70 numbers)
Variant B: 70-99 (30 numbers)

Hash % 100 result:
0-69 → Variant A (70%)
70-99 → Variant B (30%)

Example 3: 33/33/34 Split (3 variants)

Variant A: Weight 33
Variant B: Weight 33
Variant C: Weight 34

Ranges:
Variant A: 0-32 (33 numbers)
Variant B: 33-65 (33 numbers)
Variant C: 66-99 (34 numbers)

Hash % 100 result:
0-32 → Variant A (33%)
33-65 → Variant B (33%)
66-99 → Variant C (34%)

Why Weights Must Sum to 100

The algorithm divides 100 possible values (0-99):

• Each weight gets its proportion of these 100 values
• Total must equal 100 to cover all possibilities
• No overlap, no gaps

Invalid: Sum < 100

Variant A: 50
Variant B: 40
Total: 90

Problem: What happens to 90-99? No variant assigned!

Invalid: Sum > 100

Variant A: 60
Variant B: 50
Total: 110

Problem: Ranges overlap! Same user could match multiple variants!

Deterministic Routing

What is Deterministic?

Deterministic: Same input always produces same output

Inputs:

• Visitor IP address
• User agent (browser info)
• Link ID

Output:

• Specific variant assignment

Result: Same visitor on same device/browser always sees same variant.

Why Deterministic?

Benefits:

1. Consistent user experience - No jarring switches between variants
2. Accurate testing - User sees one variant through entire journey
3. Reliable conversion tracking - Can attribute conversion to specific variant
4. Stateless - No database lookup needed, algorithm is pure math

Example:

User John on Chrome from IP 192.168.1.1:
- Visit 1: Sees Variant A
- Visit 2: Sees Variant A (same)
- Visit 3: Sees Variant A (same)

User Sarah on Firefox from IP 192.168.1.2:
- Visit 1: Sees Variant B
- Visit 2: Sees Variant B (same)
- Visit 3: Sees Variant B (same)

Limitations

Scenario 1: Different Devices

• User on desktop → Assigned Variant A
• Same user on mobile → Different IP/UA → May get Variant B

Scenario 2: VPN Changes

• User without VPN → Assigned Variant A
• User turns on VPN → New IP → May get Variant B

Scenario 3: Browser Changes

• User on Chrome → Assigned Variant A
• Same user on Safari → Different UA → May get Variant B

Mitigation: Cookie persistence helps across sessions on same device.

Cookie Persistence

Cookie Details

Name: fs_variant_[linkId]

Value: Variant ID (e.g., variant_abc123)

Attributes:

• Domain: .yourdomain.com (works across subdomains)
• Path: / (works for all paths)
• Max-Age: 2592000 seconds (30 days)
• SameSite: Lax (works with redirects)
• Secure: Yes (HTTPS only)
• HttpOnly: No (accessible to JavaScript if needed)

Cookie Workflow

First Visit:

1. No cookie exists
2. Run assignment algorithm
3. Set cookie with assigned variant
4. Redirect to variant URL

Subsequent Visits:

1. Cookie exists
2. Read variant from cookie
3. Skip algorithm (faster)
4. Redirect to same variant URL

After 30 Days:

1. Cookie expires
2. Next visit: No cookie
3. Run assignment algorithm again
4. May get different variant (random)

Cookie Clearing

User clears cookies:

• Next visit runs algorithm again
• May get different variant
• New cookie set with new assignment

Private browsing:

• Cookie set but cleared when browser closes
• Each private session may get different variant
• Expected behavior

Cross-Device Consistency (Coming Soon)

Currently: Cookie-based (per-device) Future: User account-based (cross-device)

How it will work:

1. Logged-in users assigned based on user ID (not IP/UA)
2. Same variant across all devices
3. Stored in database, not just cookie

Edge Network Architecture

Why Edge Network?

Traditional approach:

User → Load Balancer → Application Server → Database → Logic → Redirect
Time: ~100-500ms

Fly Super approach:

User → Edge Runtime → Redis → Logic → Redirect
Time: ~10-50ms

Benefits:

• 10x faster - No database, no application server
• Global - Request handled at nearest edge
• Scalable - Handles millions of requests
• Reliable - No single point of failure

Technology Stack

Vercel Edge Runtime:

• Runs on Vercel's global edge network
• 70+ locations worldwide
• V8 JavaScript engine
• Middleware executes before app logic

Upstash Redis:

• Global, distributed Redis
• Read replicas worldwide
• Single-digit millisecond latency
• Automatic replication

SHA-256 Hashing:

• Cryptographically secure
• Deterministic
• Fast computation (< 1ms)
• Standard library implementation

Request Flow

1. User clicks link

https://yourdomain.com/spring-sale

2. DNS resolves to Vercel edge

Nearest edge location (e.g., San Francisco)

3. Edge Middleware intercepts

// Simplified middleware logic
export async function middleware(req) {
  const { slug } = req.nextUrl;

  // Redis lookup
  const link = await redis.get(`link:${slug}`);

  // Assign variant
  const variant = assignVariant(req, link);

  // Redirect
  return NextResponse.redirect(variant.url);
}

4. Redis lookup (< 10ms)

Key: link:spring-sale
Value: {variants: [...], weights: [...]}

5. Hash & assign (< 1ms)

const hash = sha256(ip + userAgent + linkId);
const num = parseInt(hash.slice(0, 8), 16) % 100;
const variant = findVariantByWeight(num, variants);

6. Set cookie (< 1ms)

Set-Cookie: fs_variant_abc123=variant_xyz; Max-Age=2592000

7. 307 Redirect (< 1ms)

HTTP 307 Temporary Redirect
Location: https://yourdomain.com/landing-page-v1

Total: ~10-50ms depending on global location

Statistical Distribution

Randomness vs Deterministic

Appears random:

• Different users get different variants
• Distribution matches weights

Actually deterministic:

• Same user always gets same variant
• Assignment based on hash of user attributes

Best of both worlds:

• Random distribution across population
• Consistent experience per user

Expected Distribution

With perfect randomness:

• 50/50 split → exactly 50% to each variant

In reality:

• Small samples: May vary (49/51, 52/48, etc.)
• Large samples: Approaches target (50.1/49.9)

Sample size matters:

Statistical Significance

Minimum sample sizes for A/B testing:

95% confidence, 5% minimum detectable effect:

• Baseline conversion: 2% → Need ~15,000 visitors per variant
• Baseline conversion: 5% → Need ~5,000 visitors per variant
• Baseline conversion: 10% → Need ~2,000 visitors per variant

Use an A/B test calculator to determine required sample size for your specific case.

Advanced Scenarios

Gradual Rollouts

Slowly increase traffic to new variant:

Week 1: Testing (10%)

Current: 90
New: 10

Week 2: Expanding (25%)

Current: 75
New: 25

Week 3: Increasing (50%)

Current: 50
New: 50

Week 4: Majority (75%)

Current: 25
New: 75

Week 5: Full (100%)

Current: 0
New: 100 (or delete link)

Benefit: Catch issues early with small traffic, minimize impact.

Multi-Armed Bandit (Coming Soon)

Automatically optimize weights based on performance:

Initial: Equal distribution

Variant A: 33%
Variant B: 33%
Variant C: 34%

After 1000 visitors: Adjust based on conversions

Variant A: 20% (lowest conversion)
Variant B: 50% (highest conversion)
Variant C: 30% (medium conversion)

After 5000 visitors: Further optimize

Variant A: 10%
Variant B: 70%
Variant C: 20%

Benefit: Maximizes conversions during testing phase.

Segment-Based Routing (Coming Soon)

Route based on user attributes:

By user status:

New users → Variant A (simplified onboarding)
Returning users → Variant B (advanced features)

By plan tier:

Free users → Variant A (upgrade prompts)
Paid users → Variant B (feature highlights)

By behavior:

High-intent users → Variant A (short form, direct CTA)
Explorers → Variant B (long form, detailed info)

Monitoring and Debugging

Testing Distribution

Small-scale test (10 requests):

for i in {1..10}; do
  curl -I "https://yourdomain.com/your-link"
done

Count redirects to each variant URL.

Medium-scale test (100 requests with different IPs): Use a testing service or VPN to simulate different IP addresses.

Debugging Cookie Assignment

View assigned variant:

1. Visit link
2. Open browser DevTools → Application → Cookies
3. Find cookie: fs_variant_[linkId]
4. Value shows assigned variant ID

Force reassignment:

1. Delete cookie
2. Visit link again
3. New assignment made

Redis Cache

Links are cached in Redis:

Key format: link:[domain]:[slug]

Value (JSON):

{
  "linkId": "link_abc123",
  "slug": "spring-sale",
  "domain": "yourdomain.com",
  "variants": [
    {"id": "var_1", "url": "https://...", "weight": 50},
    {"id": "var_2", "url": "https://...", "weight": 50}
  ]
}

TTL: None (cached indefinitely until link updated/deleted)

Invalidation: Automatic when link edited in dashboard

Performance Optimization

Current Performance

Metrics:

• Redirect time: ~10-50ms globally
• Throughput: Millions of redirects per day
• Availability: 99.99% uptime

Optimization Techniques

1. Edge Network:

• Request handled at nearest edge location
• Reduces latency by ~100-300ms vs centralized

2. Redis Caching:

• No database queries
• Sub-millisecond lookups
• Globally replicated

3. Deterministic Algorithm:

• Pure computation (no I/O)
• No external API calls
• Stateless (horizontally scalable)

4. Cookie Persistence:

• Skip algorithm for returning visitors
• Reduce computation by ~50%

5. 307 Redirect:

• Browser caches temporarily
• Preserves POST requests
• SEO-friendly (temporary)

Best Practices

For Accurate Testing

1. Wait for statistical significance - Don't stop tests early
2. Keep variants consistent - Don't change mid-test
3. Track at variant level - Use UTM parameters to distinguish
4. Account for cookies - Users may clear, affecting consistency
5. Monitor bot traffic - Exclude from analysis

For Performance

1. Use edge network - Already done automatically
2. Optimize variant URLs - Slow landing pages impact user experience
3. Monitor redirect time - Should stay under 50ms
4. Cache link configs - Already done via Redis

For Reliability

1. Test variants before launching - Ensure all URLs work
2. Monitor 404 errors - Indicates broken variant URLs
3. Set up alerts - Know when links break
4. Have fallback plan - If A/B test fails, direct traffic to control

Troubleshooting

"Distribution doesn't match weights"

Possible causes:

1. Small sample size - Need 100+ visitors minimum
2. Bot traffic - Bots may not trigger correct routing
3. Cached redirects - Browser temporarily caching

Solutions:

• Wait for more traffic
• Exclude bot traffic from analysis
• Test with multiple browsers/devices

"Users seeing different variants"

Expected scenarios:

1. Cleared cookies
2. Different device/browser
3. Private browsing mode (cookies cleared after session)

If unexpected:

• Verify cookie is being set (check DevTools)
• Check cookie max-age (should be 30 days)
• Verify same link is being used

"Redirect is slow"

Diagnose:

curl -w "@curl-format.txt" -o /dev/null -s "https://yourdomain.com/link"

curl-format.txt:

time_namelookup: %{time_namelookup}
time_connect: %{time_connect}
time_redirect: %{time_redirect}
time_total: %{time_total}

Typical values:

• time_redirect: ~0.010-0.050s (redirect itself)
• time_total: depends on variant URL page load time

If redirect > 100ms:

• Check Redis latency
• Check edge network status
• Contact support

What's Next?

• Creating Links - Build your first A/B test
• Links Overview - Back to links overview
• Landing Pages - Create variant pages

Quick Reference

Pro Tip: For accurate A/B testing, always use UTM parameters in your variant URLs to track performance in your analytics platform:

Variant A: /page-a?utm_content=control
Variant B: /page-b?utm_content=variant