ScryData Documentation

Why ScryData Proxy?

Database migrations are high-stakes operations. Schema changes, version upgrades, and query optimizations can break production in unexpected ways. Traditional staging environments lack the scale, traffic patterns, and edge cases that only emerge under real load.

ScryData Proxy sits transparently between your application and database, capturing every query with ~100μs overhead. This gives you complete visibility into your database workload and enables production-scale validation of changes before deployment.

                            Key Benefits for Migrations
                            Zero application changes: Drop-in replacement for direct database connection
Real traffic capture: See actual query patterns, not synthetic benchmarks
Instant validation: Circuit breakers catch breaking changes in real-time
Privacy-preserving: Cryptographic fingerprinting keeps PII safe

                        

Two Components, One Platform

ScryData provides two complementary tools for production-scale database validation:

scry-proxy

In-band query capture with <1ms overhead. Sits between your application and database to capture every query in real-time.

Drop-in replacement for direct connections
Circuit breaker and connection pooling
Privacy-preserving query fingerprinting

scry-backfill

Out-of-band data replication for shadow databases. Keeps your test environment synchronized with production data.

Logical replication with CDC support
Rate limiting to protect source database
Automatic schema change handling

                            When to Use Each
                            scry-proxy alone: Query analysis, performance monitoring, migration planning
scry-backfill alone: Creating shadow databases, data synchronization
Both together: Full production-scale validation - replay captured queries against shadow databases with real data

                        

How It Works

ScryData Proxy intercepts PostgreSQL wire protocol messages, extracts query metadata, and forwards requests to the backend with minimal overhead. Events are published asynchronously, ensuring query latency is never impacted.

┌──────────┐         ┌────────────────────────────┐         ┌──────────┐
│          │         │       ScryData Proxy       │         │          │
│  Client  │────────▶│  - Circuit Breaker         │────────▶│ Postgres │
│  (App)   │◀────────│  - Connection Pool         │◀────────│ Database │
│          │         │  - Event Publisher         │         │          │
└──────────┘         └────────────────────────────┘         └──────────┘
                                  │
                                  │ Async (best-effort)
                                  ▼
                     ┌─────────────────────────┐
                     │   Analytics Service     │
                     │   (HTTP + FlexBuffers)  │
                     └─────────────────────────┘
                                  │
                                  │ Scrape
                                  ▼
                     ┌─────────────────────────┐
                     │  Prometheus + Grafana   │
                     └─────────────────────────┘

Core Features

Deep Observability

Capture per-query metrics with value fingerprinting for hot data detection. FlexBuffers serialization for high-throughput event publishing.

Lock-Free Circuit Breaker

Three-state circuit breaker using atomic operations. Automatic failover with ~100μs overhead protects your database during failures.

Predictive Health Monitoring

Active and passive health checks with EMA baseline anomaly detection. Catch degradation before it impacts production.

Protocol-Agnostic Pooling

Connection pooling with automatic state reset. Prevents connection exhaustion even when applications have their own pools.

Production-Grade Resilience

Exponential backoff retries with jitter, integrated with circuit breaking and health monitoring for graceful degradation.

Privacy-First Design

Blake3 cryptographic fingerprinting anonymizes query values while preserving pattern analysis for compliance.

ScryData vs Traditional Poolers

ScryData provides all the connection pooling capabilities of traditional tools like PgBouncer, plus production-grade observability and resilience:

Feature	PgBouncer	ScryData
Connection Pooling	✓	✓
Connection Reuse	✓	✓
Per-Query Metrics	✗	✓
Query Anonymization	✗	✓
Hot Data Detection	✗	✓
Circuit Breaker	✗	✓
EMA-based Anomaly Detection	✗	✓
Retry with Backoff	✗	✓
Query Timeline Breakdown	✗	✓
Prometheus Metrics	Basic	Comprehensive

Performance Characteristics

ScryData is designed for production workloads with strict performance requirements:

~100μs

Target latency overhead

~500μs

Typical overhead

10K+

Queries/sec tested

~20MB

Memory for 100 connections

Lock-Free Operations

Critical path operations use no locks for predictable latency:

Circuit breaker check: 10-50ns (atomic load)
Metrics recording: <300ns (atomic increments + histogram)
Event batching: <100ns (lock-free channel send)

Quick Start

Get ScryData running in minutes:

# Configure via environment variables
export SCRY_BACKEND__HOST=localhost
export SCRY_BACKEND__PORT=5432
export SCRY_RESILIENCE__CIRCUIT_BREAKER__ENABLED=true
export SCRY_PUBLISHER__ANONYMIZE=true

# Run the proxy (listens on 127.0.0.1:5433)
./scry-proxy

# Connect through the proxy
psql -h 127.0.0.1 -p 5433 -U postgres

# View metrics
curl http://localhost:9090/metrics

For detailed configuration options, see the Configuration Guide.

Ready to Eliminate Migration-Day Anxiety?

Join platform engineers and database architects using ScryData to validate database changes against real production workloads.

Request Early Access