Skip to content
Vishal-770Public

About

Ultra-fast, Kubernetes-native EVM indexer in Rust.

Topics

Resources

Contributing

Security policy

Stars

2 stars

Watchers

0 watching

Forks

Repository files navigation

Logrix Banner

Ultra-fast, Kubernetes-native EVM indexer in Rust.
Write custom event indexing logic in TypeScript with logrix-sdk and deploy production indexer clusters in seconds.


Production Indexing Performance

Verified live on Ethereum Mainnet (Aave v3 Core Pool 0x8787...4E2):

Metric Measurement Architecture & Hardware
Blocks Indexed 9,978,560+ blocks (16,291,127 -> 26,269,700+) AWS EKS (t3.medium worker nodes)
Contract Actions Tracked 279,000+ state entities Aurora Serverless v2 PostgreSQL
Pipeline Latency < 3 blocks from real-time chain head Zero-copy vector deserialization
Ingestion Throughput > 20,000 events/sec peak Vectorized array UNNEST SQL writes
Queue Backlog (Live & Backfill) 0 messages (Drained) Amazon SQS + KEDA dual-queue scaling
Dead Letter Queue (DLQ) 0 dropped blocks Atomic cursor gap reconciliation

Highlights

  • Custom Event Logic with TypeScript: Write type-safe handlers with @logrix/sdk (compiled to sandboxed WebAssembly) or zero-code declarative YAML.
  • High-Throughput Ingestion: Vectorized PostgreSQL writes (UNNEST arrays) processing >20,000 events/sec.
  • Reorg-Safe Atomic Rollbacks: Rolling parent-hash ring buffer detects reorgs, applies atomic state rollbacks (is_reverted = TRUE), and re-indexes canonical blocks with self-healing gap reconciliation.
  • RPC Cost & Rate-Limit Protection: Multi-provider failover pool with per-node circuit breakers, decorrelated full jitter, and built-in Compute Unit (CU) budget tracking.
  • Dual Storage Tiering: Hot GraphQL entities in PostgreSQL; raw block envelopes compressed with zstd into cold S3/MinIO buckets, reducing database storage costs by up to 85%.
  • Kubernetes-Native Architecture: Active-passive Ingester leader election via Kubernetes Leases and event-driven worker autoscaling via KEDA (1 to 50+ pods).
  • Strict Core Safety Contracts: Core ports have no HTTP dependencies, queue handles preserve source lineage, and checkpoints require canonical non-zero block hashes.

Quickstart (@logrix/sdk)

1. Initialize Project

npx @logrix/sdk init my-indexer
cd my-indexer
npm install

2. Generate Typed Bindings

npm run codegen

3. Write Event Handler (src/mapping.ts)

import { TransferEvent } from "./generated/events";
import { Transfer } from "./generated/schema";

export function handleTransfer(event: TransferEvent): i32 {
  let entity = new Transfer(event.transactionHash + "-" + event.logIndex.toString());
  entity.blockNumber = event.blockNumber;
  entity.fromAddress = event.params.from.toHexString();
  entity.toAddress = event.params.to.toHexString();
  entity.amount = event.params.value;
  entity.transactionHash = event.transactionHash;
  entity.timestamp = event.blockTimestamp;
  entity.save();
  return 0;
}

4. Build & Export Values

npm run build
npm run export-values

Outputs indexer-values.yaml containing the compiled WASM handler, GraphQL schema, and contract manifest ready for deployment.


Deployment

Logrix provides two deployment paths via official Helm charts:

Option A: Self-Hosted Kubernetes (Default)

Deploys to any standard Kubernetes cluster (Minikube, Kind, K3s, EKS, GKE, or bare-metal). Includes built-in in-cluster PostgreSQL 16 and RabbitMQ 3.13 out of the box:

helm install my-indexer oci://ghcr.io/vishal-770/charts/logrix \
  -f indexer-values.yaml

Port-forward the GraphQL API server:

kubectl port-forward svc/my-indexer-logrix-api 4000:4000

Open http://localhost:4000/ for GraphiQL IDE, or query http://localhost:4000/graphql.

Optional Cold S3 Archiving:

helm install my-indexer oci://ghcr.io/vishal-770/charts/logrix \
  -f indexer-values.yaml \
  --set builtIn.minio=true

Option B: AWS Production Deployment (EKS + Terraform)

Production cloud architecture utilizing AWS EKS, AWS Aurora Serverless v2 PostgreSQL, Amazon SQS, and Amazon S3. Live deployments strictly consume the remote published OCI Helm chart (oci://ghcr.io/vishal-770/charts/logrix).

For detailed operational procedures, see the AWS EKS Production Operations Guide. For end-to-end details on zero-credential IAM, IRSA token projection, and Secrets Manager integration, see AWS Security Architecture (IRSA & Secrets Management). For multi-AZ subnetting, route tables, and S3 Gateway endpoints, see AWS Networking & VPC Architecture.

  1. Provision Infrastructure (Terraform):

    cd deploy/terraform/aws
    terraform init
    terraform apply
  2. Connect EKS & Install KEDA Autoscaler:

    aws eks update-kubeconfig --name <cluster-name> --region <aws-region>
    
    # Install KEDA for SQS queue-driven processor autoscaling
    helm repo add kedacore https://kedacore.github.io/charts
    helm repo update
    helm install keda kedacore/keda --namespace keda --create-namespace
  3. Configure Kubernetes Secret Vault:

    RAW_DB_URL=$(aws secretsmanager get-secret-value \
      --region <aws-region> \
      --secret-id "<rds-secret-arn>" \
      --query SecretString --output text | jq -r .url)
    
    # Strip ?sslmode=require for internal VPC connectivity
    CLEAN_DB_URL="${RAW_DB_URL%\?*}"
    
    kubectl create namespace logrix
    kubectl create secret generic logrix-secrets -n logrix \
      --from-literal=DATABASE_URL="$CLEAN_DB_URL" \
      --from-literal=RPC_URL="<primary-rpc-url>" \
      --from-literal=RPC_FALLBACK_URLS="<fallback-rpc-urls>"
  4. Run Initial Database Migrations:

    kubectl run logrix-migrator --rm -i --restart=Never \
      --image=ghcr.io/vishal-770/logrix:0.8.4 \
      -n logrix \
      --env="DATABASE_URL=$CLEAN_DB_URL" \
      -- command logrix migrate
  5. Deploy from Remote OCI Helm Chart:

    helm upgrade --install my-indexer oci://ghcr.io/vishal-770/charts/logrix \
      --version 0.8.4 \
      -n logrix \
      --disable-openapi-validation \
      --set config.existingSecret="logrix-secrets" \
      --set config.chainId=1 \
      --set config.queueDriver=sqs \
      --set config.sqsLiveUrl="<sqs_live_queue_url>" \
      --set config.sqsBackfillUrl="<sqs_backfill_queue_url>" \
      --set config.s3Bucket="<s3_bucket_name>" \
      --set processor.keda.enabled=true \
      --set processor.keda.sqs.queueUrl="<sqs_live_queue_url>" \
      --set serviceAccount.annotations."eks\.amazonaws\.com/role-arn"="<processor_irsa_arn>" \
      --set ingester.serviceAccount.annotations."eks\.amazonaws\.com/role-arn"="<ingester_irsa_arn>" \
      --set processor.serviceAccount.annotations."eks\.amazonaws\.com/role-arn"="<processor_irsa_arn>" \
      --set api.serviceAccount.annotations."eks\.amazonaws\.com/role-arn"="<api_irsa_arn>"

    Note on Versioning: The remote OCI Helm chart version (--version 0.8.4) automatically sets the container image tag (ghcr.io/vishal-770/logrix:0.8.4) via appVersion. Environment overlays (values-aws.yaml) omit image.tag so that bumping the chart version cleanly updates workloads without version pinning drift.

  6. Verify Indexing & GraphQL Endpoints:

    # Verify all pods are 1/1 Running
    kubectl get pods -n logrix
    
    # Port-forward and query the GraphQL API
    kubectl port-forward svc/my-indexer-logrix-api 4000:4000 -n logrix
    
    # In another terminal:
    curl -s -X POST http://localhost:4000/graphql \
      -H "Content-Type: application/json" \
      -d '{"query": "{ health { status version } checkpoint(chainId: 1) { chainId lastIndexedBlock lastBlockHash } }"}'

Configuration & Tuning Reference

All Logrix components are fully configurable through environment variables, CLI flags, and YAML manifests with sensible production defaults.

Subsystem Key Environment Variables Default Value Crate Documentation
PostgreSQL Database DATABASE_URL
DATABASE_MAX_CONNECTIONS
DATABASE_MIN_CONNECTIONS
postgres://logrix:logrix@localhost:5432/logrix
10
2
logrix-store-postgres
RabbitMQ Queue RABBITMQ_URL
RABBITMQ_PREFETCH
RABBITMQ_TIMEOUT_SECS
amqp://logrix:logrix@localhost:5672/%2f
100
10s
logrix-queue-rabbitmq
AWS SQS Queue QUEUE_DRIVER=sqs
SQS_LIVE_URL
SQS_WAIT_TIME_SECONDS
SQS_MAX_MESSAGES
rabbitmq
"" (Required if SQS)
2s
1
logrix-queue-sqs
JSON-RPC Gateway RPC_URL
RPC_FALLBACK_URLS
RPC_TIMEOUT_SECS
RPC_CB_THRESHOLD
RPC_COOLDOWN_SECS
https://sepolia-rollup.arbitrum.io/rpc
None
10s
5
30s
logrix-rpc-gateway
EVM Log Extraction CHAIN_ID
CHUNK_SIZE
CHAIN_HEAD_POLL_MS
MAX_QUEUE_DEPTH
421614
10
1500ms
1000
logrix-chain-evm
Gap Reconciler & Reorg RING_BUFFER_DEPTH
RECONCILER_INTERVAL_SECS
RECONCILER_CHUNK_SIZE
RECONCILER_LAG_THRESHOLD
128
30s
10
50
logrix-reconciler
WASM Handlers WASM_FUEL_LIMIT
WASM_MAX_MEMORY_MB
MANIFEST_PATH
1,000,000
128 MB
None
logrix-handlers
GraphQL API Server GRAPHQL_MAX_DEPTH
GRAPHQL_MAX_COMPLEXITY
GRAPHQL_DEFAULT_LIMIT
GRAPHQL_MAX_LIMIT
GRAPHQL_QUERY_TIMEOUT_SECS
7
200
100
1000
5s
logrix-api
Webhook Dispatcher ENABLE_WEBHOOKS
WEBHOOK_URL
WEBHOOK_SECRET
WEBHOOK_TIMEOUT_SECS
WEBHOOK_MAX_RETRIES
false
None
""
5s
1
logrix-webhook
Cloud Object Storage S3_BUCKET
S3_ENDPOINT
S3_PREFIX
AWS_REGION
None
None
None
us-east-1
logrix-blob-s3
Fault Tolerance Primitives Circuit breakers, Token bucket rate limiting, Jitter backoff Tuned per subsystem logrix-resilience
Unified CLI Runner Full flag & subcommand reference See CLI guide logrix-cli

For complete per-crate options, custom programmatic builders, and architecture details, refer to each individual crate README linked above.

Core safety and adapter boundaries

logrix-core contains domain models, typed errors, and pure port traits only. Runtime block ranges use BlockRangeJob::try_new, checkpoints use Checkpoint::try_new, and every MessageHandle carries the concrete source QueueType. Kubernetes Lease coordination is implemented by the CLI infrastructure adapter with valid TLS, bounded requests, fail-closed errors, and optimistic resourceVersion renewal.


Monitoring & Dashboards

1. Logrix Studio (Web Dashboard & GraphQL Explorer)

Interactive web dashboard served directly by the API container at http://localhost:4000/ (or /graphiql):

  • Schema explorer, interactive query builder, and entity inspection.
  • Real-time event streaming and live mutations over WebSockets (/ws).
  • Access via port-forward:
    kubectl port-forward svc/my-indexer-logrix-api 4000:4000

2. Pre-Built Grafana Dashboard

A production-ready Grafana dashboard is included at deploy/dashboards/logrix-overview.json. Import it directly into Grafana to monitor:

  • Indexed Head Height: Live block height vs chain head.
  • Ingestion Throughput: Rate of indexed events per second (rate(logrix_events_indexed_total[1m])).
  • Database Connection Pool: Active vs idle connections in PostgreSQL.
  • Reorg & Rollback Activity: Live alert counts on chain reorganizations (logrix_reorg_events_total).
  • Dynamic Entity Volume: Cumulative custom entity counts (logrix_entities_total).

3. Kubernetes Cluster Inspection

Monitor pods and streaming processing logs with standard kubectl:

# Check pod status and scaling
kubectl get pods -l app.kubernetes.io/instance=my-indexer

# Stream processor ingestion logs
kubectl logs -f deployment/my-indexer-logrix-processor

# View KEDA autoscaling status
kubectl get scaledobject

4. Prometheus Metrics & Operator Integration

The API service exposes Prometheus metrics at http://localhost:4000/metrics.

To enable automatic scraping via Prometheus Operator in your Helm values:

serviceMonitor:
  enabled: true
  interval: 15s

5. Production Operations & Inspection Guides

  • AWS Infrastructure Inspection: Full A-to-Z AWS CLI commands across EKS, Aurora Serverless v2, SQS, S3, VPC PrivateLink, and Cost Explorer in the AWS Infrastructure Inspection Guide.
  • Kubernetes Cluster Operations: Comprehensive kubectl commands for pod troubleshooting, rolling upgrades, and RBAC in the Kubernetes Cluster Operations Guide.
  • KEDA Autoscaling Architecture: Plain-English guide on how KEDA coordinates between AWS SQS, IAM/IRSA, and Kubernetes Pods in the KEDA Autoscaling Guide.
  • Database Browser Web UI: Step-by-step instructions for connecting a browser UI (pgweb) to AWS Aurora PostgreSQL in the Database Web UI Guide.
  • S3 Cold Storage Architecture: Detailed guide on tiered storage, Zstd block compression, and VPC Gateway endpoints in the S3 Cold Storage Guide.

License

Licensed under the Apache License, Version 2.0.

About

Ultra-fast, Kubernetes-native EVM indexer in Rust.

Topics

Resources

Contributing

Security policy

Stars

2 stars

Watchers

0 watching

Forks

Releases

Packages

Used by

Contributors

Languages