Airport Extension: Nearing Release

Our Airport DuckDB extension is approaching its first community release. Our engineering team has focused extensively on test coverage and bug resolution to ensure enterprise-grade reliability for client deployments.

Client Feedback Welcome: We're actively seeking input from beta clients and early adopters regarding any outstanding issues or feature requirements for the initial release. Your feedback directly influences our development priorities and ensures we deliver solutions that address real-world implementation challenges.

Developer Ecosystem: Supporting Tools and Frameworks

Query.Farm has developed three complementary Python modules to support Airport extension implementations and Arrow Flight server development:

query-farm-flight-server - A comprehensive framework that simplifies Airport server development for our clients. This production-ready solution includes enterprise authentication, intelligent rate limiting, type-safe RPC validation, and streamlined Arrow integration patterns.

query-farm-airport-test-server - A reference implementation and testing infrastructure built on our flight server framework. This server powers our continuous integration pipeline and is publicly accessible at ci.airport.query.farm. The deployment demonstrates seamless Google Cloud Run integration, complete with containerization and automated CI/CD workflows. Client developers can utilize this infrastructure by setting AIRPORT_TEST_SERVER=grpc+tls://ci.airport.query.farm for source builds.

query-farm-duckdb-json-serialization - A specialized parser for DuckDB's JSON serialization format used in predicate pushdown operations. This module eliminates implementation complexity for organizations building advanced query optimization capabilities.

Success Story: Airport Extension in Production

Boilstream has successfully deployed our Airport extension in their production streaming data platform, demonstrating the extension's scalability and reliability in demanding enterprise environments. This deployment validates our architectural decisions and showcases the real-world value our DuckDB extensions deliver.

Current Development: Advanced Scan Optimization Consulting

Addressing Client Infrastructure Challenges

Through our consulting engagements, we've identified a critical gap in the Python ecosystem: the lack of sophisticated query planning and file scan optimization tools. This challenge consistently emerges in client projects managing distributed file collections with complex data access patterns.

Traditional scan planning appears straightforward in theory—given file URLs with known column ranges, filtering queries should efficiently identify relevant files through interval overlap analysis. However, production environments present significantly more complex scenarios that require specialized expertise to solve effectively.

Common Production Query Patterns:

• Complex disjunctive filtering: event_time < 400 or event_time > 5600

• Conditional logic branches: case when event_time < 500 then device_id = 5 else device_id = 6 end

• Multi-dimensional constraints: event_time > 8000 and device_id in (1, 3, 4)

While Iceberg and Delta Lake provide integrated scan planning, many organizations utilize alternative storage formats that require custom optimization solutions. Our Airport extension clients often need sophisticated predicate pushdown implementations that go beyond standard offerings, so we're creating code to make it easier.

Query.Farm's Engineering Approach

We're developing an advanced Python module using sqlglot to address these optimization challenges systematically. This solution incorporates multiple indexing strategies: numeric range indexing for boundary conditions and probabilistic Bloom filters for equality and membership operations.

This development represents both a client deliverable and a strategic contribution to the broader Airport/DuckDB ecosystem, addressing fundamental infrastructure gaps that impact data-intensive applications.

Specialized DuckDB Extension Consulting

Query.Farm specializes in custom DuckDB extension development and advanced data infrastructure consulting. Our team brings deep expertise in distributed systems, query optimization, and production deployment strategies specifically within the DuckDB ecosystem.

Our Consulting Services:

• Custom DuckDB extension development and optimization

• Airport extension integration and deployment support

• Arrow Flight server architecture and performance tuning

• Advanced predicate pushdown and scan optimization solutions

• DuckDB ecosystem integration and migration planning

We work with organizations ranging from data infrastructure startups to enterprise clients implementing large-scale analytical systems. Our extensions are designed for production environments and backed by comprehensive support and maintenance services.

Ready to discuss your DuckDB extension requirements? Contact our team at hello@query.farm to explore how we can accelerate your data infrastructure goals with custom-built solutions and expert consulting services.