Jason Hoekstra

Ed-Fi Data Import

June 2023 C# .NET ETL Azure AWS Ed-Fi
Ed-Fi Data Import

Purpose-built middleware platform that reduced Ed-Fi integration timelines from months to days, deployed across major implementations nationwide including Texas, Florida, Rhode Island, and SaaS providers.

The Challenge

Direct Ed-Fi API integrations routinely took months, requiring coordination across multiple vendor, district, and state project teams. Integration work depended on scarce, highly specialized Ed-Fi engineers, and commercial ETL solutions like Boomi were cost-prohibitive for many education use cases. Several state and district initiatives were blocked entirely because data movement costs exceeded available budgets.

Role & Ownership

  • Founding Architect & Technical Lead
  • Originated and pitched the solution to the Michael & Susan Dell Foundation
  • Secured funding and was hired to design and deliver the platform
  • Owned system architecture, technical strategy, and delivery
  • Managed a ~$1M program budget and led implementation with HeadSpring as primary delivery partner
  • Full product lifecycle ownership: PoC → managed pilot → v1 → v2

Technical Approach

Architected a multi-tier middleware ETL platform optimized specifically for Ed-Fi ingestion. The key insight was abstracting Ed-Fi complexity so that a well-qualified data analyst, rather than an engineering team, could complete integrations.

  • Metadata-driven mapping engine using Ed-Fi schema metadata to dynamically generate point-and-click mapping interfaces
  • Web-based UI for configuration, mapping, and validation
  • Two CLI/agent services for transformation, processing, and ingestion
  • Shared template library with 90+ community-contributed templates for NWEA, ACT, SAT, DIBELS, i-Ready, STAR, SBAC, STAAR, and dozens more
  • Enterprise-grade logging, validation, and data quality diagnostics
  • Support for on-premises, Azure, and AWS deployment models

Key Results

  • Reduced integration timelines from months with multi-team coordination to workflows executable by individual data analysts
  • Replaced or avoided costly commercial ETL tooling and bespoke integration pipelines
  • Unlocked state and district projects previously stalled due to data movement cost and complexity
  • Supported 20+ enterprise data sources across assessments, finance, and custom domains
  • Deployed to many major Ed-Fi implementations nationwide

Deployments

  • Texas (2025) - Custom and finance domain ingestion
  • Florida - Statewide and district usage
  • Rhode Island - Assessment data imports
  • Other deployments via custom SaaS providers

Technologies

C#, .NET, ASP.NET, Ed-Fi ODS/API, Microsoft Azure, AWS