Role context: The Senior Data Engineer within Optum Data Management designs, builds, and operates scalable, metadata driven data pipelines that support the Arkansas Medicaid Decision Support System, leveraging Azure Data Factory and Snowflake to ensure reliable ingestion, integration, and delivery of State Medicaid and CMS data with a focus on data quality and state partner collaboration.
Summary
A Senior Data Engineer will design, develop, and operate end-to-end data pipelines across staging, integration, and consumption layers, using metadata driven frameworks, Azure Data Factory, and Snowflake. The role emphasizes high quality data delivery, robust data transformations, and collaboration with state partners to meet Medicaid and CMS data needs.
Responsibilities
- Work with product owners, business users, and stakeholders to translate data product and reporting needs into scalable, dependable data engineering solutions, incorporating feedback iteratively throughout the delivery lifecycle.
- Design, develop, and maintain end-to-end data pipelines across staging, integration, and consumption layers, leveraging Azure Data Factory, Snowflake SQL, and metadata driven architectures.
- Develop and optimize high performance SQL queries, views, and stored procedures to support analytics, reporting, and outbound data extracts, while guiding analytics team members on efficient and cost effective usage.
- Architect, develop, and support inbound and outbound data integration processes including batch file ingestion, automated extracts, and scheduled deliveries to internal consumers, state partners, and external agencies.
- Develop and maintain GoAnywhere workflows and monitoring for inbound and outbound file transfers.
- Collaborate with internal stakeholders to understand business objectives, regulatory needs, and study requirements, identifying suitable technical solutions, integration patterns, and data models that balance performance, scalability, and maintainability.
- Serve as a technical subject matter expert and mentor, delivering training, design guidance, and code reviews on data engineering best practices, Snowflake optimization, and Azure Data Factory pipeline design.
- Author and maintain detailed technical documentation, including data mappings, record layouts, file specifications, interface contracts, workflow diagrams, and runbooks for data loads and extracts.
- Continuously assess and enhance data quality, reconciliation controls, monitoring, and operational efficiency across analytic and decision support system data processes.
- Establish and maintain partnerships with customers, data senders, and data consumers, serving as a subject matter expert for healthcare data domains.
- Collaborate with business and technical partners to support data acquisition initiatives, including onboarding new data sources, building business justifications, and executing secure, compliant data transfers.
- Communicate complex technical concepts clearly through written documentation, presentations, and discussions with audiences from technical teams to senior leadership.
- Foster a collaborative, inclusive environment that promotes knowledge sharing, continuous improvement, and professional development across the team.
- Demonstrate adherence to regulatory, security, and data governance obligations, including HIPAA, state and federal Medicaid requirements, and controls related to external data sharing and file exchange.
- Show the ability to work independently, manage multiple priorities, and adapt to evolving business and technology requirements.
- Apply strong analytical and troubleshooting skills to diagnose pipeline failures, data anomalies, and performance bottlenecks.
Requirements
- Minimum 2 years of relevant data engineering experience.
- 5+ years of hands-on data engineering experience with a focus on enterprise data warehousing and ETL architectures.
- 5+ years of ETL development experience designing and building pipelines and dataflows using Azure Data Factory and/or Snowflake, including parameterized and metadata driven designs.
- 3+ years of experience using Snowflake database, including creating data ingestion and extraction through stored procedures and leveraging Snowflake native utilities.
- 3+ years of experience with GitHub version control, including code merges and cross environment deployments using CI/CD pipelines.
- 2+ years of scripting experience (Batch, PowerShell, or similar) to support automation and operational tasks on the Azure platform.
- Experience processing structured and semi-structured data, including fixed width and delimited files, JSON, and XML.
- Working knowledge of healthcare standards such as NCPDP vF6, HL7, and FHIR, including ingestion and transformation of these formats.
- Residence in the Little Rock, AR area and willingness to work hybrid in the office at least three days per week.
Technologies
- Azure Data Factory
- Snowflake SQL
- Snowflake
- GoAnywhere MFT
- GitHub
- CI/CD pipelines
- Batch
- PowerShell
- Python
- Microsoft Purview
- Macula Automate
- Informatica Intelligent Cloud Services (IICS)
- REST
- RPC
- NCPDP vF6
- HL7
- FHIR
Benefits
- Comprehensive benefits package
- Incentive and recognition programs
- Equity stock purchase
- 401k contribution
Location
Little Rock, AR, hybrid
Salary
USD 91,700 - 163,700 per year
