Data Governance

Data Governance Jobs

122 jobs found

1mo ago

Sr Data Engineer, Python + Spark (Data Federation skillset - Data Lakehouse - Eg: Starburst)

Photon New York, NY

New York, NY

1mo ago

$40k - $140k/yr Full time Python Azure Cloud Architecture

Apply Now

Job Description

Photon seeks a Senior Data Engineer in New York, NY (onsite) to lead data federation and lakehouse initiatives.

Responsibilities

Data Federation Architecture: Design and implement federated query layers using Starburst Enterprise or Trino to enable high-speed analytics across distributed data sources with minimal data movement.
ETL/ELT Pipeline Development: Build scalable, distributed data processing pipelines with Python and Apache Spark (PySpark).
Lakehouse Implementation: Manage and optimize Delta Lake, Apache Iceberg, or Apache Hudi to bring ACID transactions to the data lake.
Performance Tuning: Optimize Spark jobs and SQL queries across the federation layer to minimize latency and control compute costs.
Governance & Security: Implement fine-grained access control and data masking within the federation engine to safeguard privacy across connected platforms.

Requirements

Python & Spark: 5+ years of experience with Python and deep expertise in Apache Spark tuning (partitioning, shuffling, caching).
Data Federation Tools: Hands-on experience with Starburst Enterprise, Trino (Presto), or Dremio.
Lakehouse Ecosystem: Proven track record working with Delta Lake or Iceberg architectures.
Cloud Platforms: Extensive experience with AWS (EMR, S3, Glue), Azure (Databricks, ADLS), or GCP.
SQL Mastery: Expert-level SQL skills for complex analytical queries and query plan analysis.
Data Modeling: Proficiency in designing Star/Snowflake schemas and understanding Medallion Architecture (Bronze, Silver, Gold layers).

Technologies

Python
Apache Spark
PySpark
Starburst Enterprise
Trino (Presto)
Dremio
Delta Lake
Iceberg
Hudi
AWS
EMR
S3
Glue
Azure
Databricks
ADLS
GCP
SQL
dbt
Terraform
Pulumi
Kubernetes (K8s)

Benefits

Medical, vision, and dental benefits
401k retirement plan
Variable pay/incentives
Paid time off
Paid holidays

Compensation, Benefits and Duration

Minimum Compensation: USD 40,000
Maximum Compensation: USD 140,000
Compensation is based on actual experience and qualifications of the candidate. The above is a reasonable and good faith estimate for the role.
Medical, vision, and dental benefits, 401k retirement plan, variable pay/incentives, paid time off, and paid holidays are available for full-time employees.
This position is not available for independent contractors
No applications will be considered if received more than 120 days after the date of this post

View full job page →

1mo ago

Azure Data Engineer with Power BI and C# .NET exp--Need Government domain

infopeople corporation Queens, NY

Queens, NY

1mo ago

$55 - $65/hr Contract Python .NET Azure

Apply Now

Job Description

infopeople corporation is seeking an Azure Data Engineer with Power BI and C# .NET experience to design, build, and maintain cloud-based data solutions within a government domain. This onsite role in Queens, NY centers on delivering scalable data pipelines, analytics, and reporting capabilities that support government initiatives.

Responsibilities

Design and implement scalable data pipelines leveraging Azure Data Lake, Spark, Hadoop, and ETL tools such as Informatica and Talend to handle large-scale structured and unstructured data.
Create and optimize data models, data warehouses, and relational databases (MS SQL Server, Oracle, and others) to enable analytics and reporting.
Develop interactive Power BI dashboards and reports, integrating data from Looker, Apache Hive, and other linked datasets to yield actionable insights.
Collaborate with cross-functional teams to capture requirements, translate business needs into technical specifications, and build RESTful APIs for data access.
Author efficient code in C# .NET, Python, Bash, and shell scripting to automate workflows, improve performance, and support model training initiatives.
Maintain thorough documentation of data architecture, processes, and security controls aligned to government standards.
Participate in Agile development cycles, contributing to sprint planning, daily stand-ups, and ongoing improvement.

Requirements

Proven experience as an Azure Data Engineer or similar role in government or enterprise environments.
Strong proficiency in Azure data services, including Azure Data Lake and Azure Data Factory, for cloud-based data management.
Hands-on Power BI experience to create visualizations and reports; familiarity with Looker is a plus.
Proficient in C# .NET; Python for data analysis or automation tasks is highly desirable.
Deep understanding of big data technologies such as Hadoop, Spark, Apache Hive, and ecosystems like Kafka or related frameworks.
Extensive experience with SQL Server and Oracle databases; solid database design principles.
Familiarity with ETL tools like Informatica or Talend; modeling and analysis skills for training data are advantageous.
Knowledge of RESTful API development for secure data exchange; experience in an Agile environment.

Technologies

Azure Data Lake
Spark
Hadoop
Informatica
Talend
Microsoft SQL Server
Oracle
Looker
Apache Hive
Kafka
REST API
C# / .NET
Python
Bash
Shell Scripting
Power BI
Power BI Desktop
Power BI Report Builder
DAX
Data Gateway
Azure Data Factory
Azure Data Lake Storage
Azure Databricks
Azure Synapse Analytics
SSIS
Microsoft Purview
Terraform
Airflow
PySpark

Data Engineering & Architecture

Extensive ETL/ELT design and optimization experience
Advanced SQL skills (expert level)
Python / PySpark proficiency
Dimensional data modeling with star schema
REST API integrations

Azure Data Stack

Azure Data Factory
Azure Databricks
Azure Synapse Analytics
Azure Data Lake Storage

Microsoft Data Platform

Experience with SQL Server data warehouse environments
Working knowledge of SSIS and modernization or migration of SSIS workflows to Azure-based solutions

Power BI

Power BI Desktop (expert level)
Advanced DAX
Executive dashboard development
Paginated reports (Power BI Report Builder)
Data Gateway configuration
Incremental refresh
Row-Level Security (RLS)

Terraform (Infrastructure-as-Code)

Orchestration tools (Airflow or equivalent)
Security systems data integration experience
Experience with C# / .NET web application development for integration with internal systems or APIs

View full job page →

Data Governance Jobs

Sr Data Engineer, Python + Spark (Data Federation skillset - Data Lakehouse - Eg: Starburst)

Azure Data Engineer with Power BI and C# .NET exp--Need Government domain

Get Job Alerts