Job Description
Role: Senior Data Architect with Databricks experience.
Full time Professional
Remote - US
Summary: Onix is seeking an experienced Senior Data Architect/Principal Data Engineer that will play a key role in designing and implementing data solutions that meet client business needs and support the overall cloud data architecture. The ideal candidate will have a strong understanding of Lakehouse Architecture, data modeling, and data management, with a significant focus on data warehouse/Lakehouse design and development. The candidate will have experience in cloud platforms such as Azure, AWS and GCP.
Location: Remote, United States
Scope/Level of Decision Making: This is an exempt position operating under limited decision-making and supervision. Position performs a variety of assigned activities, referring more complex issues to the manager.
Primary Responsibilities:
Lead the Design, development, and implementing of Lakehouse & data warehouses architecture in a cloud based data environment that integrates with an overall data architecture.
Work closely with business stakeholders to interpret their data needs and translate them into technical requirements.
Have a good understanding of data governance and develop & implement data governance policies and procedures.
Ensure data quality and integrity by implementing data testing and validation processes.
Deploying and debugging cloud data initiatives as needed in accordance with best practices throughout the development lifecycle.
Managing cloud data environments in accordance with company security guidelines.
Lead the design and build of reusable, repeatable solutions and components for future use.
Lead and mentor a team of data engineers to implement your solution designs
Stay abreast of current and emerging trends and technologies in the data and AI/ML field. Educate delivery teams on the implementation of new cloud-based data analytics initiatives, providing associated training as required.
Partner with Delivery, and Support teams to find opportunities to reduce manual effort needed to complete deployments.
Consult on Professional Services Engagements to help our customers design and implement data warehouse solutions. Lead and develop best practices for the larger data analytics delivery team.
Provide client presentations to review project design, outcomes and recommendations.
Employ exceptional problem-solving skills, with the ability to see and solve issues before they snowball into problems.
Lead the orchestration and automation cloud-based data platforms.
Preferred Skills and Experience:
7+ years experience in Data Architecture, data engineering & analytics in areas such as performance tuning, pipeline integration & infrastructure configuration
10+ years of consulting experience
Completed Databricks Data Engineering Professional/Associate certification OR Cloud Certification (Azure, AWS, GCP)
Working knowledge of two or more common Cloud ecosystems (AWS, Azure, GCP) with deep expertise in at least one
Deep experience with distributed computing with Spark with knowledge of Spark runtime internals & Spark Structured Streaming
Working knowledge of MLOps with a strong understanding of essential components of MLOps Architecture to build, train and deploy models
Current knowledge across the breadth of Databricks product and platform features
Familiarity with optimizations for performance and scalability
Data Pipelining Experience leveraging Databricks Delta Live Tables and Data Built Tool (DBT)
Experience with terraform, Git, CI/CD tools as well as Automation and Integration testing
Thorough understanding of Databricks Delta, Iceberg, and Hudi
Experience in best Spark Best Practices & Notebook, Cluster Creation & Configuration
Ingestion Patterns & Data Quality enforcement techniques within Databricks, Spark and DLT.
Understanding of Constraints, Expectations, CDC, CDF, SCD Type 1/Type 2
Understanding of Unity Catalog & DBX Governance/Security Models
Familiarity with leveraging Databricks REST APIs - Testing and Deployment - SCIM API & DBX CLI
Education: Bachelor's Degree or equivalent experience required.
Travel Expectation: Up to 15%
It is the policy of Onix to ensure equal employment opportunity in accordance with the Ohio Revised Code 125.111 and all applicable federal regulations and guidelines. Employment discrimination against employees and applicants due to race, color, religion, sex, (including sexual harassment), national origin, disability, age (40 years old or more), military status, or veteran status is illegal. Onix will only employ those who are legally authorized to work in the United States or Canada.
