Senior Data Engineer

Role Title:Senior Data Engineer

ECLARO: A quick Summary

ECLARO is an award-winning professional services firm headquartered in New York City and operating in the U.S., Canada, UK, Ireland, Australia and the Philippines. We are dedicated to a singular purpose: providing the Right People to meet every client's needs and solve business challenges through strategic staffing, permanent placement, custom outsourcing & offshoring. Utilizing our proprietary TRINIT-E Service Maturity Model, we help clients implement programs to promote innovation, automation and process improvement.

About the Role:
We are looking for a Data Engineer to help us take our data platform to the next level. Reporting to the Lead Data Engineer or Director of Applications and Data, you will join a small but mighty team, working closely with the Lead Data Engineer, Data Analyst, and other IT professionals. In this role, you'll design, build, and maintain scalable data pipelines and integrations using Microsoft Fabric and Azure technologies, ensuring data is accessible, reliable, and optimized for analytics and decision-making.

You'll collaborate with team members to implement best practices for data governance, security, and performance, and contribute to ongoing improvements in our data architecture. This position offers the opportunity to work on impactful projects, support our data-driven culture, and grow your expertise in a modern cloud-based environment.

Duties/Responsibilities: Engineering & Integrationo Design, build, and maintain data pipelines and integration processes using Microsoft Fabric, Azure Data Factory, and related technologies.
o Implement and optimize ETL/ELT workflows, ensuring data quality, reliability, and performance.
o Test, validate, and monitor data pipelines to ensure integrity and scalability.
o Automate data workflows and support CI/CD practices for data solutions. Collaboration & Communicationo Work collaboratively with the Data & IT Team and cross-functional partners to identify data requirements, develop effective data models, and deliver high-quality data solutions that support organizational goals.
o Collaborate with external contractors to enhance data solutions; coordinate their work, provide guidance, and ensure alignment with project goals.
o Contribute to technical discussions, process improvements, and mentor junior team members as needed. Governance, Security & Documentationo Support data governance and security by following established best practices and compliance requirements; implement security controls and monitor for compliance.
o Document solutions and processes clearly for cross-functional understanding; maintain technical and user documentation. Legacy Systems Supporto Support and maintain legacy data technologies as needed (SQL Server, T-SQL).
o Maintain and enhance legacy Microsoft environments, including troubleshooting production issues and implementing tactical improvements as needed. Continuous Learningo Stay current with advancements in Microsoft Fabric, Azure, and related data engineering tools.Qualifications
Education/Experience
Bachelor's or Master's degree in Computer Science, Information Systems, or a related field.
Proven experience as a Data Engineer or in a similar role, preferably within the insurance or financial sector.
Strong understanding of data management principles, including data modeling, schema design, data lineage, and data quality assurance.
Demonstrated ability to analyze complex data sets, troubleshoot issues, and optimize data workflows.
Experience handling sensitive data such as PII and PHI, with an understanding of data privacy controls, secure data handling practices, and awareness of regulatory requirements like GDPR NYDFS, CCPA, or HIPAA.
Excellent interpersonal and communication skills, with the ability to convey complex technical concepts to both technical and non-technical stakeholders.
Creative problem-solving mindset and adaptability to evolving technologies and business needs.
Positive, collaborative attitude and willingness to tackle challenges head-on.

Technical Competencies Data Engineering & Integrationo Proficiency in designing and implementing data integration workflows using Microsoft Fabric Dataflows and Azure Data Factory.
o Expertise in managing and optimizing data within Microsoft Fabric's OneLake and Azure Data Lake Storage.
o Strong experience in designing, building, and optimizing data warehousing solutions using Microsoft Fabric Data Warehouse and Azure SQL.
o Knowledge of distributed processing frameworks (e.g., Apache Spark) within Fabric and Azure environments.
o Experience with data modeling, profiling, cleansing, and validation. Governance, Security & Complianceo Understanding of data governance principles and best practices within Microsoft Fabric and Azure.
o Experience implementing role-based access control (RBAC), permissions management, and security controls.
o Knowledge of compliance requirements (e.g., GDPR, HIPAA) related to data privacy and security. Monitoring, Automation & DevOpso Skills in monitoring, optimizing, and troubleshooting data pipelines within Fabric and Azure Data Factory.
o Familiarity with diagnostic logs, metrics, and performance tuning for Fabric and Azure services.
o Proficiency in scripting languages (Python, PowerShell) for automating tasks and supporting CI/CD practices.
o Experience with version control (Git) and automated deployment pipelines. Collaboration & Communicationo Strong communication skills to collaborate effectively with data scientists, analysts, business stakeholders, and external contractors.
o Ability to document data solutions and processes clearly for cross-functional understanding.
o Experience mentoring junior team members. Legacy Systems Supporto Knowledge of and ability to support legacy data technologies, including Microsoft SQL Server and T-SQL.
o Familiarity with operating systems such as Windows, Linux, and UNIX. Continuous Learning & Adaptabilityo Proactively keep up with the latest Microsoft Fabric and Azure data services, features, and advancements.
o Adapt to new technologies and evolving business needs.