Responsibilities
Job overview and responsibility
- Design, build, and maintain scalable data pipelines for real-time and historical data processing.
- Set up and optimize databases, data warehouses, and ETL processes to manage large-scale time-series data from sensors (energy consumption, temperature, humidity, occupancy) and external sources (weather, utility data).
- Experience in handling and processing large data input volumes (1000s of rows per second)
- Collaborate across Product, Customer Success, Installation, and Tech Support to align data architecture with business needs and ensure data quality.
- Implement monitoring and logging systems to ensure pipeline reliability and quickly troubleshoot issues.
- Work with data scientists to streamline data access, facilitate experimentation, and support predictive model deployment.
Required Skills And Experiences
- Bachelor's or Master's degree in Computer Science, Engineering, or a related field.
- 5+ years of experience as a Data Engineer, with a proven track record in building and scaling data infrastructure.
- Experience managing time-series data is a must.
- Expert knowledge in SQL (Postgres), PL/pgSQL, DBT.
- Ability to read and optimize query plans
- Strong skills in data pipeline and ETL design, with experience in tools like Metaflow, Airflow, and TimescaleDB.
- Proficiency in Python, and familiarity with data manipulation frameworks.
- Familiarity with data warehousing principles
