Senior Data Quality Engineer
Office in Chennai, Coimbatore
Data Quality Engineering
& others
We are seeking a Senior Data Quality Engineer to join our innovative team. In this role, you will ensure the accuracy, reliability, and performance of data systems through cutting-edge test automation, performance optimization, and advanced validation techniques. You will play a key role in maintaining data integrity and optimizing workflows to support critical business decisions.
Responsibilities
- Develop robust Python-based test frameworks for SQL validation, ETL verification, and stored procedure unit testing
- Automate data-driven testing with tools like pytest, Hypothesis, pandas, and tSQLt
- Implement AI/ML models for detecting anomalous behaviors in SQL transactions and for test case generation to cover edge scenarios
- Train Machine Learning models to predict slow queries and optimize database performance through indexing strategies
- Validate stored procedures, triggers, views, and business rules for consistency and accuracy
- Apply performance benchmarking with JMeter, SQLAlchemy, and AI-driven anomaly detection methods
- Conduct data drift detection to analyze and compare staging vs production environments
- Automate database schema validations using tools such as Liquibase or Flyway in CI/CD workflows
- Integrate Python test scripts into CI/CD pipelines (Jenkins, GitHub Actions, Azure DevOps)
- Design mock database environments to support automated regression testing for complex architectures
- Collaborate with cross-functional teams to develop scalable and efficient data quality solutions
Requirements
- 5+ years of working experience in data quality engineering or similar roles
- Proficiency in SQL Server, T-SQL, stored procedures, indexing, and execution plans with a strong foundation in query performance tuning and optimization strategies
- Background in ETL validation, data reconciliation, and business logic testing for complex datasets
- Skills in Python programming for test automation, data validation, and anomaly detection with hands-on expertise in pytest, pandas, NumPy, and SQLAlchemy
- Familiarity with frameworks like Great for developing comprehensive validation processes
- Competency in integrating automated test scripts into CI/CD environments such as Jenkins, GitHub Actions, and Azure DevOps
- Showcase of tools like Liquibase or Flyway for schema validation and database migration testing
- Understanding of implementing AI/ML-driven methods for database testing and optimization
Nice to have
- Knowledge of JMeter or similar performance testing tools for SQL benchmarking
- Background in AI-based techniques for detecting data drift or training predictive models
- Expertise in mock database design for highly scalable architectures
- Familiarity with handling dynamic edge case testing using AI-based test case generation