Senior Data Quality Engineer

Office in Chennai, Coimbatore

Data Quality Engineering

We are seeking a Senior Data Quality Engineer to join our innovative team. In this role, you will ensure the accuracy, reliability, and performance of data systems through cutting-edge test automation, performance optimization, and advanced validation techniques. You will play a key role in maintaining data integrity and optimizing workflows to support critical business decisions.

Responsibilities

Develop robust Python-based test frameworks for SQL validation, ETL verification, and stored procedure unit testing
Automate data-driven testing with tools like pytest, Hypothesis, pandas, and tSQLt
Implement AI/ML models for detecting anomalous behaviors in SQL transactions and for test case generation to cover edge scenarios
Train Machine Learning models to predict slow queries and optimize database performance through indexing strategies
Validate stored procedures, triggers, views, and business rules for consistency and accuracy
Apply performance benchmarking with JMeter, SQLAlchemy, and AI-driven anomaly detection methods
Conduct data drift detection to analyze and compare staging vs production environments
Automate database schema validations using tools such as Liquibase or Flyway in CI/CD workflows
Integrate Python test scripts into CI/CD pipelines (Jenkins, GitHub Actions, Azure DevOps)
Design mock database environments to support automated regression testing for complex architectures
Collaborate with cross-functional teams to develop scalable and efficient data quality solutions

Requirements

5+ years of working experience in data quality engineering or similar roles
Proficiency in SQL Server, T-SQL, stored procedures, indexing, and execution plans with a strong foundation in query performance tuning and optimization strategies
Background in ETL validation, data reconciliation, and business logic testing for complex datasets
Skills in Python programming for test automation, data validation, and anomaly detection with hands-on expertise in pytest, pandas, NumPy, and SQLAlchemy
Familiarity with frameworks like Great for developing comprehensive validation processes
Competency in integrating automated test scripts into CI/CD environments such as Jenkins, GitHub Actions, and Azure DevOps
Showcase of tools like Liquibase or Flyway for schema validation and database migration testing
Understanding of implementing AI/ML-driven methods for database testing and optimization

Nice to have

Knowledge of JMeter or similar performance testing tools for SQL benchmarking
Background in AI-based techniques for detecting data drift or training predictive models
Expertise in mock database design for highly scalable architectures
Familiarity with handling dynamic edge case testing using AI-based test case generation