backGo to search

Senior Data Quality Engineer

Office in Chennai, Coimbatore
bullets
Data Quality Engineering
& others
bullets

We are seeking a Senior Data Quality Engineer to join our innovative team. In this role, you will ensure the accuracy, reliability, and performance of data systems through cutting-edge test automation, performance optimization, and advanced validation techniques. You will play a key role in maintaining data integrity and optimizing workflows to support critical business decisions.

Responsibilities
  • Develop robust Python-based test frameworks for SQL validation, ETL verification, and stored procedure unit testing
  • Automate data-driven testing with tools like pytest, Hypothesis, pandas, and tSQLt
  • Implement AI/ML models for detecting anomalous behaviors in SQL transactions and for test case generation to cover edge scenarios
  • Train Machine Learning models to predict slow queries and optimize database performance through indexing strategies
  • Validate stored procedures, triggers, views, and business rules for consistency and accuracy
  • Apply performance benchmarking with JMeter, SQLAlchemy, and AI-driven anomaly detection methods
  • Conduct data drift detection to analyze and compare staging vs production environments
  • Automate database schema validations using tools such as Liquibase or Flyway in CI/CD workflows
  • Integrate Python test scripts into CI/CD pipelines (Jenkins, GitHub Actions, Azure DevOps)
  • Design mock database environments to support automated regression testing for complex architectures
  • Collaborate with cross-functional teams to develop scalable and efficient data quality solutions
Requirements
  • 5+ years of working experience in data quality engineering or similar roles
  • Proficiency in SQL Server, T-SQL, stored procedures, indexing, and execution plans with a strong foundation in query performance tuning and optimization strategies
  • Background in ETL validation, data reconciliation, and business logic testing for complex datasets
  • Skills in Python programming for test automation, data validation, and anomaly detection with hands-on expertise in pytest, pandas, NumPy, and SQLAlchemy
  • Familiarity with frameworks like Great for developing comprehensive validation processes
  • Competency in integrating automated test scripts into CI/CD environments such as Jenkins, GitHub Actions, and Azure DevOps
  • Showcase of tools like Liquibase or Flyway for schema validation and database migration testing
  • Understanding of implementing AI/ML-driven methods for database testing and optimization
Nice to have
  • Knowledge of JMeter or similar performance testing tools for SQL benchmarking
  • Background in AI-based techniques for detecting data drift or training predictive models
  • Expertise in mock database design for highly scalable architectures
  • Familiarity with handling dynamic edge case testing using AI-based test case generation