Note: The job is a remote job and is open to candidates in USA. Ciklum is a custom product engineering company that supports multinational organizations and scaling startups. They are looking for a Senior Data Engineer to join their team, focusing on full stack data engineering, including ingestion, transformation, and analytics enablement using modern Azure-based data platforms.
Responsibilities
- Manage OneLake structures and shortcuts for scalable enterprise data access
- Design scalable Lakehouse solutions using Medallion Architecture (Bronze, Silver, Gold)
- Build and optimize Delta Lake tables for analytics, reporting, and AI workloads
- Develop data pipelines using Fabric Data Factory, Spark, and Notebooks
- Create ingestion and transformation workflows for structured and semi-structured data
- Implement orchestration, scheduling, monitoring, and recovery for enterprise pipelines
- Design dimensional models (Star/Snowflake schemas) for BI and semantic layers
- Build curated Gold-layer datasets for analytics and AI consumption
- Support integration with Power BI semantic models and reporting platforms
- Develop batch and incremental pipelines across Azure and external systems
- Orchestrate ETL/ELT workflows using Fabric Pipelines and Azure Data Factory
- Integrate Fabric platforms with APIs, AI services, and enterprise applications
- Support MCP integration, AI workflows, and rapid prototyping initiatives
- Collaborate on ReactJS-based apps, dashboards, and AI-driven user experiences
- Automate workflows using Azure Functions, Logic Apps, Git, CI/CD, and Azure DevOps
- Develop and optimize PySpark notebooks for transformation, cleansing, and enrichment
- Build efficient SQL queries, views, and stored procedures in Fabric Warehouse / Azure SQL
- Implement optimization techniques including partitioning, caching, and query tuning
- Monitor pipeline performance, troubleshoot failures, and improve system reliability
- Implement logging, alerting, and operational best practices
- Utilize AI-assisted development tools such as GitHub Copilot and modern AI coding assistants
- Rapidly prototype and deliver scalable engineering solutions with minimal guidance
- Implement RBAC and secure data access across Fabric workspaces and Azure environments
- Apply data quality validations and governance best practices
- Support metadata management and lineage using Microsoft Purview
- Collaborate with Data Architects, Analysts, BI Developers, Product Teams, and Business Stakeholders
- Translate business requirements into scalable data and application solutions
- Participate in Agile delivery processes, code reviews, and pull request workflows
Skills
- 6 years in Data Engineering
- Hands-on exposure to Microsoft Fabric (preferred) or strong Azure Data Engineering background with willingness to learn Fabric
- Microsoft Fabric (Data Factory, OneLake, Synapse Data Engineering – basics)
- Azure Data Services: ADLS Gen2, Azure SQL, Blob Storage
- Strong SQL (joins, aggregations, performance tuning basics)
- Python (PySpark) for data transformation
- Azure Data Factory / Fabric Pipelines
- Azure Functions / Logic Apps
- Event Hubs / streaming concepts
- Microsoft Purview (basic exposure)
- Cosmos DB
Benefits
- Strong community: Work alongside top professionals in a friendly, open-door environment
- Growth focus: Take on large-scale projects with a global impact and expand your expertise
- Tailored learning: Boost your skills with internal events (meetups, conferences, workshops), Udemy access, language courses, and company-paid certifications
- Endless opportunities: Explore diverse domains through internal mobility, finding the best fit to gain hands-on experience with cutting-edge technologies
- Care: Healthcare, Basic Life Insurance, Short and Long-term disability insurance according to the Company’s Benefit Plans
Company Overview
Company H1B Sponsorship