Senior Software Engineer- Streaming Pipelines

Remote - AmericasEngineering & DataAbout the roleLooking for aSenior Software Streaming Pipelines Engineer to reimagining how merchant data flows through modern streaming architectures. This isn’t your typical ETL role – you’ll be crafting solutions that challenge conventional approaches to data processing at global scale. What makes this exciting? We’ve developed a declarative pipeline framework using Apache Beam, bolthires Cloud Dataflow, and ClickHouse that transforms how data engineers build streaming systems.Imagine YAML-driven pipelines that eliminate boilerplate code, real-time stream processing across multiple regions, and an architecture that makes complex data transformations feel effortless. You’ll work across multiple languages – Kotlin, Ruby, Python, and Rust – choosing the right tool for each challenge, alongside dbt for elegant data modelling and our custom framework that turns pipeline development into a configuration exercise rather than a coding marathon. The puzzle? Replacing entrenched batch systems with streaming-first architecture while merchants never notice the transition.You’ll tackle fascinating problems: How do you handle late-arriving data in distributed streams? What’s the most elegant approach to backfill terabytes while maintaining real-time processing? How do you architect lightning-fast real-time modelling that seamlessly combines data from multiple tables? We embrace AI and LLMs to accelerate repetitive tasks, freeing you to focus on the creative problem-solving that makes this work truly rewarding. If you love turning “impossible” requirements into beautiful solutions, this is your playground.About ShopifyOpportunity is not evenly distributed. Shopify puts independence within reach for anyone with a dream to start a business. We propel entrepreneurs and enterprises to scale the heights of their potential. Since 2006, we’ve grown to over 8,300 employees and generated over $1 trillion in sales for millions of merchants in 175 countries. This is life-defining work that directly impacts people’s lives as much as it transforms your own. This is putting the power of the few in the hands of the many, is a future with more voices rather than fewer, and is creating more choices instead of an elite option.About youMoving at our pace brings a lot of change, complexity, and ambiguity—and a little bit of chaos. Shopifolk thrive on that and are comfortable being uncomfortable. That means Shopify is not the right place for everyone. Before you apply, consider if you can:Care deeply about what you do and about making commerce better for everyoneExcel by seeking professional and personal hypergrowthKeep up with an unrelenting pace (the week, not the quarter)Be resilient and resourceful in face of ambiguity and thrive on (rather than endure) changeBring critical thought and opinionUse AI tools reflexively as part of your fundamental workflowEmbrace differences and disagreement to get shit done and move forwardWork digital-first for your daily workShopifyWe hire people, not resumes.If you think you’re right for the role,. First things firstWe want to ensure you’re the type of person who will thrive here so we can move you quickly through our hiring process. Before completing the job application, please check all the statements that apply to you below:I care deeply about what I doI am always seeking new (sometimes uncomfortable) growthI thrive on change in an ambiguous environmentI get shit done and can keep pace to ship weeklyI want to work remotely for my daily workI have a history of being resilient and resourcefulI want to make commerce better for everyoneIt looks like you might not be a great fit for Shopify at this time — while we encourge everyone to apply, we are looking for someone who can meet all these criteria.Apply tot his job