Data engineers to build data analytical solutions that will address increasingly complex business questions. A Data Engineer will be a hands-on person responsible for designing, prototyping and implementing data products that support a wide variety of data processing, data science and analytics needs. Data Engineers work closely with data scientists, product managers, data platform team to understand the functional data requirements and leverage the underlying tech stack to come up with scalable, robust data applications which can crunch terabytes of data in real-time. Data products and applications you build will enable data driven decision making across business, analytics and operations.
What you’ll do:
You should have good hands-on experience in designing, implementing, and operating stable, scalable, solutions to flow data from production systems into analytical data platform (big data tech stack MPP) and into end-user facing applications for both real-time and batch use cases
You should be able to work with business customers in a fast paced environment understanding the business requirements and implementing analytical solutions.
You should have good experience in the design, creation, management, and business use of large datasets. Do high level design with guidance; Functional modeling, break-down of a module. Thinking platforms & reuse
Build and execute data modeling projects across multiple tech stacks i.e. big data, MPP, OLAP using agile development techniques. Build and integrate robust data processing pipelines for enterprise-level business analytics.
Challenge status quo and propose innovative ways to process, model, consume data when it comes to tech stack choices or design principles. As needed, assist other staff with reporting, debugging data accuracy issues and other related functions.
Strong engineering mindset - build automated monitoring, alerting, self healing (restartability/graceful failures) features while building the consumption pipelines. Translate business requirements into technical specification (fact/dimension/filters/derivations/aggregations)
An ideal candidate will have excellent communication skills to be able to work with engineering, product and business owners to develop and define key business questions and to build data sets that answer those questions. you should bring your passion for working with huge data sets and bringing datasets together to answer business questions and drive change
What you’ll need:
3-5 years’ experience with a Bachelor's Degree in Computer Science, Engineering, Technology or related field required. 2 to 3 years of relevant software development experience with sound skills in database modeling (relational, multi-dimensional) & optimization and data architecture - databases e.g. Vertica
Good understanding of streaming technologies like Kafka, Spark Streaming. Proficiency in one of the programming languages preferably Java, Scala or Python. Good knowledge of Agile, SDLC/CICD practices and tools with good understanding of distributed systems
Experience with Enterprise Business Intelligence Platform/Data platform sizing, tuning, optimization and system landscape integration in large-scale, enterprise deployments.
Must have proven experience with Hadoop, Mapreduce, Hive, Spark, Scala programming. Must have in-depth knowledge of performance tuning/optimizing data processing jobs, debugging time consuming jobs.
Proven experience in development of conceptual, logical, and physical data models for Hadoop, relational, EDW (enterprise data warehouse) and OLAP database solutions.
Experience working extensively in multi-petabyte DW environments. Experience in engineering large-scale systems in a product environment
Seniority level
Associate
Employment type
Full-time
Job function
Engineering and Other
Industries
Information Services and Technology, Information and Internet
Referrals increase your chances of interviewing at Flipkart by 2x