1+ months

Lead Data Engineer - P206332

Riverwoods, IL 60015
  • Job Code
    P206332

At Discover, be part of a culture where diversity, teamwork and collaboration reign. Join a company that is just as employee-focused as it is on its customers and is consistently awarded for both. We're all about people, and our employees are why Discover is a great place to work. Be the reason we help millions of consumers build a brighter financial future and achieve yours along the way with a rewarding career.

As a Lead Data Engineer, you will provide technical (design & development) leadership to the Payments Data Ops Team in development of Extract/Transform/Load (ETL) applications that will interface with all key Discover applications.

Position requires excellent communication skills for understanding business vision and the ability to translate the vision into technical artifacts. Strong technical analysis and design background is also a must-have to ensure that technical deliverables are providing flexible, architecturally sound infrastructure with the ability to reuse in future Data Stores, analytical and operational in AWS.

Responsible for designing and developing data-driven platforms and next-generation analytic technologies.

Responsibilities;

  • Develop data driven solutions utilizing current and next generation technologies to meet evolving business needs.
  • Ability to quickly identify an opportunity and recommend possible technical solutions.
  • Develop application systems that comply with the standard system development methodology and concepts for design, programming, backup, and recovery to deliver solutions that have superior performance and integrity.
  • Contribute to determining programming approach, tools, and techniques that best meet the business requirements.
  • Understand and follow the PDP process to develop, deploy and deliver the solutions.
  • Be pro-active and diligent in identifying and communicating design and development issues.
  • Provide business analysis and develop ETL code and scripting to meet all technical specifications and business requirements according to the established designs.
  • Offer system support as part of a support rotation with other team members.
  • Utilize multiple development languages/tools such as Python, SPARK, Hive to build prototypes and evaluate results for effectiveness and feasibility.
  • Operationalize open source data-analytic tools for enterprise use.
  • Develop real-time data ingestion and stream-analytic solutions leveraging technologies such as Kafka, Apache Spark, Python and AWS based solutions.
  • Custom Data pipeline development (Cloud and locally hosted)
  • Work heavily within the Cloud ecosystem and migrate data from Teradata to AWS based platform.
  • Provide support for deployed data applications and analytical models by being a trusted advisor to Data Scientists and other data consumers by identifying data problems and guiding issue resolution with partner Data Engineers and source data providers.
  • Provide subject matter expertise in the analysis, preparation of specifications and plans for the development of data processes.
  • Ensure proper data governance policies are followed by implementing or validating Data Lineage, Quality checks, classification, etc.
  • Designs and develops complex and critical data projects.
  • Designs and develops real-time processing solutions.
  • Creates and enhances data solutions that enable seamless integration and flow of data across the data ecosystem.
  • Designs and develops data ingestion frameworks, leveraging open-source tools and data-processing frameworks.
  • Develops, implements, and supports application codes and analytical models.
#LI-KE1

Minimum Qualifications

At a minimum, here's what we need from you:

  • Bachelor's Degree in Computer Science, or related field
  • 4+ years of experience in Data Platform Administration/Engineering

Preferred Qualifications

If we had our say, we'd also look for:

  • 6+ years of experience in Data Platform Administration/Engineering
  • Hands on experience with Amazon Web Services (AWS) based solutions such as Lambda, Dynamodb, Snowflake and S3.
  • Experience in migrating ETL processes (not just data) from relational warehouse Databases to AWS based solutions.
  • Experience in building & utilizing tools and frameworks within the Big Data ecosystem including Hadoop, Kafka, Hive, Spark, HBase, NoSQL.
  • Deep knowledge and very strong in SQL, and Relational Databases
  • Knowledge of Data Warehouse technology (Unix/Teradata/Ab Initio)
  • Experience within the Financial industry
  • 6+ years of experience of being a lead engineer amongst a team of equal or junior level developers
  • Excellent written and verbal communication, presentation and professional speaking skills
  • Collaborative individual who comfortable working on agile-based teams, excels in working with business partners identify, develop and deliver innovative data solutions
  • Capable of collaborating with stakeholders and project leaders to understand requirements, deliverables, and set expectations on tasks that you will be responsible for.
  • 6+ years of systems development and analysis experience in designing, developing, implementing & maintaining applications
  • Knowledge of Data Warehouse technology (Unix/Teradata/Ab Initio preferred)
  • Strong analytical skills and experience with writing and performance tuning complex SQL queries
  • Hands on experience with Amazon Web Services (AWS) based solutions such as RDS, Lambda, Dynamodb, Snowflake and S3 or other cloud technologies
  • Experience with any of the following: R, SAS, Hive, Spark, Python or H2O
  • Proficiency in scripting languages such as Shell/Python/Scala/Java
  • Experience in migrating ETL processes (not just data) from relational warehouse Databases to AWS based solutions.
  • Knowledge of BI tools such as Cognos and Tableau

Discover Financial Services is an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, age, protected veteran status, among other things, or as a qualified individual with a disability.



Categories

Industry

  • Banking / Finance
Posted: 2019-08-06 Expires: 2019-10-04

Welcome to Discover
We strive to be the leading direct bank and payments services company. Our mission is to help people spend smarter, manage debt better, and save more to achieve a brighter financial future.


Why Work with Us?
You can make an impact. Whether it’s developing corporate strategy, innovating new services or supporting IT needs, every employee has the opportunity to be a vital part of our business and make a real difference in people’s lives. It’s the heart of what we do.


 

Employment Trends

Before you go...

Our free job seeker tools include alerts for new jobs, saving your favorites, optimized job matching, and more! Just enter your email below.

Share this job:

Lead Data Engineer - P206332

Discover
Riverwoods, IL 60015

Join us to start saving your Favorite Jobs!

Sign In Create Account
Powered ByCareerCast