ChicagoRecruiter Since 2001
the smart solution for Chicago jobs

Cloudera/StreamSets Big Data Engineer

Company: Aon
Location: Chicago
Posted on: May 6, 2021

Job Description:

Aon is looking for aCloudera/StreamSets Big Data EngineerAs part of an industry-leading team, you will help empower results for our clients by delivering innovative and effective solutions supporting Retiree Health Solutions.This role can be worked remotely from anywhere within the US.Your impact as aCloudera/StreamSets Big Data EngineerThe Big Data Engineer must be able to design, build and maintain enterprise level data pipelines utilizing the public cloud (AWS or Azure) based data ingestion services or ETL/ELT tools available within the big data ecosystem. The engineer will work closely with data analysts, data scientists, and database and systems administrators to create data & analytical solutions. This position is embedded within the Aon Retiree Health Solutions' Data Science team with additional support/mentorship from Enterprise Architecture.Responsibilities include the following:

  • Should have extensive experience with design, build, deployment, of ETL/ELT data pipelines within big data ecosystems (currently using Streamsets, Impala, Hue, Spark, Python, and Cloudera) to populate cloud-based infrastructure (currently using AWS). Should have experience developing technical design, and ETL specification documentsPerform complex parallel loads, cluster batch execution and dependency creation using jobs/topologies/workflows.Convert existing SQL stored procedures for optimal execution in Streamsets. Strong exposure working with relation databases DB2, Oracle & SQL Server including complex SQL constructs and DDL generation.Work with web service targets, XML/JSON Sources and Restful APIs.Manage, monitor and fine tune Hadoop cluster architecture (HDFS) jobs for performance, security and resource management.
  • Create detailed designs and POCs to enable new workloads and technical capabilities on the Platform.Work with the platform and infrastructure engineers to implement these capabilities in production.
  • Coordinate with Enterprise Analytics to manage workloads and enable workload optimization including resource allocation and scheduling across multiple tenants to fulfill SLAs. Document new/existing pipelines and datasets.
  • Participate in planning activities with Data Science and Enterprise Architecture teams.You Bring Knowledge and ExpertiseRequired Experience:
    • Six plus years of extensive experience in ETL/ELT Technologies (e.g., StreamSets/ Informatica, Talend).
    • Minimum 6 years of experience in data model, technical metadata, creating data objects, data warehouse and DataMart designs
    • Six plus years of experience in any of database technologies SQL Server, Oracle, Postgres, MySQL.
    • Four plus years of hands-on experience with big data technologies (e.g. Hadoop, Spark, Hive) deployed in the public cloud (preferably in AWS or Azure).
    • Expert level hands-on skills on Spark, and Python programmingPreferred Experience:
      • AWS or Azure certification strongly preferred
      • Experience with AWS Kinesis, Firehouse, or Confluent strongly preferred.
      • Experience with Cloudera Manager, Navigator, and Data Science Workbench.
      • Exposure to data cataloging products like Alation or Collibra.
      • Exposure to data security tools, and technologies.
      • Experience with BI reporting tools like PowerBI, and Tableau.
      • Master's Degree in Computer Science or related fieldEducation:
        • High school diploma or GED equivalent.We offer youAcompetitivetotal rewards package, continuing education & training, and tremendous potential with agrowingworldwide organization.Thesalary range for this position is $115,000 to $125,000 annually. This salary range is an estimate and the actual salary will vary based on applicant's education, experience, skills, and abilities, as well as internal equity and alignment with market data. The salary may also be adjusted based on applicant's geographic location.This position is eligible to participate in one of Aon's annual incentive plans. The amount of the incentive varies and is subject to the terms and conditions of the applicable incentive plan.Aon offers a comprehensive package of benefits for full-time and regular part-time colleagues, including, but not limited to: a 401(k) savings plan with employer contributions; an employee stock purchase plan; medical, dental and vision insurance, various types of leaves of absence, paid time off, including sick days and vacation days, short-term disability and optional long-term disability, health savings account, health care and dependent care reimbursement accounts, employee and dependent life insurance and supplemental life and AD&D insurance; optional personal insurance policies, adoption assistance, tuition assistance, commuter benefits, and an employee assistance program that includes free counseling sessions. Eligibility for benefits are governed by the applicable plan documents and policies.Our Colleague Experience:From helping clients gain access to capital after natural disasters, to creating access to health care and retirement for millions, Aon colleagues empower results for our clients, communities, and each other every day. They make a difference, work with the best, own their potential, and value one another. This is the Aon Colleague Experience, defining what it means to work at Aon and realizing our vision of empowering human and economic possibility. To learn more visit Aon Colleague Experience.About Aon:Aon plc (NYSE:AON)is a leading global professional services firm providing a broad range of risk, retirement and health solutions.Our 50,000 colleagues in 120 countries empower results for clients by using proprietary data and analytics to deliver insights that reduce volatility and improve performance.By applying for a position with Aon, you understand that, should you be made an offer, it will be contingent on your undergoing and successfully completing a background check consistent with Aon's employment policies. Background checks may include some or all of the following based on the nature of the position: SSN/SIN validation, education verification, employment verification, and criminal check, search against global sanctions and government watch lists, fingerprint verification, credit check, and/or drug test. You will be notified during the hiring process which checks are required by the position.Aon provides equal employment opportunities (EEO) to all employees and applicants for employment without regard to race, color, religion, creed, sex, sexual orientation, gender identity, national origin, age, disability, veteran, marital, or domestic partner status. Aon is committed to a diverse workforce and is an affirmative action employer.DISCLAIMER: Nothing in this job description restricts management's right to assign or reassign duties and responsibilities to this job at any time.IND-US 2483203

Keywords: Aon, Chicago , Cloudera/StreamSets Big Data Engineer, Other , Chicago, Illinois

Click here to apply!

Didn't find what you're looking for? Search again!

I'm looking for
in category

Log In or Create An Account

Get the latest Illinois jobs by following @recnetIL on Twitter!

Chicago RSS job feeds