the smart solution for Chicago jobs

Big Data Lead Engineer

Company: BayOne Solutions
Location: Chicago
Posted on: February 15, 2019

Job Description:


  • Design and develop scalable data pipelines for optimal ingestion, transformation, storage and computation using latest big data technologies.
  • Work with engineering, product, and business team(s) to understand requirements, evaluate new features and architecture to help drive decisions.
  • Design, develop and maintain automated ETL processes for efficient batch records-matching of multiple large-scale datasets.
  • Translate complex business requirements into a scalable, efficient and high availability data platform. Strong ownership of the full backend stack - from design through to deploy and beyond.
  • Experience working with continuous integration (CI/CD) framework, building regression-able code within data world using GitHub, Jenkins, Cloud Build, Spinnaker and related applications.
  • Actively participate in code review and test solutions to ensure it meets best practice specifications. Prototype ideas quickly using cutting edge and new generation technologies.
  • Build collaborative partnerships with architects, technical leads and key individuals within other cross functional groups. Work with business teams/engineers to define instrumentation and data requirements.
  • Lead projects independently and also mentor team members and other engineers in the team.
  • Regular, dependable attendance & punctuality. Qualifications: Education/Experience:
    • Bachelor or Master's degree or equivalent in computer science or related field with minimum of 5+ years of directly related work experience.
    • Good understanding of data structures and algorithms.
    • Programming experience in Python, Java or Scala.
    • Extensive experience with Elastic stack, Kafka and Spark.
    • Experience with big data technologies like Hadoop, Map Reduce, Hive, Spark/Storm, Big Query.
    • Experience with time series and NoSQL databases is a plus.
    • Experience working with containers and kubernetes is good to have.
    • Experience designing, implementing and scaling big data solutions on public cloud.
    • Experience with Unix/ Linux and shell scripting.
    • Experience working with various data sources like unstructured data files, flat files, message queues, JSON based events, databases.
    • Experience driving architectural discussions and proposing solutions for system and product changes.
    • Experience writing high-performance, reliable and maintainable code.
    • Experience with large distributed services is a plus as is building/operating highly available systems - provided by Dice Big Data, Java, Kafka, Cassandra,AWS

      Keywords: BayOne Solutions, Chicago , Big Data Lead Engineer, Engineering , Chicago, Illinois

      Click here to apply!

      Didn't find what you're looking for? Search again!

      I'm looking for
      in category

Other Engineering Jobs

Civil Design Assistant - Civil Engineer
Description: Ardmore Roderick is currently seeking a Civil Design Assistant to assist multiple departments, focusing on developing and maintaining the Illinois Tollway CAD standards and criteria requirements. This (more...)
Company: Ardmore Roderick
Location: Lisle
Posted on: 02/20/2019

QA Test Engineer
Description: Let companies apply to you, not the other way around. Sign up with Employer to get access to thousands of companies currently hiring QA test automation engineers. As a QA test automation engineer, you (more...)
Company: Hired
Location: Mount Prospect
Posted on: 02/20/2019

Lead Ab Initio Application Support Engineer
Description: Lead Ab Initio Application Support Engineer USA-IL-Greater Chicago-Chicago - Information Technology What we ll bring: A work environment that encourages collaboration and innovation. We consistently (more...)
Company: TransUnion
Location: Chicago
Posted on: 02/20/2019

TAC Sr. Engineer (SIEM)
Description: br br Trustwave helps businesses fight cybercrime, protect data and reduce security risk. With cloud and managed security services, integrated technologies and a team of security experts, ethical (more...)
Company: Trustwave
Location: Chicago
Posted on: 02/20/2019

Senior or Lead Network Engineer
Description: Will consider a Senior or Lead Network Engineer, please apply if you are a strong Network Engineer. Can be located in Chicago, IL or Knoxville, TN Prestigious Fortune 500 Firm is seeking a Lead Network (more...)
Company: Request Technology, LLC
Location: Chicago
Posted on: 02/20/2019

Cloud Native Microservices Engineer
Description: Join Accenture and help transform leading organizations andcommunities around the world. The sheer scale of our capabilities and clientengagements and the way we collaborate, operate and deliver value (more...)
Company: Accenture
Location: Chicago
Posted on: 02/20/2019

Project Engineer II - Design
Description: Ardmore Roderick is seeking a Project Engineer II Design to assist in the preparation of Phase I preliminary engineering, reports and studies and Phase II contract plans, specifications and estimates (more...)
Company: Ardmore Roderick
Location: Chicago
Posted on: 02/20/2019

Support Engineer - Industrial Networking Products
Description: Our client is an innovative and leading global supplier of connectivity solutions for industrial automation, machinery and remote asset management. They are currently seeking an individual to work as (more...)
Company: Cameron Craig Group
Location: Chicago
Posted on: 02/20/2019

Test Automation Engineer / Developer
Description: DevOps Test Automation Engineer br br Primary Responsibilities: br br A senior DevOps Test Automation Engineer, technical expert in creation and development of system test automation scripts. (more...)
Company: IT Associates
Location: Northbrook
Posted on: 02/20/2019

Engineering Inspector
Description: Application Attachments: Please include your resume and up to two additional documents. Add File The application opened in a new tab. By using this feature you agree to our Terms and Conditions and Privacy (more...)
Company: American Institute of Aeronautics and Astronautics
Location: Mount Prospect
Posted on: 02/20/2019

Log In or Create An Account

Get the latest Illinois jobs by following @recnetIL on Twitter!

Chicago RSS job feeds