Hadoop engineers

Hadoop engineers

freelance IT Skillhouse in remote
  • Post Date: December 23, 2019
Email Job
  • Share:

Job Description

Our customer, global consulting and software development company with offices in Eastern Europe, is looking for 2 Hadoop engineers to join their cloud solution team.

The ideal candidate has an eye for building and optimising the ETL system and will work closely with our systems architects and data engineers to help extract, transform, aggregate and store the data within the data pipeline and ensure consistency of data delivery and utilisation of reporting and data analytic systems.

Responsibilities:

  • Work closely with system architects and data engineer to create and optimise the architecture of ETL system
  • Design and build the ETL tools to extract, transform, aggregate and store the data
  • Always angle for greater efficiency and robustness of the ETL process to support big volume of data flow among different data systems.

Requirements:

Fluent english (one of engineers will need to travel to US customer site for couple weeks)

Bachelor degree in Computer Science, Information Systems or equivalent quantitative field and 5+ years of experience in a similar ETL role

Experience working with and extracting value from large, disconnected and/or unstructured datasets

Demonstrated ability to build processes that support data transformation, data structures, metadata, dependency and workload management

Strong interpersonal skills and ability to project manage and work with cross-functional teams

Advanced working SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working familiarity with a variety of databases, especially SQL Server and Hive

Experience building and optimizing ‘big data’ data pipelines, architectures and data sets

Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement

Experience required with the following tools and technologies:

Major Hadoop ecosystem distributions such as HDP, Cloudera etc. HDP is preferred

Public cloud such as Azure, AWS etc. Azure is preferred

JSON document processing

Apache Hive and HBase, Microsoft SQL Server

Apache NiFi, Kafka

Scala

Object-oriented/object function scripting languages such as Python, Java etc.

Other jobs you may like