Sr Data Engineer - 1583491O

Hillsboro-OR
 Job-ID :   1583491O 
  Category :  Engineers 
  Position Type : Contract Remote 

Job Description

 Sr Data Engineer

In this role, you will build and delivering scalable data and analytics solutions in Supply Chain- Global Operations and Logistics and Commercial space.

You will work on creating an Intelligent supply chain control tower enabling E2E collaboration and automated decisions for Client ’ Inventory Management.

Responsibilities:

  • You will be a part of a squad that will design, implement and integrate new technologies and evolve data and analytics products. You will be contributing to all aspects of data engineering from ingestion, transformation, and consumption in addition to designing and building test-driven development, reusable frameworks, automated workflows, and libraries at scale to support analytics products.
  • You will be establishing and maintaining Code/Build/Deployment tools
  • You will be building reusable code, with the ability to scale with very large data volumes and scale with added complexity
  • You will provide domain-specific expertise and overall engineering leadership and perspective to cross- organization projects, programs, and activities.
  • Occasionally, you will participate in architecture and design discussions, long term roadmaps and provide your point of view.
  • You will ensure you deliver on business commitments while delivering operational and engineering excellence

Day in Life

  • Develop data pipelines for projects using the leveraging standards, guidelines and data governance rules and policies, Scheduling documentation of data pipelines.
  • Move Data pipelines , workflows and data objects to production environment to support analytics solution delivery.
  • Responsible for Source code of data pipeline, Scheduling documentation of data pipelines
  • Responsible for tunning the data pipeline for performance and scalability
  • Building proof of concepts addressing current and future customer needs; anticipate.
  • Identify and solve issues concerning data to improve data quality, address operational & performance issues, remove technical bottlenecks and perform root cause analysis as needed

Skills

  • 3+years of Software/Data engineering experience
  • 3+years of building and maintaining Data Pipelines
  • Strong demonstrable skills in two of the following programming languages– Python, Java or Scala.
  • Strong Experience in processing Big Data and analyzing the data using Spark, Map Reduce, Hadoop, Apache Airflow, HDFS, Hive.
  • Strong Experience with AWS services like EMR, S3, DynamoDb
  • Should have experience in microservices.
  • Experienced in unit, functional and automated testing.
  • Experienced in GIT Jenkins
  • Good verbal and written communication and collaboration skills to effectively communicate with both business and technical teams.
  • Comfortable working in a fast-paced, results-oriented environment

Education  

  • 3+ years of Software/Data Engineering experience
  • 2+ years of experience with enterprise-level data architecture (e.g., Hive, Spark, etc.)
  • 3+ years programming experience in Python, Scala, SQL, Java
  • 3+ years of experience in developing/maintaining data pipelines, ETL and orchestrations
  • 2+ years working with Apache Spark for data transformation & aggregation
  • 1+ year experience in snowflake, data bricks
  • 2+ years working on AWS cloud services
  • Bachelor’s degree in Computer Science, Engineering or equivalent degree preferred

Skills ordered by importance -

  • Python
  • Apache Spark
  • SQL
  • AWS (S3, EMR, EKS)
  • Git
  • Deploying Spark jobs in Kubernetes
  • Apache AirFlow
  • Apachi NiFi
  • Terraform or CloudFormation
  • Hive
  • SnowFlake

 

 

 

 

Chat with us