Data Engineer | Big data

Gurugram, Haryana, India

About the job

At BluePi you will be part of a team whose mission is to transform organizations, make them more productive, better run and more profitable. Think of yourself as working with senior leadership and decision-makers at top organizations, being able to contribute positively and meaningfully to their critical decisions. You will meaningfully impact growth trajectories of some of the largest organizations. Within a short period of time, you will be ahead of your peers in terms of knowledge, the richness of experience and exposure.

You will work with a great team who will share your passion for creating value for our customers and for each other as well.

In this role, you’ll get to …

  • Solve complex business problems involving Tera Bytes of data
  • Write performant code to extract, transform and load data
  • Integrate and process data in batch and in realtime
  • Work with domain experts in delivering a high-quality solution in an agile environment
  • Discuss, debate and develop complex architectures
  • Follow internal processes while working on the tasks/projects like
  • Creating LLDs
  • Honouring the estimated timelines
  • Clear and quick articulation of dependencies and roadblocks
  • Highlighting any exception or technical debt
  • Work in collaboration of peers internal and external(customer)

Must-Have Skills-

On day one we’ll expect you to…

  • Write performant, clean code in pySpark for batch and/or realtime pipelines
  • Have an in-depth understanding of pySpark APIs to follow the best practices and write performant code
  • Integrate with different source systems to extract data like on-prem/cloud databases, website clickstream, SAP.
  • Understand the scope, design and business objective of the project
  • Own and execute the task, helping the team members as the project progresses
  • Run and build ETL on AWS environment. Hence need an in-depth knowledge of AWS.
  • Understand the Lower Level Designs and convert it into correct estimates.
  • Create design documents for the tasks along with clarity of test cases
  • Speak to the team for questions or integrations. Thus decent communication skills are required
  • Perform exhaustive debugging and provide a solution-oriented approach to a problem

Good to Have skills -

Added bonuses if you have…

  • Knowledge of AWS
  • Knowledge of SQL
  • Knowledge of any other language apart from python - Java or Scala

Team Culture -

We have an open culture that encourages opinions and feedback. I think the best way fo thinking about our culture is to look at the values we believe in:

  • Sustained excellence: excellence is not a goal, but a journey. We are always in pursuit of finding better ways of doing things and not accepting the status quo.
  • Be creative; keep learning; innovate:
    Without continuous learning, one would stagnate and would find it very hard to sustain in BluePi.
  • Challenge others to grow: We do not train you to grow; we challenge you to grow. That means you are expected to perform at an increasingly higher level of competence with each project and each passing day.
  • Work because you want to, not because you have to: Without passion for what you do, you would find it very hard to grow or even sustain in BluePi.
  • Belief: Belief in yourself and your team to prevail in the face of any challenge.
  • Team interest before individual interest: Either the team succeeds or fails, no individual alone can succeed without team success