A Date with the Data Team

It doesn’t add up. A data engineer who draws up more than just dashboards, and two others who are into Anime, hikes, and FIFA? That’s just how data engineers at Fi blow off some steam (even when there isn’t any steam to blow off).

I caught up with Gunjan Singh , Abhishek Jha , and Abhishek Meena to try and break down a bunch of things that make up their day.

First up, help me understand what a data engineer does? I’m picturing a day full of dashboards and analytics and a billion lines of code. How far off the mark am I?

Gunjan: As data engineers, we identify patterns in data and build pipelines to make the data more useful for various personas in the organisation. This also includes building a secure infrastructure, tools and in-house products that are engineered to meet the scale and new functionalities in the product.

Abhishek Jha: A typical day starts with a birds-eye view of platform/pipelines for stats, alerts/failures. Then comes the standup where catching up with the team is the first thing we do(some interesting conversations to start the day!) and then plan our daily tasks according to priority. The rest of the day goes in working on our tasks that may include development activities, PR reviews, tech discussions or debugging in case of failures/any urgent ad-hoc activities, tech talks, sharing sessions discussions with stakeholders and of course zoom games sometimes!

Right, so, pretty much what I thought, but way more fun. Speaking of routines, what’s something you do regularly that makes you better at your job?

Gunjan: Data Engineering is not just about writing ETL. We focus on abstracting, generalising, and designing for scale. We look for patterns where ETL itself can become self-service-able. To do this, we need to be aligned on what is happening in the industry. While the landscape is massive and we largely rely a lot on the vendor (eg: EMR in case of AWS), one needs to understand the tech stack well so that we are able to debug problems, add patches wherever necessary and contribute back to the OSS community.

Reading tech blogs and educating oneself about the new technologies that are upcoming in the market is part of the continuous exploration process that in-turns drives innovation. That aside, I’m finding new ways to reduce manual toil and make the system better-reliable.

That sounds like a lot of work! Do you do stuff outside of work?

Gunjan: Oh definitely! There is a misconception that data engineers need to carry their laptops everywhere to jump whenever there is an issue in the pipeline or data. Personally, I don’t recollect doing that at any point here. We focus a lot on operational automation, adding observability, monitoring and data quality checks that help us get organised at work and at the same time allow us to do what we love outside the office. In my free time, I love to sketch, draw and paint (ironically I picked this up after joining Fi).

Abhishek Meena: I watch Anime (I like a good story), I cycle to places (Beaches, Lakes, Tea Stalls, anywhere peaceful and calm), I go hiking. I love computers and building stuff (DIY) so I’m playing with raspberry pi. At home, I once assembled a server to build NAS, Media Server, DNS ad-blocker etc.

I also believe in sustainable living so I keep trying something whether it’s a rooftop vegetable garden, permaculture, planting trees in the neighborhood, composting etc.

Abhishek Jha: In my free time when I feel lazy (which is most of the time) I read or research about stock markets/crypto/finance, and dive into history books or play Fifa. Otherwise, I go outside to play Football and Cricket.

It was after reading about markets I realised that the whole ecosystem(banks, internet companies etc.) encourages millennials to spend their money (keeping money idle in a savings account itself is losing money against inflation) many times unnecessarily rather than saving smartly. That’s how I got to know of Fi for the first time and really loved how Fi is here to help us get better with our money. It felt like pieces falling into place. There’s a lot yet to achieve but as we say “ Journey of a thousand miles begins with a single step ” and the step has already been taken, do join us for the rest of the journey.

Clearly, Fi sounds like a bunch of fun. What are some things about Fi that make it very different from your past jobs?

Gunjan: As a data Engineer at Fi I get to work on multiple aspects of Engineering. I get to wear the hat of a data architect, a platform engineer & feature engineer, as we work on backend services, building infrastructure, solving interesting scaling use cases, data security & privacy alongside complying with infosec requirements.

In the last year, I had the opportunity to not just look at automation with regard to CI/CD, and building monitoring stacks, or integration with tools to push logs and metrics, but also working on micro-services and hands-on with go-lang, python, scala, java and shell scripting. Here we choose the right tool to solve a use-case or a tech problem.

At Fi, we trust the engineers enough to work on a piece independently but also provide guidance if required.

So if you are interested in working on various open-source tools, solving interesting use-cases and you like designing, building, and innovating, Fi is the place for you.

It’s interesting that you’re thinking about this from the prospect of potential employees. What are some questions you would ask a prospective candidate?

Abhishek Meena: Given that interviews are scheduled for not more than an hour, it’s important the interviewer & interviewee get comfortable with each other quickly. The best way to do that is to ask the most interesting/challenging problem that the candidate has worked on. This serves multiple purposes, first it gives candidates a chance to show their best work. And since these are problems they have struggled with or had a hard time with, they remember every detail of it and generally have learnings and positive emotions (Proud and sense of achievement) attached to it. Secondly it acts as an Icebreaker. Lastly, we get to know what are the problems they have worked on, what’s the requirement, How they tackled it, how they break down problems into smaller pieces, how do they debug problems, how they come up with solutions. Basically, it gives them an opportunity to describe their contribution based on their role and general cognitive ability to do some problem-solving as we go deep dive.

Okay, so hypothetically if you were to choose between two equally good candidates, how would you go about it?

Abhishek Meena: We do not stack-rank any candidate. The candidates come with their own area of expertise and Data Engineering is a vast area. The landscape is so huge thanks to the tons of open source components, the vendors and the programming languages. It would be unfair to stack rank them.
Given that there are multiple rounds of interviews and everyone’s feedback is taken into account, we look for the approach, long term designing thought process, how they enabled non-functional requirements and most importantly if they are a team player. Nobody likes brilliant jerks in an organization.

What’s the most important skill (hard or soft) that a data engineer needs to be successful in their role?

Abhishek Jha: Starting my career with Cloudera Open Source Platform to working with vendor-specific platform EMR/Databricks I’ve realised one thing — technologies keep changing but problem solving and analytical skills will always help in understanding technologies better and designing complex solutions.

Since in Data Engineering you have to deal with a lot of stakeholders/cross-functional teams like Business Analysts, Data Science, Marketing, Product Engineering etc., having good communication and collaborative skills will really help in implementing better solutions.

Last but not least, a person should have a love or interest for data and a passion for building pipelines/platforms that can help extract insights from this new age oil.

How’s it hiring for your team shortly after you’ve been hired yourself?

Abhishek Jha: That’s a great question!. On the tech front, having past experience with similar technologies/problems, I really enjoy talking to candidates about different sets of challenging Big Data problems that companies ranging from Fortune 500 to new-age startups are facing and solving.

I’m really thankful to the team and the company for providing me with this opportunity. This also shows the emphasis we (Fi) as a company place on culture while hiring that we trust our employees from day one to make the right decisions for us. It was this culture and trust that made the transition from interviewee chair to opposite chair easy.

So, you surely have a style. What’s your style? Do you prefer asking questions or having a chat? How do you switch between the two?

I like a good story. I just ask questions to get the story started and ask questions in between if more detail is needed or if something doesn’t fit. To evaluate skills I create a story which is riddled with problems and each problem is designed to test different aspects of Data Engineering. Interview style depends on the candidate. Some are expressive and know how to tell a story and how much detail should be provided. Some are not so expressive and we have to use questions to get the full picture. This is important to make sure that everyone gets an equal chance. I try to cover not just the coding aspects but also systems, networking, SRE as Data Engineering is a concoction of systems, platforms/infra, software programming skills.

If you’re a data engineer, and Fi sounds like a place you might enjoy working at, give us a shout.