December 2021 - Four years into my first job, I asked my manager for the opportunity to mentor at least one intern. I wanted to be a good mentor. Unexpectedly, at the start of 2022, I moved teams and began working with four people. I was appointed as the tech lead for a group tasked with building recommendation systems focusing on ‘exploration’ (addressing cold-start and sparse data problems).

Before this, I was primarily an individual contributor (IC) and a good one. I consider myself a problem solver, not just a data scientist. I handled everything from infrastructure and MLOps to frontend and backend development, and even played poker. This move was interesting for me. I spent most of my time interacting with stakeholders (product managers, engineers, analysts), defining problem statements, and setting goals and metrics. I also managed the team’s tasks and expectations.

In my previous team, we functioned as a high-throughput “Swiss army knife” team. Our manager trusted us to get the job done and gave us the freedom to set our own roadmaps and choose our own problem statements. We were very effective and had no people management overhead.

Transitioning to my new role, I had to help set team goals, map their expectations to tasks, and consider their career growth. Consequence of such responsibilities were that I had much less time to work on tasks myself. It stooped to terrifying levels were I was not writing a single line of code for an entire week. Imposter syndrome was on a all-time high. I felt like I’m getting slow and that I am losing my technical touch.

After six months, I realized I couldn’t continue, I threw in the towel. Friends and colleagues were shocked. They wanted me to stay on the managerial path and believed I was doing well. However, I judged myself based on my IC contributions and felt I hadn’t achieved much. Although the team succeeded, I didn’t see this as my accomplishment.

Returning to the IC world wasn’t as expected. I still had managerial tasks from my six-month stint. Multiple discussions with managers revealed they wanted me back on the managerial path, still leading a team. This time, I learned to judge myself on both my IC and team-lead contributions. During annual performance reviews, peers encouraged me to return to management, putting me in a dilemma.

I enjoyed my IC life—coding, debugging, building solutions, training models, and deploying them. However, people whose feedback I valued believed I’d excel in management. I consulted respected seniors who transitioned from ICs to managers. They noted that balancing significant IC work with management is challenging and impacts work-life balance. However, my organization offered to create a role combining minimal people management with substantial IC tasks. This sounded ideal but challenging to implement.

I made a soft decision that I wanted to continue doing what I like doing - being technical. So that meant that I want to continue doing technical work in an IC role, but open to handling managerial tasks as long as it doesn’t displace my primary responsibilities. With this thought, I ventured out to read up more on this to make a solid decision (perhaps a solid decision is not required at this point, but I’ll figure that out along the way).

I explored multiple books on this topic and a clear favorite was “Staff Engineer” by Will Larson, which helped me understand various Staff Engineer archetypes like the Architect and Solver, which I relate to. SE archetypes are different types of Staff Engineers. While this book talks a lot about Staff Engineers and what it means to be a Staff Engineers, it also covers topics on the Managerial roles. The book consists of interviews where most of the Staff Engineers have previously been people managers and that brought in good perspectives - how they still had pseudo-management responsibilities (do 1-1s and strategy discussions but not responsible for performance reviews). Another great take away has been to avoid snacking on easy low value work for immediate satisfaction - something that fueled my procrastinations. My notes after reading this book can easily be another article that I can write, so I will avoid dumping that here.

This entry is more of a personal journal. I plan to follow up with clearer thoughts and guidance on choosing between IC and managerial paths, what each meant for me and how I would decide to choose one over the other.

For those interested, my friend and I discussed this topic in a YouTube video.