Interview by Thomas Peham
June 25, 2015
Photos by Sunil Sadasivan
From pulling the plug with his first startup to leading Buffer’s engineering team.
Sunil found his passion for solving people’s problems quite early in his life. Though initially he wasn’t too sure about the role of software engineering.
The rise of Android and his first mobile projects then have been an eye opener and made him apply at Buffer. Now, as the CTO at Buffer, Sunil is helping millions of people around the globe.
Share this interview:
Hi Sunil, thanks so much for your time. Can you tell us a bit how a typical day in the life of Sunil looks like?
Yeah. It changed quite a bit, and how my day today might not be how it will be tomorrow. I really love coding. I try to continue that. I would say, if you asked me in September, I was coding about 10% of my time.
But now I'm actually coding a bit more. I try and shoot for 30 to 40% of my time; the rest is interviews, research and advice to the team. One of my main roles is on the hiring side with recruiting and in our chat tool - HipChat - I'm an advisor on different levels of the architecture side. I'm sort of the CTO/architect, and that's what I really enjoy.
We as a company focus on self management, which is why not much time is spent on the actual management side of things.
And you want to keep coding for 30 to 40% of your time in the future?
Yeah. I think that's really important for me in some ways. Recently, we've had this shift, we call it self-management.
The idea is that we're trying to build a natural hierarchy versus an unnatural hierarchy.
In an unnatural hierarchy my role is considered to be the CTO.
But in a natural hierarchy, someone can take on some of the roles that I have that if they feel fulfilled in that way. Therefore at Buffer, job titles have started to feel quite odd--since most of us take on quite a few different roles.
I've realized that what keeps me energized and motivated is I love to code and I love learning. I also love the people side of my job too, that's been a new experience for me.
I've been in development for almost 10 years now, but when I joined Buffer this was really the first time leading the hiring, some of the vision stuff. And that's been exciting for me, though I’ve learned that what keeps me energized is having a balance of coding and everything else.
Okay, cool. You joined Buffer in 2012, when it still was in it’s early phase. What made you choose to start at Buffer?
I was working on my own startup with two friends for about a year and a half at that point. And it was a lot of fun, a lot of learning on being a founder in addition to building a product and trying to receive traction and everything.
Throughout that time, I was reading a ton from other founders. Joel and Leo, the two co-founders at Buffer were writing quite a lot around their journey and what they've been doing. This was Joel's second startup. He had learned a lot from his first one on being Lean.
I really looked at Buffer as an example of a startup that was doing things in a Lean way and wanting to follow their example. I was reading a lot of their content and stuff and we were using Buffer to build our brand too.
So it was already in my mind a little bit. We decided to pull the plug with the startup we were working on. We were completely bootstrapped and we reached a point where we had to raise money. We had some decent traction, but no solid plan for a business model or felt we’ve truly reached product-market fit.
I wasn't too sure in what space the product we were working on would end up and wasn’t sure whether I wanted to stay there for the next few years. I knew I really wanted to work on something I was passionate about and so I decided to reach out to Joel and Leo to see if they needed help.
It was more of like, "If you need, I can help you with your Android app," that was really key for them too, to try and see whether that would work out well. I started out as a contractor and sort of evolved to this position.
Going back to your early days, has there been any “aha moment” where technology clicked for you?
Good question. There's been a few moments. When I was a kid I was really good at helping people with their computers.
When I learned how to code, I thought I was really bad at it.
I didn't think it was really right for me, because I was in a class where that person next to me was building awesome games and stuff. On the other hand, I was struggling with every assignment.
Things started to click after sticking with it in school, I landed my first job at Cisco, though I was still not too sure what I was doing. After a few months, the Android SDK came out and I decided to play around with it in my spare time.
This was the first time I was building an app completely from scratch by myself. I released two apps into the market and it hit me, "Okay, I get this. And I really enjoyed it.”
I realized that it really energized me when I have full control over something that I want to build and that I’m passionate about.
I'm more motivated by the problem I'm trying to solve, than the actual technology side of it.
You also studied computer science at university. Would you recommend others who want to get started in software engineering to go to university or start their own thing first?
Most of our engineers at Buffer never went to school for computer science.
However, I did. I took a more traditional route. I needed that because I was a bit unsure of what I wanted to do. If you're not sure what you're passionate about that’s normal and I would recommend staying in school. It's the only time in your life where you are able to experiment with so many different possibilities. Even for people who are passionate about startups, I would recommend going to school, but then working on many side projects outside of class.
In school, you have classes, that's your main pressure, but once you're out in the real world, once you're actually focused on a startup, you have to be committed to it. I think that's the same route that Bill Gates and Mark Zuckerberg--instead of simply dropping out of school and starting their companies, they worked on it while in school and chose to commit to it after a certain point.
Even Leo, our co-founder, never said, "Oh, I'm dropping out of school to work on Buffer." It was, "Ok, let me take a semester off, let me take a couple of semesters off." And eventually Leo realized he’s not going back to school (for now). You really have to be forced to dropout. I think I'm more on the side of definitely going to school, but using it as an opportunity to seek and explore the different things you're passionate about.
Can you share some technical challenges of maintaining a product which had seen an incredible growth over the last couple of years?
We've learned how to scale through challenges. We've had to learn the hard way about proper database techniques and server management at scale. For example, MongoDB often gets a bad rap for being tough to scale.
Though we’ve managed to make it work well for us. We've decided to work closely with database experts, like compose.io who help us scale our data.
You may know, Buffer was hacked in October of 2013 which was a big eye opening lesson for us. This one hit us hard. That day was a Saturday and I was scrolling my Facebook feed and I see this same spam across all of my friends and it was via Buffer.
That was a good week of a crash course in how you really have to be on top of security.
We were scaling. We were growing. We had around 500,000 access tokens for Twitter and Facebook. I didn't realize it at the time, but we're a target. We're a target for being able to post massive amounts on these networks. We didn't encrypt access tokens on our side in the database, but it was more that we trusted things. We trusted our database provider and cloud code repository hosting.
I won't go into too much details, but the lesson was that shared passwords, even across two services, you can almost consider them breached. So there are no shared passwords anymore, and we enforce two factor authentication on everything etc. But that's a good example of what we've had to learn the hard way.
You wrote a blog post about that security breach in 2013. You mentioned a couple of improvements. Are there any further lessons learned?
Yeah. We've done many further things. I think one of the biggest lessons was that security is never a one and done thing. It's an area that's always there for a startup. It's just like HR or accounting.
Each new person you hire, new feature you build, new service you use is another potential attack vector. There’s one more password, another sensitive item, and it all compounds. I think the key learning was that we need to have someone on the team who is responsible for security, and security at the top of mind of all of us on the team.
Is there any career advice you've ever received and like to share with us?
I would say probably it's more through books what I've tried to take to heart in terms of advice. Mark Cuban says "The best investment you can make is in yourself," and it's compounding really.
Even though it didn't work out with my previous startup, it was probably the best thing that I had ever done because it was an investment.
I invested that time to focus on myself. I really hadn't done much web development up until that point. It was like, "You're trying to build a startup, but to do that you need to build your own skills and learn all of the challenges by experiencing it, go through the challenges that founders face and all that kind of stuff" That's added to me being where I am today at Buffer.
In terms of ‘career’ advice, I really like Sheryl Sandberg's advice: "Find the rocket ships" which is really key. Don’t be afraid of unknown or changing things. It's in our DNA to adapt to different circumstances.
You learn so much about yourself so don't be afraid of unanswered questions. Don't be so worried about whether your next career move is a positive or negative move.
Buffer is a good example. I was a founder and then I joined Buffer as an Android contractor with a huge pay cut then my previous job, but I didn't care about that because I thought that, "I really like this team," and in time things work out.
You prove yourself and it's a growing team now. I wouldn't be too conscious about what your career progression looks like.
Can you share some tips for applying at Buffer as an engineer?
Yeah, that's a great one. There's a few things we look at. The first thing that we often check is, "Are you a Buffer user?" When I joined Buffer and pretty much from the start, we've all been Buffer users.
Then we look at your Twitter or Facebook account and check how active you are on social media. We're a social media company and the market is changing so we need to rely on our intuition on how the market is changing so that Buffer can be positioned in a good way. So if you use twitter/facebook, that’s a huge positive.
Then our values are also very important. Since we're a remote team we rely on them even more. I would recommend going through our values there and check if you feel alignment with these values. Positivity is kind of a good example. It’s our number one value. How you come across on social media is really key for us.
The other thing is side projects, especially for engineers. Side projects or your own startups or open source is a really good sign because that shows that you're self-motivated--a key component of working at a self-managed remote startup.
bugtrackers.io is also about tracking bugs. Can you share some insights how you handle testing at Buffer?
For sure. We ended up deploying anywhere from 5 to 20 times a day. At the moment we have about 13 people working on the same web repo. We're not too concerned with 100% test coverage.
It's more about using your best judgment on what's really critical and what will break, and then add testing to it on that way.
So some of our frontend doesn't have testing and some of the trivial backend doesn't. We have a unit test framework where we use Jenkins which runs the build asynchronously when you push a branch and will tell you if it's broken. Then we use Github's Hubot for deployments. And we also use Bugsnag for production exception handling.
Bugsnag will let you know immediately if something is broken. Our app is tested in so many different ways at the same time. And we also use New Relic for performance monitoring.
We also have our own alert framework to page us on any abnormal metric (response times, drop in buffered updates creates etc).
Do you see any technical challenges coming up in the next couple of months in respect of growing Buffer?
So far we've been able to scale development pretty well in terms of working on the same code base. I know that the way we can really move faster is by separating it out, being a bit more modularized and having areas that people own, which I think is where we're headed.
I think that's just a general trend towards micro-services.
This is also the first time we're close to 40 people now. This is really that tipping point where one person in the company can't know everything that's going.
Before at least we knew who to talk to. We've tried a few different project management tools, like Trello and Asana. But we haven’t found a perfect fit and so we've been working on our own tool. I’m hoping to blog about that and open source it too soon.
Yeah. In terms of technical challenges, I think the other thing is hiring. That's a hard thing to do. We have so much that we want to get done, the only way that you can get it done (in parallel) is by hiring.
Yeah. How many engineers are currently working at Buffer?
We just had one more so I believe it's about 14 now.
What's your personal passion about building products and software?
I think from an early, early age I've always been about trying to solve problems, make things easier. I didn't realize software would be used as much as it is today. I think there was a realization at some point that software is a tool to help solve problems on a large scale, faster.
I'm passionate about solving problems, and software allows me to do that on a large scale.
And that's really, really exciting. I think we've only scratched the surface of what can happen, which is awesome.