This is the Biggest Hidden Risk of AI | Traversal, Anish Agarwal
"Human Coding is Dead," said Anish Agarwal, cofounder of Traversal, in our interview. So much of today's software is already written by AI. Tools like Cursor...
Watch on YouTube →Transcript
Intro
- 00:00The code is no longer being written by
- 00:01human but by AI system. So much more
- 00:04code is being written now by cursor or
- 00:06Windsor for copilot. AI is going to
- 00:08write so much more code. No one really
- 00:10understands all of it. No one has full
- 00:12context. No team has full context about
- 00:13what's happening because it's such a
- 00:15complex system. When software breaks,
- 00:17it's going to be really difficult to
- 00:18troubleshoot it. Right? One of the
- 00:19biggest problems is downtime is
- 00:21troubleshooting. Global IT cyber outage.
- 00:24Global outage, major IT outage.
- 00:26It's also affecting hospitals, law
- 00:28enforcement departments, banks, and
- 00:30major airlines.
- 00:31The cost of downtime to annually for all
- 00:34enterprises is around $400 billion. So,
- 00:35it's a huge problem.
- 00:36It could be several hours or days before
- 00:39the situation is fully resolved.
- 00:41Fundamentally, what we're doing is when
- 00:43we have large complex software systems,
- 00:45we help figure out and they break, we
- 00:47help figure out what happened. having a
- 00:48team of on call engineers 24/7 looking
- 00:51at all of your data. So by the time an
- 00:53engineer comes onto a Slack channel, the
- 00:55root cause or current root cause is
- 00:57already given. So rather than them
- 00:59spending so much time trying to figure
- 01:01out what happened or you know calling
- 01:02more and more teams because typically
- 01:04what happens is you'll have one team
- 01:05look at the the the data. They're like
- 01:07oh it's not my fault. Then they'll call
- 01:08another team and another team and
- 01:10another team. That's how you go from
- 01:11five people to like 80 people on a
- 01:13channel, right? And so then rather than
- 01:1450 people over an hour, it's like 5 to
- 01:1710 people for a few minutes just
- 01:18verifying the answer.
The problem I'm solving
- 01:24My name is Anish. I'm the CEO and
- 01:26co-founder of Traversal. We came
- 01:27ourselves with $48 million in the series
- 01:29A funding which were led by Sequoia and
- 01:31Kleiner Perkins. We just announced our
- 01:33series A raise and we came out of
- 01:35stealth. We're building an AI site
- 01:36reliability engineer. So what that means
- 01:38is when you have large complex software
- 01:40systems and they break, we troubleshoot
- 01:41to help you figure out why it broke and
- 01:43then help fix it automatically. But it's
- 01:44sort of like perplexity when when you
- 01:46ask perplexity a question, it kind of
- 01:47gives you evidence, right? It gives you
- 01:48citations of how it got to that answer.
- 01:50But our world that the citations aren't
- 01:52web links, the citations are links to
- 01:54your observability system that we can
- 01:56now publicly talk about. We've been
- 01:57working with them for the last 6 months
- 01:58now very closely is Digital Ocean. So
- 02:00they're a large public cloud service
- 02:03provider. I think they're the third
- 02:04largest actually by number of people
- 02:05using them as their cloud provider. I
- 02:07think they have over 600,000 people um
- 02:09using them as their core infrastructure.
- 02:11And you can imagine when they break, you
- 02:12know, every person that is using them
- 02:14feels the pain because that's the main
- 02:16thing powering their their system. We've
- 02:18been working with them for 6 months now
- 02:20and we found that in that 6 month period
- 02:21we've dropped the the time to resolution
- 02:24by over 40% like 37% to be exact, which
- 02:26is incredible, right? Because as I said,
- 02:28every minute of downtime is like is
- 02:30worth thousands of millions of dollars.
The Chat GPT Moment Changed Everything
- 02:37I like sports. I like competing a lot
- 02:39and I always naturally gravitated to
- 02:41math and science. I just liked how
- 02:43abstract and clean it was and it felt
- 02:45quite universal in and what you could
- 02:47do. I came to MIT just because I thought
- 02:49machine learning and AI was really
- 02:50important and I wanted to understand it
- 02:51really deeply. This is like 2016. about
- 02:53like 8 n years ago and once I got there
- 02:55I think the one of the biggest moments
- 02:57in my like academic research career was
- 02:59Europe's 2017 and the keynote was given
- 03:02by the Google AlphaGo team and I just
- 03:04found that incredible that a system can
- 03:06learn this creative thing by itself
- 03:08because the complaint you always had
- 03:09before was that it's just copying people
- 03:11but this was purely it was learning
- 03:13creativity by by selfplay so I was like
- 03:15we should apply that everywhere this
- 03:16kind of architecture I was fortunate
- 03:18enough to get into Colombia's faculty
- 03:19and I like thinking about theoretical
- 03:21problems mathematical abstractions And I
- 03:23think university is an amazing place to
- 03:25do that. The thing that changed and that
- 03:27was the time when everything with Chad
- 03:28GPT was was happening. And so it just
- 03:30felt like something incredible has
- 03:32happened in the world. And it's like a
- 03:33once in a-lifetime thing where the world
- 03:35is fundamentally changed. People don't
- 03:36even realize it. It just felt like this
- 03:38almost religious experience as to what
- 03:39was happening in the world. I really
- 03:40like uncertainty. I like creating
- 03:42something from 0 to one. similar between
- 03:44research and entrepreneurship is that
- 03:46the uncertainty you have no idea what's
- 03:48happening most of the time and you have
- 03:50to find ways of of creating structure
- 03:52from nothing I think obviously the
- 03:53difference is in here the time spans are
- 03:57compressed right in research you get 5
- 03:59years 10 years to to make an impact here
- 04:01you get 1 month right so the feedback
- 04:03cycle is very quick but I think in this
- 04:05age of AI when AI is shooting so quickly
- 04:07being in that quick feedback cycle is
- 04:08actually very important we've kind of
- 04:10entered into the industrial age of
- 04:12artificial intelligence you And I also
- 04:13saw some of the smartest people around
- 04:14me. They were either at OpenAI or
- 04:16Enthropic or you know Meta or they were
- 04:18creating companies and that's what makes
- 04:20me really excited. And so I think
- 04:21starting a company was felt to me like a
Begin with Your Edge
- 04:23great expression of that. I guess
- 04:30I think the best AI companies are always
- 04:32going to be at the edge of where the
- 04:33models are going to be, right? That's
- 04:34how you differentiate yourself is you're
- 04:35always at the edge. If you're the edge
- 04:37and sometimes it works, sometimes it
- 04:38doesn't work. And you need to know
- 04:40quickly when it's working and when it's
- 04:41not working and correct for that. When
- 04:43we started the company in like January
- 04:44of 2024, we started without an idea. But
- 04:46we had a clear taste of the type of
- 04:48problem we wanted to take on. So we
- 04:49wanted to do something that was at the
- 04:51intersection of our research which was
- 04:52in causal machine learning and
- 04:54reinforcement learning and how it
- 04:55intersected with AI agents. Causal
- 04:57machine learning is a study of cause and
- 04:59effect. And what you want to understand
- 05:00is how do you get these AI systems to
- 05:02pick up cause and effect relationships
- 05:04from data. AB test is an example of of
- 05:06learning cause and effect relationships.
- 05:07like clinical trial is another example.
- 05:09So these are like basic ways of of
- 05:10running experiment. And so that's what
- 05:12the study of of causal machine learning
- 05:13is and we're trying to see how did that
- 05:15intersect with AI agents uh which we
- 05:17thought was like super cool and
- 05:18something we followed for like now
- 05:19almost 2 and a half years. We went
- 05:21through a few different ideas. The
- 05:22fourth person who joined us our fourth
- 05:24co-founder Ahmed and so he pitched us
- 05:26the problem of dealing with incidents.
- 05:28And as we looked into it, it kind of
- 05:29felt like a perfect problem finding this
- 05:30needle in a hay stack with many fake
- 05:32needles everywhere. So it fit with our
- 05:34research in causal machine learning and
- 05:35reinforcement learning really well. It
- 05:37fit with LLMs really well because the
- 05:39haststack is composed of like logs and
- 05:40metrics and traces and code and
- 05:42configuration files and so on and so
- 05:43forth. It fits with AI agents really
- 05:45well because you have to automate this
- 05:46complex workflow where you're, you know,
- 05:48querying all these different pieces of
- 05:50software. You're reasoning over them and
- 05:51then you're writing more queries and
- 05:52it's like this like sequential adaptive
- 05:54flow. And so it's a big market because
- 05:56everyone cares about software not going
- 05:58down. And I think it's only going to get
- 05:59bigger, right? because so much more code
- 06:02is being written now by companies like
- 06:04cursor or windsurf or copilot or what
- 06:06have you just like cambrian explosion of
- 06:09code being written no one understands it
- 06:11in some ways and so when software breaks
- 06:14it's it's going to be really difficult
- 06:15to troubleshoot it right and so I think
- 06:16that's what gave us confidence that this
- 06:18is the problem we should be taking on
- 06:20and then honestly the first VC I met in
- 06:21my life was Sequoa and I think they've
- 06:24been looking for a team to solve this
- 06:26problem they reached the same thesis
- 06:27they felt this is the problem where like
- 06:28the AI risk and technical risk is high
- 06:30and the market risk is low because if
- 06:31you can solve it, there's a big market
- 06:33and so people like us who don't come
- 06:34from this world but are good on the AI
- 06:36side are the right people to solve it.
- 06:37And so obviously that validation also
- 06:39give us confidence that this is the
- 06:40right problem.
- 06:46A lot of what these AI agent companies
- 06:48are doing is trying to replicate what
- 06:50humans have done, but there's so much
- 06:51more that can be done. And so thinking
- 06:53from first principles what AI systems
- 06:55are good at and exploiting that versus
- 06:57just trying to replicate what a human
- 06:59has done I think is also going to be
- 07:00very important to reinvent and actually
- 07:02get to the next level of innovation.
- 07:04Creating a MVP is so easy with all the
- 07:07tools out there. You can really iterate
- 07:09quickly with putting a product in
- 07:10people's hands. We built our first MVP
- 07:12probably last year in June or July like
- 07:15about 3 months in. And with small
- 07:16companies it worked great because the
- 07:18scale of the data was small. we could
- 07:19kind of look at their historical
- 07:20incidents, see what the playbook was and
- 07:22then put that into an AI agent. And so
- 07:24like our accuracy was like 90%.
- 07:26Something amazing, right? So we felt
The First Principle Saved Us From 0% Accuracy
- 07:27really confident that this is going to
- 07:28work. Just because it works in the one
- 07:30time doesn't mean it's always going to
- 07:31work because the world is constantly
- 07:32changing. And then we hit some of the
- 07:34larger enterprises including Digital
- 07:35Ocean, our accuracy went to 0%. Which is
- 07:38very difficult to see. It was a tough
- 07:39week. Creating an MVP is easy, but
- 07:41creating a production system that works
- 07:43in complex environments is really hard.
- 07:45And so I think one should not confuse an
- 07:48MDP with a production AI system. Those
- 07:50are like two very very very different
- 07:52things. But then we rearchitected a lot
- 07:54of things. We said how do we make sure
- 07:55that we're no longer trying to use our
- 07:58creativity and seeing you know get that
- 07:59into an agent but really use what these
- 08:01AI systems are good at which is using
- 08:03computation right using inference.
- 08:05That's really what unlocked us and
- 08:06suddenly our accuracy went back to up to
- 08:0890%. How do you make sure that your
- 08:10system gets better with the reasoning
- 08:13models? because there are a lot of
- 08:14people we saw in competing companies and
- 08:16so on and so forth where as the reason
- 08:18models came out they didn't get any
- 08:19better. So how do you make sure that
- 08:20you're exploiting what the reasoning
- 08:22models are good at? And the way I put it
- 08:23is that the reasoning models are very
- 08:24good at like detective stories. You have
- 08:26a mystery novel and you're trying to
- 08:27figure out who who did the crime.
- 08:29There's all these different pieces of
- 08:30evidence that you're seeing and you're
- 08:31trying to figure out who is the person
- 08:32who did it. Connecting all those dots
- 08:34and figuring out the thing, you know,
- 08:35the person who did it. that kind of
- 08:36detective story type workflow which I
- 08:39think these reasoning models are very
- 08:40good at where you have a clear answer at
- 08:42the end and you have lots of moving
- 08:43pieces that you have to like connect the
- 08:44dots between to get to the clear answer
- 08:46that felt kind of perfect for us right
- 08:48because for us you have all these
- 08:49different symptoms that happen at the
- 08:50same time you find ways to connect the
- 08:51dots to find that specific right answer
- 08:53like who did it making sure we were
- 08:55exploiting them to the maximum was was
- 08:57crucial I think and so I think that's
- 08:58the way I would put it
- 09:04is going to write so much more code and
- 09:07no one really understands all of it,
- 09:08right? If you wrote all of it, you have
- 09:10in your head just how it all fits. And
- 09:12as you get to bigger and bigger systems
- 09:13already, right, you work with some of
- 09:15the largest fortune 100 companies, no
- 09:17one is full context. No team is full
- 09:18context about what's happening because
- 09:20it's such a complex system. And that's
- 09:22just happening not just at the large
- 09:23companies, but also at the small
- 09:24companies because this the code is no
- 09:26longer being written by human but by or
- 09:29engineer but by AI system, right? So the
- 09:31lack of context means that when it's
- 09:33when an incident happens, it's just so
- 09:34much harder to debug it because you just
- 09:36don't have all of the context you need.
- 09:38And actually it's already happening like
- 09:40most people now are not developing code.
- 09:41They're starting to like validate QA
How to Survive the AI Coding Era - Do What You Love with People You Love
- 09:43code or troubleshoot code and that
- 09:45doesn't scale. So that's the big
- 09:46problem. And I think the second big
- 09:47problem I think is that you know with
- 09:49all all the developments happening with
- 09:50AI software engineering, our belief is
- 09:52that as engineers we get to do the
- 09:54really creative fun work architecting
- 09:56system design. Over time, all engineers
- 09:59will be doing will be troubleshooting,
- 10:00which would be sad in my opinion. Like
- 10:02they should be doing the most we as a as
- 10:04engineers should be doing the most
- 10:05creative work, right? And to to make
- 10:06that a reality, you you need to have
- 10:08systems, not just developing your
- 10:10software and building it, but also
- 10:11maintaining it. And so I think all of
- 10:13software maintenance needs to be
- 10:14reinvented. You have to just kind of
- 10:16persevere. If something goes wrong,
- 10:17that's okay. I think it in some ways
- 10:19it's a good thing because if it was
- 10:20easy, then everyone could do it, right?
- 10:22I think this is one of those problems
- 10:23where the problem statement is very easy
- 10:25to state, but to actually solve it is
- 10:26really hard. And I think that's where
- 10:28there's like a lot of companies trying
- 10:29it, but very few actually succeeding.
- 10:30And I think having the ability to like
- 10:32stay resilient and have grit when
- 10:34something doesn't work and just stick
- 10:35with the problem is, I think, a big part
- 10:37of what differentiates us uh as a
- 10:38company. Think about what's going to be
- 10:40important 10 years from now regardless
- 10:41of whatever happened in the world. You
- 10:43have no idea what's happening most of
- 10:45the time. And you have to find ways of
- 10:46of creating structure from nothing. And
- 10:48so like one way I say it is that you
- 10:50know in typically hard jobs whether it's
- 10:52in finance or it's in technology as an
- 10:55engineer like you have a point A if you
- 10:57get a point B and it's very hard to get
- 10:58from point A to point B in the world of
- 11:00research and also in the world of
- 11:01entrepreneurship you don't really know
- 11:02where point A is you don't know where
- 11:03you are you don't know where point B is
- 11:05you don't know where you want to go and
- 11:06if you did know it's still very hard and
- 11:07so you're constantly like in this game
- 11:09of trying to just decide where you are
- 11:11and where you're trying to go and that's
- 11:12exactly the same thing in research as
- 11:13well. What's interesting in this job is
- 11:16that the the stresses are are can be
- 11:18very high, lows can be very low. So, I
- 11:20think getting used to like just very
- 11:21high highs and low lows is important.
- 11:24But I think that one thing I've learned
- 11:25is that time it takes me to to recover
- 11:27is very fast. Even if I'm super
- 11:29stressed, I'm super tired, within one or
- 11:30two days, if I just take it off, I'm
- 11:32back to full force. And so I think
- 11:33that's been like a good learning is that
- 11:34if you're really enjoying what you're
- 11:36doing and even though there's these
- 11:37moments of massive stress, levels of
- 11:39stress that you'll never face otherwise,
- 11:40if you really love a problem and you're
- 11:42you're attracted by it, that's where
- 11:44attracts other people, right? We all
- 11:46love the problem. We're all faced in our
- 11:48lives. And so we really feel like a
- 11:50great deep desire to solve it. And
- 11:52probably the most important thing out of
- 11:53anything is surround yourself with
- 11:55people that you care that you like that
- 11:56you want to be like. And if you do that,
- 11:58life will be okay. I think about my
- 11:59research life. I found the right PhD
- 12:01adviser advisers that really molded me
- 12:04and guided me the right way. If I think
- 12:06about this world, I found the right
- 12:07investors that guided me at the start,
- 12:09the right customers that you know that
- 12:11helped define the product. You live and
- 12:13die by the people you you surround
- 12:15yourselves with and the people will kind
- 12:16of guide you through these different
- 12:17parts. And so I think building a taste
- 12:19for the right people to mentor you and
- 12:21and be a partner with you is the most
- 12:23important thing. The rest of it, I
- 12:24think, will figure itself out if you can
- 12:25find the right people around you. But I
- 12:27wouldn't just start a company for the
- 12:28sake of it. I think you should you
- 12:29should feel like some thing deep in you
- 12:32cuz it's not easy. And so you need that
- 12:34kind of deep belief or you know or
- 12:37motivation to sustain you over time.