Garrett Sussman: Inside SEO Week. I got a treat for y’all today. I got a treat. I just did not expect this, but we were able to not only schedule this interview with a major release. I’m joined today by Krishna Madhavan, who is the principal product manager at Microsoft. He’s very much in the product of data and the product of LLMs and AI search and being as transparent as possible. I am so excited to talk to him today. How are you doing, Krishna? What’s going on, man? 

Krishna Madhavan: Hey, Garrett. Complete coincidence. We didn’t schedule this to match with what we did yesterday, but somehow serendipity works out, I guess. So thanks for making the time. This is awesome. 

Garrett: So context, Bing Webmaster Tools releases pretty much the first transparent real AI search data to individual businesses and websites. Can you just set the stage? I mean, I can only imagine what’s been going on in-house and obviously there’s a lot you can’t talk about. What can you talk about? 

Krishna: Yeah, I want to talk about the ethos of why we are doing this. And I think in our community, we’ve been going through massive change and the pace of this change is relentless. Every day there is something new that pops up and people are wondering, how do these things work? What’s going on behind the scenes? 

I remember when Mike used to talk about query fan-out, people were like, what the heck is this thing? And why is Mike talking about this stuff? But these are all genuine questions that the community has around how an extremely complex system that influences livelihoods, that influences search behavior, that influences information seeking online is evolving. 

So last year at SEO Week, I had talked about how I believe in transparency of what is going on behind the scenes as much as I can, and also providing the larger community with tools and services that enable them to shape their businesses a lot better. And by the way, I’m very fortunate because I work for an organization and a company that actually puts customer feedback very high on the list of things that we do. And we’ve heard this repeatedly. Hey, can you guys give us some tools into how AI systems are citing us? Can you guys give us some sense of what queries are happening behind the scenes? This is a very complicated problem, even for us to tackle because of the scale at which it’s happening and because of the complexity of the system that we are working with. So we’ve been trying and working on things like what we announced yesterday, the AI Performance Reports that we are showing in Bing Webmaster Tools. We’ve been working on it for a while. We’ve been testing things with customers and we finally figured out the first time that we are able to offer something of value to the customers. We understand totally that this is a long journey. It’s going to take time for these things to mature. But as patient as we want to be with how the product evolves, we also understand that there is urgency here and we want to be able to serve the customers a lot better. 

Garrett: I’m so excited. We’re also thankful because as we were talking about before this, data is data. There’s value in being able to come to your own conclusions. Can you clarify for this audience, when you’re looking at the new report, you have grounded queries. You’ve talked a lot about, you talked about fan-out query at Tech SEO Connect. Can you talk to the nuances of the similarities and the differences of how an SEO should look at the grounded queries in the report? 

Krishna: Yeah. So the view that we are providing is more of an aggregated view, right? So we look at all the surfaces where grounding happens, and we are synthesizing some of the information and providing a level of detail that essentially people can digest. If I were to just show you every single little query that happens in the system, it would not be beneficial for people to process and make decisions on. So there is a level of synthesis that we do. We do a lot of work to make sure that the information that we show is respectful of privacy and it does not put people in unsafe situations. This is ongoing work. It is extremely hard to do this because people search for all kinds of very personal information. People ask Copilot for very personal information. We have to be mindful of all of these scenarios. And we try to provide a view that we think allows people to craft content strategy, right? 

Like, so if you look at the blog that we released yesterday, we have a link to how people can optimize content visibility. And we think that using the data along with the strategies of traditional SEO will go a long way in helping with visibility. And ultimately, people have to take all of this as part of their governance process. There is no one single view that will solve all the problems that a business has. But we are at least trying to provide slivers of views into the system and how it operates. And it goes a long way in actually seeing what the system is looking at. 

Garrett: I think to that point, I’m curious kind of your take on the accuracy versus precision conversation, the directionality I saw on the blog post of the optimization recommendations. What do you say to someone who says this data is not valuable because of the probabilistic nature, because it’s synthesized? 

Krishna: Well, search has always been probabilistic. Yes. If people think that search was never probabilistic, I think that they would be wrong, right? Because search has always used machine learning, has always used some sort of AI technologies. We have a different version of these things. We have evolved in the terms of what the AI systems can do. But we’ve always had a probabilistic system. So if you were looking at search performance data and you didn’t think that was probabilistic and you couldn’t do anything with that, then there is no reason for you to look at grounding queries and say, well, that’s probabilistic. I’m not going to do anything with that. 

So the thing is to understand that with these kinds of probabilistic systems, there is almost always a sense of how much tolerance for probability that you have. And we are providing the best possible views, right? Like, so, I mean, we actually saw some of these queries, right? Like we didn’t make this up. So we are providing an aggregated view and we are filtering it for some components just to make it clearer for people. So probabilistic, yes, the system is probabilistic. AI is probabilistic, but that shouldn’t hamper people from making use of data points that they have to make their content better, their strategies better. 

Garrett: It’s a perfect explanation. And Mike sends in regards, he also is like, you know, thanks for validating all his stuff on query fan-out as well. We saw that in there. I do want to ask, and you can 100% say no, but is there any more data on the way? Are there any other data points that you would hope to bring to it in the future? 

Krishna: What I will say is that in yesterday’s blog, we pointed out that this is a long journey and we are at a very early stage of this process. We understand that the community has a lot of demands for data, for insights, for views. And look, it’s all coming from a great place. Community really wants to understand their business, right? Like, there’s nothing wrong with that. And like, we want to understand our business. The process of getting all the things that people need takes time and patience to work. And there are engineering teams that are looking at things. What I will say is that we are committed to being transparent. As to whether there will be a specific data point that gets released or not, I will probably not get into that part. 

Garrett: Okay. And I think that’s fair. Now, I am curious from, I’m so excited to hear you speak at SEO Week again. One of the big things, aside from what you’ve just released, I’m curious, in terms of AI search, what is top of mind for you? What are the great problems that’s going on in Krishna’s brain right now? 

Krishna: So in terms of AI search, the big thing that I think is always top of mind for me is how do I balance how much we crawl content versus the freshness of the content that gets surfaced, right? Because as we are doing all of this, we are also seeing reports that crawl behaviors are changing. Sites are getting crawled a lot more. So one of the big things that I personally am very invested in is crawl efficiency. How do we make sure that over the long period of time, we are reducing the burden on infrastructure to support these kinds of AI search scenarios and so on, right? Because I work in web data platform. This is something that we are constantly worried about. Like, are we being as efficient as we can because we are operating at scale? The second part that I always think about is how do we offer more visibility into how these systems are behaving? How do we make sure that we bring in more insights from the engineering side into the conversation to SEO and so on? So the technical aspect of SEO doesn’t get lost. So we are constantly focused on those kinds of problems. So this is a big area of work that is evolving for us. 

Garrett: Krishna, like the next six to 12 months, what do you think the direction of SEO is going to look like? 

Krishna: I think we are going to see a lot of convergence that is happening across experiences. So right now people are like seeing, you know, hey, there is this chat-like behavior. There is this search-like behavior that’s happening. And so we are going to see quite a lot of conversions that happens across the board in terms of how these systems interoperate and behave. This essentially also means that underlying infrastructure will converge quite a bit. We will start seeing that grounding is becoming even more important than ever before, which means that tuning for visibility, looking at data around visibility, these are all going to be key aspects. 

I think that for the SEO community, this essentially means that now is the time to really dive into data, right? Like if you’ve not done so, so far, I think a lot of people have been advocating for a data-driven approach. And for example, do you understand all the grounding queries that are happening? So these are all the kinds of questions that the community will have to grapple with because the appetite for these kinds of insights is only growing a lot. 

And you will see multimodal start to become a big deal. So we’ve been talking a lot about text-based optimizations and so on. The narrative is already shifting towards multimodal and you will start seeing multimodal evolve in a pretty big way. By the way, some of the image models that MAI released, they’re just phenomenal. If you’ve not tried out Microsoft’s MAI-Image-1 with Bing Image Creator, just give it a run. We are opening up the curtains a little bit more to let the light come in to the system. Like, so the black boxy nature of things, we want to, look, I understand there is IP, there is rights, there is protection, all of that, I get it, there’s privacy. But within those frameworks, are we able to offer the most view into what we can is a big question for me. 

Garrett: I remember last year at SEO Week, and there’s always the touching on the philosophy, and you mentioned the generative AI, whether it’s image or text – I’m just curious, what is your personal take on generative AI content creation and value versus human in the loop versus just purely human created content? Do you feel like there’s a risk where the majority of content that we end up encountering on the internet is generative? 

Krishna: So Microsoft has taken on, and rightly so, the approach of humanist AI, right? Like, so we believe that AI is in the service of humans, right? And humans essentially need to benefit from all of these. So if you look at what even the company’s leadership is saying around all of this, what we’re saying is, is it delivering value? Is something that we are increasingly asking, right? So we believe in human-in-the-loop kind of systems. If you look at our agentic systems that we are building, they’re all human in the loop. They’re sort of not fully autonomous. They are not all human hands-on. They’re sort of in the middle where there’s a lot of human oversight, right? We believe that that’s sort of the right approach because then it allows you to build for people as opposed to things that replace people. I don’t think you ever can do that.

So we are doing everything that we can to place humans in the middle of all of this. and my boss and my boss’s boss, and they’re all very clear that we are building humanist AI, right? So I think that, yeah, the content that we see online is going to have a lot of AI assist. It’s going to have a lot of co-piloting, right? But I don’t think that it’ll completely be all synthetic, right? Like, because that then becomes like a boring world, right? 

Garrett: Exactly. And to bring it full circle, I mean, it all comes down to authenticity and transparency, which is really kind of a theme of your talk. Can you give us, like, a little preview in terms of what you’ll be talking about at SEO Week? 

Krishna: Yeah. So my talk is titled Invisible Converged Web. Right. So it is basically if you look at the title, it basically tells you that I am about to open up the curtain into some of the processes that happen behind the scenes. And if you also look at the kinds of things that we have been announcing, I want people to understand what is powering these systems, right? What happens behind the scene? So it is a little bit of a scientific SEO talk, but I will also have enough cat references to make it interesting for people. I have to have it. You can’t have a Krishna talk without a bunch of cats. It’s part of the contract that I make with people, so I need to have cat references. 

Garrett: I expect to see 10 cats in a trench coat at this one. 

Krishna: Now that I have a metric to reach, I’m going to shoot for it. I’m a very metrics guy, as you probably know. 

Garrett: You were at SEO Week last year. Can you give some reasons, like, what was your experience? Like, why should someone get tickets if they haven’t already? 

Krishna: So first of all, the quality of the talks is just profoundly high, right? Like, people work really hard to bring the best stuff forward. Right. That’s number one. I mean, you look at the quality of the speakers that have already been announced. It is just pretty serious people. Right. 

The second is the community is just hyper engaged. You have these conversations during coffee breaks and lunchtimes and you just meet people and you get to know people, many of who become like your very close collaborators as you work through the year and they unblock you in many ways. I mean, I’ve reached out to people who I met at SEO Week last year when I’m reviewing something or like just want to have an external perspective. I’ve talked to people. 

The other thing that I will say is just real fun, right? Like, I mean, it is just a blast. People are like goofy and there is a vibe to it. And of course, Mike hosts it. I mean, you can suddenly expect to hear somebody like rap on stage. Or somebody, some big rapper show up or some big musician show up. You can never tell how big it’s going to be. Last year, I was walking down the street and I see this big thing at Times Square of SEO Week up there! It’s just these little nuggets of fun. It just shows that there is life in this, right? Like, so SEO Week to me is all about the life and the fun that it brings to the whole serious business of what we’re doing. Right. So that’s kind of why I want to come back. 

Garrett: I’m so – I’m excited. I’m excited to see you again. I’m excited to hear your presentation. It’s like it’s so much fun. Our industry, as serious as it is, AI search is fun. Krishna, if people want to find you online, what’s the best way to get in touch before the event? 

Krishna: Oh, LinkedIn is the best way to reach me. I think my profile is kmadhavan7. Something about number 7 with me. And it’s similarly on X. People can reach out to me. I’m very responsive to LinkedIn. I also am on X. It’s @kmadhavan77, I think, my handle. So you can reach out there. Please do. People ask me a lot of questions. I usually jump in. So I’m happy to be a resource for the community. 

Garrett: Well, thank you so much. Thank you and congratulations for continuing moving our industry forward. The releases with the Bing Webmaster Tools and everything else that you’re working on. I can’t wait. 

If you guys haven’t already gotten your tickets for SEO Week, it is April 27th to the 30th in New York City again. Krishna will be there. It’s going to be awesome. We’re going to be goofy and ridiculous. And maybe you’ll see some rapping on stage, but you’ll definitely see cats in the presentation. Thanks for joining me, Krishna. I appreciate it, man. 

Krishna: Thanks, Garrett. Thank you so much. 

Garrett: See you guys.