LLM Experience Gain: How We Climbed the Animal SERPs with a Text-based Gen AI Game Featuring Zack Notes

Garrett: All right, welcome back to The Next Chapter of Search produced by SEO Week and iPullRank. If you haven’t already gotten tickets, we are like we’re under a month – it is, we are weeks away. April 28^th to May 1^st, New York City, 40 of the top speakers across SEO, AI, data, content strategy, digital marketing and general business…right you name it. And I’m joined today by someone who is in the weeds of the world of LLM AI content generation and doing a ton of really cool stuff with SEO. I’m joined today by Zack Notes, who is the founder at Sandbox SEO. Thanks for joining me today, Zack. How you doing, man?

Zack: Yeah, thank you, Garrett. It’s great to be here. And I’m excited for SEO Week. I’m working on my slides and I’m happy to be here and chatting with your group.

Garrett: You know, and we were just talking before this there’s so much going on in the world of SEO and AI. The last 12 to 24 months have been wild for practitioners everywhere from like, you know, big enterprise-level brands to startups who are just trying to like get on the map and get in the index and the challenges are real. Things are changing. What’s your perspective on the current state of search?

Zack: Well, there’s really like two sides to it, right? With search, there’s the search users and then there’s also the search marketers. And for the users, like the whole experience of search is changing. And as a result of that, their behaviors are changing and the way they’re thinking about search is shifting and, you know, the capabilities of search and what it can do for them. So, they’re using it in different ways. And then on the other side of it, there’s us who are trying to make sense of it all and make sense of the user behavior that’s changing and the traffic shifts we’re seeing. And, you know, there’s SEOs that are kind of debating a lot over like whether like, you know, SEO for LLM is really that much different than what we’ve been doing, trying to understand our traffic shifts and just like a lot of trying to kind of wrap our head around the changes and how it’s impacting us and the bottom line of our clients and our companies.

Garrett: It’s really hard. Like how are you thinking about data? Because I feel like we’re getting data from Google, from our own websites, from other marketers saying stuff, and it’s so hard to know what we can trust and what is BS. How are you kind of thinking about, you know, in terms like search behavior, like are people using search differently? Is AI overviews generating less traffic? Like where do you stand on that?

Zack: Yeah, sure. So personally, when I’m looking at AI overview traffic, you know, we don’t have like a filter in Google Search Console yet. We’re probably not going to have one for years because it’s probably just not a priority for Google. So, I’m mostly looking at SEMrush to see their AI overview search feature. One thing that’s interesting is like I hear a lot of people talking about how the traffic that’s coming from LLMs, from chatbots like ChatGPT is like so much better converting, and there’s all this attention and interest in like that new traffic. But I question like the causality there. Like if you think of it like a watermelon, right? You take a one-inch cube out of the middle of a watermelon, it’s going to be like the sweetest, juiciest part of the watermelon. And I think those are the LLM users that you’re seeing show up on your site. And like it’s the same customers you had before, it’s just a much smaller segment. And so, you know, I think you kind of have to take it with a grain of salt. And it’s – and I understand that like it’s just a new behavior. It’s the same people buying the same things or trying to get the same information. They’re just going through a different channel now.

Garrett: I think it’s a great analogy. And it’s very frustrating as marketers, like when we’re in this world of like trying so hard for you know, to find attribution and to say what we’re doing, like causality is like what’s working where we live in a world with marketers that are so focused on correlation and coming to conclusions with potentially incorrect assumptions. I say I’m right with you on that. Speaking of LLMs and kind of like what’s going on in this whole idea of experience gain, can you kind of give a preview? Like I’m excited for your presentation. You’re working on those slides. What are people going to see? What should they get excited about at SEO Week that you’re going to present to us?

Zack: Sure. So I’m going to tell the story of a site I built called Animal Matchup and launched it a few years ago, right when like the OpenAI API came out. And we were kind of like looking for a few different test ideas where we could like test the combination of programmatic SEO with GenAI content. And we saw there was a lot of search volume for like animal versus animal, right? So, like gorilla versus grizzly or like, you know, and it was a really interesting space that at the time, like what I was seeing was a lot of the content that existed was mainly just forum content, opinion pieces, people who have a strong opinion over one animal or the other. And what didn’t exist was any kind of like fight story, sort of creative story about the fight. So, what we did was we used ChatGPT to create actual fight stories. And we have since made 2,000 pages of AI content, a few pages of higher quality handwritten content.

But we’ve now been through, you know, eight algorithm updates. And we’ve seen a lot of AI sites get hammered and somehow like we’re still surviving. So, a lot of the talk is about tips that we can give, kind of theories on like why we’re still alive after all these updates. And like tips on scaling AI content. And also, more about like LLM gain, which is basically using LLMs to create features that give users an experience that they’re not already getting on the SERPs and kind of trying to offer something better than the both like the existing competitors, but also then Google and the new AI overviews. Like I think there’s a better experience that could be had if you really take the time to understand the users and the intent and the way that they’re thinking about the search.

Garrett: I love the concept. I love the whole, you know, animal matchup because it’s like, I feel I see these stats of like, you know, some disturbing amount of Americans like think they can actually like destroy a bear. Like you’ve seen that stat of like 6% of Americans think they could fight a bear. So, it makes sense that people are into these creative stories. And there’s so much…I balk at the idea of people who say like, AI generated content is not original or unique or interesting. Like yes, like it’s not at the human capacity, but the prompts and the ideas and the synthesis of creativity, you can do a lot of stuff with that, that you obviously have been working on.

I’m very curious to hear about that. And then finally, your point about, you know, just creative uses of this, like the sky’s the limit. So, you mentioned a bunch of tips, like what’s one that someone can consider doing right now for their website in the context of like search and LLMs that they can put into play?

Zack: Sure. So, one tip I have is just sort of a little bit more general outside of just LLM integration, but just internal links, right? We know that we need internal links for indexation and recently did some math that produced a pretty interesting formula. So, I learned that you need at least if you have at least five internal links pointing to random pages within the same template, your orphan page rate goes down below 1%.

Garrett: Ah, interesting.

Zack: And that’s based on like a formula that I was kind of like help, like ChatGPT helped me develop it. I cross checked it across a few different other LLMs, Gemini, Julius, they all arrived at the same formula, E^-K. So yeah, it’s just, you know, I’ve seen a lot of people like debate about how many links you need. And then we’ve got like designers and now there’s like a clear formula that we can use that like helps determine that. For LLM generation, one of the tips that I’m going to share, I love GPT in Sheets. I use it to scale content. And one of the challenges with GPT in Sheets is that every time like you refresh the Sheet, it reruns all of your prompts again. So, like A, that costs money and B, you lose like anything that you’ve saved. So, like one simple suggestion is you can use like concat function to save your prompts in the sheets. You don’t have to go searching elsewhere for the prompts. And you also don’t like rerun it again.

Garrett: That is especially for noobs. It’s like if you’re doing this out, like why learn the hard way? Like you’ve done the work and figured out like that, that like kind of little trick. I love that, man. So, what’s next? Like there’s so many directions that the future of search can go for SEO, for LLMs, for user behavior, like short term, long term, Zack, where are you looking? What do you see?

Zack: Yeah, I mean, I think that right now, the average user of LLMs is younger. And I think that like all new technology, like that’s going to scale, or not scale, but shift like older, right? So, I think it’s only going to spread. And so, I think we’re going to have to get like right now, Google is still 90% of search share. And I will see what happens with AI mode. I think that can be pretty pivotal in terms of like shifting LLM-type traffic back to Google. But yeah, I think only more and more people will learn how to use LLMs. And we’re going to have to continuously adjust to that for at least like the next, I’d say five, six years, it’ll probably grow and saturate at that point.

Garrett: It’s so interesting. One thing I learned a lot from the DOJ filings, in terms of the antitrust with Google is just how their whole, you know, kind of argument was based on default search settings, where like Google was the default on Apple and, you know, Firefox and Samsung, all these devices. And you think about like the way they rolled out AI overviews, where, you know, that was people were asking, like, how do I turn it off, but it was like a forced experience. You almost wonder with AI mode, if Google is going to take the risk that they did with AI overviews and making it a forced experience at some point, or how will people ultimately be, you know, forced to use these types of tools so that the technology kind of forces user change? Like, do you think Google has so big a moat that regardless, they’ll kind of come out on top or do you think other new emerging competitors like OpenAI will actually chip away at the AI dominance?

Zack: Yeah, I think Google’s pretty good at UX and they kind of take it slow. And I think that there will be a lot of like little mini entry points where they nudge us over to the AI mode tab and try to kind of introduce us to it and get people used to it. I don’t think that they’re just, I mean, it’s really keyword dependent, right? Like which keywords you’re on. But yeah, I think there will be a period where they’re kind of having to teach and onboard people about the existence of AI mode. But given the popularity of ChatGPT, I think that we’re close to a point where most people will know about it and use it. And I think it will probably eat away at OpenAI’s share to a big extent, just because of, like you said, like the default search engine for people, like it’s path of least resistance. You’re already going there. It’s got a muscle memory. Yeah.

Garrett: Have there been any like non-tech, non-SEO folks in your life that you’ve seen them like use ChatGPT or any of these tools for the first time and seen what their experience is like? I mean, I find really interesting right now. Like I introduced it to like, you know, my 70-year-old dad at one point or people who don’t have that sort of like latent experience with AI and it almost seems magical to them. What have you been seeing in interactions of people trying this out and do they get the utility? Are there minds blown or are they like not interested?

Zack: It’s kind of funny. So, my dad introduced me to the internet when I was like 12 years old. He called me into his office and said, “Hey, I want to show you something.” And he showed me the website of like the Autobahn. And I was like, I couldn’t care less. That’s cool, dad. I don’t care. And so, and since then, like he’s been sort of like for me, like an inspiration for someone that he builds neural networks and he still codes even though he’s like 82. Like he was a math major, like very much kind of an early adopter. My dad gives zero F’s about ChatGPT. My mom is the power user. She uses it for like shopping comparisons, making decisions, gardening questions, cooking questions. And he still like doesn’t realize how big of a deal it is, which is really like amazing to me.

Garrett: And that is so fascinating. Zack, I cannot wait to have more conversations with you in New York. Thank you so much for taking the time and, you know, kind of giving me your perspective. I can’t wait to see your deck. I don’t know if any help getting that that slides. It sounds like you are very close to being ready to just crush it on the stage. So, thank you so much, man.

Zack: Yeah, it’s good meeting you again.

Garrett: Good meeting you too. So, you guys, if you haven’t got your tickets yet, April 28^th to May 1^st, New York City, you got to be there. It’s going to be The Next Chapter in Search and learn so much. Hopefully we will see you there. And thanks for watching. Peace out.