Sociable: Meta is developing a search engine to power its AI chatbot

“Sociable” is the latest commentary on important social media developments and trends from industry expert Andrew Hutchinson of Social Media Today.

Could Meta build its own search engine, and if it did, what would that look like?

Reports have emerged that Meta is looking to build its own search tool for Meta AI, in order to reduce its reliance on Google and Microsoft’s Bing for web searches within its artificial intelligence chatbot.

There are a couple of potential angles to the news, but it seems like Meta is looking to expand its data-gathering process in order to continue to build more powerful, real-time AI data tools. Which could result in a broader Meta search tool, powering real-time insights for its apps. This is something that could have particular value for Threads, in providing up-to-the-minute notes on evolving news stories to complement engagement in the app.

When Meta first announced its Meta AI chatbot in September last year, it explained how it would be using web search results from Bing to help expand its responses:

“In text-based chats, Meta AI has access to real-time information through our search partnership with Bing and offers a tool for image generation.”

When Meta AI responses utilize Bing for information for context, Meta provides a web link to the search query.

Then in April this year, Meta announced an additional partnership with Google to expand its web search responses even further, providing a similar web link on Google-powered replies.

And now, according The Information, Meta is looking to build its own web crawler to power search queries independently, in order to reduce its reliance on its search partners for Meta AI.

As per The Information:

“As Meta Platforms tries to keep up with OpenAI in developing artificial intelligence, the Facebook owner is working on a search engine that crawls the web to provide conversational answers about current events to people using its Meta AI chatbot.”

Conceptually, this would give Meta an alternative, in case Microsoft or Google, both of whom are competing with Meta on generative AI development, decided to end their search deals.

But that would also mean that Meta would need to build a complex search system, which crawls the web in a similar way to both Google and Bing. Which is a big undertaking, and why Meta went with these partnerships in the first place.

But could Meta actually create a web crawler that functions in a similar way and would that be good enough to provide adequate answers in Meta’s chatbot?

Meta already has various web crawlers that gather information from external websites, and those crawlers have been ramping up activity in recent months, as Meta looks to gather more data for its AI projects. They could also already be gathering data for this new search engine, but a dedicated Facebook search engine, again, would be a big project.

Though a dedicated search engine is a logical move in the broader information gathering sense that would also give Meta more data for its AI language models moving forward. Maybe, that’s actually the bigger picture here, as opposed to simply powering in-stream search.

It’s more likely that Meta’s shoring up its data sources to power its AI tools and utilizing its existing crawlers to scrape whatever data it can from external providers who haven’t updated their robots.txt protections. That would help Meta build upon its already vast data stores, and if it’s already gathering that info, it makes sense to also reduce its reliance on external search providers where it can.

So maybe it’s likely less about building a competitor to these providers and more about making best use of its own data gathering processes. But even so, it’s a significant undertaking, which could provide expanded opportunities for Zuck and Co., if they can get it right.

Really, though, it’s not an unexpected move, given the competition in the space. Again, Meta is battling with these two providers for AI supremacy, and as the race continues to heat up, it’ll come as no surprise if they stop helping Meta in this regard.