Technology moats in the AI era
Introduction
I spent six years at two health data startups and always appreciated the competitive benefit those business received from building data moats. In the health data world, patient data is highly fragmented and expensive and time consuming to acquire. I’d estimate a new player in the space with infinite resources would not be able to put together a competitive dataset (depending on the use case) for 2-3 years when factoring in multi-vendor negotiations, data testing, data integration and onboarding. Not to mention, some companies (like Meta/Facebook) that wanted to get more data, were considered too risky and denied partnerships by source providers. Having seen the benefit this provided these business by limiting competition, I can get why companies and VCs are anxious to find a way to build moat in the AI era. We’ll take a look at a couple ways to build moat and a couple ways I don’t think will work in blossoming AI era.
Part 1: Sorry it’s not this easy
There’s a really tempting narrative that I’ve seen floating around where companies attempt to build moat by creating unique foundational models. The thought process goes something like this:
Most ecosystem players will use OpenAI/Anthropic/Google OOTB foundational models because…
It’s very expensive to create your own models from scratch (up to $100mln for GPT4) so…
We can create our own model that will be better because
It’s tuned to our use case
We use private data to train that foundational model API provides don’t have
No one will be able to recreate this and we will win business easily forever with out super AI model.
This isn’t completely off, but I don’t see this holding up if it’s the only approach for a few reasons:
Foundational Model providers have already shown willingness to create more use case specific models (see Code completion models, Google MedPalm and Infosec) and model interfaces. Additionally, companies like Weights and Biases and MosiacML are lowering the costs of creating new models. I’ll write another blog on this at some point, but I think 95% of indicators point to a many foundational model future vs a super GPT8 (sorry if this robs us from a GPT10 release dance honoring the Windows 95 days)
Your unique data isn’t as much of an advantage as you think. We need to update out thinking on data moats. In the pre-Generative AI era, data advantage was at the record level (and in use cases that retrieve or use data directly it still is). However, AI models don’t use data directly, they train and extract language characteristics and embedded knowledge from the data. This means that someone with a similar dataset, even one that may be less complete or missing some of the records you have can create a similar dataset. Additionally, as we saw with Alpaca, if you let your model out in the wild, it’s not extremely difficult (or illegal) to reverse engineer your model by creating synthetic data. This isn’t new for modeled data, we saw this in the Health Ad ecosystem where competitors would create look-a-like models based on outputs of targeting models created with real patient data.
People might not actually care that your model is better. This sounds counterintuitive but we see this already even within a provider - GPT4 significantly outperforms GPT3.5 in all dimensions of accuracy, but is much worse in terms of latency and cost. So even if you create a better model, people may still opt for a faster, cheaper and less accurate model. Additionally, in many use cases “accuracy” is still hard to measure and the willingness to pay for an incremental 2-3% accuracy gain may be small.
So while building the best model for your use case won’t be enough in isolation, I do think it’s a potential piece of the puzzle.
Part 2: Let’s remember the old moat
Before we talk about AI moats, let’s remember a few things that work well in traditional SaaS businesses.
Obsession with the customer - Jeff Bezos has been pitching this since 1999. In practice this means that you invest in systems for feedback directly for your Product teams and indirectly through information captured from CX/Sales that feed directly into prioritization and design. This results in a constant stream of innovation in meaningful ways to the customer and let’s remember that the customer benefit for SaaS is founded in time distributed cost and a promise of “free” innovation.
2. Increasing switch costs over customer lifecycle - Why do so few people leave Salesforce CRM when almost everyone complains about it? Because they invest in customizing the system and building out data within it that would be very time consuming to switch. It’s important to note that time-consuming here I believe is more important than expensive - switching CRMs (or Cloud Infra providers as another example) can be multiple quarters of drag on business and internal leaders that create a situation where the negatives almost always outweigh the provides and competitors can’t counter with discounts and onboarding support.
3. Partner & Integration Ecosystem - If it’s daunting to switch a core system, it becomes even more daunting if that system is a driver for many other applications within the organization.
Part 3: Building Moat with AI systems
As you may have surmised from the lead-up, building moat in AI systems is not as novel as we want it to be, but there are some key areas of focus that we recommend for our clients. These recommendations are based on the assumptions that the Generative AI ecosystem will continue to evolve rapidly over the next several years and that we are headed towards a world with many models and intelligent routing.
Customer Obsessed AI Innovation - Do your customers care if you add a feature that expands an input form in your UI from a sentence to a paragraph? Maybe, maybe not (hint: probably not, since you are just replacing a copy-paste from ChatGPT). Creating a system of AI innovation based in customer feedback is a bit trickier since the capabilities of AI are changing and not well defined. To create advantage, companies need to invest in systems that showcase AI capabilities to their customers and solicit feedback on usefulness that can be rapidly acted upon. If you get this right, you will be wowing customers with more creative AND useful uses of AI (see: https://genaiproductevaluator.com/ and https://genaidataanalysis.com/ for some examples we produced) while your competitors are still expanding and summarizing text.
Build Interchangeable AI systems - Given the expectation of rapid evolution, it’s important to build AI driven systems using toolkits like Langchain that abstract out models and providers, that make it easy to switch. This will create a situation where you can update to new models in days post release, and competitors take months to capitalize on new features. With AI as such a hot topic, being first as the ecosystem evolves matters to customers. As a bonus it also lowers internal cost of development.
Build Generative AI systems that adapt to the client - This is the AI flavor of increasing switching cost. Generally it would consist of one of two things. Fine-tuning or customer/user specific prompt insertion. Fine-tuned models per customer are much more expensive and rigid in provider changes, so prompt insertion likely the best choice in most cases. What this means is behind the scenes you prompt may include variables that change per client (example for a prompt for a marketing brief) such as:
“(Fixed template - hidden from user): Act as a marketing agency creative director and write a 1 page marketing campaign brief with sections ... the tone should be {customer variable 1}, don’t mention competitors {customer variable 2}
(User Input:) Spring marketing campaign for NextGen energy drink
Base the format and output on these examples:
(Example:) {customer brief example 1}{customer brief example 2}”
4. Build downstream integrations based on AI - Building on the prior point, if you have value-able AI creative capabilities in your system based on customer data, don’t lock it into your system alone. Find downstream systems that can’t mimic that ability without the customer data and make sure your system is the source for those.
5. Build great solutions to the tricky problems that MATTER - While LLMs can do a lot out of the box, when we talk to most companies with experience releasing custom systems using AI, there is significant iteration an multiple tools involved in getting performance to a level that wows customers. It may take multiple quarters of iteration to get it right…this builds REAL advantage and if focused on problems that matter it will provide meaningful advantage. Tactically, this usually involves having an updated stack that allows for interchangeability, monitoring, performance evaluation. Right now having an updated stack is a competitive advantage in itself as Sequoia indicated less than 10% of companies have sufficient logging and monitoring (Sequoia trends article) to really monitor live performance and conduct A/B testing on model or prompt updates.
Conclusion:
To steal a line from Greylock’s Jerry Chen :
“THE NEW MOATS ARE JUST THE OLD MOATS.”
This is mostly true, all the old principles of GTM competition are still valid, companies need to figure out how to build technical and process systems that help them excel on those old principles in the AI world. If you found this article helpful, and want to work with us on specific plans for your company feel free to reach out to info@genaipartners.com
For some additional opinions on moat and tech stack - there’s a couple great articles that came out this week:
Greylock - https://greylock.com/greymatter/the-new-new-moats/
Sequioa - https://www.sequoiacap.com/article/llm-stack-perspective/
A16z - https://a16z.com/2023/06/20/emerging-architectures-for-llm-applications/
