Building an
AI-Native Startup
Founder Survey 2024
Martina Lauchengco
Partner, Costanoa Ventures
Amanda Paolino
Investment Fellow,
Costanoa Ventures
Today’s early-stage startup founders are the first generation to launch companies in the era of GenAI-native applications. If they aren’t building on top of AI, they’re rethinking what and how to build because of it.
But what is the real impact of GenAI on the startup ecosystem? How are founders determining which models they use, what talent they need, and how to think about defensibility? And what decisions are they making right now that will impact how the world interacts with AI for years to come?
This summer, Costanoa surveyed 30* founders about their experiences and challenges building in the AI space — 21 of whom built their companies within the last two years. These are qualitative, not quantitative, insights, but they provide a helpful lens into how the startup landscape is evolving alongside GenAI.
It’s our hope that entrepreneurs can use this information to better understand how their peers are leveraging AI — the mix of models they’re using, the decisions they’re making, and the challenges they’re facing. These insights can help founders calibrate their own adoption of GenAI and build the kind of technical moat needed to create lasting value.
For everyone else, it’s important to understand how these decisions are being made — because the outcomes will impact our daily lives in ways we won’t even realize.
Think, for a moment, about how our smartphones have changed the way we engage with the world. If you lose your wallet, it’s an inconvenience. But if you lose your phone, your way to pay, ride home, or ability to reach someone — all feels gone. And that’s not because of the phone itself, but because of the 5.5 million apps enabled by it.
Right now, we’re on the precipice of a new GenAI app ecosystem. The average consumer may not directly interact with the latest LLMs or MMMs directly, but they’ll rely on applications that are built on top of them.
So take a moment to explore how these new capabilities are being applied in startups. A better understanding of the current moment will help us be more thoughtful stewards of our future.
Respondent Profile
The companies surveyed encompass a broad range of AI use cases, from planning and booking trips to customizing sales outreach to improving IT workflows. Key demographics of the company founders surveyed:
*Only companies with outside funding were included in the final results
$700M
raised across full cohort
$70M
collectively raised by seed and pre-seed companies who shared their funding
50%
building vertical-specific SaaS
17%
targeting consumers
47%
building vertical-specific SaaS
90%
U.S. based companies
68%
pre-Series A funding
57%
repeat founders
Key Findings
Talent
AI founders aren’t considered necessary, but they do impact hiring decisions.
Technology
Most founders use multiple foundational models, but GPT is a mainstay.
Challenges
Experienced founders are more attuned to go-to-market and cost considerations.
Talent and Culture:
How founders think about hiring AI expertise
The common wisdom is that hiring at an early stage startup is intensely competitive and difficult, especially for engineering roles — but these AI founders reveal a more nuanced landscape.
Is it important to be headquartered in the
San Francisco Bay Area?
The majority of global investment in AI startups happens in Silicon Valley — and half of survey respondents call the Bay Area home. But many o thers choose alternate geographies, including New York City, Seattle, Washington D.C., and even Lehi, Utah.
That said, second-time founders are 33% more likely to start their company in the Bay Area versus first-timers, citing proximity to talent and investors as a driving force.
There is no better place in the world to work on the most ambitious projects of this generation than San Francisco.”
Is an “AI co-founder” needed?
There doesn’t appear to be a “one size fits all” model for seeking out an AI cofounder. Just under one-third of companies founded after 2022 note they either have or feel the need for a co-founder with a PhD or background in applied models. Founder history, product type, location — there’s no detectable pattern to the prioritization of an AI co-founder.
Some founders note an AI co-founder helps with product development and fundraising, while others say it’s more important to have this skill set represented on the engineering team.
I don’t feel I need an ‘AI co-founder’ for this work, as the technology approach is more at the systems level, utilizing various AI tools. That said, it’s clear to me that in the not-so-distant future, building up that NLP/computer vision/related skill sets will be important to have on the team.”
Costanoa Insight
“In the current cohort of startups, many founders have strong technical networks already. It’s great to see how many are investing early in GTM hires, and hiring when that’s not your sweet spot is genuinely challenging.”
Which roles get hired first?
Over 70% of startups already have an engineer with an AI research background and/or prompt and applied model experience — but the presence of several key roles vary across stages.
Finally, the founders’ backgrounds seem to be a considerable factor in talent prioritization. Second-time founders are marginally more likely to hire across all roles, while companies with an AI co-founder are significantly more likely to employ an AI research engineer — 90% vs. 61%, respectively.
Do you have a(n)…(check all that apply)?
Is engineering talent really that hard to find?
Despite the assumption that most startups struggle to hire good engineers, fewer than half of founders reported that filling technical roles is their biggest talent challenge.
The rest either reported that they didn’t have any challenges around hiring, or showed more diversity in the roles they struggled to fill — such as UX, product design, and sales.
Engineers who are both strong technically and have a strong Product + UX instinct. This is especially important to us because we have a very non-technical ICP. Engineers often struggle to build products for non-engineers.”
Costanoa Insight
“This is about as much challenge as we’d expect to see in hiring strong technical talent. But it was surprising and reassuring to see how many were investing in GTM hires early. Many companies don’t invest in this early enough.”
Technology:
How founders use models, data and tools to build AI products.
The founders surveyed display a wide range of approaches to how they select, fine-tune, and leverage the latest foundational models — or not.
What foundational models are most popular?
The easy answer is OpenAI’s GPT. But the reality is more complicated — and varies by growth stage.
GPT is ubiquitous, but building on multiple foundational models is almost as prevalent:
We continually compare models on both performance and cost, and are willing to switch if we can get as-good performance at a cheaper cost.
As startups mature, they’re more likely to diversify beyond GPT — 80% of Series A and later use alternate models, compared to 60% of pre-seed and seed stage.
Gemini, Llama, and Claude are the most popular models after GPT. But there’s an interesting caveat: pre-seed and seed stage companies are more than twice as likely to use Mistral compared to Series A and beyond.
Which foundational models are you using? Check all that apply.
When diving into what would prompt founders to switch from using GPT as their primary foundational model, 75% mention cost and/or performance. But respondents share a wide range of opinions on what would really motivate them to transition.
Investor Insight
“The foundation model landscape is changing fast. On one hand, the converging trend on model performance has led to people choosing a variety of models; on the other hand, lots of people still choose the top 3 shops as a more future-proof option as the state of frontier keeps moving forward.”
How do founders choose models?
Founders cite similar reasons for choosing a model as switching from GPT to an alternative: cost, performance, ease of use, and accuracy.
However, repeat founders are nearly twice as likely than first-time founders to mention cost as a chief consideration (44% vs. 25%).
Performance and value are the biggest factors for us using AI. Low latency only matters for some specific use cases that we would like to pursue.
How common is fine-tuning models — or building from scratch?
Building an entirely new model involves deep work and is exceedingly rare, with only one founder reporting they are exclusively building their own model from scratch.
But roughly half of startups are fine-tuning models — training an off-the-shelf model on new data for specific use cases. Of those, 70% are creating products for vertical SaaS or consumer use cases.
Are you fine-tuning or building your own models to complement off-the-shelf LLMs?
Engineers who are both strong technically and have a strong Product + UX instinct. This is especially important to us because we have a very non-technical ICP. Engineers often struggle to build products for non-engineers.”
Investor Insight
“Choosing RAG vs. fine-tuning is still an evolving conversation. In fact, it’s probably not choosing one or the other but figuring what is relevant for a specific use case. How companies choose what they do to refine models, improve latency, and reduce cost will be more interesting to watch over time, especially as new capabilities emerge in frontier models.”
What data sources are most often used for training?
Most startups are leveraging proprietary data (61%) or public datasets (54%) to train their models.
But the likelihood of companies using synthetic or third-party data varies by stage:
Easy API and cheap to deploy for testing ideas at a currently low user volume. Good enough for our relatively simple purposes.
Notably, there are a few companies that don’t train their models, and instead implement them off-the-shelf.
We started with OpenAI models as they were the early leader. We are increasingly experimenting with Claude models because our customers are asking for VPC deployments, and Claude can be deployed inside a customer account
Ease of deployment and usage from Azure OpenAI. General robustness and capability of 4o model above other models.
Investor Insight
“Even with LLM’s trained on internet scale data, companies with a proprietary data asset are still in the best position to have a long term moat. Any company that isn’t thinking about how they can cultivate a unique data asset over time will get left behind.”
How are multimodal models impacting applications?
Multimodal capabilities describe a model’s ability to cover multiple “modalities”, like text, video, or image — but we’re still in the infancy of adoption.
Only 30% of founders surveyed are using multimodal capabilities in their products, with adoption rates varying by sector:
Investor Insight
”I really see this changing more, especially as it applies to voice and vertical SaaS applications. I’m already seeing companies think differently about the type of customer experience you can create by incorporating these new found capabilities.”
Which AI tools are most valuable to AI founders?
Just shy of 90% of respondents are using some type of AI tooling either to build their platform or to enhance their own work, with the majority of tools supporting infrastructure and MLOps.
Coding Tools
Cursor
Github Copilot
Supermaven
Agents
Aomni
Julius
Framework
LangChain
PyTorch
Infrastructure
Databricks Delta Lake
Kubernetes
LanceDB
Pinecone
Postgres
RunPod
Vertex A
MLOps
Aquarium
LangSmith
Parea
Wandb
There are so many out-of-the-box tools that are great to expedite prototyping (and sometimes production tools), such as LangChain and Pinecone. That said, broadly speaking, at this stage any tool using AI is helpful — because having our teams use these tools helps us all be more empathetic and curious to the current capabilities of AI. That creates valuable insights for how to build better products using these tools.
Investor Insight
“80% of these tools are closed vs. open source. That’s not surprising at this stage, but I wonder how much that will change over the coming years and if that represents more opportunity.”
Challenges:
What keeps AI founders up at night
When asked about their challenges, startup founders tended to cite predictable pain points (such as talent and product-market fit) while unexpectedly dismissing others (like defensibility).
What are the biggest challenges in building a company?
These founders give equal weight to acquiring customers and talent when it comes to naming their biggest challenges.
- Only 16% of pre-seed and seed mention talent as a major challenge
- But of those who did mention talent, 80% are repeat founders
What are the biggest challenges you’re facing as you build your company?
Trust. There is an increased scrutiny and skepticism of AI outputs. How do we start delivering value immediately while building trust and growing scope over time?
Investor Insight
“The core challenges of early stage startups are the same whether you’re an AI-native startup or not. These problems have been the same for decades, but now we’re adding a layer of complexity as teams learn how to harness the power of AI.”
How are founders dealing with model regression?
Models are constantly updated, making model regression a unique yet increasingly important issue for AI startups. If a new version of a model performs worse on relevant tasks than its predecessor, it could introduce unexpected changes in their product’s behavior.
While some founders seem lost on where to begin, the majority (60%) are already making an effort to keep up with model regression.
A few mention Kubeflow or Langsmith, but most respondents addressing model regression are doing so through their own manual setups.
We typically constrain our use cases such that the underlying data distribution doesn’t change very much, and we pin our model versions so the behavior doesn’t change without us knowing. Cases where we’ve experienced performance regression have been when we’ve fine-tuned our own LLMs on domain-specific tasks.
Investor Insight
“Read Twitter and try it out” was such an honest answer. And so GenZ. But it’s spot on and a smart way to learn fast.”
Do founders worry about defensibility?
Sam Altman already warned that OpenAI would “steamroll” AI startups, but founders do not seem worried about defensibility (the possibility of what they have built becoming part of a foundational model over time).
Only 13% expressed even ‘slight concern’ about defensibility.
Rather, most founders expressed confidence that their products delivered unique value or had such a niche purpose that they wouldn’t make sense for foundational models to absorb.
I worry about it some because that is my job. But I think the industry we’re in, the problem we’re solving, isn’t as likely to be fundamentally disrupted by foundation models, at least not for years.
Investor Insight
“This surprised me the most — but maybe shouldn’t have. You have to have impossible belief in your idea as a founder, or why else would you do it?”
What’s the most painful part of building in AI right now?
When asked about the difficulties of building in AI at the current moment, a few common themes emerged across all founders’ open-ended responses. First time versus experienced, early or later stages, most respondents mention:
Rapid pace of change
Inability to access data
Lack of quality tooling
Uncertainty
However, repeat founders are 2x more likely to mention “noise in the market” compared to first-timers.
The ground is shifting under everyone’s feet, and it’s not clear where we’ll be in six months. It makes it hard to fully commit to specific strategies with a small team.
Managing the entire process/workflow is tough. All the products out there to do so are severely lacking in simplicity. Also training models is hard today, since you typically need multiple nodes and many GPUs
Learnings and takeaways
As a whole, the survey reveals a diversity of perspectives — in other words, there is no formula to build an AI startup.
But for other founders and entrepreneurs, there are valuable learnings to use as you build your own business:
1.
Calibrate yourself
Are you exploring the landscape of models, or are you potentially missing out? Are you leveraging your own real-world data sets for training? Will adopting multimodal capabilities give you a competitive edge?
2.
Make decisions with conviction
From where you choose to headquarter your company to how you approach defensibility, being a founder is all about making choices. It takes courage to start a business, and part of that means making decisions that may be wrong.
3.
Celebrate your place in this historic moment
There are naysayers all around who only see the risk or negative applications of AI. But founders who are building companies right now are optimistic — and so are we. There’s enormous potential for AI to make our jobs better and our future brighter, and you’re part of it.