Building an
AI-Native Startup

Founder Survey 2024

Martina Lauchengco

Partner, Costanoa Ventures

Amanda Paolino

Investment Fellow,
Costanoa Ventures

Today’s early-stage startup founders are the first generation to launch companies in the era of GenAI-native applications. If they aren’t building on top of AI, they’re rethinking what and how to build because of it. 

But what is the real impact of GenAI on the startup ecosystem? How are founders determining which models they use, what talent they need, and how to think about defensibility? And what decisions are they making right now that will impact how the world interacts with AI for years to come?

This summer, Costanoa surveyed 30* founders about their experiences and challenges building in the AI space — 21 of whom built their companies within the last two years. These are qualitative, not quantitative, insights, but they provide a helpful lens into how the startup landscape is evolving alongside GenAI. 

It’s our hope that entrepreneurs can use this information to better understand how their peers are leveraging AI — the mix of models they’re using, the decisions they’re making, and the challenges they’re facing. These insights can help founders calibrate their own adoption of GenAI and build the kind of technical moat needed to create lasting value. 

For everyone else, it’s important to understand how these decisions are being made — because the outcomes will impact our daily lives in ways we won’t even realize.

Think, for a moment, about how our smartphones have changed the way we engage with the world. If you lose your wallet, it’s an inconvenience. But if you lose your phone, your way to pay, ride home, or ability to reach someone — all feels gone. And that’s not because of the phone itself, but because of the 5.5 million apps enabled by it.

Right now, we’re on the precipice of a new GenAI app ecosystem. The average consumer may not directly interact with the latest LLMs or MMMs directly, but they’ll rely on applications that are built on top of them. 

So take a moment to explore how these new capabilities are being applied in startups. A better understanding of the current moment will help us be more thoughtful stewards of our future. 

Respondent Profile

The companies surveyed encompass a broad range of AI use cases, from planning and booking trips to customizing 
sales outreach to improving IT workflows. Key demographics of the company founders surveyed:

*Only companies with outside funding were included in the final results

raised across full cohort

collectively raised by seed and pre-seed companies who shared their funding

building vertical-specific SaaS

targeting consumers

building vertical-specific SaaS

U.S. based companies

pre-Series A funding

repeat founders

That said, second-time founders are 33% more likely to start their company in the Bay Area versus first-timers, citing proximity to talent and investors as a driving force. 

There is no better place in the world to work on the most ambitious projects of this generation than San Francisco.”

Is an “AI co-founder” needed?

There doesn’t appear to be a “one size fits all” model for seeking out an AI cofounder. Just under one-third of companies founded after 2022 note they either have or feel the need for a co-founder with a PhD or background in applied models. Founder history, product type, location — there’s no detectable pattern to the prioritization of an AI co-founder.

Some founders note an AI co-founder helps with product development and fundraising, while others say it’s more important to have this skill set represented on the engineering team. 

I don’t feel I need an ‘AI co-founder’ for this work, as the technology approach is more at the systems level, utilizing various AI tools. That said, it’s clear to me that in the not-so-distant future, building up that NLP/computer vision/related skill sets will be important to have on the team.”

Which roles get hired first?

Over 70% of startups already have an engineer with an AI research background and/or prompt and applied model experience — but the presence of several key roles vary across stages.

Finally, the founders’ backgrounds seem to be a considerable factor in talent prioritization. Second-time founders are marginally more likely to hire across all roles, while companies with an AI co-founder are significantly more likely to employ an AI research engineer — 90% vs. 61%, respectively.

Do you have a(n)…(check all that apply)?

Is engineering talent really that hard to find? 

Despite the assumption that most startups struggle to hire good engineers, fewer than half of founders reported that filling technical roles is their biggest talent challenge.

This image has an empty alt attribute; its file name is 01_17hd.png

The rest either reported that they didn’t have any challenges around hiring, or showed more diversity in the roles they struggled to fill — such as UX, 
product design, and sales.

Engineers who are both strong technically and have a strong Product + UX instinct. This is especially important to us because we have a very non-technical ICP. Engineers often struggle to build products for non-engineers.”

The founders surveyed display a wide range of approaches to how they select, fine-tune, and leverage the latest foundational models — or not.

What foundational models are most popular?

The easy answer is OpenAI’s GPT. But the reality is more complicated — and varies by growth stage.

GPT is ubiquitous, but building on multiple foundational models is almost as prevalent:

We continually compare models on both performance and cost, and are willing to switch if we can get as-good performance at a cheaper cost.

As startups mature, they’re more likely to diversify beyond GPT — 80% of Series A and later use alternate models, compared to 60% of pre-seed and seed stage. 


Gemini, Llama, and Claude are the most popular models after GPT. But there’s an interesting caveat: pre-seed and seed stage companies are more than twice as likely to use Mistral compared to Series A and beyond.

Which foundational models are you using? Check all that apply.

When diving into what would prompt founders to switch from using GPT as their primary foundational model, 75% mention cost and/or performance. But respondents share a wide range of opinions on what would really motivate them to transition. 

How do founders choose models?

Founders cite similar reasons for choosing a model as switching from GPT to an alternative: cost, performance, ease of use, and accuracy. 


However, repeat founders are nearly twice as likely than first-time founders to mention cost as a chief consideration (44% vs. 25%). 

Performance and value are the biggest factors for us using AI. Low latency only matters for some specific use cases that we would like to pursue.

How common is fine-tuning models — or building from scratch?

Building an entirely new model involves deep work and is exceedingly rare, with only one founder reporting they are exclusively building their own model from scratch.


But roughly half of startups are fine-tuning models — training an off-the-shelf model on new data for specific use cases. Of those, 70% are creating products for vertical SaaS or consumer use cases.

Are you fine-tuning or building your own models to complement off-the-shelf LLMs?

Engineers who are both strong technically and have a strong Product + UX instinct. This is especially important to us because we have a very non-technical ICP. Engineers often struggle to build products for non-engineers.”

What data sources are most often used for training?

Most startups are leveraging proprietary data (61%) or public datasets (54%) to train their models.


But the likelihood of companies using synthetic or third-party data varies by stage:

Easy API and cheap to deploy for testing ideas at a currently low user volume. Good enough for our relatively simple purposes.

Notably, there are a few companies that don’t train their models, and instead implement them off-the-shelf.

We started with OpenAI models as they were the early leader. We are increasingly experimenting with Claude models because our customers are asking for VPC deployments, and Claude can be deployed inside a customer account

Ease of deployment and usage from Azure OpenAI. General robustness and capability of 4o model above other models.

How are multimodal models impacting applications?

Multimodal capabilities describe a model’s ability to cover multiple “modalities”, like text, video, or image — but we’re still in the infancy of adoption.

Only 30% of founders surveyed are using multimodal capabilities in their products, with adoption rates varying by sector: 

Which AI tools are most valuable to AI founders?

Just shy of 90% of respondents are using some type of AI tooling either to build their platform or to enhance their own work, with the majority of tools supporting infrastructure and MLOps.

Coding Tools

Cursor
Github Copilot
Supermaven

Agents

Aomni
Julius

Framework

LangChain
PyTorch

Infrastructure

Databricks Delta Lake
Kubernetes
LanceDB
Pinecone
Postgres
RunPod
Vertex A

MLOps

Aquarium
LangSmith
Parea
Wandb

There are so many out-of-the-box tools that are great to expedite prototyping (and sometimes production tools), such as LangChain and Pinecone. That said, broadly speaking, at this stage any tool using AI is helpful — because having our teams use these tools helps us all be more empathetic and curious to the current capabilities of AI. That creates valuable insights for how to build better products using these tools.

When asked about their challenges, startup founders tended to cite predictable pain points (such as talent and product-market fit) while unexpectedly dismissing others (like defensibility). 

What are the biggest challenges in building a company?

These founders give equal weight to acquiring customers and talent when it comes to naming their biggest challenges. 

  • Only 16% of pre-seed and seed mention talent as a major challenge
  • But of those who did mention talent, 80% are repeat founders 

What are the biggest challenges you’re facing as you build your company?

Trust. There is an increased scrutiny and skepticism of AI outputs. How do we start delivering value immediately while building trust and growing scope over time?

How are founders dealing with model regression?

Models are constantly updated, making model regression a unique yet increasingly important issue for AI startups. If a new version of a model performs worse on relevant tasks than its predecessor, it could introduce unexpected changes in their product’s behavior. 

While some founders seem lost on where to begin, the majority (60%) are already making an effort to keep up with model regression. 

A few mention Kubeflow or Langsmith, but most respondents addressing model regression are doing so through their own manual setups.

We typically constrain our use cases such that the underlying data distribution doesn’t change very much, and we pin our model versions so the behavior doesn’t change without us knowing. Cases where we’ve experienced performance regression have been when we’ve fine-tuned our own LLMs on domain-specific tasks.

Do founders worry about defensibility?

Sam Altman already warned that OpenAI would “steamroll” AI startups, but founders do not seem worried about defensibility (the possibility of what they have built becoming part of a foundational model over time).
 Only 13% expressed even ‘slight concern’ about defensibility. 

Rather, most founders expressed confidence that their products delivered unique value or had such a niche purpose that they wouldn’t make sense for foundational models to absorb.

I worry about it some because that is my job. But I think the industry we’re in, the problem we’re solving, isn’t as likely to be fundamentally disrupted by foundation models, at least not for years.

What’s the most painful part of building in AI right now?

When asked about the difficulties of building in AI at the current moment, a few common themes emerged across all founders’ open-ended responses. First time versus experienced, early or later stages, most respondents mention:

Rapid pace of change

Inability to access data

Lack of quality tooling

Uncertainty

However, repeat founders are 2x more likely to mention “noise in the market” compared to first-timers.

The ground is shifting under everyone’s feet, and it’s not clear where we’ll be in six months. It makes it hard to fully commit to specific strategies with a small team.

Managing the entire process/workflow is tough. All the products out there to do so are severely lacking in simplicity. Also training models is hard today, since you typically need multiple nodes and many GPUs

As a whole, the survey reveals a diversity of perspectives — in other words, there is no formula to build an AI startup.

But for other founders and entrepreneurs, there are valuable learnings to use as you build your own business:

Are you exploring the landscape of models, or are you potentially missing out? Are you leveraging your own real-world data sets for training? Will adopting multimodal capabilities give you a competitive edge?

From where you choose to headquarter your company to how you approach defensibility, being a founder is all about making choices. It takes courage to start a business, and part of that means making decisions that may be wrong.

There are naysayers all around who only see the risk or negative applications of AI. But founders who are building companies right now are optimistic — and so are we. There’s enormous potential for AI to make our jobs better and our future brighter, and you’re part of it.