State of AI 2025 Preview

[SachaG]

Here is a preview link for the upcoming State of Web Dev AI 2025 survey, the first ever edition of this new survey:

https://survey.devographics.com/en-US/survey/state-of-ai/2025

I would love to get feedback on:

• Better formulations for existing questions
• Missing options for existing questions
• Other questions to ask
• Anything else!

[SachaG]

• Removed Image Generation section and added it as a question in "other tools"
• Added "pain points" questions for all main tool categories

[SachaG]

Suggestion: put IDE/code generation sections first to avoid people filling out "cursor" in "coding assistant" section?

[captainbrosset]

Since this is targeted at web devs, I wonder if there should be questions about APIs as well.
The survey currently focuses mostly on tools that developers might use while coding. With more and more ways to run AI model on the web (either server-side, or client-side with WebGPU, WebNN, or the newly proposed Prompt/Translator/Rewriter/... client APIs), it would be great if we could ask if web devs are using, which ones, and what their feedback is.

Edit: I now see that there's an AI browser API which I missed.
I see it's a list of checkboxes right now, but it doesn't say how to answer. Are people expected to say what they want? What they already use?
It'd be great to add the positive/negative flags and free text comments to this as well.

[SachaG]

• Add DeepSeek

[altryne]

Braindump of notes as I run through the survey:

• Coding Assistants should start with Cursor, Windsurf, Copilot then everything else (I'd add continue.dev) - ok I scrolled forward to IDEs and see Cursor and Windsurf there.. maybe worth opening with IDEs and then assistants afterwards to remove confusion

• Code Generation is a strange naming, not sure what's a better one? defintiely add lovable.dev and for open source there's openui

• Image generation (add these):

• • Ideogram
• • Google Imagen 3
• • Grok

• What proportion of the code you produce is AI-generated?
  This seems like a hard question to quantify

---

The things I'm missing the most out of this as someone who's following AI very closely (weekly podcast for the past 2 years) are:

• There's no question about Agents, either building them, running them or using them

• Agents like Operator from OpenAI, Goose from Block, CrewAI

• Asking a question about Agentic frameworks (Langghraph, Crew, Eliza) and experience with building with those agentic frameworks

• "Do you run open source AI locally? If so how"

• • Ollama
• • LM Studio
• • ggml

• Models survey

• • Heard of "LLama, Deepseek, Qwen, Phi"

• Reasoning models, interacted with them?, what's the experience been like

• • DeepSeek R1, o1 from OpenAI, QwQ, OpenThinker

• Ai Hardware devices, heard, bought, experience?

• • Rabbit R1
• • Friend
• • Humane Pin
• • Meta Raybans

[eric-burel]

I think we miss "GitHub Spark" alongside v0 and bolt in the code generation section. It's in technical preview but will probably aim at being a big player : https://githubnext.com/projects/github-spark/

[eric-burel]

To answer about agents: the problem is that we are entering the build-your-own LLM solution category. That could be inserted somehow into the "Usage" question, asking whether people are crafting their own AI pipelines/agents/whatever advanced solutions. This could be relevant given that with frameworks such as langchain it can be actually relatively easy and I see a lot of developers crafting their homemade tools.

[SachaG]

Another question: should we ask about specific models, rather than just "apps"? e.g. GPT 4o, o1, deepseek r1… I guess I'm not sure how many people are aware of which model they're even using?

[SachaG]

• Added lovable
• Added openUI
• Moved IDE section first
• Renamed "Chatbots" to "Model Providers"
• Added "Have you tried running AI models locally on your own machine?" question

[SachaG]

@altryne

Thanks so much for all the feedback! I'll have to do some more research into all these topics! Generally I would say these are all great suggestions but maybe still a bit too niche/early-adopter-focused?

The things I'm missing the most out of this as someone who's following AI very closely (weekly podcast for the past 2 years) are:

There's no question about Agents, either building them, running them or using them

Agents like Operator from OpenAI, Goose from Block, CrewAI

Asking a question about Agentic frameworks (Langghraph, Crew, Eliza) and experience with building with those agentic frameworks

I'm not sure I know enough about agents to write that question, and I feel maybe it's still a bit too early? Operator isn't even fully available yet.

"Do you run open source AI locally? If so how"

I think we could ask this as a yes/no question for now? And then maybe get more granular next survey?

Reasoning models, interacted with them?, what's the experience been like
Ai Hardware devices, heard, bought, experience?

These also seem a bit too niche given that we're not surveying only AI-industry people, but "regular" web developers.

[eric-burel]

@SachaG an agent is more a pattern than a specific solution, basically it's an agent as soon as your LLM starts talking to itself and using tools, it can be as simple as a while loop around an API call. So web developers may implement their own agent, or use one at company level. I think tracking agent usage/development it could be a good indicator of devs and ecosystem maturity. Or at least whether they have some sort of homemade/company-wide LLM solution around.

Apart from that I've discovered the open source project Cline, which seems pretty popular. It works as a VS code extension like GitHub Copilot and could be mentioned alongside other assistants.

Also I suspect Microsoft Copilot will be conflated with GitHub Copilot.

[Aschen]

Hello 👋

@eric-burel told me about this exciting initiative, great job guys and thanks for doing it 😄

I'm not sure to understand if you want to do a survey about AI tools for developers, about AI tools used by developers to create product or both.

If you also aim AI tools used by developers to create product, then I think you could add questions/sections for:

• observability, evaluation: yes/no, then which tool: gentrace, langfuse, langsmith, etc
• fine tuning: yes/no, SaaS/local/cloud, which model

Also, I have few remark about existing sections, feel free to comment on my comment 😁

Models part

• ChatGPT is not the name of a models but the name of an end user application. The name of this family of models is "GPT"

Also it could be interesting to ask user which kind of model they are using, usually we can classify LLM in 3 categories:

• SLM (Small Language Models): usually models with <7B parameters like gtp-4o-mini, phi-2, Gemini flash, Claude 3.5 haiku, Mistral small
• reasoning models: models with internal chain of thoughts like o1, DeepSeek r1, Gemini 2.0 flash thinking
• LLM (Large Language Models): every other model > 7B parameters like gpt-4o, claude 3.5 Sonnet, Gemini 1.5 pro, LlaMa 3, etc

Other tools part

(Could be a library part)

Some suggestion on interesting tools from the ecosystem:

• DSPY
• Outlines
• @google/generative-ai
• LangGraph
• LangChain
• LlamaIndex

I think for each tool, an interesting question could ask if the user use this tool for a POC, personnal project or a production project:

• "Never heard of it"
• "Heard of it"
• "Used it for personnal project"
• "Used it in my company for a R&D project"
• "Used it in my company for a project in production"