Hey Everyone,

This is the second part in my summary of the State of AI Report 2024. You can see the first part of this summary here.

The State of AI Report analyses the most interesting developments in AI and it’s by Nathan Benaich and Air Street Capital. They are based in the UK so it may have some European perspectives a bit more highlighted.

Read the Report

Nearing the end of 2024 the question of whether scaling laws in LLMs are slowing down, what the acceleration of AI infracture and AI chips will do, the lack of quality data bottleneck, and many other related topics are top of mind among AI enthusiasts. People like Sam Altman, Eric Schmidt or Kevin Scott can insist all they like that the scaling laws of LLMs are not slowing down, but there are clearly many variables to take into account for the near future of Generative AI. How effective AI agents will be in 2025 is a big part of that. It’s no longer enough to talk about “sparks of AGI” to get a true sense of AI’s progress.

As an emerging tech analyst (with several related Newsletters) I make it my business to dive deeply into such reports and report back to you on what I find most salient. Air Street Press has a Substack and the section I like to follow the most is called Guide to AI. All to say some things will slow down but some scaling laws will continue, both are true. With a new AI Arms race in hyper-datacenter compute by the likes of xAI’s Colossus or Microsoft’s “Stargate” set to open in 2028, we’ll soon have some more tangible answers.

Even as Generative AI struggles a bit to add real-world value and displaying difficulty coming up with new “emergent capabilities” or tangible ROI in real applications and vertical industries, the pace of progress is nascent and will impact the decades to come. What’s going on in AI and LLMs is important, but perhaps not as important as the media and BigTech have tried to make it out to be the last two years.

We might be stuck in 2023 still in some ways halfway between chatbots and AI agents, and whatever else comes next. Another deep learning plateau or winter is entirely possible. If a lot of the hype was manufactured that doesn’t mean AI and exponential tech won’t be impactful, it just means you cannot fully trust reports from VC firms, social media and PR based channels. A lot of influencers, writers and evangelists become sucked into all of that. It becomes their job to be professional techno-optimists and cherry pick academic papers and ideas that support their growing following.

That’s not how I tend to operate. I try my best to provide AI news and content that’s skeptical, grounded and pragmatic. BigTech’s centralization of Generative AI is ripe for abuse of which we are somewhat likely to witness soon. So when we talk about “The State of AI”, it cannot be divorced from concerns of its impact on culture, rule of law, society, geopolitics, the future of energy, capitalism and even political leadership, the media and democracy itself. Divorcing the hype from the trends is part of my job as an independent analyst.

✨ Holiday Discount Deal 🥂

During the month of December, I’m giving away annual subscriptions at a discounted rate. Check out the offer:

Get 35% off for a 1-year discount deal if you use this code in the month of 🎄December. ❄️ Or use the code to give a Gift to friend, colleague, family member or acquittance who is a fellow AI Enthusiast.

Get 35% off for 1 year

What dominant use cases and what mega trends?

Racing to open up more data centers and make bigger ones for frontier LLMs

Code generation

Support chatbots

Enterprise Search, the Google for Work

Goofy viral products

Using Generative AI to upgrade military systems

Merging Generative AI into general purpose humanoid robots

Chinese LLM startups and their open-source models are rising

Clearly 2024 and 2025 will see a lot of important foundational work. Even if the magic of ChatGPT has worn off and faded for products like Claude, for search like Perplexity, for an admiration of NotebookLM, and how Cloud providers like Amazon are supporting Anthropic to move forwards. Some of the pieces are starting to come together.

In a Trump administration will America choose to centralize Generative AI into authoritarian and surveillance capitalism instruments that will mostly benefit BigTech and their shareholders (the stock market going up), or something else? Can we trust Elon Musk as a Chief AI Officer of the United States? I have a lot of unanswered questions going into 2025. Whether we are covering AI tools or the future of AI’s impact on society, we’ve covered some ground recently here on this Newsletter:

🌟📋🌱 (November, 2024)

Popular Articles on AI Supremacy last month

How Silicon Valley is prepping for War

Major AI Functionalities 🧩 / with Apps🎮

“Who to follow in AI” in 2024? 🎓

AI for Non-Techies 🛠️: Top Tools for Meeting Notes

🌟📋🌱 (October, 2024)

Popular Articles on AI Supremacy from the month before

Prompt Engineering from 2020 to 2025 ✨

AI for Non-Techies 🛠️: Top Tools for Meeting Notes

Goldman Sachs and Economists are Backtracking on Generative AI’s Value

NotebookLM Custom Instructions

East vs. West in AI

When you consider the evolution of Qwen (Alibaba Cloud) and what Alibaba has done now with QwQ-32B-Preview (Read the blog), it really does make you wonder in the spectrum between China’s open-source ecosystem and the West’s closed models (Europe is not really even in the picture). I think it’s important to tell both sides and not only focus on U.S. Generative AI innovation, though writing about OpenAI has been entertaining in its own peculiar way as the B2C chatbot monopoly full of internal controversy and lawsuits. OpenAI’s revenue growth in 2024 was astounding, even as their leadership and direction felt increasingly unhinged. But Ads really?

In 2017, researchers at Chinese technology firm Baidu demonstrated that pouring more data and computing power into machine learning algorithms yielded mathematically predictable improvements—regardless of whether the system was designed to recognize images, speech, or generate language.

Now eight years later as we’re about to enter 2025 we’re about to pass into new territory of scale, AI chips and AI infrastructure. Lines are being drawn, geopolitics is heating up, national competition is fierce and monopoly capitalism (American BigTech) will supercharge the ecosystem will hundreds of Billions more investment to brute-force the next era of AI into reality. Ultimately these tensions will be good for civilization and innovation.

Share

Some scaling laws will continue, even as new kinds of bottlenecks appear.

Scaling Laws – Training Compute Expanding at 4x YOY

Image credit: Epoch AI

The consistent and predictable improvements from scaling have led AI labs to aggressively expand the scale of training, with training compute expanding at a rate of approximately 4x per year. – Epoch AI

The AI Startup Eleven Labs recently launched GenFM, a NotebookLM clone on their reader app.

It will take at least a decade from 2022 to even know the full impact of Generative AI on industry and vertical landscapes in the Enterprise AI sense:

The Big Three of Enterprise AI

Search and IT

Product & Engineering

Customer support

Reading the industry reports of Generative AI on the future of work and productivity of 2022 and 2023 becomes a somewhat laughable exercise in retrospect. Generative AI has made a lot of real economists, analysts, VC funds, professors, and investment funds say a lot of fairly inaccurate stuff. It’s unfortunate.

Two years since ChatGPT first went live, we don’t have any real clue even what the actual “modern Generative AI” stack will look like even a few years from now.

Big Three of AI Agent Frameworks

LlamaIndex

LangChain

CrewAI

Great Read: Agent-Responsive Design: Rethinking the web for an agentic future via (Sahar Mor).

Read Part I of the State of AI Report summary here.

AI Supremacy
State of AI Report 2024 Summary
Hey Everyone…
Read more

Let’s return to the report State of AI 2024, that’s so valuable in providing us summaries of the year before.

Generative AI Needs to Generate more revenue

The multiples of investment to sales are very concerning as the report points out:

The buzziest Gen AI startups don’t have much longer to prove their worth in actual revenue generation compared to the investment in them. Revenue can ramp up even substantially for OpenAI and Anthropic in 2025 and they might even begin to see profitability on the horizon until early next decade! It’s daunting future for any business, nevermind the Gen AI duopoly players.

With a Google acquisition and bankruptcy, I think we can assume Character.AI and Stability AI are shadows of their former selves. Cohere, Runway and Perplexity are running out of leaway and fast. Anthropic acquiring Cohere, who also specialize in Enterprise AI, seems likely. The “OpenAI of Canada” and whatever Sovereign AI props Canada has with it, are likely not so sustainable on these multiples.

Perplexity and text-to-video hype like Runway ML, still has room to run in 2025. In fact, text-to-video is yet another area where I expect Chinese products to get a lot better and faster. Whatever the case, how OpenAI and Nvidia navigate the 2025 to 2030 period has the power to color the entire Generative AI ecosystem and narrative. BigTech’s demand can fuel Nvidia’s AI chip growth and datacenter profits, but for OpenAI 2025 is much more dangerous going.

The unsung winner of Generative AI in 2024 is actually Meta Platform.

Bro, is everything okay? Mark Zuckerberg realizes that if Google begins to lose marketshare in Search, it’s a great moment for Meta to also go after the ultra lucrative Search Advertising market. Llama models are just the beginning for Meta, how they build B2B products for Enterprise is actually going to be a thing in 2025. After spending Billions on Metaverse efforts and a year of efficiency, Meta is super well positioned in Generative AI heading into 2025. Meta Platform’s stock is up 71% year to date in 2024.

Price to Latency will continue to Democratize AI

New kinds of techniques to make “open-source” models more efficient will also challenges some of the frontier models who will want to charge more for more advanced and their latest systems.

It’s clear that these Benchmarks being used are a musical chairs of the PR hype game. The competition from no matter how few players, is however ultimately good for developers building stuff with LLMs.

Read the Full Report

Inference and LLMs getting Better and Cheaper

The overall trend each year is for vastly better models and better price to performance ratios.

Can Google Deepmind Create the Best Price to Performance Ratio in 2025?

Google DeepMind is in a solid position now with LLMs even after many early failures.

Google’s Gemini 3, Jarvis (browser) and Astra (multimodal) are under a lot of pressure after the good performance by Google Labs’ project NotebookLM. Our guide was created by – you can read it here:

Read our Guide

Google remains fairly woeful in product and marketing, but at least Google DeepMind still has some good researchers and engineers.

Chat agents as interactive developer sidekicks…

The idea of Universal agents hasn’t gone well so far, as Google’s Astra remains mostly a prototype.

Big Three Browser Agents

Anthropic’s Computer Use (should get better rapidly)

OpenAI’s Operator (coming in January, 2025)

Google’s Jarvis (coming this month, apparently)

Note: Claude has gotten a lot better in writing recently with custom styles.

Read the Blog

In fact in 2025 tailoring LLMs and frontier models to our preferences will continue to get a lot better even as No-code AI agents begin to take steps to being actually useful across more workflows.

European AI is in Crisis

However you see the fate of the EU AI Act, the EU’s capability to build technology companies or Generative AI startups is very much behind the times even as Germany and France look like they are in political and economic instability.

According to the State of AI Report, Mistral is the bright light.

“European leaders have been desperate to point to a domestic success story as US labs have occupied the spotlight. For now, Mistral remains the continent’s primary bright spark.”

Mistral also appears better at raising funds, than generating revenue. As Aleph Alpha in Germany no longer afford to make LLMs. This “Sovereign AI” movement looks like more ideology and theory than practice. The U.S. and China really dominate and Europe looks like it’s entered a technological dark age. To imagine otherwise is hopefully naive at the end of 2024. Even as a BLOC the EU cannot compete in what comes next.

Even as ASML is being held hostage in their sales to China by the U.S. ASML is expecting a drop-off in sales to China as a result of U.S. trade restrictions. The firm expects China to return to taking up a smaller share of its overall global sales in 2025, CFO Dassen said in a transcript of a video interview of October 16th, 2024.

Who to Watch in European Gen AI?

Mistral (France)

Hcompany (France)

Dust AI (France)

Poolside (France)

Synthesia (United Kingdom)

DeepL (Germany)

Paris may be a bright spot, but the EU is badly behind in Generative AI or LLM frontier model leaders. The UK has a good number of Gen AI startups, but few good ones.

The VCs who are from the UK are overly optimistic about Mistral’s prospects in their report.

“With over €1B in the bank, Mistral has emerged as the undisputed European foundation model champion, demonstrating both impressive computational efficiency and multilingual capabilities. Au Large, its flagship model is available via Azure as part of the company’s new partnership with Microsoft.”

I like what Air Street Capital is doing but the EU bias in their report is fairly ubiquitous. A year later and Aleph Alpha and AI21 Labs (Israel) are barely worth mentioning anymore. It’s honestly embarrassing if you live in the EU!

Share

Databricks and Snowflake pivot to build their own models

Databricks is ahead of Snowflake in AI, but Snowflake partnering with Anthropic says a lot.

Snowflake announced its acquisition of Datavolo, a company specializing in advanced data integration technology. This acquisition is expected to facilitate the integration of structured and unstructured data into the Snowflake platform, further supporting AI and data analytics tasks. Databricks is likely to IPO in 2025 or 2026. Snowflake somehow has a market cap of nearly $57 Billion.

Databricks have also done better with AI in their acquisitions. The Mosaic research team, now folded into Databricks, open-sourced DBRX in March. A 132B MoE model, DBRX was trained on just over 3,000 NVIDIA GPUs at a cost of $10M. Databricks is pitching the model as a foundation for enterprises to build on and customize, while remaining in control of their own data.

Snowflake and Databricks aren’t growing like normal Tech or SaaS companies, these will be major players in the years to come. They will also increasingly intersect with Generative AI and new emerging AI architectures, just like Nvidia is getting into everything via their investments, the Cloud, partners and software ecosystems.

Emerging Big Three

Nvidia

Databricks

Snowflake

Whatever you think of Apple’s slowness to develop Apple Intelligence or Microsoft’s woeful Copilot Era, more nimble younger companies will begin to come to the foreground in 2025.

Snowflake calls itself an AI Data Cloud, while Databricks seems to be emphasising data ownership in the AI era. Nvidia, is just dominating. Nvidia is also easily the best investor I’ve ever seen in Generative AI, and I follow this as carefully as I can with .

Regulators scrutinize the relationships between key generative AI players…

BigTech’s PR media ecosystem doesn’t like to cover antitrust problems in BigTech, big surprise there. However Google and Microsoft have targets on their backs.

Google and the DOJ will be back in a DC federal court in April for the remedies trial. The Department of Justice says that Google must divest the Chrome web browser to restore competition to the online search market, and it left the door open to requiring the company to spin out Android, too. This is a big problem for Google’s search Ads dominance and a huge opportunity for everyone from Meta, to OpenAI, Perplexity, Microsoft Bing and others.

Meanwhile Microsoft is facing wide-ranging antitrust probes especially around how it bundles software and its AI behaviors. The probe was approved by FTC Chair Lina Khan ahead of her likely departure in January.

Regulators have particularly zeroed in on the close relationship between OpenAI and Microsoft, along with Anthropic’s ties to Google and Amazon. Amazon has invested $4 Billion more into Anthropic recently and have alone invested more than 50% of all funding they have received. This looks ugly for Monopoly Capitalism and Cloud players having more control in the U.S. frontier model duoply that OpenAI and Anthropic now represent.

Is this the model of innovation they really think is going to beat China? Totally inept as a strategy by the way.

Google’s own arguments against the DOJ look really weak. Selling Chrome might not be the most painful part of the DOJ’s antitrust demands for Google. Google is too reliant on Search Advertising revenue and Google DeepMind could lose a lot of talent as this verdict and trial progresses, even amid appeals. Google’s internal culture and leadership is already fairly problematic.

After Judge Amit Mehta ruled this summer that Google is an illegal monopolist in general search services and search text advertising, the government has finally laid out its plan for how to restore competition, with proposals ranging from relatively simple tweaks in business practices to large structural changes. – The Verge

Microsoft’s abuse of power in bundling and in their deal with OpenAI is even more alarming. While Trump might want to protect BigTech from attacks, Elon Musk with xAI might advise him to do the opposite.

Nvidia and TSMC are unlikely to gain much antitrust scrutiny, even as they carry Generative AI on their hardware and datacenter backs. Countries like Japan, the U.S. and Germany are heavily subsidizing TSMC infrastructure in their backwards as Samsung and Intel are in crisis. Intel even fired their CEO. Generative AI is like a game of thrones moment in the industry and semiconductor supply chain, where it’s obvious Taiwan and (and to some extent Japan and South Korea) are more serious and efficient.

Pseudo-acquisitions as an exit strategy

BigTech have also been acquiring Generative AI startups via shady means or acqui-hire strategies to get access to their models and best staff fast. It’s been horrible to watch this consolidation as many of the well-funded startups imploded in spectacular fashion including:

Inflection AI (Microsoft)

Adept AI (Amazon)

Stability AI (bad leadership)

Character AI (Google)

These companies have lost their best talent, leadership and often their best models. If they are rebooted it’s mostly for show in pathetic attempts to avoid scrutiny. Suffice to say Generative AI consolidation occurring so quickly is not a great sign for the massive investments in these types of companies and AI startups. BigTech acquiring you might a valid exist strategy, but it’s not good for the ecosystem.

It’s the equivalent of a SPAC but for an acquisition, and it should be illegal.

“Given some of the regulatory hurdles, we’ve seen the rise of pseudo-acquisitions, where a Big Tech company i) hires the founders and much of the team of a start-up; ii) the start-up exits the model-building game to focus on its enterprise offer; iii) investors are paid out via a licensing agreement.” – The Report

This winner-takes-all capitalism is so unhealthy but is part of the U.S.’s strategy to maintain dominance economically and technologically. If this is the future of Silicon Valley where a few companies are propping up the entire U.S. stock market there’s going to be a surprise one day.

Github reigns supreme, but an ecosystem of AI coding companies is growing

There are a bunch of code AI startups that really do have potential.

The one with the purple umbrella is Poolside. The one below it is Coedium. Augment is backed by Eric Schmidt, but he also backs Magic.dev. What’s wrong with that picture?

Big Four of AI Code

Poolside

Augment Code

Cognition AI

Magic dev

You have to admit though, Anysphere (Cursor AI) and Codeium are fairly impressive in their own right. Github Copilot certainly was a first mover advantage and Microsoft’s acquisition of Github was perhaps mostly for this reason (the deal should have never been granted).

Generative AI Search

We will have to watch Perplexity and Glean very closely in the future of RAG and Search.

Read our Guide on Perplexity

Signs of Google’s dominance slipping in the EU might be showing. According to Growth Memo on Substack.

Will 2025 finally be the year when OpenAI’s ChatGPT Search, Perplexity and others make headway against Google? Just as social media networks are bifurcating so too does it seem to be occurring in Search and various search experiences within research as opposed to topical search on mobile.

As ChatGPT says it has 250 million weekly users, in some ways that’s going to begin to impact Google search as a whole. AI search engines could begin to take marketshare from Google at faster rates accelerated by the DOJ case against them. If this were to occur, it would reshape the entire internet and it’s something I’ve written about several times.

The stock market has exaggerated how competitive Google, Microsoft and Amazon might be in Generative AI in the long-run. Tech companies on the S&P 500 have seen a 30% gain since January 2022. The Nasdaq Composite closed 6.21% higher in November, 2024 alone. But a young person is more likely to search on TikTok or ChatGPT than Google.

Share

OpenAI is betting all of its marbles on ChatGPT with a search product that lives inside of it. That makes sense because ChatGPT is how they make most of their revenue and where they are seeing most of their growth. If that 250 million weekly users goes to 1 Billion in 2025 or even 2026, that will begin to materially impact Google search and the future of Ad revenue even more.

The likes of Meta or Amazon could also seek to acquire Perplexity or build their own Search engine, since Amazon is taking advertising very seriously now the last few years. It would be a fairly good time to do it, Google could very well be in a different market position next year or a few years from now.

I’m out of time, I’ve gotten to slide 121 of the State of AI Report. I barely made a dent in this report covering topics related to just a few slides on it. The entire report reads 213 slides long.

Read the Report

Read More in  AI Supremacy