Continuing with the series on Foundation Models – key facts, we will discuss the main direction of progress in these models.
Foundation models are evolving rapidly, but you can’t understand their progress by looking only at model size. The more important story is that foundation models are advancing along several dimensions at once. They are becoming broader in capability, more flexible in the types of data they process, more specialized for particular industries, more connected to tools and workflows, and increasingly subject to evaluation, safety, transparency, and governance requirements.
This shift is helping move artificial intelligence from isolated model performance toward complete, usable, and accountable AI systems.
From Narrow AI to Reusable Capabilities
Traditional machine-learning systems were commonly designed for narrowly defined tasks. A model might be developed specifically for fraud detection, image classification, demand forecasting, or another individual use case. Each new application often required its own training process, dataset, and model.
Foundation models introduced a different paradigm. Instead of building a separate model for every task, you can train a model broadly and then adapt it for many downstream applications.
The shift is therefore:
Narrow task-specific models → Reusable capabilities across multiple tasks
This reuse is a defining characteristic of the foundation-model approach. A single underlying model can potentially contribute to many applications when combined with appropriate prompting, retrieval, fine-tuning, tools, and domain-specific adaptation.
From Supervised to Self-Supervised Learning
Another major direction of progress has been the move toward self-supervised learning. Traditional supervised machine learning depends heavily on labeled datasets. Humans identify categories, outcomes, or examples and provide the labels the model learns from.
This approach can be effective, but labeling large datasets is expensive and time-consuming. Foundation models increasingly learn from very large quantities of unlabelled or weakly labeled data.
Through self-supervised learning, models identify patterns, relationships, structures, and representations in the data itself. This makes it possible to train models on much larger datasets and develop capabilities that can later transfer to many different tasks.
The shift can be summarized as:
Manually labeled examples → Learning transferable representations from broad data
From Unimodal to Multimodal AI
Early AI systems frequently worked within a single type of information. A language model processed text. A computer-vision system processed images. A speech model processed audio.
Foundation models are increasingly becoming multimodal. Multimodal systems can process and sometimes generate several types of information, including:
- Text
- Images
- Audio
- Video
- Code
- Structured data
The significance of multimodality goes beyond convenience. Many real-world problems are naturally multimodal.
A healthcare application, for example, may need to work with clinical notes, medical images, laboratory results, and structured patient information. A scientific system may combine papers, datasets, images, simulations, and code.
The move toward multimodality therefore brings AI systems closer to the way information actually appears in professional environments.
From General-Purpose to Domain-Specific Foundation Models
At first glance, the foundation-model paradigm may appear to be moving entirely toward increasingly general-purpose models. In practice, another important trend is occurring at the same time: specialization.
Domain-specific foundation models are being developed for areas such as:
- Healthcare
- Biology
- Scientific research
- Geospatial analysis
- Earth observation
- Industry
- Cloud operations
These systems can be trained or adapted around the terminology, data structures, workflows, and requirements of particular sectors.
The objective is not necessarily to replace general-purpose models. Instead, the emerging ecosystem is likely to contain both broad general models and more specialized models designed for specific environments.
This reflects an important principle:
Broader capability and deeper specialization can develop simultaneously.
From Centralized to Distributed Learning
Foundation models are also beginning to evolve beyond completely centralized training and data architectures. Many organizations work with information that cannot easily be pooled into one location. Healthcare records, financial information, industrial data, and government datasets may face privacy, regulatory, security, or commercial restrictions.
Federated approaches aim to enable learning across distributed data environments without transferring all sensitive information to a single centralized repository.
The general idea is:
Shared learning without necessarily sharing the underlying raw data
This direction is particularly relevant in sectors where privacy and data sovereignty are critical. Federated systems introduce their own technical and governance challenges, but they represent an important avenue for expanding AI while limiting unnecessary movement of sensitive information.
From Static Outputs to Action
Generative AI initially became widely visible through systems that produced outputs such as text, images, code, or summaries. The next stage increasingly involves connecting models to tools and workflows.
Foundation models can be integrated with:
- Search systems
- Databases
- APIs
- Software applications
- Calculators
- Enterprise systems
- Automation platforms
- Agents
This allows AI to move beyond generating a response.
A model may participate in a sequence such as:
Interpret → Plan → Retrieve → Use Tools → Generate → Review → Act
This transition is central to the emergence of agentic AI. However, moving from output generation to action also raises the stakes. An incorrect paragraph may be inconvenient. An incorrect automated action can have operational, financial, legal, or safety consequences.
As AI becomes more capable of acting within workflows, human oversight and governance become increasingly important.
From Model Scale to System Quality
Perhaps the most important shift in the field is the movement away from treating model size as the dominant measure of progress. For several years, much public discussion focused on larger parameter counts, larger datasets, and greater computational scale. Scale remains important, but it is only one part of AI performance.
A useful AI system also depends on:
Data quality
Poor or inappropriate data can limit even a highly capable model.
Adaptation
The model must be configured for the task, domain, and user.
Efficiency
Smaller or more efficient systems may be preferable when cost, latency, privacy, or local deployment matters.
Evaluation
Capabilities must be tested against realistic use cases rather than assumed from general benchmarks.
Safety
The system should operate within defined boundaries appropriate to the task’s consequences.
Transparency
Users and decision-makers need sufficient understanding of system behavior, limitations, and evidence.
Governance
Policies, responsibilities, controls, monitoring, and escalation mechanisms must accompany deployment.
Human oversight
People remain responsible for judgment, accountability, and consequential decisions.
The emerging direction is therefore:
Bigger model → Better system
But the two are not the same.
Foundation Models Are More Than Large Language Models
A common misconception is that the terms foundation model and large language model mean the same thing. They do not. Large language models are an important class of foundation models, but the foundation-model paradigm extends much further.
Foundation models are being developed across:
- Language
- Vision
- Speech
- Code
- Biology
- Scientific computing
- Geospatial analysis
- Earth observation
- Multimodal reasoning
These models differ in the information they process and the domains they operate in. What they share is the broader principle of training broadly enough to produce reusable capabilities that can be adapted for many downstream tasks.
Foundation models should therefore be understood as a broad family of reusable AI models, not simply another name for language models.
Seven Directions of Foundation-Model Progress
The main directions of change can be summarized as follows:
Narrow → General
Models are moving from single-task specialization toward reusable capabilities.
Supervised → Self-supervised
Large quantities of unlabelled data are being used to learn transferable representations.
Unimodal → Multimodal
AI is expanding from individual data types toward integrated processing across multiple modalities.
General-purpose → Specialized
Domain-specific foundation models are emerging for professional and scientific fields.
Centralized → Distributed
Federated and distributed approaches are being explored where sensitive data cannot easily be centralized.
Static outputs → Action
Models are increasingly connected to tools, agents, and workflows capable of executing tasks.
Model scale → System quality
The focus is broadening toward data, efficiency, adaptation, evaluation, safety, transparency, governance, and human oversight.
The Larger Implication
The next phase of foundation-model development will probably not be defined by a single technological breakthrough. Progress is occurring across the entire AI system. Models are becoming more general in some contexts and more specialized in others. They are becoming multimodal, connected to tools, embedded in workflows, and increasingly capable of supporting action.
At the same time, the requirements surrounding them are becoming more demanding. The question is no longer simply:
How capable is the model?
Increasingly, the more important questions are:
How well is the model adapted?
What information can it access?
What tools can it use?
How is it evaluated?
What authority does it have?
Who remains accountable?
That is the central direction of progress. The future of foundation models will be determined not merely by how large or intelligent they become, but by how effectively their capabilities become reliable, efficient, specialized, governed, and useful systems.