Recently, when driving past a data center being constructed near Port Washington, Wisconsin, I was stunned. The size of this facility is breathtaking, designed for 2.5 million square feet of data center space across 672 acres (with the potential to expand to 1,900 total acres). Upon completion, it calls for an estimated 50,000 to 150,000 AI servers drawing an estimated 900 MW to 1+ GW of IT load and power demand. And this single project is just one of dozens going up around the country.
Presently, investors are enthralled by the enormity of the aggregate AI Infrastructure build. Huge companies are making extensive investments in infrastructure, components, and software to stand up a complex inferencing ecosystem. Everything is supersized.
As the example above illustrates, the footprints themselves are gigantic, covering vast tracts of real estate and requiring substantial energy capabilities to power them. Inside will sit thousands of server racks, each weighing as much as a small pickup truck or a male walrus. Even the silicon (once known as “MICROchips”) has subscribed to the bigger-is-better philosophy, with a high-profile, newly public company building a single system-on-chip about the size of an LP vinyl record.
All these assets, and the trillions of dollars spent on them, are being assembled to power large language models (LLMs). The operative word is large.
We marvel, of course, at this entire situation. But this is not where the story ends. As is often the case in technology, good things come in small packages. Another frontier for investors to consider is on the horizon, beyond the bulldozers: Edge AI.
In simple terms, Edge AI places workloads as close to the “edge” where data is created and actions are executed. It is implemented via any unique device built to perform a specific function but upfitted with onboard compute capacity. In practice, these devices ingest information from their operating environment, process it using artificial intelligence, and execute the best actions in real time.
Today’s massive data center buildout is largely about constructing the “brain,” the centralized intelligence responsible for training increasingly sophisticated AI models. The greater the computing power devoted to this infrastructure, the more advanced the intelligence frontier becomes.
While much of today’s AI investment is focused on centralized data centers and cloud infrastructure, Edge AI pushes intelligence outward onto dynamic endpoint devices operating at the edge of the network. The full potential of AI investment will only be realized when that centralized intelligence works in concert with highly capable Edge AI systems. Together, they enable a continuous, 360-degree machine learning ecosystem, where intelligence is not only trained centrally, but also deployed locally to deliver real-time insights, autonomous decision-making, and instantaneous action.
Edge AI: Coming Soon to a Device Near You
The best place to start this discussion is with the applications it will enable. We can identify many, but like all amazing technological enhancements, the ultimate extensions are unknowable. In 2000, when the internet was being imagined, was it possible to predict that 25 years later, 27% of married couples would meet on dating apps?
AI is enabling Edge 2.0 applications beyond today’s traditional robotics and Internet of Things (IoT). We introduce a few here, aware that opportunities without boundaries are infinite.
Robotics
Much of today’s AI discussion centers on software agents that help knowledge workers and consumers automate repetitive digital workflows. These applications are naturally well-suited for cloud-based computing resources. The automation of physical tasks will explosively expand the scope. Robots of all shapes, sizes, and capabilities are being developed and deployed in dynamic real-world operating environments, such as factory floors, warehouses, and logistics networks.
While industrial robots have long been capable of repetitive functions, such as material handling, welding, and assembly-line execution, the next generation aims to navigate imperfect operating environments that require real-time adaptation and decision-making. This opportunity is substantial given the more than half a billion manufacturing workers globally who today must augment physical processes with continuous human insights and on-the-fly adjustments to get the job done.
This ambition requires on-board intelligence.
Autonomous Vehicles
A thrill for some, driving is a boring task for most. Music and podcasts are welcome distractions, but at the end of the day, driving is a colossal waste of life’s most precious resource: time. Handing this tedious yet high-stakes task over to the car itself is a holy grail.
In addition to the benefits afforded to the commuter, flawless autonomous vehicles would make our roadways much safer, ridding highways of the imperfections humans bring to the driver’s seat, including tiredness, distraction, rage, and chemical impairment. There are also network benefits to be gained from an elegant orchestration of cars optimally traversing shared, crowded roads.
This ambition requires on-board intelligence.
Wearables / Health Care
As a parent of a Type 1 diabetic, I have witnessed firsthand the profound impact localized technology can have on personal health management. Continuous glucose monitors—wearable devices that provide real-time insight into blood sugar levels before symptoms emerge—have transformed daily life for patients living with this chronic condition. Early identification of physiological changes enables wearers to proactively manage, or even prevent, serious health episodes.
This is only the beginning. Intelligent, untethered wearable devices will enable continuous monitoring and real-time analysis.
This is only the beginning. Intelligent, untethered wearable devices will enable continuous monitoring and real-time analysis, enabling earlier detection/diagnosis, more personalized care, physician oversight, and variable therapeutic administration (e.g., dosing, muscle stimulation), resulting in a healthier population.
This ambition requires on-board intelligence.
Military Applications
Syncing Central Intelligence with troop engagement has been at the heart of military hierarchies from the beginning. How do assets on the battlefield execute in concert with the ambitions of central command? Complicating the equation is a field of engagement that is consistently changing. And above all, the #1 job is to keep soldiers safe.
As Major Michael Zequeira noted in Military Review, AI can serve as a “crucial combat multiplier” by “unburdening planning staffs to provide more focused efforts on the uniquely human aspects of operational planning,” while avoiding unnecessary civilian casualties or fratricide.
Assets include drones and other unmanned aircraft, AI-enabled weapons systems, missile interceptors, and C4ISR, each of which will employ a hybrid approach that leverages the broader network, supplemented by on-device processing, to make crucial split-second decisions.
This ambition requires on-board intelligence.
Space
Still in the early days, space has captured the imagination of futurists and innovators and is rapidly garnering investor attention as well. Early applications surround communications, Earth observation, and defense, but visionaries are pursuing resource harvesting, offshoring, and even colonization.
Around 15,000 satellites are in orbit today, but forecasts have projected 100,000+ by 2030. It is ironic that, given the extreme physical distance of assets from the cloud, routing all data to Earth for analysis is inefficient, requiring payment of what some have described as a lofty “round-trip tax.” Therefore, real-time processing aboard spacecraft, aka edge computing, must play a role.
This ambition requires on-board intelligence.
The unique requirements of this new computing era are already forcing architectural change and creating opportunities today for investors imagining what comes next.
This is a small sampling of the multitude of prosperity-rich domains for Edge AI to unlock. At this stage of the revolution, it is nearly impossible to predict the winning businesses or the winning business models. Though the market remains young and the road ahead uncertain, the unique requirements of this new computing era are already forcing architectural change and creating opportunities today for investors imagining what comes next.
How To Facilitate On-Board Intelligence
A consistent characteristic of most Edge AI devices is that they are small and untethered. In contrast to today’s Edge 1.0 (IoT) use cases, which are often stationary and connected to fixed infrastructure, tomorrow’s intelligent machines will increasingly operate remotely, autonomously, and in dynamic real-world environments.
Earlier generations of robots, for example, could rely on abundant grid power, high-bandwidth connectivity, and physically robust rigs—think of the massive robotic systems found on automotive production lines. The next wave of devices, by contrast, will often be mobile, compact, difficult to access, and deployed in harsh or unpredictable outdoor settings.
Such devices will require specialized design and engineering, incorporating a peculiar blend of components and architectures. Highly performant technical requirements, combined with physical limitations, create opportunities for technical innovation. We have identified several features critical for Edge AI.
Form Factor
The tech industry is masterful at shrinking capabilities into smaller form factors. In health care, the physical constraints of the human body make this trend even more essential. However, adding robust processing capabilities to small devices requires very efficient computing architectures. Not only will the processor need to be incredibly efficient, but it will also have to source the requisite data where there is limited room for memory.
This will require novel on-board storage and memory interfaces to minimize latency, technologies fundamentally distinct from the high-bandwidth memory systems currently absorbing billions of dollars in hyperscale AI data centers.
Power Efficiency
“Untethered device” means that persistent power access is unavailable. Everything about these Edge devices must be low power, a refinement that is far from trivial. Making an asset low-power is not simply an incremental adjustment; the device and its components must be purposely designed. In addition, mobile or remote devices will be battery-powered.
Innovators are meeting the call to produce long-duration batteries that strike an optimal tradeoff between power consumption and energy efficiency, all within footprint limitations
Environmental Robustness
Data centers and factories are controllable environments, but the Edge AI world we envision will be pervasive. Devices will exist everywhere—exposed to rain, extreme cold, intense heat, radiation in space, high-speed motion, and even the complexities of the human body—all while functioning across vastly different ambient conditions and operating intensities. There will be no one-size-fits-all solutions, as each application will mandate its own delicate and purpose-built sensitivities.\
Security
Assets that contain onboard intelligence and can trigger an action pose a significant security risk. Reminding me of the preposterous scene from The Naked Gun (1988) when Hall-of-Fame outfielder Reggie Jackson was robotically reprogrammed to become an assassin during a major league baseball game. Thank goodness Sergeant Frank Drebin was on the scene that day.
But in all seriousness, any autonomous asset with its own intelligence that lacks oversight or an override feature is a potentially dangerous one. Security will be paramount to prevent catastrophes resulting from either technology failures or malfeasance by bad actors. In addition, some collected data will be highly classified or personally sensitive and need to be protected to comply with privacy regulations.
Sensing
A critical ingredient in enabling intelligent action is data. Much like how humans rely on their senses to interpret the world around them, Edge devices use embedded sensing technologies to perceive and understand their environments. These devices collect raw data, convert it into actionable information, and feed it into an intelligence engine capable of making decisions locally and in real time. Examples include accelerometers, thermometers, current sensors, cameras, and lidar/radar.
Importantly, machines equipped with these sensing capabilities are often able to detect patterns and process information beyond the limits of human perception or at speeds humans cannot achieve. While sensing technologies have long been a fragmented field populated by niche innovators, the rise of Edge AI is rapidly expanding the addressable market.
Inferencing
Finally, once compute processing is in place and data is gathered, the AI “magic“ begins: inferencing. This is the stage where AI models process language, commands, and contextual data in real time to generate decisions and actions. Given the power, latency, and connectivity constraints inherent to edge environments, Small Language Models (SLMs) rather than massive LLMs are often best suited for the task.
While less capable than frontier-scale models, SLMs using model compilers are optimized for efficiency, speed, cost, and autonomy, enabling highly effective performance for targeted, real-world applications at the edge.
Investment opportunities are unfolding before our eyes. Companies old and new alike are being called upon to make this new distributed computing vision a reality.
Investment opportunities are unfolding before our eyes. Companies old and new alike are being called upon to make this new distributed computing vision a reality.
As the graphic here illustrates, novel capabilities in new and improved applications raise the opportunity set for many vendors. In some instances, the inflection has begun, but for many, the game has yet to kick off. Regardless, this is a large wave not yet receiving the full investment attention it deserves.
Above, we discussed some of the challenges associated with incorporating Edge AI into today’s technology ecosystem. Yet viewed from another angle, many of these “challenges” are in fact advantages in disguise. Not only will Edge AI considerably enhance the functionality and utility of both existing and newly created applications, but its distributed architecture may also alleviate several of the most contentious issues emerging alongside the rapid adoption of artificial intelligence.
As one industry observer aptly noted: “As the digital world evolves into the resource-hungry domain of AI, it finds itself bottlenecked by the physical reality of the grid. The world of bits has finally collided with the limits of the world of atoms.”
Centralized AI infrastructure has created immense demands on power generation, transmission networks, cooling systems, communications infrastructure, and physical land use. By distributing compute intelligence outward toward the edge of the network, many of these constraints can be partially mitigated. Edge AI has the potential to reduce latency, lower bandwidth requirements, improve resiliency, enhance data privacy, and ease concerns surrounding energy availability, affordability, and NIMBYism associated with hyperscale data center expansion.
More importantly, Edge AI represents the natural evolution of artificial intelligence from centralized cognition toward distributed autonomy. The cloud may remain the “brain” responsible for training increasingly sophisticated models, but the edge becomes the nervous system, where sensing, inferencing, and real-world action occur instantaneously. In that sense, the next phase of the AI revolution may not simply be about building larger models or larger data centers, but rather about embedding intelligence directly into the physical world itself.
This communication contains the personal opinions, as of the date set forth herein, about the securities, investments and/or economic subjects discussed by Mr. Bies. No part of Mr. Bies’s compensation was, is or will be related to any specific views contained in these materials. This communication is intended for information purposes only and does not recommend or solicit the purchase or sale of specific securities or investment services. Readers should not infer or assume that any securities, sectors or markets described were or will be profitable or are appropriate to meet the objectives, situation or needs of a particular individual or family, as the implementation of any financial strategy should only be made after consultation with your attorney, tax advisor and investment advisor. All material presented is compiled from sources believed to be reliable, but accuracy or completeness cannot be guaranteed. © Silvercrest Asset Management Group LLC
