More organizations are beginning to operationalize generative artificial intelligence (AI). Along with this, many of them will be looking to managed service providers (MSPs) to help them optimally deploy the inference engines that are needed to analyze data in production environments.
Training a generative AI model and deploying the inference engines that enable it to run are two very distinct tasks. The former requires a data science team that understands how to create a generative AI model. However, you can treat it just like any other software artifact that needs deployment.
The rise of GPUs and AI accelerators
Today, most organizations use graphical processor units (GPUs) with parallel processing capabilities to train and deploy AI models. However, due to the limited supply and high cost of GPUs, many organizations deploy generative AI models on traditional CPUs. There is also an emerging class of AI accelerators, also known as xPUs, that can run an inference engine.
According to a recent report, GPUs in 2023 accounted for 74 percent of chipsets used in AI applications within data centers. Forecasts expect them to grow on a compound annual growth basis (CAGR) by 30 percent over the next five years. This will cause them to reach $102 billion by 2028. In comparison, CPUs held a 20 percent market share in 2023 and will see a 28 percent CAGR attributable to AI application usage to reach $26 billion in 2028. The xPU market, meanwhile, had an estimated three percent of the market share in 2023. However, it will see a 31 percent CAGR over the next five years to reach $3.7B in 2028.
Evolving AI ecosystems
As generative AI evolves, organizations of all sizes will seek MSPs to guide them in choosing the right model and processor class for deployment. Those models will span everything from customized instances of foundational models provided by OpenAI, Microsoft, Google, Amazon Web Services, and Hugging Face, to smaller models that have been specifically trained to optimize tasks in a specific domain. In all probability, organizations will deploy many more small models at the network edge than they will deploy customized foundational models in large data centers or public clouds.
The shortage of data science expertise for building and deploying AI models is shifting the deployment responsibility to IT operations. An MSP often manages these in many organizations. In fact, there will come a day soon when organizations decide which MSP to rely on based on the amount of generative AI expertise they can bring to bear.
Navigating the generative AI maze
It may take organizations up to 12 months to determine how they specifically want to operationalize generative AI. While invoking a generative AI service might seem easier, the cost of using a service priced by inputs and outputs quickly surpasses the expense of customizing an open-source AI model.
Also, not every model will necessarily require a GPU to run it when less expensive alternative processors are available. The tradeoff will be determining that alternative processor by surface results that are not quite as accurate as a GPU.
One certainty is that any MSP helping organizations navigate the maze of generative AI options will become indispensable for many years to come.
Photo: SuPatMaN / Shutterstock
