Run:AI co-founders, Omri Geller & Ronen Dar.
Some appliances and tools consume a larger than expected amount of energy. In the home, HVAC is often cited as the most energy-hungry drain that consumers experience at close quarters. In the world of software, there are superpower energy drainers too. Big data analytics software engines are called engines for a reason, they consume a lot of processing, data storage and memory power.
AI is also a hungry greedy beast.
Running Artificial Intelligence (AI) processes isn’t a lightweight computing operation. The so-called ‘data science workloads’ that AI solutions need to process require more than your average laptop and, typically, more than your average company server stack in many cases.
AI works a long day shift
Data science workloads often need greedy access to multiple computing resources such as Graphical Processing Units (GPUs are ultra-fast processors originally built for high-end graphics, but now used elsewhere while still retaining their original name). These AI-tasked workloads need access to GPUs for what is usually hours on end if they are to perform their intelligence intelligently.
The trouble is, access to GPUs in even the most expansive IT stacks is often subject to bottlenecks. The problem is compounded when data scientists are ‘statically’ allocated a few GPUs each — and those expensive hardware resources end up sitting idle when not used. The end result is that data scientists trying to create our next-generation of AI systems are faced with three possible scenarios:
● AI has trouble getting enough GPU capacity.
● AI has too much GPU capacity and so inevitably wastes some of it.
● AI has the right kind of capacity but is limited in terms of when it can use it, plus it has poor visibility into when the resources are available in the first place.
If we accept the above, then we can suggest that getting deep learning and AI to work properly means adapting our approach to computing. Tel Aviv based Run:AI thinks it has some (or indeed all) of the solution to this juggling conundrum via its deep learning virtualization platform.
This software is designed to support Kubernetes-based infrastructures, a technology that handles the orchestration & scaling of cloud computing resources. Run:AI enables IT departments to set up and manage a critical AI infrastructure for data science teams with control and visibility, while maximizing hardware utilization and development velocity.
Stretching the compute pool
Instead of statically assigning GPUs to data scientists, Run:AI creates a pool of GPU resources and will elastically ‘stretch’ a workload to run over multiple GPUs if they’re available. Important jobs can be given guaranteed quotas (which they can also exceed) and Run:AI’s software will automatically scale the workloads to the available hardware, based on defined priorities.
To simplify workflows, Run:AI’s virtualization platform plugs into Kubernetes with a single line of software code. The platform’s visibility tools enable organizations to understand how their GPU resources are being used by their data science teams, helping with infrastructure scaling and identifying bottlenecks.
“Deep learning is creating whole new industries and transforming old ones,” said Omri Geller, co-founder and CEO of Run:AI. “Now it’s time for computing to adapt to deep learning. Run:AI gives both IT and data scientists what they need to get the most out of their GPUs, so they can innovate and iterate their models faster to produce the advanced AI of the future”.
Global CTO of software integration and analytics company TIBCO Nelson Petracek works in a much bigger corporate space to the comparatively start-up world that Run:AI operates in. Agreeing that optimizing the use of GPUs for AI is interesting, Petracek suggests that this is somewhat like what VMware did for CPUs (central processing units), but with AI as the ‘consuming client application type’… and with virtualization approaches that are better suited to AI workloads.
“One differentiator here is Run:AI’s integration with Kubernetes and its proprietary scheduling/virtualization and configuration approaches, as other vendors obviously offer ‘GPU in the cloud’ capabilities. Lambda is one such company, as are the obvious ones like AWS. It appears that Run:AI can utilize an organization’s bare metal/machines (I mean private cloud) as well, so not just cloud infrastructure or Infrastructure-as-a-Service (IaaS) in this case,” said TIBCO’s Petracek.
Car sharing on the IT superhighway
Clearly then, you don’t just fire up an AI engine and expect the gas (petrol) to flow freely unless you’ve done some preparation for pit stops and services.
As we work increasingly remotely and across many more disparate disconnected (or only occasionally connected) digital channels, the need to manage the traffic running in the virtualization layer becomes more pressing. With AI engines on the road (software- and highway-based double pun deliberately intended), we need to be able to car share (pun still intended) across what resources we do have.
Run:AI’s work here essentially concentrates on abstracting workloads from the underlying computing infrastructure. This way, no single AI deep learning workload needs to be too greedy as the dynamic provisioning of resources should mean there’s a seat for everybody to travel.