A startup named Gimlet Labs has secured an $80 million Series A funding round led by Menlo Ventures to tackle one of the most problematic aspects of AI: Its slowdown during inference. The company is co-founded by Zain Asgar, an adjunct professor at Stanford and the founder of several companies; Michelle Nguyen; Omid Azizi; and Natalie Serrino.
The startup has built something it calls a multi-silicon inference cloud system. The software enables AI workloads to execute concurrently on various types of chips, rather than a single chip. The platform is able to illustrate what each step of the process requires, dividing it between CPUs, GPUs and high-memory machines.
AI agents typically carry out numerous steps, and each step requires distinct resources, according to the company. Some processes require more computing power, some more memory, and some are conditional on network speed. Since no single chip can do everything well, the company developed software that can work with whatever hardware is available to it.
The idea is arriving as companies are spending heavily on data centers, investors say. Some estimates say worldwide spending could hit almost $7 trillion by 2030. Gimlet Labs says that many systems today are only 15 to 30 percent utilized, wasting expensive hardware. Its software, the company says, can dramatically increase efficiency of AI workloads by distributing them across many machines.
Gimlet Labs says the system can make inference run between three and 10 times faster while adding no additional cost or power usage. It can even partition sections of a model so that each segment runs on the chip best suited for it. The company is currently partnering with leading chip makers such as NVIDIA, AMD, Intel, ARM, Cerebras and d-Matrix.
The product is available as software or via an application programing interface on the company’s hosting cloud service. It is primarily aimed at big A.I. labs and data-center operators, not small developers.
The startup publicly launched in October and said it had already achieved revenue in the eight-figure range. The company also said its customer count has more than doubled in recent months with, among others, a large AI model developer and an unnamed big cloud supplier.
The founders had worked together previously at Pixie, a startup that developed tools for monitoring Kubernetes systems. Pixie launched then was acquired by New Relic in 2020.
Including this round and prior funding at the seed stage, Gimlet Labs has raised in total $92 million. Backers include Factory, Eclipse Ventures, Prosperity7, Triatomic and a number of individual technology industry backers. The company now has around 30 workers.
Business Outstanders brings you sharp insights on tech, business, entrepreneurship, law, crypto, and more. We uncover what’s next. Stay updated, sign up for our newsletter and be part of the future!
