In a significant development for the AI infrastructure sector, General Compute, an AI inference cloud startup, has successfully secured a $400 million loan from Upper90, a tech investment firm. This deal is notable as it marks the first instance of inference-specific chips being used as collateral for financing. These chips are designed to run pre-trained AI models efficiently, contrasting with the more expensive chips typically used for model training. The financing reflects a growing trend in the market, where there is increasing concern over the costs associated with AI tools and tokens. As a result, investors are shifting their focus towards infrastructure that can operate open-source models at a lower cost compared to the latest large language models (LLMs) developed by leading tech labs.
Founded by CEO Finn Puklowski, General Compute had previously raised a $15 million seed round in May to establish an inference neocloud utilizing silicon from SambaNova, a chipmaker backed by Intel. Neoclouds are specifically designed for AI workloads, distinguishing them from the general-purpose infrastructure provided by traditional hyperscalers like Amazon Web Services (AWS) or Microsoft Azure. The company’s SN50 chips are engineered for inference tasks, boasting power efficiency and eliminating the need for costly water-cooling systems, which allows for quicker deployment across a wider range of data centers. General Compute claims that these new chips can deliver inference speeds that are 16 times faster than those offered by GPU-based cloud services.
However, the startup faces challenges in acquiring a sufficient quantity of these chips, particularly as a new entrant in the market. Upper90's co-founder and CEO, Billy Libby, who previously worked as a quantitative trader at Goldman Sachs, has a history of financing advanced chip purchases. In 2021, his firm financed GPU acquisitions for Crusoe, an energy-focused data center startup, which was believed to be the first loan secured against the value of advanced chips. At that time, traditional lenders were hesitant to engage in such deals due to the perceived risks and uncertainties surrounding GPU depreciation.
As the market has evolved, companies like CoreWeave have successfully turned chip-backed loans into a viable business model, leading to a successful IPO. Libby noted that when they financed Nvidia GPUs, the market was inefficient, allowing them to capitalize on the risk. With GPUs now more widely understood and possibly over-purchased, Upper90 is pivoting towards companies like General Compute to capitalize on the next wave of AI advancements. Libby emphasized the importance of open-source models and the necessity for businesses to focus on inference rather than requiring supercomputers. This shift in focus is gaining traction, as companies providing access to open models, such as OpenRouter and Fireworks, are raising significant funding at high valuations. New models like Kimi’s K3 are also emerging, demonstrating competitive performance against the latest offerings from companies like Anthropic and OpenAI. Additionally, TensorWave, another player in the AI infrastructure space, is pursuing a partnership with AMD, indicating a growing trend of alternatives to Nvidia. As the landscape evolves, compute providers that are not tied to Nvidia may find themselves in a favorable position to offer cost-effective inference solutions.
