When it comes to AI, many enterprises seem to be stuck in the prototype phase. Teams can be constrained by GPU capacity and complex and opaque model workflows; or, they don’t know when enough training and customization is enough, and whether they’ve reached the highest level of performance and accuracy (or not). This is because they’re doing fine-tuning wrong, according to RapidFire AI. The company says it can get enterprises over that hump with its “rapid experimentation” engine. Now in open-source release, the platform is designed to speed up and simplify large language models (LLMs) customization, fine‑tuning and post‑training. Hyper-parallel processing is at its core; instead of just one configuration, users can analyze 20 or more all at once, resulting in a 20X higher experimentation throughput, the company claims. “That ability to see multiple executions on representative samples is the underlying key to our performance,” RapidFire AI CEO and co-founder Jack Norris told VentureBeat in an exclusive interview.

Why hyper-parallelization leads to faster results

With RapidFire AI, users can compare potentially dozens of configurations all at once on a single or multiple machines — various base model architectures, training hyperparameters, adapter specifics, data preprocessing and reward functions. The platform processes data in “chunks,” switching adapters and models to reallocate and maximize GPU use. Users have a live metrics stream on an MLflow dashboard and interactive control (IC) ops; this allows them to track and visualize all metrics and metadata and warm-start, stop, resume, clone, modify or prune configurations in real time. RapidFire isn't just spinning up additional resources; it’s using the same resources — so if users have just one, two or four GPUs, they can count 8, 16 or 32 variations in parallel. “You get this emulation of a cluster even with the same GPU, and that incentivizes exploration,” explained Arun Kumar, RapidFire CTO and cofounder. “We are bringing this philosophy of abstracting away lower-level systems execution details from the user and letting them focus on application knowledge, metrics and knobs.” The platform is Hugging Face native, works with PyTorch and transformers and supports various quantization and fine-tuning methods (such as parameter efficient fine tuning, or PEFT, and low-rank adaptation, or LoRA) as well as supervised fine tuning, direct preference optimization and group relative policy optimization. Data scientists and AI engineers don’t need to be concerned with what's on the back end, how to shard data, swap models or maximize GPU utilization, Norris explained. This means junior engineers can be as effective as senior ones because they can see what's working, quickly adjust, narrow down and eliminate less promising derivatives. “This basically democratizes the approach,” he said. He emphasized that organizations should be looking to compete just on model complexity or performance. Rather, “it's the ability to properly leverage data, to fine tune, take advantage of that data. That will ultimately be what they rest their competitive advantage on.” RapidFire AI is released under the Apache 2.0 license, meaning it can be downloaded, modified and re-licensed by anyone. Its open‑source Python package, documentation and guides are available now. Open source is critical to the company’s philosophy; as Kumar put it, open source has “revolutionized the world” over the last 20 years. “There's good business value in open source, but also the transparency and the ability of the community to contribute, stand on each other's shoulders rather than stepping on each other's toes, is fundamentally valuable,” he said.

Projects sped up 2-3X Using RapidFire AI, the Data Science Alliance has sped up projects 2-3X, according to Ryan Lopez, director of operations and projects. Normally, multiple iterations would take a week; that timeline has been shortened to two days or less. The nonprofit, which is focused on community and social projects, has been experimenting with computer vision and object detection analysis, Lopez explained. With RapidFire, they can simultaneously process images and video to see how different vision models perform. “What RapidFire has allowed us to do is essentially iterate at hyper speed,” said Lopez. It gives his team a “really structured, evidence-driven way to do exploratory modeling work.” RapidFire’s hyperparallelism, automated model selection, adaptive GPU utilization and continual improvement capabilities give customers a “massive increase” in speed and cost-optimization, noted John Santaferraro, CEO of Ferraro Consulting. This is compared to in-house hand coding or software tools that focus just on the software engineering aspect of model acceleration. “Hyperparallelism accelerates the AI enablement of model selection, the ability to identify high-performing models and shut down low-performing models,” he said, all while minimizing runtime overheads. RapidFire competitors include specialized software vendors, GPU infrastructure companies and MLOps and AIOps vendors such as Nvidia or Domino, Santaferraro noted. However, its acceleration at both the model and GPU level is key to its differentiation. RapidFire is “unique in the way it has AI enabled the model training, testing, fine-tuning and continual improvement process.”

Iterating to innovate RapidFire’s platform has supported a whole spectrum of language use cases, including chat bots for Q&As, search on internal documentation and financial analytics, said Kumar. Design partners and potential customers have deployed up to three dozen use cases using in some cases 40 or even 10 billion parameter models. “The models are more right-sized for their application and the inference volume, rather than using a trillion parameter model for everything,” he said. “That reins in their total cost of ownership.” The three biggest roadblocks between AI experimentation and deployment are data readiness, accuracy and trust, Santaferraro noted. AI requires traditional data quality for structured data and “veracity” for unstructured data. Hallucinations are a challenge, particularly with public models, because they occur in a black box. Trust requires rapid testing in a time-consuming fashion. Drift, or when a trained model changes based on new inputs and introduces unforeseen risks, is another concern. Enterprises must be able to reduce the risk of inaccurate answers, hidden threats and runaway activity; faster model cycles can shrink the gap between unsafe research prototype and a deployable system aligned with corporate governance and goals, said Santaferraro. “Unfortunately, most enterprises are spending large amounts of money, using massive resources, to plow through these models and eliminate risk,” he said. “There is no other quick way forward, except, of course, speeding the iteration process.” Leading organizations should focus their compute on the aspects of public LLMs they find most useful, he advised, then add their IP, knowledge base and unique point of view to develop private small language models (SLMs). When looking at tools like RapidFire, it is critical to consider organizational and personnel readiness, as well as infrastructure fit and investment, said Santaferraro. They must be able to support iteration acceleration and fit tools into their existing infrastructure in a way that streamlines process feeding and outputs. Ultimately, how quickly a company can innovate with AI correlates to how fast they can improve business processes and support new products and services, he said, noting: “The speed of iteration is the key to all innovation.”

Read the full article on VentureBeat