How FactSet cut model training and deployment from days to hours with Databricks AI Runtime

來源原標題

How FactSet cut model training and deployment from days to hours with Databricks AI Runtime

機構

Databricks

原始標籤

Company · Customers

原始連結

https://www.databricks.com/blog/how-factset-cut-model-training-and-deployment-days-hours-databricks-ai-runtime

摘要原文

FactSet’s Text-to-Formula (TTF) system has a deceptively difficult job: take a natural-language request from an investment professional and turn it into the correct FactSet Query Language (FQL) formula. FactSet initially built TTF around retrieval-augmented generation (RAG), using retrieval to find relevant examples and information for a given request. But as consumption grew, queries became more specialized and formulas more complex; the approach hit a quality ceiling. FactSet began incorporating fine-tuned open models into its Compound AI architecture, allowing the team to train models on the structure and patterns of FQL rather than relying primarily on retrieval at inference time. In its earlier work on TTF, FactSet found that incorporating fine-tuned models reduced end-to-end latency by about 60%. As the system evolved, so did the role of fine-tuning. FactSet accumulated more training data and began updating its models regularly, turning what had been an experimental workflow into a recurring part of developing TTF. Today, the team fine-tunes models roughly every two to three weeks. Training every few weeks changes the infrastructure requirements. Preparing an environment and configuring GPU compute may be manageable for an occasional experiment. Repeating that work every time new data or a new model is ready to test adds overhead directly to the development cycle. Model requirements can change from one experiment to the next as well. FactSet has worked with Llama-based models and continues to evaluate newer open models. Different models, prompt lengths, and output characteristics can require different compute configurations, so tying the training environment to a fixed GPU setup would make it harder to adapt as the team experiments. FactSet uses Databricks AI Runtime to run these fine-tuning workloads on serverless GPU infrastructure. Engineers can choose compute based on the requirements of a particular training run without first provisioning and configuring the underlying GPU environment. TTF’s training workflow also sits alongside data, experimentation, model management, and governance already running on Databricks. FactSet uses MLflow to track model development and experiments, while its underlying data remains governed within its existing Lakehouse environment. Running fine-tuning with AI Runtime keeps GPU training inside that workflow. Engineers can work with governed data, train a new model, track the resulting artifacts, and move toward deployment without maintaining a separate GPU stack for fine-tuning. For FactSet, that reduced the time required to set up, train, and deploy a custom TTF model from days to hours. The ability to change models has been part of TTF’s evolution from the beginning. Advanced RAG, especially reliance solely on externally hosted models, becomes a bottleneck on consistent accuracy and latency as the consumption grows and more complex formulas. RAG worked until the system encountered more complex formulas. Fine-tuned open models gave FactSet another way to improve quality and latency. Now, as newer models become available, the team can evaluate them against the workload rather than treating its current model choice as permanent. TTF itself is becoming more specialized. As FactSet accumulates more data, the team is exploring additional fine-tuned models for tasks such as query decomposition, component extraction, and library identification. That creates more opportunities to use the model best suited to each part of the workflow, but also makes fast, repeatable training more important. A model change is much easier to test when it does not require rebuilding the training environment around it. With GPU compute available on demand through AI Runtime, the engineering work shifts toward the questions that actually affect TTF: which model to use, how to train it, and whether the resulting version improves the application. Additionally, the fine-tuning pipeline was built on Databricks Lakeflow Jobs and Pipelines as a plug-and-play, reusable framework — designed to be easily adopted across different TTF tasks and other teams with minimal setup effort, providing a standardized foundation that reduces onboarding friction and accelerates integration. Open models are moving quickly. The model that offers the right balance of quality, latency, and cost today may not be the one a team wants six months from now. Preserving model choice therefore requires more than the ability to download a new set of weights. The surrounding training and deployment workflow has to keep up. FactSet’s experience with TTF shows what that looks like in production. The team can continue testing new open models as its data and application evolve, while AI Runtime provides serverless GPU infrastructure within its existing Databricks workflows. Instead of spending days setting up, training, and deploying each iteration, FactSet can complete that cycle in hours. For a system that is retrained every few weeks, that difference compounds. New data can move into a new model sooner, and engineers can spend more of each iteration improving TTF rather than preparing the infrastructure required to train it. Subscribe to our blog and get the latest posts delivered to your inbox.