Microsoft Analysis Releases OptiMind: A 20B Parameter Mannequin that Turns Pure Language into Solver Prepared Optimization Fashions

Microsoft Analysis has launched OptiMind, an AI based mostly system that converts pure language descriptions of advanced choice issues into mathematical formulations that optimization solvers can execute. It targets a protracted standing bottleneck in operations analysis, the place translating enterprise intent into blended integer linear applications normally wants knowledgeable modelers and days of labor.

What OptiMind Is And What It Outputs?

OptiMind-SFT is a specialised 20B parameter Combination of Specialists mannequin within the gpt oss transformer household. About 3.6B parameters are energetic per token, so inference price is nearer to a mid sized mannequin whereas retaining excessive capability. The context size is 128,000 tokens, which permits lengthy specs and multi step reasoning traces inside a single request.

The mannequin takes a pure language description of an optimization downside as enter. The output is a mathematical formulation together with executable Python code that makes use of GurobiPy. The generated script defines choice variables, constraints, and goal, calls the Gurobi solver, and prints the optimum goal worth and selections.

OptiMind acts as a formulation layer between area specialists and customary MILP solvers. It doesn’t exchange the solver, it generates the MILP that the solver will optimize.

Structure, Coaching Setup, And Datasets

The bottom mannequin is openai/gpt-oss-20b, wonderful tuned into microsoft/OptiMind-SFT utilizing cleaned optimization datasets. The structure is a Combination of Specialists transformer, with routing that prompts a subset of specialists per token. The mannequin is launched below the MIT license.

Coaching makes use of 8 NVIDIA B200 GPUs, and inference and analysis within the reference setup use 8 NVIDIA H100 GPUs. Reported wonderful tuning time is about 8 hours. For normal use, the workforce recommends a minimum of 32 GB of GPU reminiscence on {hardware} comparable to A100, H100, or B200.

For supervised wonderful tuning, the analysis workforce assemble cleaned variations of OR Instruct and OptMATH Practice. For testing, they use knowledgeable validated and re-cleaned variations of IndustryOR, Mamo Advanced, and OptMATH. These benchmarks cowl arduous formulation duties the place present fashions usually attain solely 20 to 50 p.c accuracy on the unique noisy variations.

Class Based mostly Error Evaluation And Knowledge Cleansing

A key technical concept in OptiMind is to mix optimization experience with LLM coaching. The analysis workforce classifies issues from OR-Instruct and OptMATH into 53 seed courses, for instance set cowl, circulate store scheduling, or touring salesman downside.

For every class, they run the gpt-oss-20b-base mannequin on a pattern of issues and choose situations the place the mannequin output disagrees with the bottom fact. Optimization specialists examine this stuff, determine the recurring formulation errors, and write quick error descriptions and preventive hints. These hints describe right constraints, variable bounds, or modeling methods, comparable to the right Miller Tucker Zemlin constraints for TSP.

The analysis workforce then makes use of a semi-automated pipeline. They regenerate options with a bigger mannequin that’s prompted with the category particular hints, apply majority voting throughout samples to enhance answer high quality, and drop objects that stay inconsistent. Additionally they detect lacking parameters and ambiguous statements and regenerate downside descriptions when wanted. The result’s a cleaned coaching corpus that’s higher aligned with right mathematical formulations.

Inference Pipeline, Hints, And Check Time Scaling

At inference time, OptiMind behaves as a multi stage system, not only a single immediate. The default pipeline first classifies every check occasion into one of many 53 optimization courses used throughout error evaluation. It then augments the immediate with the error abstract and trace pairs related to that class.

The mannequin then generates a reasoning hint, the mathematical formulation, and the GurobiPy code. When extra compute is accessible, the system can apply self consistency with majority voting. It generates a number of candidate scripts, executes them, and selects the answer that seems most frequently inside set numerical tolerances.

A multi flip correction mode can be enabled. The system runs the generated code, captures solver logs or execution errors, feeds this suggestions again to the mannequin, and lets the mannequin revise the formulation and code for a number of rounds. This closes some modeling and coding errors at the price of increased latency.

Quantitative Beneficial properties On Optimization Benchmarks

On cleaned variations of IndustryOR, Mamo-Advanced, and OptMATH, the OptiMind framework considerably improves answer accuracy. The fine-tuned mannequin improves formulation accuracy by 20.7 p.c throughout a number of optimization benchmarks, with additional features when check time scaling methods comparable to self consistency and multi flip suggestions are utilized.

Throughout these benchmarks, OptiMind improves absolute accuracy over the gpt-oss-20b-base mannequin and outperforms different open supply fashions of comparable or bigger dimension. It reaches efficiency that’s aggressive with proprietary frontier fashions comparable to GPT-o4 mini and GPT-5 below the evaluation settings.

These outcomes depend on cautious cleansing of each coaching and check knowledge. The analysis workforce report that many obvious mannequin errors on authentic benchmarks truly got here from lacking knowledge, ambiguous descriptions, or incorrect reference options, and that re-cleaning can raise obvious accuracy for a set mannequin from about 40 to 60 p.c into the 70 to 90 p.c vary on the corrected units.

Key Takeaways

1. OptiMind is a 20B parameter Combination of Specialists transformer within the gpt-oss-family that takes pure language optimization issues as enter and outputs each a mathematical formulation and executable GurobiPy code, with about 3.6B parameters activated per token and a 128,000 token context size.
2. The mannequin is okay tuned from openai/gpt-oss-20b on cleaned optimization datasets comparable to OR-Instruct and OptMATH, and evaluated on knowledgeable validated benchmarks together with IndustryOR and Mamo Advanced, specializing in blended integer linear programming formulations.
3. OptiMind makes use of class based mostly error evaluation and knowledgeable written hints for 53 optimization courses, then applies these hints each in knowledge cleansing and at inference time, which systematically reduces frequent modeling errors in generated MILPs.
4. The framework improves formulation accuracy by 20.7 p.c throughout a number of optimization benchmarks in comparison with the bottom mannequin, and with check time scaling strategies comparable to self consistency and multi flip suggestions it reaches efficiency that’s aggressive with bigger proprietary techniques.
5. OptiMind-SFT is launched as microsoft/OptiMind-SFT on Hugging Face and as microsoft-optimind-sft in Azure AI Foundry, the place it may be served through SGLang as an OpenAI appropriate endpoint, enabling sensible integration into choice help pipelines for provide chains, manufacturing, logistics, and scheduling.

Take a look at the Model Weights and Technical details. Additionally, be happy to comply with us on Twitter and don’t neglect to affix our 100k+ ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well.