Can LLMs predict fluid simulations?

It has been quite a while since I was last researching whether it was possible to speed up fluid simulations via ML models. More specifically: Can I replace the expensive pressure projection step of a CFD with a neural network, and produce a result that looks very similar? The plan was to couple the model and the solver. The coupling considers data from the solver as features for the model, and the model’s prediction serves as input to the solver again. The imagination of creating an eval around this idea came to me when I was recently watching a podcast between Florian (@xeophone) and Yacine, stuff Florian was doing at Prime Intellect. The talk was very enlightening, and I was wondering if I could create a private eval that I can personally use to judge the LLMs. The question was kind of simple: Can an LLM understand the data and create a fluid model that predicts the pressure projection step? I think I can provide the training data, and it should be able to create a model, whose performance can be measured as a proxy for measuring its current capabilities. This could serve as a personal baseline for me to compare between different models, and I think it is a non-trivial and open problem that I can waste a few tokens at.

I went with Mantaflow for generating the data as it has quite a nice API to get everything out that is required for training. I generated the following features from multiple different scenes:

• Velocity Divergence \(\nabla . u_t*\)
• Occupancy which says which part of the grid is fluid and which is solid geometry.
• Intermediate Velocity which is the pre-projection velocity.
• Previous Pressure from the previous projection step. It is initialized with zero and updated every step of the simulation.
• Advected Density or Smoke field which is copied before pressure projection.

I wrote up an initial_prompt.md that specified that the LLM does not need to use every available feature for the fluid model. Additionally, they were not allowed to use internet access, which was specified in the prompt and which also meant that I had to look at the tool calls and make sure to invalidate runs that were doing something fishy. This was done because, I did not want to contaminate its context with things I have no control over; I am pretty sure that there is already some research on how it is best done, which honestly might have already popped up during pre-training, but I did not want the internet/latest research to influence its decisions. They were also not allowed to look at the solutions of other runs, which is pretty self-explanatory, as I did not want any leakage. I had recently subscribed to ChatGPT Plus and wanted to burn my tokens, so I decided to eval GPT5.5 and GPT5.6-sol. I ran the experiments multiple times for each model, and no additional prompts except for the initial prompt were provided.

I also created, in addition to the training and validation dataset, private evaluation scenes in which I evaluated the fluid model’s performance by replacing Mantaflow’s numerical pressure solver and comparing the result against a numerical reference. I would then take an average of all the metrics I extracted from these scenes. The metrics that interested me were:

• Individual pressure NRMSE
• Individual gradient RMSE
• Individual post-projection divergence
• Coupled pressure NRMSE
• Coupled divergence RMS
• Coupled velocity RMSE
• Coupled density RMSE

The initial prompt did not reveal that I was using Mantaflow, or that I have a collection of private metrics and scenarios. I did, however, state that the fluid model’s predicted pressure could be used as an input during deployment. Below is the summary of the best results and some of the video samples:

I also included GPT-5.6-Sol Max’s results because it tried something different, and I think they are worth taking a look at.

GPT 5.5 High

GPT 5.5 came up with a ridge model, and the way it came up with this is kind of funny: It first did a CNN smoke run (I love that word) on CPU, found out that it was slow, and then rejected the architecture entirely. This happened even when I was pretty explicit about it in the prompt: “Hey! do not train on the CPU”. But regardless, it gave up on the run and decided on training 2 ridge regression models:

1. Stateless, which took in Divergence, Occupancy, Density, and Mask, and predicted the pressure
2. Temporal, which took the same features as stateless model but in addition to that, it took the predicted pressure of the previous time step.

After training, it ran a validation test to select the best one. It did 2 kinds of validation tests: self-fed and reference fed. The self-fed validation is when the prediction gets fed into the input. The reference-fed is when it takes the previous time step pressure from the reference numerical solution, in this case the validation dataset, and calculates the loss on pressure over the entire simulation. I was pleasantly surprised that it was clever to check both the metrics: the individual step error metrics and the long horizon simulation metrics, that I discussed earlier. Unsurprisingly, the temporal model did not work out because the self-fed validation NRMSE shot up to infinity. However, it did well on reference fed validation test. In the end, the LLM had to choose the stateless model because it was the only model left that did not blow up the validation metrics.

In my opinion, both of them are bad. This is because the LLM chose a ridge model with a 3x3 neighbourhood of divergence, occupancy, velocity, density and mask. The problem with this approach, as my maths professors in college painfully reminded me, is that you cannot break up the grid into discontinuous chunks and expect that the simulation would work. Even worse, during private evals, only 18 out of 30 scenarios finished to completion, everything else seg-faulted because velocity, divergence, and pressure blew up.

Why self-fed model does not work

Let’s assume, hypothetically, that it did pick a proper temporally conditioned fluid model, which takes in four inputs:

• Velocity Divergence
• Previous Pressure
• Occupancy
• Valid Mask

If such a hypothetical situation existed, the individual pressure NRMSE would look very good; and I managed to verify this with some of the runs where I removed the instruction “predicted pressure could be used as an input during deployment…” from the initial prompt. This is probably because most of the answer is already present in the input. The previous pressure is usually already close enough to the current pressure. The network might therefore learn something like: \[ p_t ≈ p_{t-1} + \epsilon \] The error \(\epsilon\) increases because the previous predicted pressure, with errors, was fed into the model again, producing even more error. It should have learned the divergence and should have tried to minimise it, but the network gets one-shotted by the previous pressure, and thinks it should optimise on that, which, I think, the individual pressure metric encourages it to. This was the reason why I included the coupling metrics for eval and the line “predicted pressure could be used as an input during deployment…”

I am not particularly impressed with what I got from GPT 5.5, but this is a good baseline for what GPT-5.6-Sol High is about to do next.

GPT 5.6 High

GPT5.6 built a U-Net with four downsampling stages. The fluid model takes in 3 inputs:

• Normalised Velocity Divergence
• Occupancy
• Valid Mask

The overall architecture here is pretty good because it now receptive for the entire grid rather than a smaller 3x3 neighbourhood like before. And notice how it does not take pressure into account at all. This is approximately what the numerical solver is doing as well: calculating \(\nabla^2 p_t\) using \(\nabla . u_t*\), without \(p_{t-1}\). Consequently, like we described earlier, the model is not picking up any error ridden pressure during simulation, which leads to much better coupled stability, and is shown by the coupled metrics. When the LLM removed the previous pressure from the input, the fluid model forced itself to infer pressure from current divergence and geometry.

Like the two different ridge models GPT 5.5 came up with, GPT 5.6 also decided on training 2 different types of models: the stateless model and the temporal model. It was encouraged, in the prompt, that it should try to run different experiments and choose the one with the best result, which happened to be the stateless model like before. Additionally, GPT 5.6 also correctly inferred from the initial prompt that the prediction at time \(t\) is going to be fed into the fluid model at time step \(t+1\), and it incorporated that in its model validation choice. The temporal model’s individual pressure RMSE was much better than the stateless model, and yet, it rejected the more attractive “short-term” temporal model, and chose the “long-term” stateless model. The stateless model proved to be far more stable during private evals. I think this demonstrates GPT-5.6’s ability to reason and focus on the long term target horizons rather than chasing short term metrics. I am also impressed by its choice of the U-Net architecture; this signals its ability to understand the underlying mechanics of fluid simulation: one corner of the grid affects the other corner. It would have been a lot better if we had access to the CoT reasoning for further analysis but alas we do not.

GPT 5.6-Sol Max

GPT 5.6-Sol Max, to me, felt smarter than the GPT 5.6-Sol High model. It directly trained three stateless U-Net models, did not even bother with a temporal model, with three different feature sets and went straight to validating it:

It selected the one with the lowest NRMSE, which in this case happens to be the simplest one. I am assuming it already guessed that occupancy and normalisations will not improve its primary validation metric, but did the experiments anyway just to be sure.

Interestingly, Max chose a smaller model than High: GPT 5.6 High’s U-Net had around 1.89 million parameters while Max’s had around 0.5 million. It compensated this with an additional numerical correction step after the prediction, which gave it an edge during private evals. The numerical correction is actually solving the pressure Poisson equation after the end of each forward pass. Max’s strategy was essentially: get the best approximation for the pressure, and then do the iteration on that. The U-Net supplied the best approximate as predicted pressure, and subsequent algorithmic steps refined it. This could be called cheating, but it is a working solution and something that works better than High’s model. Reflecting back on the original question “Can I replace the expensive pressure projection step of a CFD with a neural network, and produce a result that looks very similar?” maybe the right question all along was “Can I come up with the best guess for the pressure very quickly and then refine it in subsequent steps?”.

If we take a look at the code, we can see what it is essentially doing: summing up the neighbourhood pressure, subtracting divergence from it, normalising it, and then using that for convergence, which as described before is what the solver does as well (simplifying).

class UNet:
    ...
        
    def forward(
        self,
        model_input: torch.Tensor,
        standardized_rhs: torch.Tensor,
        valid_mask: torch.Tensor,
    ) -> torch.Tensor:
        # unet pass, omitted for brevity
        ...
        pressure = self.output(last_layer_of_unet) * valid_mask
        degree = F.conv2d(valid_mask, self.neighbor_kernel, padding=1).clamp_min(1.0)
        rhs = standardized_rhs * valid_mask
        for _ in range(self.jacobi_steps):
            neighbors = F.conv2d(
                pressure * valid_mask, self.neighbor_kernel, padding=1
            )
            proposal = (neighbors - rhs) / degree
            pressure = (
                (1.0 - self.jacobi_relaxation) * pressure
                + self.jacobi_relaxation * proposal
            ) * valid_mask
        return pressure

At 64x64, this approach is actually slower than the numerical projection but when we scale it to a 256x256 grid, we see that, while benchmarking, this speeds up quite alot. It is nowhere near realtime, but nonetheless still good. We already start at a reasonable guess and then do more iterations on it.

Unsurprisingly, model inference on GPU is slower than CPU because additional refinement steps, in the implementation, require the data to be loaded onto the CPU.

Final Thoughts

To answer the question from the title: Yes, LLMs can predict fluid simulations (but with some caveat)! But all in all, I think this eval might be one of my go-tos to measure the frontier. There are still interesting things we can do from here, such as answering:

• How long does it take for the predicted pressure to converge vs starting from complete scratch?
• If we substitute the shallower fluid model of GPT 5.6-Sol max with deeper model made by GPT 5.6-Sol high, will it improve the metrics?
• Can both of the above models scale to larger grids?
• How does the inference time change with more iterations during forward pass?
• Could GPT5.6-Sol max have picked up that we were doing long horizon simulations, if we had not put it in the prompt?
• How will the open source models or Claude behave? Will they reach the same conclusions, and if we had access to CoT, how would they reason about it?