12. NeuralDEM: Real-time Simulation of Industrial Particulate Flows
NeuralDEM: A UPT follow up for particulate flows.
Introduction
While most deep learning models for physics simulations are still just tested on academic benchmarks, we discussed in the last blog post the Universal Physics Transformer (UPT) that went a step further with respect to the competitors in CFD (fluidodynamics) simulations.
Today we discuss the next iteration of UPT, NeuralDEM that takled a particularly hard problem literally at the scale of problems in the industry involving granular and discontinuous materials.
The problem: What does “DEM” means?
The problem we consider consists in predicting how a system of particles evolves in time, knowing the mass, the radius and the velocity of each particle.
This system of particles follows Newtons second law (yes the same you saw in your first course of Physics!) and, for each particle, reads as follows:
The equation says that each particle experience three types of forces: The first is of external origin, the second is a contact force with the nearby particles and walls and the third one represent the influence of a surrounding fluid phase, if present.
This problems are usually solved with the Discrete Element Method (DEM) that tracks each particle separately and computes its motion from Newton’s laws.
More concretely, as DEM simulation, they consider particles in an Hopper, that is commonly used in the industry for the storage of particulate material.
A second, more complex and more realistic, variant of this problem that they consider is the prediction of the same system of particles but in the context where the surrounding fluid has a role. In this case, the fluid velocity evolves following the Navier-Stokes equations that need to be coupled with the DEM equation discussed previously. This problem is called CFD-DEM simulations and I don’t discuss this here to keep the blog short but you can find all the details in the original paper [1].
More practically, as CFD-DEM simulation, they consider particles in a Fluidized bed reactor, a device widely used in industry for processes such as catalytic cracking, combustion, and chemical synthesis.
These problems are particularly complicated due to the very high number of particles and the material properties that strongly influence the behaviour of the system. And it is an interesting problem to approach with deep learning techniques since the timestep you need to use if you want to simulate it with numerical solvers needs to be extremely small, significantly smaller than the duration of contact of colliding particles particles. For this reason, numerical methods are particularly slow on this specific problem.
The Model: NeuralDEM
NeuralDEM builds on the Universal Physics Transformer (UPT) introducing two novelties:
Physics representation: It models the Lagrangian discretization of DEM as an underlying continuous field.
Multiple branches: It adopts a main branch for the modeling of system quantities that are fundamental to predicting the evolution of the state while auxiliary off-branches are added to directly model macroscopic quantities by retrieving information from the main-branch state and further refining the prediction using relevant inputs.
Multi-branch transformer
The multi-branch design contains two types of branches: main branches, which model the primary dynamical variables, and off-branches, which predict auxiliary macroscopic quantities. Both are a stack of standard Transformer blocks.
The main branches are coupled through joint self-attention: before each attention layer, their tokens are concatenated, allowing the different primary fields to exchange information, and are then split back into separate branches.
Off-branches are coupled only one way. Their tokens attend both to themselves and to the main-branch tokens through cross-attention, but gradients are stopped toward the main branches. Thus, off-branches can read from the learned dynamics without modifying them.
Why NeuralDEM is convenient over classical solvers?
NeuralDEM operates (in both training and inference) at orders of magnitude (at least 3) coarser timescales than what a classical solver requires to be stable and accurate.
The timescale relation is at least
More concretely, simulating a granular flow of 250k particles through a hopper, with a trajectory spanning 40s or 4M numerical timesteps, requires 3 hours on 16 cores of high-performance CPUs when using traditional DEM. In contrast, on a single state-of-the-art GPU, the fastest NeuralDEM inference model faithfully reproduces the physics rollout in just 1.4 s. The 1.4s is what make it possible to actually use NeuralDEM in real time.
Conclusions
We have seen how UPT has been adapted to simulate the dynamics of large particle systems. To me, the most interesting innovation is the introduction of multiple branches, which feels like a promising way to make neural operators more modular and physically meaningful.
More broadly, I think NeuralDEM is interesting not only as a faster surrogate for DEM, but also as a design idea for future architectures in complex multiphysics systems. I will certainly bring the ideas of this work with me!