Benchmark Results
This page shows the results from benchmark runs on various datasets.
Click on a dataset to expand the available plots.
coupled_oscillators
| Metric |
MultiONet |
FullyConnected |
LatentPoly |
LatentNeuralODE |
| MSE |
1.12e-05 |
1.42e-07 |
1.42e-03 |
7.58e-04 |
| MAE |
2.56e-03 |
2.79e-04 |
2.95e-02 |
1.97e-02 |
| MRE |
6.16 % |
0.61 % |
68.80 % |
54.88 % |
| Epochs |
15000 |
6000 |
25000 |
4000 |
| Train Time (hh:mm:ss) |
00:59:19 |
00:14:34 |
03:33:50 |
02:32:25 |
| Inference Times |
3.51 ms ± 0.20 ms |
986.41 µs ± 52.16 µs |
645.90 µs ± 28.87 µs |
219.48 ms ± 5.01 ms |
| Gradient PCC |
0.2599 |
0.5290 |
0.1430 |
0.1215 |
| # Trainable Params |
1972163 |
504668 |
195100 |
659403 |
| Model Memory (MB) |
8.14 MB |
1.93 MB |
1.49 MB |
5.85 MB |
| Forward Pass Memory (MB) |
26.20 MB |
13.69 MB |
121.56 MB |
110.21 MB |
| Avg. Uncertainty |
3.17e-03 |
3.52e-04 |
3.25e-02 |
2.44e-02 |
| DE MAE |
1.35e-03 |
1.96e-04 |
1.61e-02 |
9.91e-03 |
| DE MRE |
4.58 % |
0.60 % |
34.78 % |
22.03 % |
| UQ PCC |
0.3819 |
0.2765 |
0.1654 |
0.3767 |
| UQ Confidence |
-33.99 % |
-5.47 % |
2132.27 % |
1864.76 % |
Comparative Plots
FullyConnected
MultiONet
LatentNeuralODE
LatentPoly
lotka_volterra
| Metric |
MultiONet |
FullyConnected |
LatentPoly |
LatentNeuralODE |
| MSE |
3.23e+02 |
1.75e-01 |
1.68e+02 |
4.54e+01 |
| MAE |
3.72e+00 |
1.68e-01 |
6.63e+00 |
2.83e+00 |
| MRE |
4736.57 % |
1.42 % |
234782.95 % |
230.28 % |
| Epochs |
15000 |
20000 |
24000 |
8000 |
| Train Time (hh:mm:ss) |
01:05:04 |
00:27:23 |
01:24:02 |
13:07:48 |
| Inference Times |
3.27 ms ± 0.15 ms |
609.17 µs ± 59.44 µs |
750.66 µs ± 115.50 µs |
466.40 ms ± 3.67 ms |
| Gradient PCC |
0.2744 |
0.5486 |
0.6950 |
0.4760 |
| # Trainable Params |
800775 |
52620 |
78456 |
200537 |
| Model Memory (MB) |
3.06 MB |
0.20 MB |
0.60 MB |
1.54 MB |
| Forward Pass Memory (MB) |
14.17 MB |
7.94 MB |
75.67 MB |
105.78 MB |
| Avg. Uncertainty |
2.60e+00 |
1.68e-01 |
2.08e+00 |
3.51e+00 |
| DE MAE |
3.26e+00 |
9.61e-02 |
6.43e+00 |
2.95e+00 |
| DE MRE |
2246.18 % |
0.85 % |
234892.26 % |
10799946.34 % |
| UQ PCC |
0.8776 |
0.6317 |
0.3078 |
0.6307 |
| UQ Confidence |
80921.77 % |
-0.44 % |
4203140.56 % |
1331634245.40 % |
Comparative Plots
FullyConnected
MultiONet
LatentNeuralODE
LatentPoly
osu2008
| Metric |
MultiONet |
FullyConnected |
LatentPoly |
LatentNeuralODE |
| MSE |
8.64e+02 |
9.90e+01 |
6.94e-01 |
8.58e-01 |
| MAE |
3.93e+00 |
1.54e+00 |
8.42e-02 |
9.33e-02 |
| MRE |
3.81 % |
0.29 % |
7.96 % |
8.04 % |
| Epochs |
7000 |
20000 |
20000 |
15000 |
| Train Time (hh:mm:ss) |
00:58:08 |
00:21:39 |
02:21:23 |
02:57:50 |
| Inference Times |
13.62 ms ± 8.40 ms |
454.38 µs ± 52.27 µs |
727.38 µs ± 55.81 µs |
35.56 ms ± 1.87 ms |
| Gradient PCC |
-0.0068 |
-0.1830 |
0.4264 |
0.4223 |
| # Trainable Params |
2488384 |
29609 |
163221 |
688968 |
| Model Memory (MB) |
9.50 MB |
0.11 MB |
1.25 MB |
6.06 MB |
| Forward Pass Memory (MB) |
94.11 MB |
16.30 MB |
106.23 MB |
106.52 MB |
| Avg. Uncertainty |
7.52e+00 |
1.04e+00 |
3.95e-01 |
1.11e-01 |
| DE MAE |
4.79e+00 |
7.72e-01 |
4.36e-01 |
5.91e-02 |
| DE MRE |
2.98 % |
0.18 % |
6.71 % |
2.77 % |
| UQ PCC |
0.8964 |
0.7517 |
0.9914 |
0.7024 |
| UQ Confidence |
-18.90 % |
-0.09 % |
0.65 % |
22.88 % |
Comparative Plots
FullyConnected
MultiONet
LatentNeuralODE
LatentPoly
simple_ode
| Metric |
MultiONet |
FullyConnected |
LatentPoly |
LatentNeuralODE |
| MSE |
6.53e+04 |
3.67e+05 |
1.06e+07 |
6.82e+06 |
| MAE |
3.50e+00 |
1.08e+01 |
3.69e+01 |
2.34e+01 |
| MRE |
1.49 % |
1.55 % |
4.85 % |
1.69 % |
| Epochs |
7500 |
10500 |
12000 |
6000 |
| Train Time (hh:mm:ss) |
00:53:33 |
00:59:27 |
00:41:16 |
04:15:57 |
| Inference Times |
2.96 ms ± 0.14 ms |
1.75 ms ± 0.14 ms |
595.67 µs ± 84.73 µs |
50.16 ms ± 0.72 ms |
| Gradient PCC |
0.9513 |
0.9943 |
0.9210 |
0.8400 |
| # Trainable Params |
1282024 |
4495205 |
86167 |
846324 |
| Model Memory (MB) |
4.89 MB |
17.99 MB |
0.66 MB |
7.31 MB |
| Forward Pass Memory (MB) |
20.48 MB |
26.00 MB |
79.99 MB |
249.78 MB |
| Avg. Uncertainty |
5.66e+00 |
8.10e+00 |
2.65e+01 |
8.40e+00 |
| DE MAE |
2.22e+00 |
1.56e+00 |
2.74e+01 |
1.74e+01 |
| DE MRE |
1.59 % |
0.56 % |
3.36 % |
1.53 % |
| UQ PCC |
0.8393 |
0.9531 |
0.8079 |
0.7782 |
| UQ Confidence |
-4.83 % |
1.53 % |
-3.45 % |
7.36 % |
Comparative Plots
FullyConnected
MultiONet
LatentNeuralODE
LatentPoly
simple_reaction
| Metric |
MultiONet |
FullyConnected |
LatentPoly |
LatentNeuralODE |
| MSE |
1.18e-03 |
5.27e-06 |
3.33e-03 |
2.96e-01 |
| MAE |
1.73e-02 |
1.12e-03 |
2.81e-02 |
2.61e-01 |
| MRE |
0.29 % |
0.02 % |
0.55 % |
5.23 % |
| Epochs |
12000 |
10000 |
10000 |
7000 |
| Train Time (hh:mm:ss) |
01:41:39 |
00:35:33 |
03:22:24 |
05:53:19 |
| Inference Times |
3.09 ms ± 0.70 ms |
1.06 ms ± 0.07 ms |
736.66 µs ± 21.15 µs |
69.49 ms ± 0.90 ms |
| Gradient PCC |
0.1708 |
0.4613 |
0.5081 |
0.5332 |
| # Trainable Params |
3109252 |
312821 |
517411 |
1100426 |
| Model Memory (MB) |
11.87 MB |
1.20 MB |
4.02 MB |
9.25 MB |
| Forward Pass Memory (MB) |
26.53 MB |
13.25 MB |
197.74 MB |
249.79 MB |
| Avg. Uncertainty |
1.29e-02 |
1.35e-03 |
5.53e-02 |
2.72e-01 |
| DE MAE |
8.56e-03 |
5.42e-04 |
1.95e-02 |
2.15e-01 |
| DE MRE |
0.16 % |
0.01 % |
0.41 % |
3.86 % |
| UQ PCC |
0.7300 |
0.6513 |
0.3883 |
0.6622 |
| UQ Confidence |
-0.45 % |
0.05 % |
1.64 % |
-6.54 % |
Comparative Plots
FullyConnected
MultiONet
LatentNeuralODE
LatentPoly
