Making multivariate training data out of univariate series

The data problem

In the previous post I described how TiRex-2 routes targets, past covariates and future-known covariates through different paths. All of that machinery is useless if the model never sees training examples where covariates actually carry information about the target.

That's harder than it sounds. Large univariate time series corpora are plentiful: sales, traffic, energy, weather, plus synthetic series from Gaussian processes. Multivariate corpora with genuine, diverse dependencies between variates are scarce and narrow. Train on them alone, and the model learns the few dependency patterns they happen to contain, and little else.

So we went the other way: keep the big univariate pool, and couple its series into multivariate samples during training, with dependency structures we control. Every training step sees a freshly composed sample, so the model never sees the same combination twice.

Step 1: augment each series on its own

Before any coupling happens, each series drawn from the pool is perturbed independently:

• Piecewise-linear amplitude trends, so the scale of a series can drift over time.
• Quantile censoring, which clips values above or below a random quantile, like a saturated sensor.
• Synthetic spikes with Gaussian, triangular or rectangular kernels.

The result is cropped or NaN-padded to the target length \(T\). Missing data is part of the job, so the model should see it during training too.

Step 2: couple them

Next, one mechanism is drawn uniformly from a small set. Each mechanism turns \(F\) augmented series \(u_1,\dots,u_F\) into \(F\) output variates \(x_1,\dots,x_F\) with a known kind of dependency. Try them out:

Each mechanism stands in for a pattern we see in real systems:

• Identity / pass-through is the control. Some samples stay univariate, so the model doesn't learn to expect a partner for every series.
• Functional coupling: \(x_{f,t} = \phi_f(u_{0,t}) + \varepsilon_{f,t}\), with monotone, compressive, discretizing or piecewise-linear \(\phi_f\). Think of redundant sensors.
• Linear mixing: \(x_{f,t} = \sum_i a_{fi}\, u_{i,t}\). Shared latent drivers, as in factor models. The singular values of \(A\) are sampled from dominant, uniform or power-law regimes.
• Cointegration: shared random-walk trends plus stationary AR(1) residuals. Non-stationary series tied together by a long-run equilibrium.
• Linear structural causal model: a random DAG with lagged edges, which produces lead-lag structure.
• Nonlinear structural causal model: nonlinear parent functions and an optional multiplicative gate, so the strength of coupling depends on the state of the system.

Step 3: make it look like the real world

Coupled series are still too clean. The final step adds the mess that real covariates come with:

• Variate permutation, so position in the input carries no meaning.
• Smooth time warping via Brownian-bridge lags. Real covariates are rarely perfectly aligned with the target, and the misalignment itself drifts.
• Patch masking with contiguous NaN blocks per variate.
• Partial future observability: the future part of random covariates is truncated. This is what separates past covariates from future-known ones, and it teaches the model to handle both from the same raw material.
• Discretization in value (uniform, quantile, power-law) and in time (freezes, staircases, duty cycles). Binary flags, stepwise tariffs and on/off schedules are everywhere in practice.

Does the model actually use its covariates?

The whole point of the pipeline is a model that exploits covariates instead of ignoring them. The experiments in the paper evaluate the full model, so they don't single out the pipeline, but they do show that TiRex-2 learned exactly that skill:

• Covariates carry real weight. In the fev-bench ablation, withholding the future-known covariates worsens MASE by 0.114 on the tasks that have them, against 0.033 over all tasks. The model relies on the signal in the forecast window.
• Misaligned covariates still help. Pairing a random-walk target with a noisy copy shifted by \(\Delta\) steps, TiRex-2 keeps extracting useful signal well beyond the shift at which Chronos-2 falls back to its no-covariate baseline. That's the kind of lag and warping the post-processing step trains on.
• Structured systems. On 135 chaotic systems from the dysts benchmark, with one channel as target and the rest as future-known covariates, TiRex-2 leads Chronos-2 at short to moderate horizons.

Takeaway

If you want a foundation model to use covariates, you have to show it many different ways in which covariates relate to targets, far more than any curated multivariate corpus contains. Composing that data on the fly from univariate series is cheap, controllable and effectively unlimited. And because every mechanism is known, it also gives us a clear way to ask which kinds of structure the model actually needs.