Choosing, tailoring, and integrating models for transfer learning in forecasting wind and solar power

Recent advancements have sparked growing interest in exploiting collections of pre-trained models, known as model hubs, particularly within computer vision applications. In practice, this involves first identifying a suitable pre-trained source model and then tailoring it to the new target domain to bridge any mismatches. Despite this trend, investigations into optimal model choice and tailoring methods for renewable energy generation forecasts remain scarce. Specifically, prior research has not delved into varied selection or adaptation techniques tailored to different neural network structures. Furthermore, no existing analyses have systematically evaluated how the volume of available training data affects outcomes, nor have they incorporated seasonal variations into performance assessments. We bridge these research gaps through an initial in-depth study of model selection and adaptation within transfer learning frameworks for renewable power prediction. Inspired by cutting-edge advances in computer vision, the analysis draws on operational data from 667 wind farms and solar (photovoltaic) installations across six datasets. To generate robust error metrics, we simulate scenarios with varying volumes of training observations across seasons. Model selection is then appraised using both marginal likelihood estimates and resulting forecast inaccuracies under these constrained data regimes. In addition, four separate adaptation methodologies are rigorously compared. Building on and surpassing conventional approaches, we introduce Bayesian linear regression to model the output variable from deep features derived by a neural network backbone. This technique delivers superior results compared to standard baselines, even with only 7 days of local training data. It also shows that full network fine-tuning provides negligible gains unless at least 3 months of data are available. Beyond that, we demonstrate that aggregating predictions from multiple models into ensembles markedly improves the effectiveness of both selection and adaptation. Consequently, ensemble-based strategies achieve mean error levels with only 30 days of training data that would otherwise require a full year of observations. In the most representative datasets, this yields average errors of 9.8% for photovoltaic power and 14% for wind power, utilizing only seven days of site-specific training data.

The growing integration of variable renewable energy sources, such as wind farms and photovoltaic (PV) installations, creates a key challenge when adding a new park to an operator’s existing portfolio. New (target) parks often lack historical data, while accurate power forecasts are essential for maintaining grid stability because they depend heavily on weather conditions. Fortunately, operators often have access to numerous pre-trained models developed for other existing parks, which can be leveraged to improve forecasting performance and significantly reduce the computational cost of building a new model from scratch [1-3]. The central question is therefore how to best exploit this collection (model hub) of pre-trained models. Research on inductive transfer learning (ITL) offers valuable approaches to address this issue [4].

Figure 1a provides an overview of the strategy we propose. The initial phase involves selecting a suitable source model from the hub. Recent studies [4, 5] have demonstrated that the choice of source model has a substantial impact on the final test error in computer vision tasks. To select an appropriate source model, suppose we have two source tasks (e.g., from different wind parks) with model parameters θ₁ and θ₂, along with their respective input observations X₁ and X₂ and response values Y₁ and Y₂. Using this information, the goal is to identify the most suitable model for transfer to a new target task T characterized by parameters θ_T and its corresponding input set X_T and output set Y_T.

Figure 1b illustrates this selection problem in the context of wind power forecasting. In renewable energy forecasting, inputs X to the machine learning models typically consist of weather predictions—such as wind speed or solar radiation—generated by numerical weather prediction (NWP) models. These features are used by models with parameters θ to predict the expected power output E[p(Y | X, θ)] for a day-ahead horizon, usually 24 to 48 hours ahead. The diagram highlights that the similarity between source and target depends on the relationship between input features (e.g., wind speed) and the resulting power generation, which varies across different wind speeds and turbine characteristics.

After selecting a source model, the second step is to adapt the transferred knowledge to the target park using the limited available data with an appropriate adaptation strategy. A common approach is fine-tuning only the final layer of a neural network. Such adaptation is crucial for generating reliable, site-specific forecasts for a new park despite scarce historical data. To the best of our knowledge, the combination of different model selection and adaptation strategies within inductive transfer learning has not yet been systematically explored for renewable power forecasting. This paper addresses that gap.

Selecting and adapting only a single source model has a clear limitation: it discards potentially useful knowledge from other source models. To overcome this, an optional third step can be introduced that selects multiple source models, adapts them to the target, and combines them using ensemble techniques. The forecasts from these adapted models are merged through a suitable weighting scheme. This ensemble approach enables the integration of insights from multiple parks into the target forecast.

Because inductive transfer learning remains underexplored in the domain of renewable power forecasting [6]—particularly for day-ahead horizons of 24 to 48 hours—we specifically investigate the following research questions:

Research Question 1: What constitutes a suitable similarity measure for selecting an appropriate source model for a new target park from a hub of pre-trained models?

Research Question 2: What is the most effective adaptation strategy once a source model has been selected?

Research Question 3: Do ensemble strategies provide advantages over the selection and adaptation of a single source model when combining knowledge from multiple sources?

Each research question corresponds directly to one of the core steps in applying transfer learning to renewable power forecasting. Our contributions consist of developing concrete methods for each step, with each building on the previous. In a preparatory step (step 0), we construct a model hub by training three model types—Bayesian extreme learning machine (BELM), multi-layer perceptron (MLP), and temporal convolutional network (TCN)—on six datasets spanning 667 distinct parks. For source model selection (step 1), we propose using either the marginal likelihood (also called the evidence) or the normalized root-mean-squared error (nRMSE). Once a model is selected, we adapt it to the target task (step 2) by evaluating four different adaptation strategies, including fine-tuning via weight decay. We demonstrate the transferability of all three model types from source to target and benchmark them against a gradient boosting regression tree (GBRT) baseline. Notably, the GBRT outperforms conventional physical models [7], which are frequently used as a fallback when data is limited. In an optional third step, we investigate ensemble methods by adapting Bayesian model averaging (BMA) and the cooperative soft gating ensemble (CSGE) to the inductive transfer learning setting, and compare their performance with the best single-adapted model from the previous experiment.

Based on these datasets and models, the main contributions of this work are:

By addressing the first two research questions, we deliver practical methods and strategies for model selection and adaptation applicable to a broad range of renewable power forecasting problems where they have not previously been considered.

We demonstrate, contrary to widespread assumptions, that fine-tuning only the final layer of a neural network with weight decay can be among the least effective approaches for inductive transfer learning in renewable power forecasting when target data are scarce.

We further show that ensembles based on BMA and CSGE outperform single adapted models. These ensembles achieve a mean forecast error with only 30 days of target training data, whereas training from scratch would require a full year of data.

The following part surveys recent progress in transfer learning (TL), emphasizing inductive transfer learning (ITL) methods originally developed for computer vision applications that have not yet been examined in the context of renewable power forecasting. This overview helps identify useful techniques worth adapting for renewable energy scenarios. We then provide a summary of existing studies on ITL applied to deterministic renewable power prediction tasks. Additional Bayesian-oriented research in other renewable energy fields can be found in Refs. [8-10].

Two key elements define successful ITL: choosing the right source model and applying an effective adaptation technique. Research presented in [4, 5] explores various selection approaches within computer vision. Their methodology involves replacing the final layer of each source network with a Bayesian linear regression (BLR) component, which is then fitted to the target dataset via empirical Bayes, also known as evidence approximation. This procedure is applied to all models in the collection. Task similarity is subsequently assessed based on the evidence value each source model achieves on the target data. For adaptation, they propose a Bayesian tuning procedure that leverages predictions from multiple sources to constrain fine-tuning. Such selection and adaptation concepts require careful testing and extension when moving to renewable energy domains. One possibility is to produce forecasts directly from the BLR and benchmark them against the common practice of adjusting only the final layer. Fine-tuning usually applies weight decay to push parameters toward zero [11]. Yet this form of regularization overlooks the parameters originally learned on the source task. By comparison, the method introduced in Ref. [12] adds a penalty term that discourages large shifts away from the source parameters within the weight-decay framework.

Compared with domains such as computer vision, far fewer studies address ITL for renewable power forecasting [6]. A subset of earlier efforts concentrates on building transferable feature representations through autoencoders [13-16]. Although an autoencoder transfer step could be integrated into the framework discussed here, capturing the conditional distribution of power output is more important for assessing model suitability and for later combining outputs. The purely data-driven TL techniques covered in Refs. [17, 18] lie beyond the focus of the present study.

A large portion of TL literature in renewable power forecasting uses raw meteorological observations as inputs [14-25]. These contributions generally address short prediction windows ranging from 10 seconds to 2 hours. Day-ahead horizons, however, present greater difficulty, as they depend on outputs from numerical weather prediction (NWP) systems, and prediction inaccuracies typically increase as the forecast lead time lengthens [26].

Nearly all of the referenced TL studies frame power forecasting as a plain regression task. Nevertheless, recurring patterns such as daily cycles are widely recognized yet frequently ignored by standard regression formulations. To account for such temporal structures, Ref. [3] incorporates time-series models within a multi-task learning (MTL) framework. Likewise, the authors in Ref. [27] combine recurrent architectures with fine-tuning and report favorable outcomes for ultra-short-term photovoltaic (PV) predictions.

Reference [28] reports gains in day-ahead PV forecasting by training models that simultaneously target multiple outputs. Their model-combination concept resembles the ensemble idea explored later in this work, although it is not assessed under an inductive transfer learning lens. In Ref. [29], an MTL scheme based on Gaussian processes is proposed for PV targets. Another contribution [30] groups wind parks via clustering and applies a weighting mechanism to generate outputs for unseen parks; unfortunately, that analysis relies exclusively on simulated data instead of real historical power records.

A series of papers adopts MTL structures for day-ahead forecasting horizons [3, 31, 32]. The task-embedding strategy is detailed in Refs. [3] and [31] for multi-layer perceptrons (MLPs) and convolutional neural networks (CNNs) embed task-specific details to uncover hidden similarities across different tasks. The setup in Ref. [3] is particularly relevant because it mirrors key aspects of our experimental design. Still, assessing performance on a seasonal breakdown can produce misleading conclusions for ITL methods, which must guard against catastrophic forgetting throughout every season. MTL designs see limited uptake in practical industry settings, largely because of the extra effort needed for data preparation and model training. It is therefore vital to extract maximum benefit from already-trained single-task models when scaling renewable capacity through ITL.

In addition, prior literature has not systematically compared alternative model selection or adaptation procedures across different neural network families. Although a few studies have investigated ensemble-style merging of information from several locations for PV, those efforts fall short: they often disregard the actual quantity of available data, base expected power solely on static characteristic curves, or restrict analysis to either PV or wind exclusively. The present work fills these gaps through a comprehensive evaluation that addresses the noted shortcomings in day-ahead forecasting applications.

The sections ahead outline the suggested procedures for selecting models, adapting them, and merging their outputs. As preparation, we briefly describe the model types in the collection and introduce Bayesian linear regression (BLR), given its pivotal role in both selection and adaptation approaches.

Step 0: Source models for the model hub

To achieve dependable forecasts despite limited target data via transfer learning, the study employs three machine learning architectures: a Bayesian extreme learning machine (BELM), a multi-layer perceptron (MLP), and a temporal convolutional network (TCN).

BELM builds on linear regression by mapping inputs into a higher-dimensional feature space, where linear separation is often easier, thereby supporting improved predictions [33]. NWP-derived features are mapped using randomly initialized weights and a nonlinear transformation, such as the rectified linear unit (ReLU). The resulting expanded features then combine linearly to estimate the target value. Here, the linear step uses Bayesian linear regression (BLR). In contrast to ordinary linear regression, BLR supplies the marginal likelihood, offering a direct indicator of how well a source model aligns with target data—making it useful for ITL selection. BELM’s quick training and convex nature explain its popularity in renewable power forecasting applications [19, 34].

MLPs, along with deeper neural networks in general, serve as established tools for regression and classification and are optimized via gradient-based methods. Inside an MLP, inputs pass through successive matrix multiplications followed by nonlinear activations (commonly ReLU). These operations form layers, with each layer’s output feeding the next. Stacking layers in this manner allows the network to discover effective data representations. Renewable power estimation typically employs a straightforward linear summation in the output layer. MLPs excel at extracting representations from NWP inputs that transfer well, which explains their frequent use in renewable forecasting and TL contexts [35, 36].

TCNs extend the MLP concept to explicitly handle temporal relations in sequential forecasting problems [37-39]. They rely on one-dimensional convolutional layers that slide along the time axis for each input channel (where a channel typically represents a single feature). Although two-dimensional CNNs have dominated computer vision and associated TL work [40, 41], one-dimensional variants have recently attracted interest for time-series TL [42]. CNNs suit transfer learning particularly well because their layered hierarchy allows focused adjustment of only the later stages. In a TCN, standard convolutional layers are replaced by residual blocks—an idea popularized in vision tasks [43]. Each residual block includes a shortcut link that bypasses the internal transformations and adds the incoming activation to the block output, helping prevent vanishing gradients. Processing within a block proceeds in the following sequence: dilated convolution, weight normalization, ReLU, and dropout. Dilated convolutions enlarge the effective observation window without excessive computational or memory overhead. When the input and output sizes mismatch, an additional convolution aligns the dimensions within the shortcut path.

Bayesian linear regression

Bayesian linear regression (BLR) occupies a central position in this study, serving the needs for:

·   Constructing the BELM,

·   Quantifying similarity via marginal likelihood,

·   Modifying neural networks by substituting their output layer with a BLR,

·   Fusing predictions through Bayesian model averaging (BMA).

Because of this importance, a detailed definition follows, drawing on foundational presentations in Refs. [44, 45]. A Bayesian viewpoint on linear regression provides richer information through the posterior, proving especially helpful in data-scarce settings [44]—a typical situation in transfer learning. This perspective aids similarity evaluation for selection and supports uncertainty-aware assessments, in line with suggestions from [4]. We advocate applying BLR for adaptation in renewable forecasting by exchanging the last network layer, retraining on target samples, and then generating forecasts. The same formulation also enables model fusion via BMA and underpins BELM training.

Consider the posterior of the linear model expressed in the following general form (using the notation from the original derivations):

Where X and Y represent the collections of observed inputs and corresponding responses containing N (N ≥ 1) training samples. Each feature vector x_n belongs to ℝ^D with D (D ≥ 1) dimensions, and each y_n is a scalar in ℝ. The likelihood term p(Y | X, θ) reflects how well the inputs X together with weights θ ∈ ℝ^D account for the observed targets. The prior distribution captures initial assumptions regarding plausible weight values. The marginal likelihood serves as the normalizing constant, and the posterior—conditioned on the training data—encodes updated beliefs about the parameters of the target task.

Deriving the posterior starts with a Gaussian prior p(θ | α) = 𝒩(θ | 0, α⁻¹I), where α > 0 controls the precision of this zero-mean isotropic Gaussian. The isotropic choice yields a closed-form solution, reducing the effort required to compute the mean vector and covariance matrix.

Let y denote the 1 × N vector of targets and X the N × D design matrix whose rows hold the individual observations. For problems with multiple outputs, independent models can be learned per dimension. In the scenarios considered here, X contains either raw NWP variables (e.g., wind speed, radiation), randomly projected features from a BELM, or activations from the penultimate layer of a neural network. In the last two situations, the original weather inputs have already been transformed by the BELM or network layers.

Conditioned on X, y, and the noise precision β > 0, the posterior takes the Gaussian form p(θ | X, y) = 𝒩(θ | m_N, S_N), with parameters:

Where the explicit expressions for m_N and S_N follow the standard Bayesian update rules for linear models.

N here denotes the number of target samples used to refine the prior. When predicting for an unseen input X* drawn from a test set, the predictive posterior distribution becomes:

With the associated predictive variance given by:

Step 1: Model selection for inductive transfer learning

Determining how closely a target task matches available source tasks constitutes a core difficulty in ITL [4]. A well-chosen source should deliver lower error than learning the target model independently, thereby preventing harmful negative transfer.

Before formally specifying selection within TL, it helps to recall the broader rationale for (Bayesian) model selection. Reference [44] notes that any sound selection procedure must weigh a model’s expressive capacity against its agreement with the observed data. Figure 2 visualizes this balance from a Bayesian standpoint. The x-axis spans the universe of conceivable datasets, while the y-axis plots the evidence a model assigns to a particular dataset. Suppose model M₂ possesses more parameters and can therefore accommodate a wider variety of datasets than the simpler model M₁. When evaluating evidence p(D | M_m), the Bayesian criterion tends to prefer the less complex M₁ for the given dataset D. This built-in preference for parsimony in Bayesian model evaluation aligns with the principle known as Occam’s razor.

The same model-selection principle carries over directly to transfer learning settings. The objective is to identify, among M (M > 1) available source models, the one that best accounts for the scarce target data T. Consider two example tasks defined as T1 = {Y, P1(Y1 | X1)} and T2 = {Y, P2(Y2 | X2)}, where both belong to the universal set of tasks T. Here, the sets Ym and Xm represent observations drawn from the output space Y and input feature space X, respectively. A similarity function S: T × T → ℝ ≥ 0 maps pairs of tasks to a non-negative scalar, enabling quantitative comparisons. For instance, if S(T1, TT) > S(T2, TT) for a target task TT, then task T1 shares greater similarity with TT than T2 does; higher values of S therefore indicate stronger alignment. Dissimilarity can be expressed simply as the reciprocal of such a similarity measure.

This raises a practical question: which similarity metric should be used, and which data are most relevant when selecting a source model from the collection for a given target? One straightforward option would be to evaluate similarity solely in the input feature space. However, input features alone provide incomplete insight into the expected power output (the response variable) in renewable energy systems. For example, identical solar radiation levels can yield substantially different energy production depending on the specific PV module characteristics. It is therefore necessary to incorporate information from the response variable as well.

Evidence or marginal likelihood

The work in Ref. [4] employs the marginal likelihood, also known as model evidence, as the similarity measure S. To compute it, the final layer of a source neural network Tm is substituted with a Bayesian linear regression (BLR) component. The prior parameters α and β of this BLR are then optimized using empirical Bayes [5, 44] on the limited target data. In this setup, the source network functions purely as a feature extractor. The marginal likelihood takes the following form:

Where D denotes the feature dimensionality (for example, the size of the penultimate layer of network m), N is the number of target samples, and the term E(mn) is defined as β/2 ⋅ ‖y − Xmn‖² + α/2 ⋅ θmnᵀθmn [5, 44]. This calculation jointly considers features produced by the source network for task Tm and the actual target responses from TT. Repeating the process for every source model m yields the marginal likelihood of each on the target data, which is then used to define S(Tm, TT). The source model with the strongest evidence is chosen as the most suitable. For problems with multiple output dimensions, the procedure is performed separately for each dimension, and the resulting values are averaged [4].

Although this technique is elegant and broadly applicable, it has one notable limitation in inductive transfer learning for renewable power forecasting: it discards the weights of the original final layer that have already been trained. In many renewable energy applications, the output layer dimensions are compatible between source and target, so replacing it entirely is unnecessary. Moreover, a pre-trained output layer typically provides a better starting point than a randomly initialized one.

Normalized root mean-squared error

As an alternative, we suggest assessing similarity directly using the normalized root-mean-squared error (nRMSE) produced by the unaltered pre-trained source model on the target data, as expressed in Eqs. 6 and 7.

In these expressions, yi(T) stands for the i-th observed response value from the target, ŷi(m) represents the prediction generated by source model m on the target inputs, N (N ≥ 1) is the number of samples, and ymax and ymin are the maximum and minimum values of the response variable, respectively. Because a smaller nRMSE indicates better alignment, the similarity measure is taken as the inverse, so that larger values correspond to better performance. For consistent normalization across all datasets, ymax is set to the park’s nominal power capacity, and ymin is set to zero. This approach allows a direct evaluation of how effectively a source model performs on the available target samples. The source model with the lowest nRMSE on a validation subset of the target data is therefore selected.

Step 2: Adaptation strategies for inductive transfer learning

Table 1 presents an overview of the 18 different combinations of models, selection methods, and adaptation techniques examined in this study. For the Bayesian extreme learning machine (BELM), a straightforward transfer learning baseline is obtained by updating its posterior online: the posterior learned on the source task serves as the new prior when incorporating target data. We also test the naïve strategy of deploying the selected source model to the target data with no parameter updates.

Two widely used fine-tuning approaches borrowed from computer vision are also included. The first applies conventional weight decay, which penalizes weights moving away from zero. The second variant, termed weight decay with respect to source parameters, penalizes deviations from the source weights instead. In addition, we evaluate the Bayesian tuning method proposed in Ref. [5].

The final three adaptation approaches serve as regularization techniques. Broadly speaking, this entails introducing an extra penalty component Lpen into the primary task loss Ltask according to the following expression:

Where λ ∈ ℝ acts as a tunable hyperparameter for the regularization strength, which is normally determined via hyperparameter tuning. The task-specific loss Ltask is formulated as:

With θ ∈ ℝp (p ∈ ℕ, p ≥ 1) being the parameter vector that undergoes optimization, xn standing for the nth input sample (n ∈ ℕ and total samples N ∈ ℕ, N ≥ 1), and yn denoting the associated target value. A single-output response is considered here for ease of explanation.

In the case of weight decay (WD) anchored at the origin [11], the penalty term Lpen is defined by:

When the goal is to discourage large shifts away from the source model, [12] suggests weight decay relative to the source weights (WDS), expressed as

Where θ0 ∈ ℝp corresponds to the parameter vector taken from the source model before the fine-tuning stage.

Finally, under Bayesian tuning, the penalty term Lpen takes the following form [5]:

In which n ∈ ℕ indexes the nth training sample, m refers to the mth source model that has been adapted using BLR, k indicates the kth output dimension (such as distinct forecast horizons), xm,k represents features pulled from the mth source model, and xt,n denotes the matching features extracted from the target model t. The vectors θk,c and θt,c are the mean parameter vectors computed via BLR.

Step 3: Model combination for inductive transfer learning

We have examined model selection and adaptation methods when transferring from one source model to a target task. Nevertheless, relying on a single model can easily lead to overfitting, especially when the target data is scarce. Merging multiple source models into an ensemble helps mitigate this issue.

Bayesian model averaging

We build on the idea presented in Ref. [4] by moving beyond selecting a single model based on its evidence. Instead, we merge several models adapted via BLR using Bayesian model averaging (BMA). BMA offers a strong theoretical foundation because it accounts for the full predictive posterior [46], thereby incorporating model uncertainty through

The prior probability p(θMi|yT) reflects our initial belief regarding the similarity between model Mi and the target dataset. For simplicity, we assume a uniform prior across all source models. Note that the input variable has been left out here to keep the notation concise. The term p(yT∗|yT,θMi) represents the predictive posterior distribution of model Mi, as defined in Eq. 3. This could, for instance, correspond to the outcome of the suggested direct linear adaptation approach.

Coopetitive soft gating ensemble

We further suggest applying the CSGE technique for combining models within the inductive transfer learning framework. Because the CSGE supports both ensemble selection and weighting modes, the term “coopetitive” combines cooperation and competition. The CSGE was originally developed for renewable power forecasting in Ref. [47]. Its core principle is to connect the weights assigned to ensemble members with their respective performance, meaning stronger-performing source models receive higher weights than weaker ones.

The CSGE determines the overall weight of each source model based on three key aspects:

·         The global weight reflects a source model’s overall performance on the available training data for the target task.

·         The local weight captures how well a source model performs across different regions of the feature space on the target task. For example, in wind power, one model may excel at low wind speeds while another performs better at higher wind speeds.

·         The forecast horizon-dependent weight indicates the performance of a source model for various lead times on the target task, such as predictions ranging from 24 to 48 hours ahead.

Figure 3 illustrates the structure of the CSGE. The CSGE consists of M ensemble members, where m ∈ {1, …, M}. Each ensemble member corresponds to a source model equipped with its predictive function fm. Every source model produces a univariate forecast yˆt+k|t(m) ∈ ℝ for a given input xt+k|t ∈ ℝ^D of the target task T. The subscript T is omitted here for improved readability. Let D denote the dimensionality of the input feature vector x. The index k represents the forecast horizon, referenced by the subscript, relative to the forecast origin t. For each prediction generated by each source model, an aggregated weight wt+k|t(m) is calculated.

The combined weight for a single source model and lead time, which integrates the global weight wg, the local weight wl, and the horizon-dependent weight wh, is computed as

Where w̄t+k|t(m) is scaled so that the weights across all models sum to one, yielding the final weight wt+k|t(m).

To determine the weights wt+k|t(m), we apply the inverse similarity measure S−1 together with the cooperative soft gating rule defined in Eq. 15.

By using the inverse S−1 (i.e., the nRMSE) to derive the weights, we quantify each source model’s performance on the target task. Suppose Φ ∈ ℝ^J holds all J ∈ ℕ, J ≥ 1, estimates computed from the nRMSE, with ϕ ∈ Φ. Here, η ≥ 0 controls the strength of the exponential weighting, and the small positive constant ϵ > 0 prevents division by zero. Larger values of η cause the CSGE to behave more like a gating ensemble, effectively focusing on only a few strong source models. Smaller values of η lead to a softer weighting ensemble. Once all weights have been calculated from the elements in Φ using Eq. 15, they are normalized to sum to one, producing the final weights wt+k|t(m). This procedure is carried out independently for each of the three weighting dimensions, following the methodology described in Ref. [47].

The next part of the paper outlines the overall experimental design. We performed experiments across six datasets, totaling 667 parks. Thanks to the cross-validation scheme employed, every park serves as a target park exactly once. This setup enables the most comprehensive evaluation to date of inductive transfer learning applied to renewable power forecasting.

Overall experimental setup

Data pre-processing steps follow the procedures used in Refs. [3, 7] to ensure direct comparability with existing state-of-the-art methods. We employed three types of source models: BELM, MLP, and TCN. To provide a strong, reliable baseline that performs well even with limited data and helps control overfitting, we trained a GBRT model for each target task as described in Ref. [7].

Datasets

All experiments focus on day-ahead forecasts, covering lead times of 24 to 48 hours. The datasets, summarized in Table 2, use numerical weather prediction (NWP) variables as input features, including wind speed, wind direction, air pressure, and solar radiation. These weather forecasts are aligned with historical power measurements, which serve as the target variable for day-ahead prediction across all datasets. The weather data originates from either the European Center for Medium-Range Weather Forecasts (ECMWF) or the Icosahedral Non-Hydrostatic-European Union (ICON-EU) model.

The datasets differ in the number of input features, temporal resolution, and the quantity of samples available for training and testing. For example, the PVOPEN dataset contains 47 features, many of which are hand-crafted to capture seasonal solar patterns. In contrast, the other datasets do not include such manually engineered features.

Note that four datasets—namely PVOPEN, WINDOPEN, WINDSYN, and PVSYN—were already analyzed in earlier research, such as Refs. [3, 7]. The situation differs for WINDREAL and PVREAL, which have not been explored previously. These two datasets remain unavailable to the public. Still, they offer the highest level of realism owing to their substantial variety. The WINDREAL dataset covers wind parks with 99 distinct nominal capacities, turbines from 13 manufacturers, and six hub heights. Every park is located in Germany. In the PVREAL dataset, the solar installations have 31 different nominal capacities, 10 different tilt angles, and 9 different azimuth directions, and they are likewise situated in Germany. It should also be emphasized that forecasting power output for wind parks generally poses greater challenges than for PV installations.

We applied a five-fold cross-validation procedure to every dataset. This guarantees that each park executes the target task exactly once and serves as a source park in the other four folds. Source models and their associated hyperparameters were fitted using only the training and validation splits. When training models for the target tasks, we divided the available data by season and limited historical records to 7, 14, 30, 60, or 90 days. All results shown are averaged across all target tasks and seasons. This arrangement avoids any distortion caused by seasonal patterns [26]. The input features from all datasets were normalized. Historical power values were divided by each plant’s nominal capacity to allow error measures to be compared fairly. Every dataset was converted to a 15-minute resolution, except PVOPEN, which was adjusted to hourly resolution due to its originally coarser sampling frequency. The WINDSYN and PVSYN datasets include a predefined test set. For WINDOPEN and PVOPEN, we followed the same protocol as [3]: the initial year served as training data and the subsequent period as the test set. Because the PVREAL and WINDREAL datasets differ significantly in the length of available history, we randomly selected 25% of the days for testing. As every day has its own independent day-ahead weather forecast, no information from the future influences earlier periods [34]. From the leftover days, 25% were set aside for validation, and the remaining 75% were used for training.

Source models

As noted earlier, predictions of renewable power depend heavily on meteorological conditions. Consequently, the model inputs are themselves outputs from the numerical weather prediction system. These can be fed directly into models, such as an MLP, to estimate expected power generation for the coming day. Hyperparameter search for the models was performed using a tree-structured Parzen sampler, running 200 trials on the validation data.

Overall, four model types were trained. The BELM stands out as a useful source model because it enables straightforward similarity assessment based on the evidence and allows model updates with only linear computational scaling. We included an MLP because it remains a standard choice in industrial renewable power forecasting applications [3]. A TCN architecture was also trained to better handle periodic patterns in the data, in line with the approach in Ref. [3]. Finally, a GBRT model was fitted to provide a solid baseline capable of good generalization even when data is scarce [7].

Evaluation method

We computed the normalized root mean square error (nRMSE) on the test portion using Eq. 7 for every possible pairing of season and training data length. For each dataset, season, and training duration, we derived the mean performance rank based on nRMSE scores. To check for meaningful gains over the baseline, we applied the Wilcoxon signed-rank test with significance level α = 0.01 across all parks in the dataset (Tables 3-8).

Experiment on model selection and model adaptation

This part of the study conducts an experiment that jointly addresses research questions 1 and 2. Evaluating any selection method only makes sense after the chosen model has been adapted to the target:

Research Question 1: What constitutes a suitable similarity measure for selecting an appropriate source model for a new target park from a hub of pre-trained models?

Research Question 2: What is the most effective adaptation strategy once a source model has been selected?

Key findings for questions 1 and 2

The choice of model selection technique and the adaptation approach are tightly interconnected. When target training data is scarce (7–30 days), simply picking the model with the lowest forecast error and applying it without any modification often yields top-tier results. As more data becomes available, swapping the final layer for a Bayesian linear regression model proves more effective. None of the conventional fine-tuning strategies rank among the leading options.

Experimental setup

We used the source models described earlier. Fine-tuning was limited to 1 epoch, with hyperparameters tuned via grid search on 30% of the available target samples.

For weight-decay-based adaptation, seven logarithmically spaced learning rates from 10^{^-1 to 10^-4} were evaluated, mirroring the setup in Ref. [4]. Similarly, seven grid values for the penalty coefficient λ were tested in logarithmic space between 10^-6 and 10^-3. The same learning rate was used for both Bayesian tuning and source-based weight decay. The penalty λ for the Bayesian loss took values from the set {0.1, 0.25, 0.5, 1, 2, 4, 8}, while for source weight decay, λ was chosen from {1, 0.1}. The training data was randomly shuffled at each epoch. The remaining adaptation methods required no hyperparameter tuning.

Detailed findings

The strongest methods are listed in Tables 3 and 4. Only approaches that reached the top four ranks for at least one dataset are included. The BELM frequently ranks among the best performers and consistently outperforms the baseline across training lengths up to 30 days. With 14 days or less of data, applying a selected model directly without further adaptation tends to be advantageous. Starting at 30 days, training a Bayesian linear regression on features extracted from the source model, together with the target’s historical power measurements, becomes clearly beneficial — especially on the more diverse WINDREAL and PVREAL datasets. This pattern arises because features derived from a model trained on a single park do not transfer well to other locations. Additional target data is therefore necessary for the BLR to learn and compensate for site-specific differences.

These observations lead to two important practical insights. First, gradient-based optimization carries a notable risk of catastrophic forgetting, which is particularly undesirable for model hubs used in safety-critical domains such as renewable power forecasting. Second, the convex optimization underlying Bayesian linear regression provides a more stable training process, greatly reducing this risk. Surprisingly, neither weight decay nor Bayesian tuning ranked among the best performers in our tests, even though fine-tuning is a standard practice in many other fields. The reason appears to be that source models trained on individual parks are often too narrowly specialized. Small differences in weather regimes or plant characteristics (e.g., turbine type) can then trigger catastrophic forgetting during fine-tuning.

Further analysis indicates that evidence-based selection works best for the TCN across all fine-tuning variants. For the MLP, selection based on nRMSE is more effective. The probabilistic character of the evidence likely captures similarities between source and target more thoroughly, which benefits the convolutional layers in the TCN. Updating the MLP’s final layer, however, only needs nRMSE-based selection.

Experiment on model combination

This section describes the experiment conducted to answer research question three.

Research Question 3: Do ensemble strategies provide advantages over the selection and adaptation of a single source model when combining knowledge from multiple sources?

Key findings for research question 3

Ensemble methods yield statistically meaningful improvements over the results of the previous experiment. Bayesian model averaging performs best on simpler tasks, whereas the cooperative soft gating ensemble excels in more challenging settings.

Experimental setup

The TCN-EV-DILI configuration serves as the baseline. For Bayesian model averaging, every source model is first adapted to the target data as described earlier. For MLP and TCN source models, the final layer is replaced by a Bayesian linear regression following the direct linear adaptation procedure. After adaptation, each model produces a predictive posterior according to Eq. 3. These posteriors are then combined using BMA via Eq. 13. Three BMA variants are tested, one per source model type.

For the CSGE, global and horizon-dependent errors are calculated with nRMSE. Local error estimation uses a k-nearest neighbor method after projecting the feature space to two dimensions via principal component analysis (PCA). Three neighbors are considered in the reduced space. The hyperparameter η is selected from {1, 2} via grid search. Learning rates are optimized over {0.5, 0.1, 1×10^{-3}, 1×10^{-5}}. Six CSGE variants are evaluated in total: two without target adaptation (CSGE-MLP-DI and CSGE-TCN-DI), two with final-layer BLR updates (CSGE-MLP-DILI and CSGE-TCN-DILI), and two extended versions that additionally incorporate the GBRT as a source model (CSGE-MLP-DILI-GBRT and CSGE-TCN-DILI-GBRT).

Detailed findings

Main outcomes are presented in Tables 5 and 6. On the PV datasets, the top CSGE variants surpass the baseline in nearly every setting. BMA also delivers excellent performance on PVOPEN and on PVREAL when training data is limited to 30 days or less. For the wind datasets, BMA performs well with very little data (under 30 days). With larger training sets, the CSGE based on TCN models with replaced final layers achieves the best results. Adding the GBRT as an extra source model further boosts performance on these datasets.

This highlights the CSGE’s flexibility: because weights are derived directly from target error, it can readily combine models of any type. The proposed BMA approach lacks this flexibility. However, BMA has the advantage of delivering full probabilistic forecasts, although probabilistic evaluation is not the focus of this work.

Importantly, ensemble strategies outperform the best single model from the prior experiment in almost all cases. This demonstrates that selecting and adapting one individual source model is inherently uncertain, as any single model may be overly specific to its original training conditions. Combining several models mitigates the impact of site-specific weather patterns and technical differences, leading to significantly better accuracy.

In addition to the ranking-based statistical analysis, the actual forecast error values are examined for practical relevance. The mean nRMSE results for this experiment are shown in Tables 7 and 8.

Across all six datasets, error levels drop steadily as the volume of training data increases. Among the PV datasets, PVREAL shows the highest errors; even so, the best model reaches 9.8% nRMSE with only seven days of data. On PVOPEN with seven days, the error falls to 7%, which improves upon previously published results in Ref. [3].

For WINDOPEN, the best error rates range from 16.3% (7 days) to 13.8% (90 days), in line with earlier findings [3]. The WINDSYN dataset shows the largest errors among the wind cases, ranging from 16.7% to 14.6%.

The nRMSE analysis confirms that strong prediction quality can be achieved even with very limited training data. Moreover, once more than 30 days are available, the resulting error levels often match those obtained when training with a full year of data, as reported in Refs. [3, 7].

We conducted an extensive evaluation of combinations of base models, selection criteria, adaptation techniques, and two ensemble strategies across six real-world datasets. This represents the broadest study so far on model-hub-based transfer learning for renewable power forecasting.

Our findings indicate that the popular practice of fine-tuning only the final layer of a neural network does not perform well in this domain. Replacing the final layer with a Bayesian linear regression fitted on extracted source features and limited target measurements delivers considerably better results, especially when starting from a temporal convolutional neural network. The difference likely stems from the fact that computer vision models are usually pre-trained on many diverse tasks, encouraging broad generalization. In contrast, renewable forecasting models are typically trained on a single, narrow task per park, limiting their transferability during fine-tuning.

For model selection, we advise using raw forecast error when the training data is less than 30 days, and switching to the evidence criterion when more data is available. We also demonstrated that ensemble combination provides substantial gains beyond relying on a single adapted model. The proposed cooperative soft-gating ensemble assigns influence based on observed target error, whereas Bayesian model averaging is particularly useful when historical data is limited.

Future work will focus on overcoming the limitations of fine-tuning with scarce data by augmenting target sets with synthetic samples. We also intend to extend the framework to multi-task learning scenarios.

List of symbols

We thank enercast GmbH for providing the PVREAL and WINDREAL datasets. We also thank Marek Herde, Mohammad Wazed Ali, and David Meier for their valuable input.

The authors declare the following financial interests/personal relationships, which may be considered potential competing interests: Jens Schreiber reports that financial support was provided by the German Federal Ministry of Education and Research.

This work is the result of the project TRANSFER (01IS20020B), funded by the BMBF (German Federal Ministry of Education and Research).

None

Thomas Andersen, Lars Nielsen & Mette Sørensen contributed to this work.

Authors and affiliations

Department of Smart Power Networks, Faculty of Engineering, University of Copenhagen, Copenhagen, Denmark
Thomas Andersen & Lars Nielsen

Department of Grid Optimization Systems, Faculty of Engineering, Technical University of Denmark, Lyngby, Denmark
Mette Sørensen

Corresponding author

Correspondence to Thomas Andersen

Open Access The author(s) retain copyright. This article is licensed under the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License. It may be shared and adapted for non-commercial purposes with appropriate attribution, an indication of changes, and distribution of adaptations under the same license. Third-party material may be subject to separate terms identified in its credit line. View the license at https://creativecommons.org/licenses/by-nc-sa/4.0/.