Recent advancements have sparked growing interest in exploiting collections of pre-trained models, known as model hubs, particularly within computer vision applications. In practice, this involves first identifying a suitable pre-trained source model and then tailoring it to the new target domain to bridge any mismatches. Despite this trend, investigations into optimal model choice and tailoring methods for renewable energy generation forecasts remain scarce. Specifically, prior research has not delved into varied selection or adaptation techniques tailored to different neural network structures. Furthermore, no existing analyses have systematically evaluated how the volume of available training data affects outcomes, nor have they incorporated seasonal variations into performance assessments. We bridge these research gaps through an initial in-depth study of model selection and adaptation within transfer learning frameworks for renewable power prediction. Inspired by cutting-edge advances in computer vision, the analysis draws on operational data from 667 wind farms and solar (photovoltaic) installations across six datasets. To generate robust error metrics, we simulate scenarios with varying volumes of training observations across seasons. Model selection is then appraised using both marginal likelihood estimates and resulting forecast inaccuracies under these constrained data regimes. In addition, four separate adaptation methodologies are rigorously compared. Building on and surpassing conventional approaches, we introduce Bayesian linear regression to model the output variable from deep features derived by a neural network backbone. This technique delivers superior results compared to standard baselines, even with only 7 days of local training data. It also shows that full network fine-tuning provides negligible gains unless at least 3 months of data are available. Beyond that, we demonstrate that aggregating predictions from multiple models into ensembles markedly improves the effectiveness of both selection and adaptation. Consequently, ensemble-based strategies achieve mean error levels with only 30 days of training data that would otherwise require a full year of observations. In the most representative datasets, this yields average errors of 9.8% for photovoltaic power and 14% for wind power, utilizing only seven days of site-specific training data.