The increasing deployment of wind power plays a key role in addressing climate change. Nevertheless, the intermittent nature of wind generation often leads to curtailment, resulting in substantial financial losses for wind farm operators. Battery energy storage systems (BESS) can help alleviate curtailment by providing local backup power. However, dedicating storage primarily to this supporting function can greatly diminish its profitability in energy markets. Optimal BESS operation requires carefully balancing local curtailment mitigation with participation in market bidding, yet achieving this in practice remains difficult owing to the intricate coordination required and the uncertainty of both electricity prices and wind output. This study explores a combined bidding strategy for a co-located wind-plus-battery setup operating in the spot market and the Regulation Frequency Control Ancillary Service markets. We introduce an innovative deep reinforcement learning method that separates the overall market engagement into two interconnected Markov decision processes—one for each asset. This structure allows the BESS to capture surplus wind energy that would otherwise be curtailed while simultaneously participating in joint bidding across markets to optimize total operational income. Validation was performed using real-world wind farm data. The proposed coordinated bidding approach outperformed a conventional optimization-based reference case, delivering roughly 25% higher revenue and achieving 2.3 times greater reduction in wind curtailment. Findings demonstrate that participating jointly in multiple markets substantially enhances the economic viability of wind-battery installations relative to engaging in individual markets in isolation. Additional simulations indicate that charging the BESS with otherwise curtailed wind power yields further economic benefits. Effective deployment of the developed algorithm is expected to promote greater co-location of renewable generation and storage facilities, thereby realizing broader advantages for the power system.