Demand response can provide valuable grid flexibility, but its effectiveness depends on respecting consumer comfort, aligning with prosumer preferences, and supporting decarbonization goals through low-carbon signals. Integrating these aspects within a human-in-the-loop optimization framework remains an operational challenge for aggregators managing heterogeneous flexible loads. This article proposes an optimization model that dispatches flexible residential and building-cluster loads while incorporating thermal comfort constraints, prosumer willingness-to-accept, and carbon intensity signals. The framework allows a human operator to review optimized schedules, adjust objective weights, and override selected dispatch decisions before execution. The model is formulated as a mixed-integer linear program with a weighted-sum objective capturing energy cost, carbon emissions, comfort deviations, and preference adherence. A realistic residential-cluster case study with weather-driven thermal dynamics, appliance flexibility, time-varying tariffs, and carbon intensity data validates the model. The proposed dispatch reduces feeder peak demand by 18.6% and carbon emissions by 11.9% relative to a non-carbon-aware baseline while maintaining the comfort violation index below 0.035 degree-hours per occupied interval. Operator overrides occur in 6.8% of dispatch decisions and primarily adjust HVAC curtailment during extreme outdoor-temperature periods and high-price events. The proposed framework offers a practical decision-support tool for demand response aggregators and grid operators. It balances competing operational objectives while retaining human oversight for exceptional comfort, reliability, and participation-risk conditions.