You can find a working example in rllib-energyplus. It integrates EnergyPlus python API with OpenAI Gym (actually, Gymnasium, a fork by the original authors of Gym - Gym isn't maintained anymore) and Ray RLlib.