Your best bet is to do this in the OpenStudio application vs. the plugin. This page describes construction sets. Just drag a construction from your model or the library into the "Exterior Surface Construction" "Walls" drop zone.

Make sure the construction set you choose is used. If you want to apply it to the entire building go to the "Facility Tab" select the building object, and drag the construction set into the construction set drop zone.