This is actually because of something called thermal inertia. Materials such as walls and furniture absorb or release heat slowly.