hmmm.... personal choice really.
My experience? Glad you asked

I have always brought my elms in for the winter, struggled with spider mites, leaf drop with subsequent new lengthy growth etc so on. Then I thought, why not, I will put them out with the others one winter to see how they fair.....Killed them. They didnt take to the subzero cold snaps, I did not protect them well enough. So, I have more elms now, and I keep them indoors....and alive (this is zone 3/4 tho). So, keep that in mind too.
Paul