We are returning to Telluride for the third year in a row. I wholeheartedly agree with those who recommend the town, particularly since your wife, like mine, isn't into skiing all that much. In town, my wife likes the restaurants, the shops, the bakery, the coffee shops, the theaters (and I don't mean movie theater). Last year, we went to a performance of Pippin. The year before, we went to a concert (not a partying concert).
The town is not a party town. Comparing to the East, I'd describe the town, in feel, to maybe Portsmouth NH? MV would be like a very large version of Stowe's Spruce Peak development, everything there, but sterile.
I agree with the earlier recommendation of staying at the Oak Street/Gondola area, and second the nomination of Camels Garden. When I ski with my wife, we go up the gondola, ski blues and easy blacks, meandering around the mountain, lunching either at MV or one of the restaurants on the mountain.
You, your wife, and your kids will appreciate the variety of options that staying in town will afford.