"But I confess it is an alien notion to me."

Actually it's alien to his whole family. Tony doesn't have it quite so bad, but he still thinks of relaxing as what you do in between the fun stuff. And Jarvis, of course, is only ever one-third sleeping at most (except for major hardware upgrades) and therefore describes himself as being at rest either all the time or none of it.