"I think any man in business would be foolish to fool around with his secretary. If it's somebody else's secretary, fine."