This was asked recently in the thread:
http://www.swingingheaven.co.uk/swingers-forum/viewtopic/19577.html
but to save you the trouble of looking it up I'll post the answer again:
The terms “swinging†and “the swinging lifestyle†were coined by a minister during the 19th century in order to label individuals who chose to have sex with other individuals to whom they were not married. The minister was trying compare these consenting adults to monkeys, swinging from bed to bed as a monkey might swing from tree to tree. Little did he know that soon the term would be reclaimed by the swingers themselves!