I'm a 4 handicap golfer myself and grew up in a family heavily involved in golf (Father and brother played in college, Grandfather had a stint on the pro tour, and my uncle made golf his living as a head pro at a course.)
I've heard about 100 ways to ruin your swing over the years and here are a few:
Playing Baseball, tennis, lacrosse, badminton, ect
Practicing other "Stick sports" obviously will effect your swing and I can't recommend playing these religiously if you want to take golf seriously.
Hobbies like skiing, weightlifting, and construction work supposedly affect your posture. I haven't experienced any evidence of this since I lift weights and water ski.
Reading golf magazines and watching the golf channel put way too many swing images, motifs, ideas, and corrections into your head that will ultimately take you far away from improvement. This is the one I think hurts players (especially new players) swings THE MOST. Each person has a distinct swing and there are a ton of moving parts in golf that can never fully be discussed in an article or show. These mediums will have you second guessing yourself a ton. Self correcting a golf swing using info meant to be read by millions of golfers with millions of problems and differences is ridiculous. If you want to be a good golfer you need to have someone (who knows what he's talking about) watching your swing and helping you. Even tiger woods has an instructor and he's the best ever.
as for the WII game:
I doubt it will affect your swing. I've played it and you really dont need to make a real golf swing to do well. You dont shift your weight and turn like the other sports i have mentioned so you wont experience and adverse muscle memory. Its more like a flick of the wrist then and athletic move.
Play on.