Well, they are the Socialist Party... So wouldn't that be accurate?
Man, the Left are so bad at analogies...
The difference is that most on the right (and libertarians for that matter) start off talking about policy, whereas most on the left start from a position where all of the motives on the other side are based on bigotry and "fascism". It's usually after the left says that they want to nationalize health care, transportation, education, etc, that the "socialist" label comes out, but in this case it's entirely accurate.