Yeah, that's pretty much a question you known the answer to: no, and not many of them ever did. But the cliche came from somewhere, and Slate explains.
Before Communists, the first “card-carrying” groups tended to be members of labor unions, and often members used the cards openly to claim union jobs. Perhaps the earliest mention of “card-carrying” union members came in a 1912 Daily People article, which described how “ ‘Union-card’ carrying members” of certain labor unions were occasionally able to get elected to Congress. A 1918 New York Tribune article described members of the Industrial Workers of the World as “red-card-carrying ‘wobblies,’ ” referring to the members by their popular nickname and red membership cards. In 1927, all “card-carrying members” of the secret organization United Farmers reportedly received discounts from local merchants.
Read the rest of the article to learn how the phrase became linked to Communists of the Red Scare. Link -via mental_floss