I was so concerned about the mildew smell on the breast pump kit parts (thanks for those who responded with good ideas to solve the problem), I actually called the company who makes them: Medela. The conversation went something like this:

Me: I have one of your kits for the symphony, and the membrane and
protective cap smell like mildew. I have sanitized them twice according
to the directions, and they still smell.

Medela Lady: You said you had the kit for the symphony model, was that
the one with the Lactina parts as well.

Me: Yes that is correct.

Medela Lady: That smell is from the rubber seals on the Lactina part.
It is normal and will go away over time.

Me: Oh...

Medela Lady: Yes we get this call a lot.

So I guess I never had a reason to be concerned, but I do find it odd that they would sell a part which smells like mildew when it is new.