Agreed but we need to remember that most of the "rubber" in the blowers are actually not rubber but some type of elastomer (i.e. a plastic). Any elastomer can deteriorate and silicone is more robust in its ability maintain flexibility and not break down as it ages, so it is definitely a superior material. With those properties, it is also a much more expensive material.
That does not mean other materials in blower bulbs are necessarily inferior, but they should be replaced if there is any visible deterioration of the surface of the bulb itself. The outside surface is more likely to be exposed to UV and chemicals that cause the elastomer to break down, versus the interior that is not exposed to those conditions. If the bulb shows signs of breakdown (a gritty feeling) then it is time to replace it.
Mine is several years old and I have do not see any signs of the material breaking down yet. When I do, there will be a replacement in my camera bag quite quickly.
All that being said, I have yet to see any contaminants introduced into the camera body by my cleaning the sensor with a bulb-type blower. The "dust" has to adhere to the sensor to be a problem and so far as I know there is an adhesive strip below the sensor to capture any random dust and dirt that falls off the sensor. The biggest problem I have found to date is spring, when there is a lot of pollen in the air. Pollen is quite sticky and adheres to anything including the sensor stack.