How to stop VoiceOver from announcing the caller ID when receiving a phone call?

in settings > phone, i set announce incoming call to "never, and in settings > general > accessibility > voiceover, i set always speak notifications to off. well, vo still speaks caller's name out loud. this is not desirable in many situations, and i think this is a bug apple over looked. in the mean time, is there any way to overcome this?
i do want to know who is calling, so disable phone notification on the lock screen is not an option.