Can keeping my smart phone in a shirt pocket affect my ICD?

Yes, it could. You should avoid carrying your cell phone less than 6 inches away from the device.