In general, AT&T will keep the message on their servers for 72 hours and will keep trying to send it to your phone. After 72 hours, the system "gives up" and the message is deleted. The sender likely will not receive any sort of notification that the message did not go through.
I found the following information in the AT&T developer documents:
"The AT&T Wireless network retries deliveries of SMS messages to the AT&T Wireless customer for up to 72 hours; after which time any undeliverable messages are deleted from the AT&T Wireless network."