The thing with Destroyers is, you have to SPAM them. In groups of at least 10 with considerable backup they pack a huge amount of hitpoints and armor which makes up for the crappy damage they do. They are really only vulnerable to hack units (mainly infantry), so build ranged units to accompany them. Go Heroic, build a Palace and:
AQ FU Murmillo
AQ FU Destroyer
AQ Arcus (kills most Destroyer counters)
(AQ Cheiro when facing massed infantry)
(AQ Turma when facing massed archers)
Heavy Myth (Prommies, Automata, Behemoth) to serve as meatshield/distraction is always nice
Charge into his forward base. Micro Destroyers on buildings, shooting ones first, and DONT let them wander off fighting troops. Use Murms and myth as general combat units, micro Arcus to kill anything dangerous to your Destroyers. As his buildings fall he'll get increasingly desperate, focusing his units on your well-armored Destroyers, so the rest of your units can kill them.
That's basically it, an attacking Attie army is a lot like a Norse infantry push. As soon as you have more units than him, he's powerless against your strong infantry killing his base and stuff.
All Hail Giant Squid World Domination