After a tone of ready and messing around we have working AES 256 or AESV3 support. The PDF 3200 ISO has a supplement document that explains how to handle the AESV3 encryption where R = 5. I should now that for PDF 2.0 or ISO 3200-2 there a a new R=6 which once again defines an alternative method for calculating the AES 256 document key. This new method has yet to be publicly released as far as I can tell and the ISO is still in review.
In order to support AES256 Java Cryptography Extension (JCE) Unlimited Strength Jurisdiction Policy Files have to be installed on the JDK running ICEpdf. The files can be downloaded here, http://www.oracle.com/technetwork/java/javase/downloads/jce-6-download-429243.html
This seems so 1996 but there is a potential work around if needed. The bouncy castle crypt library support the strong encryption out of the box. In the future we could consider moving back to this library as we once used it the 2.x days on JDK 1.3. There aren't a lot of files out there using such strong encryption but I'll wait for community feedback.