Xueming Shen's Blog
Friday Apr 17, 2009
ZIP64, The Format for > 4G Zipfile, Is Now Supported
We heard you! finally:-)
The support for ZIP64, the format for > 4G ZIP file, has finally been added into the latest OpenJDK7 build(b55). This RFE (request for enhancements) had been buried in the 200+ Jar/ZIP bug/rfe pile so deep that I was not even aware of / remembered its existence until a recent call from A custom strongly asking for it. Given everyone now has 200G+ disk space (and yes, most of my kid's video clips are now around 1G after I got that new camcorder), it is relatively easy for the Jar/ZIP user to run into this 4G ceiling these days. The RFE was quickly climbing itself to the top of my to-do list and is now in b55. From now on only the sky is the limit for your Jar/ZIP file:-)
So if you have > 4G stuff (either the total size of files zipped in > 4G or the individual files themselves are > 4G) to jar/zip, try out the OpenJDK7-b55 (and later), either via the java.util.jar/zip APIs or the Jar tool. Let me know if you find anything not working or broken, I hope it's perfect though:-)
Here is the brief background info regarding the 4G size problem in Jar and ZIP file.
(1)Various size and position offset related fields in original ZIP format are 4 bytes, so by nature ZIP has this 4G size limitation.
(2)The field for total number of files zipped/stored in ZIP's Central directory record is 2 bytes, so it has the 65536 limit (Java's ZIP file implementation has some hacky code to workaround this issue though)
(3)ZIP64 format extensions was introduced in (in spec 4.5) to address above size limitation.
(4)JDK7 now fully supports the ZIP64(tm) format extensions
defined by the PKWARE's ZIP specification
If you are interested in source code, here is my ZIP64 "to-do" list (a copy/paste note from the spec) and the code diffs.
Posted at 02:56PM Apr 17, 2009 by xuemingshen in Java | Comments[7]
Hi,
Nice to see one more arbitrary limit removed. :)
Best,
Ismael
Posted by Ismael Juma on April 17, 2009 at 03:45 PM PDT #
Time to update the Wikipedia entry [1] - "while others, such as Java's built-in java.util.zip, still lack it"
[1] http://en.wikipedia.org/wiki/ZIP_(file_format)
Posted by Kirill Grouchnikov on April 27, 2009 at 03:25 PM PDT #
Hi,
I downloaded jdk-7-ea-bin-b59-windows-i586-14_May_2009.exe used the zip api to compress a >4 GB file. But it did not work. Do i need to make any other changes (in the code or setting) to make it work?
Jackson.
Posted by Jackson on May 20, 2009 at 09:58 PM PDT #
Jackson, can you describe a little more about what "it did not work" means, for example an exception? Is it possible to send me your code or a snapshot of the place where you use the "zip api"? which definitely will help me to trace down the issue. One of the test case I used is at http://cr.openjdk.java.net/~sherman/4681995/webrev.02/test/java/util/zip/LargeZip.java.html. Also the jar tool is built on top of the java.util.jar/zip APIs, I also used it to test.
Posted by Xueming Shen on May 26, 2009 at 11:33 AM PDT #
Does the zip library (java.util.zip) in Java 7 automatically use the Zip64 extensions when the zip file exceeds 4GB? Does existing code need to be modified to take advantage of the new Zip64 format?
Posted by Dem on July 31, 2009 at 01:08 AM PDT #
Yes, the java.util.jar/zip will pick up the Zip64 extension "automatically. User code does not need to change.
Posted by xueming.shen on August 15, 2009 at 12:45 AM PDT #
That's great to hear. Thanks!
Posted by Dem on August 18, 2009 at 10:38 AM PDT #