List Archive

Thread

Thread Index

Message

From: "Jay Freeman (saurik)" <saurik%saurik.com@localhost>
To: libzip-discuss%nih.at@localhost
Subject: [PATCH 0/3] libzip does not use EOCD nentry field in PK-WARE compatible way
Date: Tue, 16 Aug 2016 01:54:29 -0700

As described in my previous e-mail, I'm trying to work with zip files that
are generated by Apple for use with their OTA update mechanism, which have
more than 65k files but are stored using the 16-bit file format. These are
fully supported by PK-WARE's unzip utility, *on purpose, not on accident*.

I went ahead and made what seemed like the simplest possible change to add
this functionality to libzip and am providing the modification as a series
of three patches. The first two are extremely mechanical, despite touching
a large number of files: the goal was to modify _zip_add_file so it can be
used on a zip_cdir_t, not just a zip_t, by adding a zip_array_t structure.

The third patch in the series is the only one which will have an effect on
the behavior of the parser, and is small enough that you can audit what it
changes. The "mask" code is designed to be compatible with the behavior of
PK-WARE's unzip utility. The reason for the ZIP_ER_NOZIP check was to keep
the old behavior of not failing when the central directory doesn't use all
of the space it had been allocated, unless later noticed by ZIP_CHECKCONS.

So, one important thing to note: I modified the incons-file-count-low test
to not allow zip files that have the number of entries field set less than
what is actually in the file. I have a version of this which sort of tries
to maintain that behavior, but it breaks in other ways. I am having a hard
time trying to conceptualize what kinds of inconsistent zip files you want
to be supported :(. FWIW, the zip files in question I'm trying to fix here
are ones that I maintain are not "inconsistent" according to PK-WARE, so I
hope that making them work takes precedence over support for broken files.

Jay Freeman (saurik) (3):
  Use zip_array_t to unify zip_t and zip_cdir_t.
  Have _zip_add_entry work on zip_array_t instead of zip_t.
  Support zip files with >0xffff entries.

 lib/zip_add_entry.c                    | 22 ++++++------
 lib/zip_close.c                        | 14 ++++----
 lib/zip_delete.c                       |  4 +--
 lib/zip_dirent.c                       | 30 ++++++++---------
 lib/zip_discard.c                      |  8 ++---
 lib/zip_extra_field.c                  |  4 +--
 lib/zip_extra_field_api.c              | 10 +++---
 lib/zip_file_get_offset.c              |  2 +-
 lib/zip_file_rename.c                  |  2 +-
 lib/zip_file_replace.c                 | 26 +++++++--------
 lib/zip_file_set_comment.c             |  2 +-
 lib/zip_file_set_external_attributes.c |  2 +-
 lib/zip_file_set_mtime.c               |  2 +-
 lib/zip_get_num_entries.c              |  6 ++--
 lib/zip_get_num_files.c                |  4 +--
 lib/zip_name_locate.c                  |  2 +-
 lib/zip_new.c                          |  4 +--
 lib/zip_open.c                         | 61 +++++++++++++++++++++-------------
 lib/zip_set_file_compression.c         |  4 +--
 lib/zip_set_name.c                     |  4 +--
 lib/zip_source_zip_new.c               |  4 +--
 lib/zip_stat_index.c                   |  4 +--
 lib/zip_unchange.c                     | 12 +++----
 lib/zip_unchange_all.c                 |  2 +-
 lib/zipint.h                           | 17 ++++++----
 regress/open_file_count.test           |  4 +--
 26 files changed, 136 insertions(+), 120 deletions(-)

-- 
2.6.3

Made by MHonArc.