How to Compress and Extract Multipart Zip Files on Linux

This blog post explains how to create multipart zip files and then extract them in another host which runs on Linux, in case that the single zip file is too big to transport from one host to another. I will demonstrate this on CentOS host, other distributions will be similar, apart from installation command.

1. Firstly, you will need to install p7zip utility:

sudo yum -y install p7zip

2. Then, you can create archive file using below command:

7za -v100m a test.zip test

The above command tells 7za to create files with volume of 100MB each, the archive file name is test.zip and the source of the directory is “test”, which contains the files that you need to create archive from.

My sample output looks like below:

[root@localhost ~]# 7za -v100m a test.zip test

7-Zip (a) [64] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21
p7zip Version 16.02 (locale=en_AU.UTF-8,Utf16=on,HugeFiles=on,64 bits,4 CPUs Intel(R) Xeon(R) CPU E5-2686 v4 @ 2.30GHz (406F1),ASM,AES-NI)

Scanning the drive:
1500 folders, 7363 files, 1347676548 bytes (1286 MiB)

Creating archive: test.zip

Items to compress: 8863


Files read from disk: 7363
Archive size: 473633025 bytes (452 MiB)
Everything is Ok

3. The following files will be created under the current directory:

-rw-r--r-- 1 root root 104857600 Sep 16 20:26 test.zip.001
-rw-r--r-- 1 root root 104857600 Sep 16 20:26 test.zip.002
-rw-r--r-- 1 root root 104857600 Sep 16 20:26 test.zip.003
-rw-r--r-- 1 root root 104857600 Sep 16 20:27 test.zip.004
-rw-r--r-- 1 root root  54202625 Sep 16 20:27 test.zip.005

4. After all the files being transported to the destination host, you can run below command to unzip those files:

7za x test.zip.001

All you need to specify is the first splitted file with “.001” extension and 7za will manage to find the rest. My sample output looks like below:

[root@localhost ~]# 7za x test.zip.001

7-Zip (a) [64] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21
p7zip Version 16.02 (locale=en_AU.UTF-8,Utf16=on,HugeFiles=on,64 bits,4 CPUs Intel(R) Xeon(R) CPU E5-2686 v4 @ 2.30GHz (406F1),ASM,AES-NI)

Scanning the drive for archives:
1 file, 209715200 bytes (200 MiB)

Extracting archive: test.zip.001
--
Path = test.zip.001
Type = Split
Physical Size = 209715200
Volumes = 3
Total Physical Size = 473633025
----
Path = test.zip
Size = 473633025
--
Path = test.zip
Type = zip
Physical Size = 473633025

Everything is Ok

Folders: 1500
Files: 7363
Size:       1347676548
Compressed: 473633025

Hope above helps.

ZIP Incompatibility Issue: Need PK compat. v4.5 (can do v2.1)

Today I downloaded a zip file from one of our clients at work and tried to unzip the archive file, however, I got the following error:

$ unzip test.zip
Archive:  test.zip
   skipping: test.csv  need PK compat. v4.5 (can do v2.1)

To solve this issue, I had to install p7zip.

On Red Hat/CentOS, use yum:

$ sudo yum install p7zip

On Ubuntu, use apt-get:

$ sudo apt-get install p7zip

And then use command 7za to extract the file:

$ 7za e test.zip
7-Zip (A) [64] 9.20  Copyright (c) 1999-2010 Igor Pavlov  2010-11-18
p7zip Version 9.20 (locale=en_US.UTF-8,Utf16=on,HugeFiles=on,2 CPUs)

Processing archive: test.zip

Extracting  test.csv

Everything is Ok

Size:       56433
Compressed: 7529

Create ZIP file using PHP

When developing web applications, it is very common that you will encounter to deal with a list of different file formats. I have been doing web development for about one and half years, the file formats that I have encountered are pure TXT file, CSV, XML, INI, and different types of image file formats, like GIF, PNG and JPG. The common tasks to deal with those file formats are basically reading and writing to them. Since they are so common for day to day web development, it is not surprised that PHP comes with a wide variety of both built-in functions and external libraries to connect to, and work with, almost any file format you can name.

I haven’t encountered this situation before, but let’s think about one scenario. You need to write a script to transfer a large amount of files your script created to a remote server. It is simple, right? All you need to do is to create a SSH key to allow script to remotely login to the server and upload the files one by one. But the problem is that you are dealing with large amout of files, and it is not efficient to copy file one by one. The ideal solution is to zip all the files into one zip file, transfer it across to the server, and unzip from the server. Can PHP create a zip file for you? Luckily we have PEAR library available, and it can just do the job for us in a few lines of code.

Here is what you need to do.
[ul]
[li]Download the PEAR base system from here[/li]
[li]Download the PEAR Archive_Zip package from here[/li]
[/ul]
Unzip the files to the appropriate location, and change your php include_path accordingly. Now you are ready to start the action.

Creating ZIP archives

This is your script needs to do, get a list of file names, and create a zip file.

create( $filesList ) )
{
[tab]echo "Zip file created successfully!";
}
else
{
[tab]echo "Unable to create zip file";
}
?>

Now once you have copied the zip file created across to the server machine, you can create a separate script to unzip the file.

listContent();

foreach ( $filesList as $file )
{
[tab]foreach ( $file as $key => $value )
[tab]{
[tab][tab]echo "$key: $value<br />";
[tab]}
[tab]echo "<br />";
}
?>

The output of listContent() is a structured array of arrays, with each array element representing an individual file from the archive. Typically, each element holds information on the name of the corresponding file, its index position, its status, its size (both compressed and uncompressed) and its time of last modification. It’s fairly easy to extract this information with a loop and re-format it to make it more presentable.

As the examples above illustrate, PEAR’s Archive_Zip class is quite versatile and allows you to perform some fairly complex ZIP file interaction with just a few lines of code. Hopefully, the sample scripts above sparked some ideas about how you can use this class in your daily development activities and got you interested in experimenting with it. Have fun!