How to Compress and Extract Multipart Zip Files on Linux

This blog post explains how to create multipart zip files and then extract them in another host which runs on Linux, in case that the single zip file is too big to transport from one host to another. I will demonstrate this on CentOS host, other distributions will be similar, apart from installation command.

1. Firstly, you will need to install p7zip utility:

sudo yum -y install p7zip

2. Then, you can create archive file using below command:

7za -v100m a test.zip test

The above command tells 7za to create files with volume of 100MB each, the archive file name is test.zip and the source of the directory is “test”, which contains the files that you need to create archive from.

My sample output looks like below:

[root@localhost ~]# 7za -v100m a test.zip test

7-Zip (a) [64] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21
p7zip Version 16.02 (locale=en_AU.UTF-8,Utf16=on,HugeFiles=on,64 bits,4 CPUs Intel(R) Xeon(R) CPU E5-2686 v4 @ 2.30GHz (406F1),ASM,AES-NI)

Scanning the drive:
1500 folders, 7363 files, 1347676548 bytes (1286 MiB)

Creating archive: test.zip

Items to compress: 8863


Files read from disk: 7363
Archive size: 473633025 bytes (452 MiB)
Everything is Ok

3. The following files will be created under the current directory:

-rw-r--r-- 1 root root 104857600 Sep 16 20:26 test.zip.001
-rw-r--r-- 1 root root 104857600 Sep 16 20:26 test.zip.002
-rw-r--r-- 1 root root 104857600 Sep 16 20:26 test.zip.003
-rw-r--r-- 1 root root 104857600 Sep 16 20:27 test.zip.004
-rw-r--r-- 1 root root  54202625 Sep 16 20:27 test.zip.005

4. After all the files being transported to the destination host, you can run below command to unzip those files:

7za x test.zip.001

All you need to specify is the first splitted file with “.001” extension and 7za will manage to find the rest. My sample output looks like below:

[root@localhost ~]# 7za x test.zip.001

7-Zip (a) [64] 16.02 : Copyright (c) 1999-2016 Igor Pavlov : 2016-05-21
p7zip Version 16.02 (locale=en_AU.UTF-8,Utf16=on,HugeFiles=on,64 bits,4 CPUs Intel(R) Xeon(R) CPU E5-2686 v4 @ 2.30GHz (406F1),ASM,AES-NI)

Scanning the drive for archives:
1 file, 209715200 bytes (200 MiB)

Extracting archive: test.zip.001
--
Path = test.zip.001
Type = Split
Physical Size = 209715200
Volumes = 3
Total Physical Size = 473633025
----
Path = test.zip
Size = 473633025
--
Path = test.zip
Type = zip
Physical Size = 473633025

Everything is Ok

Folders: 1500
Files: 7363
Size:       1347676548
Compressed: 473633025

Hope above helps.

Recovering Windows Boot Loader in GRUB

I have two disks with two different OSes installed on each – one for Windows XP and one for Fedora 19. They work happily with each other and don’t cause any issues.

Two weeks ago, I decided to upgrade Windows XP to Windows 7. Installation worked fine, Windows 7 boots without problems (although I need to unplug the Fedora disk because Windows 7 just couldn’t allow me to proceed with the installation with two disks connected).

However, when I re-plugged the other disk back and boot from Fedora as usual, Windows won’t start if I select it from the GRUB menu (I already suspected this problem before I tried to upgrade Windows).

The simplest solution is to re-install Fedora again and the installer should be able to pick up the new Windows 7 without problems, but I will have to re-install everything under Fedora, which is not desired.

To fix this issue, I followed the following steps:

1. Find out the UUID in the Windows partition. In most cases (and in my case too), it is /dev/sda1, to be sure, you can run “gdisk” as root:

[root@localhost]# gdisk -l /dev/sda

GPT fdisk (gdisk) version 0.8.8

Partition table scan:
MBR: MBR only
BSD: not present
APM: not present
GPT: not present

Disk /dev/sda: 1953523055 sectors, 931.5 GiB
Logical sector size: 512 bytes
Disk identifier (GUID): 39402158-CA16-42CC-AC6F-2320B82B2498
Partition table holds up to 128 entries
First usable sector is 34, last usable sector is 1953523021
Partitions will be aligned on 8-sector boundaries
Total free space is 6979 sectors (3.4 MiB)

Number Start (sector) End (sector) Size Code Name
1 2048 104855551 50.0 GiB 0700 Microsoft basic data
5 104857264 167771286 30.0 GiB 0700 Microsoft basic data
6 167771583 1006633151 400.0 GiB 0700 Microsoft basic data
7 1006633215 1953520127 451.5 GiB 0700 Microsoft basic data

My Windows is installed on the 50GB partition, which is the first one.

2. Seondly, I needed to find the UUID for this partition, run

[root@localhost]# blkid /dev/sda1

/dev/sda1: UUID="0C6E54886E546C88" TYPE="ntfs"<

3. Finally I need to update the grub.cfg file with the new UUID. Open file /boot/grub2/grub.cfg, and scroll to the part that have Windows information


### BEGIN /etc/grub.d/30_os-prober ###
menuentry 'Microsoft Windows (on /dev/sda1)' --class windows --class os $menuentry_id_option 'osprober-chain-16D8D903D8D8E253' {
insmod part_msdos
insmod ntfs
set root='hd0,msdos1'
if [ x$feature_platform_search_hint = xy ]; then
search --no-floppy --fs-uuid --set=root --hint-bios=hd0,msdos1 --hint-efi=hd0,msdos1 --hint-baremetal=ahci0,msdos1 16D8D903D8D8E253
else
search --no-floppy --fs-uuid --set=root 16D8D903D8D8E253
fi
drivemap -s (hd0) ${root}
chainloader +1
}
### END /etc/grub.d/30_os-prober ###

Simply replace “16D8D903D8D8E253” with “0C6E54886E546C88”,


### BEGIN /etc/grub.d/30_os-prober ###
menuentry 'Microsoft Windows (on /dev/sda1)' --class windows --class os $menuentry_id_option 'osprober-chain-0C6E54886E546C88' {
insmod part_msdos
insmod ntfs
set root='hd0,msdos1'
if [ x$feature_platform_search_hint = xy ]; then
search --no-floppy --fs-uuid --set=root --hint-bios=hd0,msdos1 --hint-efi=hd0,msdos1 --hint-baremetal=ahci0,msdos1 0C6E54886E546C88
else
search --no-floppy --fs-uuid --set=root 0C6E54886E546C88
fi
drivemap -s (hd0) ${root}
chainloader +1
}
### END /etc/grub.d/30_os-prober ###

then reboot your computer.

Now you should be able to select the Windows entry from the GRUB menu in the boot up.

How to set the timezone on Ubuntu Server

I recently bought a new VPS from a hosting company in New York and I installed Ubuntu 13.04 on the new machine. As expected, it chose Americ/New_York as the default time zone for the new server. To change it, it is as easy as run the following command:


sudo dpkg-reconfigure tzdata

And the just follow the prompts to choose your currently region and city.

If you want your cron jobs to be using the updated time zone, you will also need to restart the cron demaeon.


/etc/init.d/cron restart

And syslog


sudo service rsyslog restart

Or maybe just reboot the server to make sure everything is in synced.

Too easy…

File Size Limitations For ZIP

I have just learned that in order to use ZIP, both the archive file as well as the individual files inside the ZIP file should have the size of less than 4GBs, if it is not the case, you will get the segmentation fault error.

So if you have much bigger files that need to be archived, it is recommended to use tar ball and then GZ it.

tar czvf archive.tar.gz archive1 archive2 archive3 

Both TAR and GZ should work fine so long as your OS’ filesystem supports very large files. In my test I can tar and then gz 4 6GB individual files without problems, although it took quite a while.

VirtualBox Shared Folders with CentOS Server Guest

I have recently switched from MacBook Pro to Mac Mini for my development machine and I have also switched from VMWare to VirtualBox simply for the sake of OpenSource. Because the new Mac Mini gives me 16GB of DDR3 RAM, which gives me the power to run my own virtual server machine as my sandbox to develop on. To make my life a bit easier, I want to share my coding folder on Mac Mini with the server in the virtual machine, so that I don’t need to keep syncing files manually. Luckily VirtualBox supports this, but need a bit of work to get it going.

In this article I will be creating a shared folder within VirtualBox that will link my /Users/ericlin/Projects directory with one on the CentOS box under /mnt/projects.

Firstly, within VirtualBox, select the guest machine you wish to contain the shared folder.

Press the Settings button and choose the folder that you want to share on your host machine, and give appropriate access level:

Start your guest machine and login as normal.

Select Devices -> Install Guest Additions, this will insert an iso image into the CDROM, but we need to mount it manually:

sudo mkdir /media/cdrom
sudo mount /dev/cdrom /media/cdrom
mount: block device /dev/sr0 is write-protected, mounting read-only

Now if you do “ls” on /media/cdrom, you will see the following contents:

Run file “VBoxLinuxAdditions.run” to install the addition:

sudo /media/cdrom/VBoxLinuxAdditions.run

Running it the first time I got the following error:


To fix it, simply do what it asks you to do:

yum install kernel-devel-2.6.32-279.el6.x86_64

Wait for it to finish and try again with previous command:

sudo /media/cdrom/VBoxLinuxAdditions.run

Don’t worry about the FAILED message in the last line, we are running the server version, not the desktop version, so no Window System is required. Once everything is finished, reboot the virtual machine:

sudo reboot

Lastly, you just need to mount the folder. I created the directory that is going to be mounted to: /mnt/projects

sudo mkdir /mnt/projects

Now you just need to mount the shared folder to the newly created directory using the following command.

sudo mount –t vboxsf /Users/ericlin/Projects /mnt/projects

Then you should be able to

cd /mnt/projects
ls -sh

If you want the foler to be mounted automatically on start up, add the following line to your /etc/fstab file:

Projects        /home/ericlin/em    vboxsf    defaults    0 0