Zsync HTTP-based File Transfer Utility Transfers Large Files Efficiently

Zsync is an opensource file transfer utility built on top of rsync algorithm. This helps to download partial/differential files over the HTTP protocol. The utility allows downloading only new parts of a file from a centralized location,  where the older version of the file is already within your computer.

While rsync is for syncing data from one computer to another,  zsync allows file distribution, where the file hosted in a server using any web server can be distributed to many and downloaded seamlessly.

How it works

The command-line utility will do all the differential calculations in the client, instead of doing it in the server as in rsync. Server metadata will be created only once and stored as part of the control file. And rest of the operations and decision making will be handled by the client-side application. This will reduce the huge processing needed on the server-side, even when thousands of clients are trying to fetch the file.

Server

The server will calculate the checksum and create the control file in advance. This contains checksums,  block size of the file, length and file permissions. Since the rest of the comparisons happens at the client-side, there aren’t any checksum calculations happening on the server, and each time the client requests for data.

Client

The Zsync client is written in “C”, and capable of pulling the metadata from the server first. This will compare with the current files checksum data (generated by rsync) and requests the remaining data from the server.

How to Create .zsync files

There are multiple ways to create and transfer files using Zsync –

• Transfer Compressed Files  (gzip, tar, etc)

By running the command “zsyncmake” on the target file, the utility will automatically look into the compressed file and produce a .zsync file for the content. This is one the major differentiation with “rsync”, which doesn’t support compressed files. If you want to share compressed individual files such as rpm, deb, etc. may not help to utilize the capabilities of zsync. At the same time, it will be useful for ISO files, which contain multiple types of files and for each release some changes may happen.

• Compress and Transfer

Zsyncmake is capable of compressing the file in gzip format before sharing the files. And this compression approach will the best, better than using any standard compression tool, since this may offer better transfer efficiency.

The above command will create a gzipped version of the file, a .zsync file, and a URL by assuming that these two files will be served from the same document root of a  web server.

Zsync Example – Debian/Ubuntu ISO Downloads

One of the best use cases of zsync is the distribution of Ubuntu/Debian iso images as .zsync downloads. You can choose Ubuntu iso .zsync from the below link, and the tool will download only the new changes in the file, no need to download the whole ISO image.

Download using zsync

If you are downloading this file for the first time,  you will be able to see the messages as below and starts the download –

References

• Official Website 
• Technical Paper by  Colin Phipps, the developer
• Man page for Zsync
• Man page for Zsyncmake

Siji Sunny

An Entrepreneur, an Opensource Enthusiast  and Researcher in the domain of Embedded Systems, Wireless and IoT – Has over 16+ years of experience in managing and contributing enterprise Research Projects, in Embedded Systems, Software Technologies, Product Conceptualizations and development, Telecommunication, Media and Entertainment and Consumer Electronics.

Support CNX Software! Donate via cryptocurrencies, become a Patron on Patreon, or purchase goods on Amazon or Aliexpress