How To Compress The Data That Saved In Hdf5?

I am using python 2.7 to read a video and store in hdf5. This is my code import h5py import skvideo.datasets import skvideo.io videodata = skvideo.io.vread('./v_ApplyEyeMakeup_g01_

Solution 1:

Your problem should be solved using a suitable encode for your video file. Based on your business, there are various encoding algorithms for example there is x265 which will compress the video but requires high resource to do that. Take a look here.

Recently I have heard about another interesting encode which is good for online streaming called Daala you can get more information here.

Generally it depends on what you expect from the encoding, but choosing a good encoder is the way you should go, try search for that.

Solution 2:

By adding chunking I was able to make the output 7.2M compared to 10M without. So it definitely improves, but still far from dedicated video formats. You may play with other filters from https://support.hdfgroup.org/services/filters.html but I doubt they will improve the compression by an order of magnitude. So if you want to continue with h5py, you probably need to accept larger file size. In case this is not acceptable, just try another file format.

import h5py
import skvideo.datasets
import skvideo.io
videodata = skvideo.io.vread('./v_ApplyEyeMakeup_g01_c01.avi')

print(videodata.shape)
with h5py.File('./video.hdf5','w') as f:
    f.create_dataset('data',
                      data=videodata,
                      compression='gzip',
                      compression_opts=9,
                      chunks=(164, 20, 20, 3))
    f.create_dataset('label', data=1)