How to read a gzip netcdf file in python? -
i have working python program reads in number of large netcdf files using dataset command netcdf4 module. here snippet of relevant parts:
from netcdf4 import dataset import glob infile_root = 'start_of_file_name_' infile in sorted(glob.iglob(infile_root + '*')): ncin = dataset(infile,'r') ncin.close()
i want modify read in netcdf files gzipped. files gzipped after creation; not internally compressed (i.e., files *.nc.gz). if reading in gzipped text files, command be:
from netcdf4 import dataset import glob import gzip infile_root = 'start_of_file_name_' infile in sorted(glob.iglob(infile_root + '*.gz')): f = gzip.open(infile, 'rb') file_content = f.read() f.close()
after googling around maybe half hour , reading through netcdf4 documentation, way can come netcdf files is:
from netcdf4 import dataset import glob import os infile_root = 'start_of_file_name_' infile in sorted(glob.iglob(infile_root + '*.gz')): os.system('gzip -d ' + infile) ncin = dataset(infile[:-3],'r') ncin.close() os.system('gzip ' + infile[:-3])
is possible read gzip files dataset command directly? or without otherwise calling gzip through os?
because netcdf4-python wraps c netcdf4 library, you're out of luck far using gzip module pass in file-like object. option is, suggested @tdelaney, use gzip extract temporary file.
if happen have control on creation of these files, netcdf version 4 files support zlib compression internally, using gzip superfluous. might worth converting files version 3 version 4 if need repeatedly process these files.
Comments
Post a Comment