Crate grib

Expand description

This is a GRIB format parser library written in Rust programming language. This project aims to provide a set of library and tools which is simple-to-use, efficient, and educational.

GRIB is a concise data format commonly used in meteorology to store historical and forecast weather data. It is intended to be a container of a collection of records of 2D data. GRIB files are huge and binary and should be processed efficiently. Also, since GRIB is designed to support various grid types and data compression using parameters defined in external code tables and templates, some popular existing softwares cannot handle some GRIB data.

§Template support

GRIB2 can contain grid point values for various grid systems. This diversity is supported by a mechanism called “templates”.

Although GRIB2 contains a large number of grid point values, the coordinates and values of individual grid points are not encoded directly as numerical data. Since the grid points are regularly arranged, the coordinates can be defined by the type of projection method used for the grid system and the specific parameters for that projection method, so only a simple definition of the grid system is encoded in the data.

Also, since the best encoding method for values varies from data to data, there are multiple methods that can be used to encode values, and the method used and the specific parameters needed to encode it are defined along with the data itself.

These definitions of grid systems and data representation are represented by sequences of bytes called templates, which should be supported in order for the reader to read GRIB2 data. grib-rs supports the following templates. We would love to support other templates as well, so please let us know if there is any data that is not readable.

§Supported grid definition templates

For data using the following grid systems, latitudes and longitudes of grid points can be computed.

Template number	Grid system	Notes
3.0	latitude/longitude (or equidistant cylindrical, or Plate Carree)	supporting only regular grids
3.20	Polar stereographic projection	enabling feature `gridpoints-proj` required
3.30	Lambert conformal	enabling feature `gridpoints-proj` required
3.40	Gaussian latitude/longitude	supporting only regular grids

§Supported data representation templates

For data using the following encoding methods, grid point values can be extracted.

Template number	Encoding method	Notes
5.0	simple packing
5.2	complex packing
5.3	complex packing and spatial differencing
5.40	JPEG 2000 code stream format	enabling feature `jpeg2000-unpack-with-openjpeg` required
5.41	Portable Network Graphics (PNG)	enabling feature `png-unpack-with-png-crate` required
5.42	CCSDS recommended lossless compression	enabling feature `ccsds-unpack-with-libaec` required
5.200	run length packing with level values

§Examples

This is an example of accessing the grid point values and latitude/longitude coordinates for a specific submessage within a GRIB2 file.

use grib::{codetables::grib2::*, ForecastTime, Grib2SubmessageDecoder, Name};

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let fname = "testdata/Z__C_RJTD_20160822020000_NOWC_GPV_Ggis10km_Pphw10_FH0000-0100_grib2.bin";
    let f = std::fs::File::open(fname)?;
    let f = std::io::BufReader::new(f);
    let grib2 = grib::from_reader(f)?;

    let (_index, submessage) = grib2
        .iter()
        .find(|(_index, submessage)| {
            matches!(
                submessage.prod_def().forecast_time(),
                Some(ForecastTime {
                    unit: Name(Table4_4::Minute),
                    value: minutes,
                }) if minutes == 30
            )
        })
        .ok_or("message with FT being 30 minutes not found")?;

    let latlons = submessage.latlons()?;
    let decoder = Grib2SubmessageDecoder::from(submessage)?;
    let values = decoder.dispatch()?;

    for ((lat, lon), value) in latlons.zip(values) {
        println!("{lat} {lon} {value}");
    }

    Ok(())
}

This is an example of accessing the parameter values contained within a section and template parameters of a specific submessage in a GRIB2 file.

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let fname = "testdata/Z__C_RJTD_20160822020000_NOWC_GPV_Ggis10km_Pphw10_FH0000-0100_grib2.bin";
    let f = std::fs::File::open(fname)?;
    let f = std::io::BufReader::new(f);
    let grib2 = grib::from_reader(f)?;
    let (_index, first_submessage) = grib2.iter().next().unwrap();

    let actual = first_submessage.section5();
    let expected = Ok(grib::def::grib2::Section5 {
        header: grib::def::grib2::SectionHeader {
            len: 23,
            sect_num: 5,
        },
        payload: grib::def::grib2::Section5Payload {
            num_encoded_points: 86016,
            template_num: 200,
            template: grib::def::grib2::DataRepresentationTemplate::_5_200(
                grib::def::grib2::template::Template5_200 {
                    num_bits: 8,
                    max_val: 3,
                    max_level: 3,
                    dec: 0,
                    level_vals: vec![1, 2, 3],
                },
            ),
        },
    });
    assert_eq!(actual, expected);

    Ok(())
}

You can also dump parameters contained within submessages. Currently, this is only implemented for certain sections. Below is an example of dumping the same submessage as in the previous example.

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let fname = "testdata/Z__C_RJTD_20160822020000_NOWC_GPV_Ggis10km_Pphw10_FH0000-0100_grib2.bin";
    let f = std::fs::File::open(fname)?;
    let f = std::io::BufReader::new(f);
    let grib2 = grib::from_reader(f)?;
    let (_index, first_submessage) = grib2.iter().next().unwrap();

    // In the following example, the dump output is written to a buffer for testing purposes.
    // To dump to standard output, use the following instead:
    //
    //     let mut stream = std::io::stdout();
    //     first_submessage.dump(&mut stream)?;
    let mut buf = std::io::Cursor::new(Vec::with_capacity(1024));
    first_submessage.dump(&mut buf)?;
    let expected = "\
#  SUBMESSAGE (total_length = 10321)
##  SECTION 0: INDICATOR SECTION (length = 16)
##  SECTION 1: IDENTIFICATION SECTION (length = 21)
1-4       header.len = 21  // Length of section in octets (nn).
5         header.sect_num = 1  // Number of section.
6-7       payload.centre_id = 34  // Identification of originating/generating centre (see Common Code table C–11).
8-9       payload.subcentre_id = 0  // Identification of originating/generating subcentre (allocated by originating/generating centre).
10        payload.master_table_version = 5  // GRIB master table version number (see Common Code table C–0 and Note 1).
11        payload.local_table_version = 1  // Version number of GRIB Local tables used to augment Master tables (see Code table 1.1 and Note 2).
12        payload.ref_time_significance = 0  // Significance of reference time (see Code table 1.2).
13-14     payload.ref_time.year = 2016  // Year (4 digits).
15        payload.ref_time.month = 8  // Month.
16        payload.ref_time.day = 22  // Day.
17        payload.ref_time.hour = 2  // Hour.
18        payload.ref_time.minute = 0  // Minute.
19        payload.ref_time.second = 0  // Second.
20        payload.prod_status = 0  // Production status of processed data in this GRIB message (see Code table 1.3).
21        payload.data_type = 2  // Type of processed data in this GRIB message (see Code table 1.4).
##  SECTION 3: GRID DEFINITION SECTION (length = 72)
##  SECTION 4: PRODUCT DEFINITION SECTION (length = 34)
##  SECTION 5: DATA REPRESENTATION SECTION (length = 23)
1-4       header.len = 23  // Length of section in octets (nn).
5         header.sect_num = 5  // Number of section.
6-9       payload.num_encoded_points = 86016  // Number of data points where one or more values are specified in Section 7 when a bit map is present, total number of data points when a bit map is absent.
10-11     payload.template_num = 200  // Data representation template number (see Code table 5.0).
12        payload.template.num_bits = 8  // Number of bits used for each packed value in the run length packing with level value.
13-14     payload.template.max_val = 3  // MV - maximum value within the levels that are used in the packing.
15-16     payload.template.max_level = 3  // MVL - maximum value of level (predefined).
17        payload.template.dec = 0  // Decimal scale factor of representative value of each level.
18-23     payload.template.level_vals = [1, 2, 3]  // List of MVL scaled representative values of each level from lv=1 to MVL.
##  SECTION 6: BIT-MAP SECTION (length = 6)
##  SECTION 7: DATA SECTION (length = 1391)
##  SECTION 8: END SECTION (length = 4)
";
    assert_eq!(String::from_utf8_lossy(buf.get_ref()), expected);

    Ok(())
}

The examples directory in the source repository may help you understand the API.

§Crate features

Following crate features are available in this crate. These descriptions are extracted from the manifest of the crate.

[features]
default = [
  "jpeg2000-unpack-with-openjpeg",
  "png-unpack-with-png-crate",
  "ccsds-unpack-with-libaec",
  "gridpoints-proj",
]

# Enable JPEG 2000 code stream format support in the GPV decoder with the help
# of OpenJPEG written in C.
jpeg2000-unpack-with-openjpeg = ["dep:openjpeg-sys"]

# Enable experimental performance improvements in the JPEG 2000 code stream
# format decoder.
jpeg2000-unpack-with-openjpeg-experimental = ["jpeg2000-unpack-with-openjpeg"]

# Enable Portable Network Graphics (PNG) support in the GPV decoder with the
# help of png crate.
png-unpack-with-png-crate = ["dep:png"]

# Enable CCSDS recommended lossless compression support in the GPV decoder with
# the help of libaec written in C.
ccsds-unpack-with-libaec = ["dep:libaec-sys"]

# Enable computation of latitudes and longitudes of grid points points using
# PROJ for some grid systems.
gridpoints-proj = ["dep:proj"]

# Enable date and time validity checks and date and time calculations using
# chrono crate.
time-calculation = ["dep:chrono"]

Re-exports§

pub use crate::codetables::Code;
pub use crate::codetables::Code::Name;
pub use crate::codetables::Code::Num;

Modules§

codetables
cookbook: A cookbook of examples for GRIB2 data handling.
def: Definitions of parameters contained in GRIB data.
utils: Utility functions not directly related to the information in the GRIB2 data.