Parallel NetCDF Release Notes:

=====================================

-------------------------------------

version _PNETCDF_VERSION_ (_PNETCDF_RELEASE_DATE_)

-------------------------------------

o New features

* The configure environment has been migrated to use GNU automake and

libtool.

* Building shared libraries is now supported with configure option

--enable-shared. However, the default configuration is to build static

library only. Note linking shared libraries is not recommended due to

the likely performance degradation.

o New optimization

* Use hash function for faster attribute name lookup. See r3335.

o New Limitations

* PnetCDF now requires the underlying MPI library define data type

MPI_Offset an 8-byte integer. Configuration process will be aborted if

MPI_Offset is detected to be of size 4 bytes. See r2944.

* The maximum number of files that can be simultaneously opened by an MPI

process is now limited to 1024 (a new constant NC_MAX_NFILES defined in

pnetcdf.h). When this limit is reached, error code NC_ENFILE will be

thrown.

* NC_MAX_VAR_DIMS, the maximum number of dimensions per variable is raised

to 2147483647, i.e. NC_MAX_INT

* NC_MAX_DIMS, the maximum number of dimensions per file is raised to

2147483647, i.e. NC_MAX_INT

* NC_MAX_ATTRS, the maximum number of attributes per file is raised to

2147483647, i.e. NC_MAX_INT

* NC_MAX_VARS, the maximum number of variables per file is raised to

2147483647, i.e. NC_MAX_INT

* NC_EMAX_REQ error code will be thrown when a single request of a get/put

API call or the aggregated size of nonblocking requests in a wait API

call exceeds INT_MAX (max of 4-byte integer). Note the size limit is per

MPI process, not across all processes. This is due to MPI-IO may not

support a single MPI-IO request of size larger then INT_MAX. A new

configure command-line option "--enable-large-req" has been added

available to disable the checking. Use this option is recommended only

when the underneath MPI-IO library supports large single requests.

o New constants

* NC_FORMAT_UNKNOWN, NF_FORMAT_UNKNOWN, and NF90_FORMAT_UNKNOWN are new

constants that are one of the returned values of argument formatp in the

API ncmpi_inq_file_format when the file under inquiry is unrecognizable

by PnetCDF library.

* NC_MAX_NFILES, NF_MAX_NFILES, and NF90_MAX_NFILES define the maximum

number of files that can be kept opened by an MPI process.

o New APIs

* none

o API syntax changes

* Add C type qualifier "const" to argument fill_value of ncmpi_def_var_fill

to indicate the contents pointed by fill_value is immutable.

o Semantics updates

* Calling ncmpi_end_indep_data() in collective data mode is no longer

considered illegal, starting from this release 1.9.0. In this case, the

API simply returns NC_NOERR. Note calling ncmpi_begin_indep_data() in

independent data mode has been allowed since PnetCDF 1.2.0. Thanks to

Gregory Sjaardema for pointing out the asymmetry between the two APIs.

See r3517.

* For CDF-5 files, the maximum size of a variable or a variable's record

is NC_MAX_INT64-3 (i.e. 2^64-4).

* When the file create/open modes are not consistent among processes, the

modes on all processes will be replaced by rank 0's mode and PnetCDF

library will still try to create/open the file. Error code

NC_EMULTIDEFINE_CMODE or NC_EMULTIDEFINE_OMODE will be returned on

processes whose modes disagree with rank 0's. If safe mode is on, then

all processes will also get the same error code.

* API ncmpi_inq_nreqs() - allows argument nreqs to be NULL. In this case,

the API is equivalent to checking the validity of argument ncid. In

previous releases, NC_EINVAL is returned when argument nreqs is NULL.

* API ncmpi_inq_buffer_usage() - allows argument usage to be NULL. In this

case, the API is equivalent to checking the validity of argument ncid. In

previous releases, NC_EINVAL is returned when argument usage is NULL.

* API ncmpi_inq_buffer_size() - allows argument usage to be NULL. In this

case, the API is equivalent to checking the validity of argument ncid. In

previous releases, NC_EINVAL is returned when argument buf_size is NULL.

o New error code precedence

* none

o Updated error strings

* The error string corresponding to error code NC_EMAXDIMS is changed to

cover the case when NC_MAX_DIMS or NC_MAX_VAR_DIMS is exceeded.

o New error code

* NC_EMAX_REQ - indicates a single request of a get/put API call or the

aggregated size of nonblocking requests in a wait API call exceeds

INT_MAX (max of 4-byte integer). The corresponding F77 and F90 error

codes are NF_EMAX_REQ and NF90_EMAX_REQ, respectively.

* NC_ENULLPAD - indicates the file fails strict null-byte header check. The

corresponding F77 and F90 error codes are NF_ENULPAD and NF90_ENULPAD,

respectively. The same error code has also been defined in NetCDF since

4.5.1. Note this error is not considered fatal. PnetCDF will continue

the file open procedure if there is no other error found. Therefore, when

encountering this error code returned from the file open API, ncmpi_open,

user can safely ignore it. See section "Conflict with NetCDF library"

below for further discussion.

o New PnetCDF hint

* none

o New run-time environment variables

* none

o New build recipe

* doc/README.Ubuntu provides build instructions for Ubuntu Linux OS.

o New/updated utility program

* Utility program ncmpivalid is renamed to ncvalidator to reflect its

purpose. It is now running in serial and thus no longer requires an MPI

compiler to build it. In fact, its source codes are all in a single file,

ncvalidator.c, which can be compiled using gcc, i.e. command "gcc -o

ncvalidator ncvalidator.c". This utility program can be used to validate

a classic CDF file, and report the locations that fail to comply with CDF

formats. It can also repair a file whose header fails to use null bytes

for paddings, when the command-line option "-x" is used. See its man page

for further description of usage.

o Other updates:

* Update UTF8 encoding utility to use utf8proc library version 2.1.0 from

the Julia Language project, https://github.com/JuliaLang/utf8proc

License https://github.com/JuliaLang/utf8proc/blob/master/LICENSE.md

* Add quiet mode command-line option "-q" to utility program ncvalidator.

* Add an internal check for whether the file starting offsets of all

variables are in an increasing order as the variables defines. See r3369.

* All build recipes (README files) have been moved to folder doc. Most of

the recipes have been updated with instructions for building shared

libraries.

* Rename configure-time variables TEST_SEQRUN, TEST_MPIRUN, and TEST_OUTDIR

to TESTSEQRUN, TESTMPIRUN, and TESTOUTDIR, respectively. This change

avoids conflict to the name convention reserved by automake.

* Fix TESTMPIRUN default value to use full path specified in --with-mpi at

configure command line

o Bug fixes

* Fix a bug in nonblocking APIs when two separate nonblocking requests that

can be coalesced into a contiguous MPI fileview or buffer type but the

coalesced length becomes too big and overflows a 4-byte integer, data

type of argument blocklengths of MPI_Type_create_hindexed, coalescing

should be skipped. I.e. such two requests should be treated as two

separate blocks in MPI_Type_create_hindexed. Thanks Jim Edwards for

reporting. See r3565.

* Fix a bug in nonblocking APIs when one request's access region covers

more than one other non-interleaved requests. PnetCDF fails to identify

them as a single group of interleaved requests. Test program

test/nonblocking/interleaved.c has been revised to test the fix.

See r3389.

* Fix a bug in string parsing the environment variable PNETCDF_HINTS and

print warning messages when hint strings are ill-formed. See r3284.

* Fix a bug in F90 APIs for reading and writing scalar variables, i.e.

number of dimensions is zero. The bugs appear only when using Intel

Fortran compiler which does not allow checking presence of an argument

and its contents in the same if condition. See r3009, r3012, r3013.

* Conform with CDF-2 file format specification on using 2^32-1 for vsize

when the variable size is larger than 2^32-4 bytes. See r2957.

* Fix a bug in nc_test when running on an ARMv7 architecture, a raspberry

pi, where char is unsigned and which is little Endian, 32-bit machine.

See r2922.

* Fix a bug that fails to do byte-swap when retrieving user-defined fill

values. See r2864. This fix is critical when PnetCDF is configured with

option erange-fill enabled (default mode) and the target platform is a

Little Endian.

o New example programs

* none

o New programs for I/O benchmarks

* none

o New test program

* test/testcases/tst_vars_fill.m4 - tests strided put (vars) APIs with

fill mode on.

* test/largefile/large_coalesce.c - tests a bug fixed in r3565 when two

separate nonblocking requests that can be coalesced into a contiguous MPI

fileview or buffer type but the coalesced length becomes too big and

overflows a 4-byte integer.

* test/testcases/tst_info.c - tests an internal bug in heap memory

allocation trace mechanism. See r3514.

* test/cdf_format/tst_corrupt.c

1. tests whether NC_EMAXDIMS can be thrown when opening a corrupted file

header with number of dimensions being larger than NC_MAX_DIMS.

2. tests whether NC_EMAXATTS can be thrown when opening a corrupted file

header with number of attributes being larger than NC_MAX_ATTRS.

3. tests whether NC_EBADDIM can be thrown when opening a corrupted file

header with dimension ID of a variable is beyond the number of

dimensions defined in the file.

4. tests whether NC_EBADTYPE can be thrown when opening a corrupted file

header with nc_type of a variable is not within the legal NC atomic

data type allowed by the file format.

* test/testcases/tst_max_var_dims.c - tests the number of dimensions

against NC_MAX_VAR_DIMS when calling ncmpi_def_var API.

* test/largefile/tst_cdf5_begin.c - tests CDF-5 file header corruption for

the case when the starting file offsets of variables are not in an

increasing order as they defined.

* test/cdf_format/tst_open_cdf5.c - tests whether NC_ENOTNC can be

correctly thrown when opening a corrupted CDF-5 (bad_begin.nc5) which

contains two variables and the second variable's starting file offset is

less than the first variable's end offset.

* test/testcases/large_var_cdf5.c - tests whether NC_EVARSIZE can be

correctly thrown when a variable or a variable record is larger than

NC_MAX_INT64-3.

* test/testcases/mix_collectives.c - tests collective get/put APIs that

allow different processes to call different kinds of APIs (i.e. var1,

vara, vars, varm, etc.) and use different var IDs. In other words, the

API kinds can be different among processes in a collective call.

* test/largefile/high_dim_var.c - tests blocking vars APIs for high-

dimensional variables. See r3164.

* test/largefile/large_dims_vars_attrs.c - tests large number of

dimensions, large number of variables, and large number of attributes.

* test/testcases/put_all_kinds.m4 - tests various blocking put APIs.

* test/testcases/tst_dimsizes.c - borrowed from NetCDF library to test

define/inquire APIs for the maximum allowable dimension sizes. This test

program is similar to (or a subset of) test/cdf_format/dim_cdf12.c

* test/testcases/null_args.c - tests whether the correct error codes can

be returned when using NULL arguments for start, count, stride, or imap

o Conformity with NetCDF library

* API ncmpi_open now checks the number of large variables that is legal

for CDF-1 and CDF-2 files. See r2961. For the information about the

number of large variables allowed in the classic formats can be found in

http://www.unidata.ucar.edu/software/netcdf/docs/file_structure_and_performance.html#classic_format_limitations

* Add checking for error code NC_ELATEFILL. Setting a variable's attribute

_FillValue must be done in the variable's initial define mode.

* NetCDF-4 documents in 4.4.1 and prior state that setting attribute

_FillValue for global variable NC_GLOBAL is not allowed and error code

NC_EINVAL will be thrown. Revisions r2954, r2955, and r2956 enforce this

rule. However, because the rule has never been implemented in any NetCDF

official release, NetCDF developers have decided to abandon this rule

by revising the NetCDF document starting from v4.5. See discussion in

NetCDF GitHub pull request 458. To be consistent with NetCDF, PnetCDF

follows the same decision by wrapping the checking with preprocessor

conditional #ifdef NO_NC_GLOBAL_FILLVALUE (see r3403). Note NC_EGLOBAL

will be thrown when using NC_GLOBAL in ncmpi_def_var_fill(). This is

due to no data type information for the fill value can be passed.

* From the releases of 1.6.1 through 1.8.1, putting attribute _FillValue

automatically turns on the fill mode for the variable. Similarly,

deleting attribute _FillValue turns off the fill mode. However, this

behavior does not conform with NetCDF library. This release fixes this

inconsistency, so putting or deleting attribute _FillValue does not

affect a variable's fill mode. See r2951.

o Conflict with NetCDF library

* Null-byte padding for attribute values in file header -- The classic CDF

specifications require header padding uses null (\x00) bytes. However,

prior to version 4.5.0, NetCDF did not implement this specification

entirely. In particular, it has never enforced the null-byte padding for

attribute values (it has for others, such as names of dimension,

variables, and attributes.) It also appears that files created by SciPy

NetCDF module or NetCDF Java module, both developed independent from

NetCDF-C, also fail to respect this padding specification. This becomes

a problem for PnetCDF to read such netCDF files, because PnetCDF enforces

the header padding from its very first release. The files violating the

padding specification will not be readable by PnetCDF of all releases

prior to 1.9.0 and error code NC_EINVAL or NC_ENOTNC will be thrown when

opening such files. Note if the sizes of all attribute values of your

files are aligned with 4-byte boundaries, then the files are readable by

PnetCDF. In order to keep the files in question readable by PnetCDF,

checking for null-byte padding has been disabled in 1.9.0 by default. A

new configure command-line option "--enable-null-byte-header-padding" has

been made available to enable the checking and error code NC_ENULLPAD

will be thrown when opening files. Note NC_ENULLPAD is not considered a

fatal error. Users can safely ignore it. However, we keep this checking

in ncvalidator no matter the new configure option is set or not.

ncvalidator is a utility program reports whether a CDF file violates the

file format specification, including this null-byte padding. See r3528

for source codes changed and discussion in NetCDF Github issue

https://github.com/Unidata/netcdf-c/issues/657.

o Issues related to MPI library vendors:

* When using OpenMPI 2.0.0 through the latest 2.1.2 to build PnetCDF 1.8.1

and prior, the following message may appear when running make check:

"MPI error (MPI_File_open) : MPI_ERR_FILE: invalid file"

but the test programs report "pass". This message can be safely ignored.

It is due to OpenMPI failing to translate system error code ENOENT into

MPI_ERR_NO_SUCH_FILE, but returns MPI_ERR_FILE instead. Thanks to Edgar

Gabriel, a trick to silence the messages is to add the following to the

command line of make check:

TEST_SEQRUN="/path/to/OpenMPI/2.x.x/bin/mpirun --mca io romio314 -n 1"

See discussion thread in OpenMPI:

https://github.com/open-mpi/ompi/issues/4412

Note this error message does not appear in PnetCDF 1.9.0 and after.

* When using OpenMPI 2.1.1 and priors built with Intel compilers 17.0.0,

the test program nonblocking/mcoll_testf.f90 fails with segmentation

fault. This is due to a bug in OpenMPI. See discussion thread in OpenMPI:

https://github.com/open-mpi/ompi/issues/3695

* PnetCDF can no longer be built on 32-bit machines if using OpenMPI

versions 2.0.2 and 1.10.6 and earlier versions. This is because OpenMPI

configures type MPI_Offset as a 4-byte integer on all 32-bit machines,

even if the machines allow 64-bit file offsets, known as large-file

support. We suggest to use other MPI libraries to build PnetCDF on 32-bit

machines. See the discussion thread in OpenMPI:

https://github.com/open-mpi/ompi/issues/3195

* Fix test/nc_test/test_read.m4 and test/testcases/modes.c which test

whether the correct error codes can be returned when opening a

non-existing file. Thanks to Mark Dixon for testing various MPI

libraries, C compilers, and file systems, we found that when using

MVAPICH2 2.2, its Lustre driver always adds O_CREAT flag to the open()

calls. Because of this behavior, the non-existing file will be created

with zero length, which causes PnetCDF to spew NC_ENOTNC instead of

NC_ENOENT. A bug report has been filed to MVAPICH. The fix shall come in

the next release of MVAPICH. The bug report can be found in

http://mailman.cse.ohio-state.edu/pipermail/mvapich-discuss/2017-February/006300.html

o Issues related to Darshan library:

* When using Darshan in conjunction with PnetCDF, if you encounter

link-time or run-time errors with messages shown below, then we recommend

to use Darshan library version 3.1.5 and later.

During compile/link time:

/soft/perftools/darshan/lib/libdarshan-mpi-io.a(darshan-pnetcdf.o):

In function `__wrap_ncmpi_create': darshan-pnetcdf.c:62: undefined

reference to `ncmpi_create'

During run time:

WARNING: Darshan ncmpi_create() stub called; this is probably the result

of a link-time problem.

o Clarifications

* The NetCDF CDF formats define a set of external NC data types and

describe their intents of use, shown below.

External type No. Bits Intent of use

------------- -------- ---------------------------------

NC_CHAR 8 text data (the only non-numerical type in NetCDF)

NC_BYTE 8 1-byte integer

NC_SHORT 16 2-byte signed integer

NC_INT 32 4-byte signed integer

NC_FLOAT 32 4-byte floating point number

NC_DOUBLE 64 8-byte real number in double precision

NC_UBYTE 8 unsigned 1-byte integer

NC_USHORT 16 unsigned 2-byte integer

NC_UINT 32 unsigned 4-byte integer

NC_INT64 64 signed 8-byte integer

NC_UINT64 64 unsigned 8-byte integer

Note NC_CHAR is the only non-numerical data type available in NetCDF

realm. All other external types are considered numerical, which are

illegal to be converted (type-casted) to and from a NetCDF variable

defined in NC_CHAR type (error code NC_ECHAR will be thrown). The only

legal APIs to read/write a variable of type NC_CHAR are the "_text" APIs.

* Starting from 1.7.0, PnetCDF translates internal data types (i.e. data

types of the I/O buffer and also used in the API name such as text,

schar, uchar, short, int, etc.) to MPI data types based on the table

below. Note MPI_BYTE does no correspond to any internal data type used

in NetCDF/PnetCDF APIs. Thus, when MPI_BYTE is used to construct an MPI

derived data type which is later used as argument buftype in a flexible

API, the error code NC_EBADTYPE will be thrown (Not a valid data type).

Data type of Corresponding

internal I/O buffer Example API MPI datatype

------------- ---------------------- -----------------

text ncmpi_put_var_text MPI_CHAR

schar ncmpi_put_var_schar MPI_SIGNED_CHAR

uchar ncmpi_put_var_uchar MPI_UNSIGNED_CHAR

short ncmpi_put_var_short MPI_SHORT

ushort ncmpi_put_var_ushort MPI_UNSIGNED_SHORT

int ncmpi_put_var_int MPI_INT

uint ncmpi_put_var_uint MPI_UNSIGNED

long ncmpi_put_var_long MPI_LONG

float ncmpi_put_var_float MPI_FLOAT

double ncmpi_put_var_double MPI_DOUBLE

longlong ncmpi_put_var_longlong MPI_LONG_LONG_INT

ulonglong ncmpi_put_var_ulonglong MPI_UNSIGNED_LONG_LONG

-------------------------------------

version 1.8.1 (January 28, 2017)

-------------------------------------

This release is mainly for fixing the following bug.

o Bug fixes:

* Fortran parameters NF_MAX_DIMS, NF_MAX_ATTRS, NF_MAX_VARS, and NF_MAX_NAME

defined in the Fortran header file, pnetcdf.inc, were set to obsolete

values and have been fixed in this release to conform with their

corresponding C constants. Because these four parameters were not used

by PnetCDF internally, the end users can safely skip this release and

1.8.1 should perform the same as 1.8.0. If your programs explicitly use

these four parameters, then updating to 1.8.1 is recommended.

o Other updates:

* To conform with GNU Coding Standards, the installation location for man

pages are changed to ${prefix}/share/man.

* Support of DESTDIR for Building Binary Packages has been added, as

suggested by [email protected]. Ticket #25

-------------------------------------

version 1.8.0 (December 19, 2016)

-------------------------------------

o New features

* Copy fill values to data elements that cause out-of-bound error (i.e.

NC_ERANGE.) In netCDF v4.4.1 and priors, type conversion for out-of-

bound elements proceeds even if the out-of-bound error is detected, and

the contents (in both files and read buffers) are left up to the user

to handle. See the URL below for details.

http://www.unidata.ucar.edu/software/netcdf/docs/group__error.html

Instead of carrying out the type conversion, PnetCDF fills the out-of-

bound data elements with their "fill values", as specified by the

variable's attribute "_FillValue" or default CDF fill values if the

attribute is not defined. Specifically, for PnetCDF put APIs, the value

of attribute _FillValue, if defined, will be used when writing to files.

For get APIs, the default CDF _FillValue corresponding to the internal

data type will be used to fill the read buffer. This feature can be

disabled by adding option "--disable-erange-fill" to the configure

command line. In this case, the conventional NetCDF method described

above is used.

* A new configure option --enable-relax-coord-bound is added to provide a

more relaxed boundary check for argument start. Traditionally, both

NetCDF and PnetCDF detect the error code NC_EINVALCOORDS when the value

of start is larger than or equal to the defined dimension size. This can

be inconvenient for some applications that make a zero-length request

with start being the dimension size, usually the last process. The new

configure option relaxes the boundary check by allowing the above

exception, i.e. start[i] can be the size of dimension i only when

count[i] is zero. The default of this feature is disabled. For more

detailed information, see discussion in:

http://lists.mcs.anl.gov/pipermail/parallel-netcdf/2016-March/001839.html

* New macros have been added to PnetCDF header file, pnetcdf.h, to indicate

the configurable options that were enabled/disabled. These macros are

PNETCDF_ERANGE_FILL, PNETCDF_SUBFILING, PNETCDF_RELAX_COORD_BOUND, and

PNETCDF_DEBUG_MODE. Their values are set to 1 when enabled, 0 otherwise.

* API ncmpi_inq_file_format can now detect HDF5 file signature and report

NC_FORMAT_NETCDF4.

* Enable ncmpidump to check whether input file is in HDF5 format when -k

option is used at the command line.

* In ncmpi_open, add a checking for HDF5 file signature and return error

code NC_ENOTNC3 if detected.

o New APIs

* ncmpi_strerrno, nfmpi_strerrno, nf90mpi_strerrno return the string name

of a NC error code.

* ncmpi_inq_path, nfmpi_inq_path, nf90mpi_inq_path for inquiring the file

pathname which was used to open/create the CDF file.

o New optimization

* Use Bernstein hash function for checking whether the name of a variable

or dimension has already been used.

* Replace the use of linked lists with arrays for storing the pending

nonblocking requests. This could save time for traversing the linked

list.

o Syntax changes

* ncmpi_cancel can now be called in define mode.

o Semantics updates

* For collective APIs, the following errors are considered fatal:

NC_EBADID, NC_EPERM, NC_EINDEFINE, NC_EINDEP, and NC_ENOTINDEP. Once

detects these errors the calling APIs return immediately. If the fatal

errors happen only to a subset of processes, the running program may

hang, due to other processes being waiting at an MPI collective call.

(Note hanging will never occur to independent APIs.) To debug a hanging

problem, users can run the program in safe mode by setting the

environment variable PNETCDF_SAFE_MODE to 1. In safe mode, error codes

are synchronized among all processes. Error codes other than the above,

if detected, are not as fatal. The processes encountering the non-fatal

errors will continue to the collective call but with zero-length

requests. Thus, the collective calls can nicely complete without hanging.

* Consistency check for header data (defining dimensions, variables, and

attributes) is moved to individual APIs and only enabled when safe mode

is enabled. Header data consistency check is no longer performed in

ncmpi_enddef. As a reminder, PnetCDF APIs that change header data are

all collective and their arguments are required to be consistent among

the calling processes.

* In netCDF-3, whether the contents of an NC_BYTE variable in CDF-1 or

CDF-2 files are signed or unsigned is determined by the calling APIs. See

http://www.unidata.ucar.edu/software/netcdf/docs/data_type.html#type_conversion

"In netcdf-3, we treat NC_BYTE as signed for the purposes of conversion

to short, int, long, float, or double. (Of course, no conversion takes

place when the internal type is signed char.) In the _uchar functions, we

treat NC_BYTE as if it were unsigned. Thus, no NC_ERANGE error can occur

converting between NC_BYTE and unsigned char." In other words, if called

from signed APIs, NC_BYTE variables are treated as signed. If called from

unsigned APIs (i.e. _uchar APIs, the only unsigned APIs in netCDF-3) they

are unsigned. NetCDF-3 specifically makes an exception to skip NC_ERANGE

check when calling _uchar APIs on NC_BYTE variables. However, in netCDF-4

and CDF-5, because of the introduction of the new data type NC_UBYTE, an

unsigned 8-bit integer, which makes NC_BYTE a signed 8-bit integer and

thus renders the above exception less sense. Starting from this release,

for CDF-5 files, regular NC_ERANGE check is performed in PnetCDF for all

APIs that access NC_BYTE variables. For CDF-1 and 2 files, PnetCDF still

honors that exception.

* Prior to version 1.7.0, any outstanding nonblocking requests detected at

file close are automatically cancelled without reporting an error.

Starting from this 1.8.0 release, error code NC_EPENDING will be returned

when pending requests are detected at the file close.

* API ncmpi_set_default_format is changed to an independent API. However,

it is expected to be called by all processes that intend to create a file

later. Inconsistency in default format setting will be detected at file

create time.

o New utility program

* Added pnetcdf-config shell script to display the options used in building

PnetCDF.

o New error code precedence

* When two or more errors are detected by the PnetCDF library (e.g. due to

the use of more than one bad API argument), the error code returned from

an API will follow the precedence below. In principle, the errors related

to ncid is the most serious, followed by varid.

For put attribute APIs:

NC_EBADID, NC_EPERM, NC_ENOTVAR, NC_EBADNAME, NC_EBADTYPE, NC_ECHAR,

NC_EINVAL, NC_ENOTINDEFINE, NC_ERANGE

For get attribute APIs:

NC_EBADID, NC_ENOTVAR, NC_EBADNAME, NC_ENOTATT, NC_ECHAR, NC_EINVAL,

NC_ERANGE

For put/get variable APIs:

NC_EBADID, NC_EPERM, NC_EINDEFINE, NC_ENOTVAR, NC_ECHAR,

NC_EINVALCOORDS, NC_EEDGE, NC_ESTRIDE, NC_EINVAL, NC_ERANGE

o New error code

* NC_EPENDING: Pending nonblocking request is found at file close.

* NC_EINVAL_OMODE: invalid file open mode. This separates the case from the

file create API which uses NC_EINVAL_CMODE mode. The error messages for

both error codes are revised accordingly.

* NC_EMULTIDEFINE_CMODE: inconsistent file create mode. This separates the

case from NC_EMULTIDEFINE_OMODE. The error messages for both error codes

are revised accordingly.

* The Fortran error codes corresponding to the above are also added.

o New test program

* test/testcases/scalar.c to test whether arguments start, count, stride,

and imap are properly ignored when getting/putting a scalar variable

* test/testcases/erange_fill.m4 to test the new configure-time feature of

--enable-erange-fill

o Other updates:

* Check file open/create mode for NC_DISKLESS and NC_MMAP and return error

code NC_EINVAL_OMODE/NC_EINVAL_CMODE if either of these two modes is

used. These modes are currently not supported.

* Clarify support for CDF-2 and CDF-5 formats. When the size of MPI_Offset

is less than 8 bytes, support for CDF-2 and CDF-5 is disabled.

* When using OpenMPI 2.0.1 to build PnetCDF, users may encounter an error

message of "test_write.c: file ./scratch.nc should not exist" or "FAILURE

at line 429 of test_ncmpi_abort in test_write.c: expecting NC_ENOENT or

NC_EFILE but got NC_NOERR" while running "make check". The cause is due

to a bug in OpenMPI 2.0.1 that fails to return the correct MPI error

class MPI_ERR_NO_SUCH_FILE when deleting a non-existing file. The bug fix

will appear in OpenMPI 2.0.2. See the OpenMPI bug report in

https://github.com/open-mpi/ompi/issues/2232

* When using OpenMPI 1.10.2 and 1.8.5 to build PnetCDF, users may encounter

an error message of "An error occurred in MPI_Type_create_hindexed" when

running "make check". This is due to a bug in OpenMPI that fails to

create a zero-size datatype in MPI_Type_create_hindexed. The bug fix has

appeared in the releases of 1.10.3, 1.8.6, and 2.0.1. See the bug report

in https://github.com/open-mpi/ompi/issues/1611

* When using SGI MPI with MPT 2.14 to build PnetCDF, users may encounter an

error message of "function:MPI_TYPE_CREATE_HVECTOR, Invalid argument".

The bug fix will appear in 2.15 to be released in November 2016.

* The quiet mode of utility program ncmpidiff, command-line option -q, now

print no message when two files or two variables in comparison are the

same.

o Interoperability with netCDF-4

* When using 1.8.0 release to build netCDF version 4.4.1.1 and earlier,

users may encounter errors with message like "test_put.c: NetCDF: Numeric

conversion not representable" from running command "make test". This is

due to an incorrect implementation of uchar APIs for accessing variables

of type NC_BYTE for CDF-2 files in both PnetCDF and NetCDF. If not using

such a special case, users can ignore this error. See more information

above under "Semantics updates".

o Bug fixes

* Fix to allow argument no_fill in API ncmpi_inq_var_fill to be NULL. See

r2609

* Fix to allow putting global attribute _FillValue. See r2560

* Fix some missing character string normalizations for names of attributes,

dimensions, and variables.

* Fix the return unlimited dimension ID of value -1, not 0, by Fortran API

nfmpi_inq_unlimdim when no unlimited dimension is defined. This conforms

with netCDF convention. See r2396 and r1372

* Fix string processing of the environment variables for PnetCDF I/O hints

See r2385

* Fix utility ncmpidiff to use collective read APIs in collective data mode

See r2382

-------------------------------------

version 1.7.0 (March 3, 2016)

-------------------------------------

o New features

* Nonblocking requests now can be posted while the program is in the define

mode. This feature enables applications to add new variables and post the

nonblocking write requests without separating the codes by

ncmpi_enddef(). Note ncmpi_wait_all and ncmpi_wait must still be called

in the data mode.

* When using immutable write buffer in put APIs on a Little Endian machine,

the in-place byte swap operation performed internally in PnetCDF can

cause a fatal error (by trying to change the contents of an immutable

memory space). The solution of copying user's write request to a

temporary buffer and perform byte swap there conflicts with PnetCDF's

design principle of avoiding internal memory allocation as much as

possible. Since the size of immutable buffer in most cases is small,

PnetCDF now compromise the two by making a copy of write requests that is

less than 4KB. Users are warned that using immutable write buffer larger

than 4KB will still cause the fatal error in the PnetCDF default

configuration. The way to completely disable in-place byte swap is to

build PnetCDF with --disable-in-place-swap option at the configure time.

In this case, the internal memory allocation in PnetCDF will increase.

o Syntax changes

* In ncmpi_wait_all(), ncmpi_wait(), and ncmpi_cancel(), the API's third

argument, array of requests IDs, is changed to INOUT. Upon successful

completion or cancellation of individual nonblocking requests, the

corresponding request IDs are set to NC_REQ_NULL.

o New run-time environment variables

* PNETCDF_VERBOSE_DEBUG_MODE environment variable can be used to print the

location in the source code where the error code is originated, no matter

the error is intended or not. This run-time environment variable only

takes effect when PnetCDF is configure with debug mode, i.e.

--enable-debug is used at the configure command line. Set this variable

to 1 to enable. Set it to 0 or keep it unset disables this mode. Default

is 0, i.e. disabled. Users are warned that enabling this mode may result

in a lot of debugging messages printed in stderr.

o New example programs

* examples/C/nonblocking_write_in_def.c shows an example of posting

nonblocking write requests in the define mode.

* examples/C/req_all.c shows an example of using NC_REQ_ALL to flush all

pending nonblocking requests without providing the requests IDs.

o New test program

* test/cdf_format/dim_cdf12.c tests defining maximal dimension size for

CDF-1 and CDF-2 file formats

* test/testcases/test_erange.c tests if the error code NC_ERANGE can be

correctly returned for two cases 1) get a NC_UBYTE value of 255 from a

netCDF file to a memory buffer of type signed char and 2) put a value

of -1 of signed char to a NC_UBYTE variable in a netCDF file

* test/testcases/check_type.c tests if the error codes can be correctly

returned when conflicted in-memory and external data types are used.

* test/testcases/put_parameter.f tests the use of immutable write buffer

(e.g. a buffer declared as PARAMETER). Note the buffer size must be

smaller than 4KB.

* test/nonblocking/i_varn_indef.c tests posting nonblocking requests in

define mode.

* test/nonblocking/req_all.c tests the use of NC_REQ_ALL for flushing

all pending nonblocking requests without providing the requests IDs.

* test/last_large_var.c tests the special case when there is no record

variable, the last fixed-size variable can be larger than 2GiB in size

if its starting file offset is less than 2GiB.

* test/testcases/buftype_free.c and test/testcases/buftype_freef.f test

the bug in r2160.

* testcases/add_var.c checks the starting file offsets of newly added

variables from re-entering the define mode.

* testcases/attrf.f checks NF_ERANGE is returned instead of coredump. This

is particularly for NAG Fortran compiler that may report "Arithmetics

exception".

* testcases/check_striping.c checks if the file striping unit and factor

returned from MPI-IO hints are consistent among processes.

* test/nonblocking/column_wise.c checks if PnetCDF detects interleaved

fileviews from multiple nonblocking requests and correctly breaks and

reconstructs the fileviews so the combined fileview is monotonic

non-decreasing in file offsets.

o New optimization

* Filling variables at ncmpi_enddef() is now done by aggregating all write

requests into one MPI collective write call. In v 1.6.1, this is done by

filling one variable at a time.

o New utility program

* ncoffsets reports the file offset information, including the starting and

ending file offsets, of variables stored in a netCDF file. ncoffsets is

compiled with gcc if gcc is presented on the build system. Additional

command-line options are: (-v) reports only for a selected list of

variables in interest, (-s) prints the variable sizes, (-g) outputs the

file space gap size from the end of previous variable, (-x) reports

whether there is a gap between any two adjacent fixed-size variables. See

the man page for descriptions of all command-line options and examples.

o Semantics updates

* All nonblocking APIs now take a NULL pointer for the request ID argument,

meaning users do not wish to keep track of the request ID. If NULL

request IDs are used, NC_REQ_ALL should be used when calling

ncmpi_wait_all/ncmpi_wait to commit all the pending nonblocking requests.

This feature relinquishes users from the responsibility of tracking the

IDs of pending requests.

* Using NC_REQ_ALL as the 2nd argument "num" in ncmpi_wait_all/ncmpi_wait

APIs will flush all the pending nonblocking requests. In this case, the

3rd and 4th arguments "array_of_requests" and "array_of_statuses" will be

ignored and thus these two arguments can be NULLs.

* Using NC_REQ_ALL in ncmpi_cancel() will cancel all the pending

nonblocking requests.

* Using NC_GET_REQ_ALL or NC_PUT_REQ_ALL in ncmpi_wait_all(), ncmpi_wait(),

and ncmpi_cancel() for all the pending get-only or put-only requests,

respectively.

o Other updates:

* Conform with netCDF on the maximal dimension size for CDF-2 file format

to be (2^32 - 4)

* NC_ERANGE checks have been removed from nc_test for text APIs and

variables that are defined as NC_CHAR type

* Add README.K-Computer build recipe for using Fujitsu MPI compilers on the

K computer at RIKEN in Japan

* Add README.INTEL build recipe for using Intel MPI compilers 4.x

* Build dependency rule is added for files configure and configure.in

* PnetCDF checks MPICC/MPICXX/MPIF77/MPIF90 instead of CC/CXX/F77/F90/FC.

If MPICC/MPICXX/MPIF77/MPIF90 are set, PnetCDF will ignore

CC/CXX/F77/F90/FC. If CC/CXX/F77/F90/FC is set instead of

MPICC/MPICXX/MPIF77/MPIF90, PnetCDF will now copy them to

MPICC/MPICXX/MPIF77/MPIF90.

* Enforce netCDF convention on error code priority: NC_ECHAR trumps

NC_EINVALCOORDS, NC_EEDGE, and NC_ESTRIDE.

* Return error code NC_EGLOBAL instead of NC_ENOTVAR for APIs where using

NC_GLOBAL as the variable ID argument is prohibited.

* All Fortran 77 test and example programs (files with .f and .F

extensions) have been revised to conform with 77 standard, and was tested

using pgf77.

* Now provides a pkg-config file

(http://www.freedesktop.org/wiki/Software/pkg-config/), making it

slightly easier to set the correct pnetcdf include and library paths.

o Bug fixes

* Fix the bus error of invalid address alignment when build with Fujitsu

compiler. See r2171 and r2180.

* Fix the bug for the special case when there is no record variable, the

last fixed-size variable can be larger than 2GiB in size if its starting

file offset is less than 2GiB. See r2166.

* Fix the nonblocking flexible APIs that fail to save (duplicate) the user

MPI derived data type that later is needed to unpack read data to the

user buffer (a call to MPI_Unpack). See r2160.

* Fix Fortran 77 constants nf_fill_uint and nf_fill_int64 (thanks Jim

Edwards) that pgf77-based MPI compiler does not like "_8" modifier.

See r2045 and r2051. The same issue for NAG Fortran compiler is also

resolved. See r2089 and r2093.

* In the example program examples/tutorial/pnetcdf-write-nb.c, the write

buffers used in two iput API calls should be different. See r2095.

* Fix the error reporting mechanism for NC_ERANGE, for when an arithmetic

overflow happens. Overflow checking is now performed before I/O.

-------------------------------------

version 1.6.1 (June 1, 2015)

-------------------------------------

o New features

* PnetCDF now supports fill mode. ncmpi_set_fill() can be used to set the

fill mode for the entire data set. Note the differences from netCDF:

1. The default mode in PnetCDF is NC_NOFILL.

2. Setting fill mode for the entire file or individual variables must be

done in define mode.

3. For non-record variables, they are filled at the time ncmpi_enddef()

is called.

4. For record variables, users must explicitly call ncmpi_fill_var_rec()

to fill one record at a time before writing to the record of that

variable.

o New APIs

* ncmpi_def_var_fill() sets fill mode for an individual variable. This API

must be called in define mode.

* ncmpi_inq_var_fill() inquires fill mode of a variable.

* ncmpi_fill_var_rec() is a collective API that fills a record of a record

variable. This API must be called at data mode.

* ncmpi_inq_default_format() for inquiring the default file format for

new file creation. Note the default format can be changed by

ncmpi_set_default_format().

* The above new API are also available in Fortran and C versions.

o New error code

* NC_ENOTRECVAR when attempting operation only for record variables

* NC_ENOTFILL when attempting to fill a variable when its fill mode is off

* NC_EMULTIDEFINE_FILL_MODE when inconsistent dataset fill mode is detected

* NC_EMULTIDEFINE_VAR_FILL_MODE when inconsistent variable fill mode is

detected

* NC_EMULTIDEFINE_VAR_FILL_VALUE when inconsistent variable fill value is

detected

* Fortran versions of the above error codes are also added.

o New example programs

* C/fill_mode.c shows the use of ncmpi_set_fill(), ncmpi_def_var_fill(),

ncmpi_inq_var_fill() and ncmpi_fill_var_rec()

F77/fill_mode.f is the Fortran version.

F90/fill_mode.f90 is the Fortran 90 version.

CXX/fill_mode.cpp is the C version.

* C/ghost_cell.c shows how to use varm API for writing from a user buffer

as a 2D array with ghost cells on both ends of every dimension.

o New test programs

* nc_test/tst_nofill.c borrowed from netCDF

* testcases/ivarn.c tests bug fix r2023 when the request IDs stored in

argument array_of_requests[] of ncmpi_wait_all() are not in an

increasing order.

o Other updates:

* Change the chunk size used for moving variable data when the file header

extent expands. The default is now 1MB. If the file's striping unit

size is known (from MPI-IO hint striping_unit) then the chunk size is

set to the striping unit size.

o Bug fixes

* Add missing F90 function overloading for f90mpi_put_var_all,

f90mpi_get_var_all, f90mpi_put_vard_all, and f90mpi_get_vard_all,

when the user buffer is a scalar.

* Fix when the request IDs passed in argument array_of_requests[] of

ncmpi_wait_all() are not in an increasing order. See r2023.

* Fix C compile error for converting NULL to string. See r2039.

-------------------------------------

version 1.6.0 (February 2, 2015)

-------------------------------------

o Format conformation updates:

* Conform with netCDF4 on CDF-1 and CDF-2 formats. The only difference now

between the two formats are the OFFSET item in the format spec (32 vs.

64 bit integers.) All names (variable, dimension, attribute) now allow

extended characters (e.g. special2 and MUTF8).

o New APIs

* Nonblocking buffered varn API family.

For C, ncmpi_bput_varn_<type>()

For F77, nfmpi_bput_varn_<type>()

For F90, nf90mpi_bput_varn()

For C , NcmpiVar::bputVarn()

* Nonblocking varn API family.

For C, ncmpi_iput_varn_<type>() and ncmpi_iget_varn_<type>()

For F77, nfmpi_iput_varn_<type>() and nfmpi_iget_varn_<type>()

For F90, nf90mpi_iput_varn() and nf90mpi_iget_varn()

For C , NcmpiVar::iputVarn() and NcmpiVar::igetVarn()

* Blocking vard API family takes an argument of MPI derived data type that

describes the file access layout, as opposed to vara and vars APIs that

use start[] and count[].

For C, ncmpi_put_vard() and ncmpi_get_vard()

For F77, nfmpi_put_vard() and nfmpi_get_vard()

For F90, nf90mpi_put_vard() and nf90mpi_get_vard()

For C , NcmpiVar::putVard() and NcmpiVar::getVard()

* Collective var1 API family

For C, ncmpi_put_var1_all() ncmpi_get_var1_all()

ncmpi_put_var1_<type>_all() ncmpi_get_var1_<type>_all()

For F77, nfmpi_put_var1_all() nfmpi_get_var1_all()

nfmpi_put_var1_<type>_all() nfmpi_get_var1_<type>_all()

For F90, nf90mpi_put_var_all() nf90mpi_get_var_all()

For C , NcmpiVar::putVar_all() NcmpiVar::getVar_all()

* ncmpi_inq_buffer_size() returns the size of buffer previously attached

for use of bput APIs. With ncmpi_inq_buffer_usage() one can calculate

the space remaining for additional bput requests.

For F77, nfmpi_inq_buffer_size()

For F90, nf90mpi_inq_buffer_size()

For C , NcmpiFile::Inq_buffer_size()

* ncmpi_inq_recsize() returns the size of record block, i.e. the sum of

single records of all record variables.

For F77, nfmpi_inq_recsize()

For F90, nf90mpi_inq_recsize()

For C , NcmpiGroup::getRecSize()

* ncmpi_inq_num_rec_vars() and ncmpi_inq_num_fix_vars() report the number

of record and fixed-size variables, respectively.

For F77, nfmpi_inq_num_rec_vars() and nfmpi_inq_num_fix_vars()

For F90, nf90mpi_inq_num_rec_vars() and nf90mpi_inq_num_fix_vars()

For C , NcmpiGroup::getRecVarCount() and NcmpiGroup::getFixVarCount()

o New PnetCDF hint

* pnetcdf_subfiling -- it can be set in an MPI info object or in the

environment variable PNETCDF_HINTS to enable/disable subfiling.

The value is either "enable" or "disable".

o PnetCDF hint priority

* The alignment hints set in the environment variable "PNETCDF_HINTS" have

the highest priority, which overwrite the alignment hints set in

ncmpi__enddef(), which overwrite the alignment hints set in the MPI_Info

object used in the call of ncmpi_create() and ncmpi_open().

o New error code

* NC_ESTRICTCDF2 for attempting CDF-5 operation on CDF-2 file. For

example, define a variable of type NC_INT64 in a CDF-2 file.

* NC_ETYPESIZE when filetype size is bigger than the variable size

* NC_ETYPE_MISMATCH when the element type of filetype mismatches the

variable type

* NC_ETYPESIZE_MISMATCH when filetype size mismatches buffer type size

* NC_ENULLSTART when argument start is a NULL pointer

* NC_ENULLCOUNT when argument count is a NULL pointer

* NC_EINVAL_CMODE when invalid file create mode is set, (e.g. cannot have

both NC_64BIT_OFFSET & NC_64BIT_DATA. In PnetCDF 1.5.0 and earlier

versions, if both flags were set, then NC_64BIT_DATA triumphs

NC_64BIT_OFFSET.)

o New example programs

* C/bput_varn_uint.c and F77/bput_varn_int8.f show the use of

nonblocking bput_varn APIs

* C/i_varn_int64.c and F77/i_varn_real.f show the use of nonblocking

iput_varn and iget_varn APIs

* C/vard_int.c F77/vard_int.f F90/vard_int.f90 CXX/vard_int.cpp show the

use of vard API to write/read record and fixed-size variables.

* C/transpose.c shows the use of ncmpi_put_vara_int_all to write a 3D array

that is dimensional-transposed from the one stored in memory. Six

transposed 3D arrays are saved whose dimensions are organized as ZYX,

ZXY, YZX, YXZ, XZY, and XYZ. The C , Fortran 77, and Fortran 90

versions are also provided.

o New test program

* nonblocking/wait_after_indep.c tests if ncmpi_end_indep_data() returns

properly when nonblocking APIs are called in independent data mode and

the wait call is made after exiting the independent mode.

* nonblocking/flexible_bput.c tests flexible bput APIs that use

noncontiguous buffer type, noncontiguous imap and require type conversion

* testcases/flexible2.c tests flexible APIs that perform type conversion

* testcases/flexible_varm.c tests flexible varm APIs

* testcases/varn_contig.c tests the case when the fileview is actually a

contiguous chunk. PnetCDF should be able to merge all.

* nonblocking/bput_varn_uint.c tests nonblocking bput_varn APIs

* nonblocking/i_varn_int64.c tests nonblocking iput_varn and iget_varn APIs

* test/testcases/test_vard.c test/testcases/test_vardf.f

test/testcases/test_vardf90.f90 test the new vard APIs.

* test/testcases/inq_recsize.c tests if one can correctly inquire the

size of record block from in a netCDF file. A similar program in F90,

named inq_recsizef.f90, is also added.

* In test/nc_test, the test programs below are borrowed from netCDF test

programs: t_nc.c tst_misc.c tst_norm.c tst_small.c tst_names.c

tst_atts3.c tst_atts.c

* test/testcases/one_record.c tests the special case defined in CDF-1 and

CDF-2 specification that "A special case: Where there is exactly one

record variable, we drop the requirement that each record be four-byte

aligned, so in this case there is no record padding."

* test/testcases/modes.c tests if correct error codes are returned when

various file create/open modes are used.

* Under test/testcases, varn_int.c varn_intf.f varn_real.f90 test varn APIs

* test/testcases/inq_num_vars.c test if one can correctly inquire the

numbers of record and fixed-size variables defined in a netCDF file.

A similar program in F90, named inq_num_varsf.f90, is also added.

* test/nonblocking/interleaved.c tests a combination of interleaved

file types. This is to test the bug fix in r1758.

o New optimization

* Prior to this release, PnetCDF wraps each MPI read/write call in put/get

APIs with two MPI_File_set_view(). One is before the MPI read/write call

to take advantage of MPI's fileview feature for accessing non-contiguous

file locations. The other is after the MPI read/write call to make the

whole file visible, as the root process may write to file header later

in the data mode and it alone cannot make a call to MPI_File_set_view()

because the function is collective.

In this release, the second MPI_File_set_view() has been removed. The

root process's fileview is changed to always keep the whole file header

visible. Saving a collective call to MPI_File_set_view() is expected to

improve some performance.

o Semantics updates

* Header consistency mechanism has been updated. See README.consistency for

details.

* The use of NC_SHARE is also revised. See README.consistency for details.

o Other updates:

* The subfiling feature is now disabled in ncmpidump and ncmpidiff until

a bug fix to allow reading the master file with the number of processes

that is smaller than the number of subfiles.

* The attribute names reserved for subfiling feature are now changed to

use a prefix of "_PnetCDF_SubFiling". The leading "_" underscore is a CDL

convention.

* The flexible APIs now allow argument buftype to be MPI_DATATYPE_NULL.

In this case, argument bufcount is ignored and argument buf's data type

must match the data type of variable defined in the file - no data

conversion will be done. This extension makes the flexible APIs be

able to correspond to the netCDF APIs whose names do not contain a data

type, e.g. nc_put_vara().

* Type conversion between NC_BYTE and unsigned char no longer checks for

out of range error (NC_ERANGE). CDF file format specification make a

special case for this kind of data type conversion. See: "Note on byte

data" in the format specification.

* Conform with the CDF-2 and CDF-5 formats that names are normalized

according to Unicode NFC normalization rules during encoding as UTF-8

for storing in the file header.

* A new configure option, --enable-large-file-test, to enable testing

I/O on large files and large variables. Note the testing programs will

run very slowly.

o Bug fixes

* fix the bug for flexible get_varn API. When buftype is noncontiguous, the

bug forgot to "unpacks" the temporary buffer that reads data from file to

the user buffer using buftype.

* fix the bug in blocking flexible get APIs when buftype is noncontiguous,

swap is needed, type conversion is not. The bug sets a NULL pointer to

the read buffer and passes it to MPI_File_read functions. See r1815.

* fix the bug for the NetCDF special case when there is only one record

variable and the record size is not four-byte aligned. In this case,

NetCDF spec says the alignment must dropped (no padding). See r1791.

* fix the bug in nonblocking APIs when requests are resorted into

nonoverlapping groups and the first group of file types are interleaved,

it was mistakenly identified as non-interleaved. See r1758.

* fix the setting for PNETCDF_RELEASE_DATE in configure.in. Using read

command together with IFS does not work as expected in bash 4.3.11. See

http://lists.mcs.anl.gov/pipermail/parallel-netcdf/2014-July/001586.html

for further detailed discussion.

-------------------------------------

version 1.5.0 (July 8, 2014)

-------------------------------------

o New features

* A new configure option "--disable-in-place-swap" is added. It disables

the byte-swap operations running in-place on the user's write buffers.

The purpose of providing this option is to deal with the problem when a

Fortran program uses a immutable buffer for put APIs, e.g. the buffer is

declared as a PARAMETER, and in-place byte swap on this buffer causes

segmentation fault. See discussion threads of

http://lists.mcs.anl.gov/pipermail/parallel-netcdf/2013-July/001498.html