# # Copyright (C) 2012, Northwestern University and Argonne National Laboratory # See COPYRIGHT notice in top-level directory. # # $Id$ # This directory contains several example programs. Detailed descriptions of each program, compile and run instructions, and sample outputs are provided at the beginning of each file. Most of the example programs are developed with C, C++, F77, and F90 versions. ./tutorial A directory contains example programs that use different parallel I/O strategies, including: Parallel I/O from the master process Concurrent I/O on separate files Real parallel I/O on shared files Using "flexible" APIs Using non-blocking APIs Using non-blocking buffered write APIs Detailed descriptions for these programs can be found in https://parallel-netcdf.github.io/wiki/QuickTutorial.html ./C/block_cyclic.c ./CXX/block_cyclic.cpp ./F77/block_cyclic.f ./F90/block_cyclic.f90 This example makes a number of nonblocking API calls, each to write a block of columns into a 2D integer variable in a file. In other words, data partitioning pattern is a block-cyclic along X dimension. The pattern is described by the rank IDs if run with 4 processes. 0, 0, 1, 1, 2, 2, 3, 3, 0, 0, 1, 1, 2, 2, 3, 3, 0, 0, 1, 1, 2, 2, 3, 3, 0, 0, 1, 1, 2, 2, 3, 3, 0, 0, 1, 1, 2, 2, 3, 3, 0, 0, 1, 1, 2, 2, 3, 3, 0, 0, 1, 1, 2, 2, 3, 3, 0, 0, 1, 1, 2, 2, 3, 3, 0, 0, 1, 1, 2, 2, 3, 3, 0, 0, 1, 1, 2, 2, 3, 3, 0, 0, 1, 1, 2, 2, 3, 3, 0, 0, 1, 1, 2, 2, 3, 3, 0, 0, 1, 1, 2, 2, 3, 3, 0, 0, 1, 1, 2, 2, 3, 3, 0, 0, 1, 1, 2, 2, 3, 3, 0, 0, 1, 1, 2, 2, 3, 3, 0, 0, 1, 1, 2, 2, 3, 3, 0, 0, 1, 1, 2, 2, 3, 3, 0, 0, 1, 1, 2, 2, 3, 3, 0, 0, 1, 1, 2, 2, 3, 3 ; ./C/collective_write.c ./CXX/collective_write.cpp This example defined NUM_VARS 3D integer non-record variables in a file. All variables are partitioned among processes in a 3D block-block-block fashion. The I/O is carried out by making NUM_VARS calls to ncmpi_put_vara_int_all(), one for each variable. Performance measurements are reported in the standard output. ./C/column_wise.c ./CXX/column_wise.cpp ./F77/column_wise.f ./F90/column_wise.f90 This example makes a number of nonblocking API calls, each writes a single column of a 2D integer variable defined in a file. The data partitioning pattern among processes is a cyclic along dimension X, illustrated below by the process rank IDs if run with 4 processes 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3, 0, 1, 2, 3 ; ./C/flexible_api.c ./CXX/flexible_api.cpp ./F77/flexible_api.f ./F90/flexible_api.f90 This example shows how to use PnetCDF flexible APIs, ncmpi_put_vara_all() and ncmpi_iput_vara() to write two 2D array variables (one is of 4-byte integer byte and the other float type) in parallel. Local buffers have a ghost cell of length 3 surrounded along each dimension. ./C/get_info.c ./CXX/get_info.cpp ./F77/get_info.f ./F90/get_info.f90 These two example programs print the PnetCDF and MPI-IO hints to the standard output. ./C/hints.c ./CXX/hints.cpp ./F77/hints.f ./F90/hints.f90 This example sets two PnetCDF hints: nc_header_align_size and nc_var_align_size and prints the hint values, the header size, header extent, and variables' starting file offsets. ./C/mput.c This example shows how to use a single call of ncmpi_mput_vara_all() to write a sequence of requests with arbitrary array indices and lengths. It is intended to run on 4 processes. If more processes were allocated, the extra processes write zero-length requests. The offsets and lengths are just some random numbers to make the output look like: 3, 3, 3, 1, 1, 0, 0, 2, 1, 1, 0, 2, 2, 2, 3, 1, 1, 2, 2, 2, 1, 1, 2, 3, 3, 3, 0, 0, 1, 1, 0, 0, 0, 2, 1, 1, 1, 3, 3, 3 ; ./C/nonblocking_write.c ./CXX/nonblocking_write.cpp ./F77/nonblocking_write.f ./F90/nonblocking_write.f90 This example is almost the same as to collective_write.c but using nonblocking APIs instead. ./C/put_vara.c ./CXX/put_vara.cpp ./F77/put_vara.f ./F90/put_var.f90 This example shows how to use nfmpi_put_vara_int_all() to write a 2D 4-byte integer array in parallel. The data partitioning pattern among processes is a *-block in Fortran order. It is described by the process rank IDs as below if run on 4 processes. 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3 ; ./C/put_varn_float.c ./CXX/put_varn_float.cpp ./F77/put_varn_real.f ./F90/put_varn_real.f90 This example makes a single call of ncmpi_put_varn_float_all() to write a sequence of one-element requests with arbitrary array indices. All subrequest indices, starts[], are within the boundaries of a single variable. See comments at the beginning of the source file for compile, run instructions, and example output. ./C/put_varn_int.c ./CXX/put_varn_int.cpp ./F77/put_varn_int.f ./F90/put_varn_int.f90 This example makes a single call of ncmpi_put_varn_int_all() to write a sequence of requests with arbitrary array indices and lengths. All subrequests (starts[] and counts[]) are within the boundaries of a single variable. See comments at the beginning of the source file for compile, run instructions, and example output. ./C/create_open.c This example shows how to use to create a new file, close it, and open the file for read only. ./C/global_attributes.c This example creates a new file and add 2 global attributes, one is of text type and the other 10 short integers. The file is closed and re-opened to read back the attributes. ./C/get_vara.c ./CXX/get_vara.cpp This example is the read counterpart of example put_vara.c. It shows how to use ncmpi_get_vara_int_all() to read a 2D 4-byte integer array in parallel. It also reads a global attribute and two attribute of variable named "var". The data partitioning pattern is a column-wise partitioning across all processes. ./C/transpose.c ./CXX/transpose.cpp ./F77/transpose.f ./F90/transpose.f90 This example writes dimensional-transposed 3D arrays using varm APIs. For example, when Z=4, Y=6, and X=8, an array partitioned among 4 processes (P0,P1,P2,P3) and organized in dimension ZYX are illustrated below: P0: var[Z=0][*][*]= 0, 1, 2, 3, P1: var[Z=0][*][*]= 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, P2: var[Z=0][*][*]= 48, 49, 50, 51, P3: var[Z=0][*][*]= 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, P0: var[Z=1][*][*]= 96, 97, 98, 99, P1: var[Z=1][*][*]=100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143, P2: var[Z=1][*][*]=144, 145, 146, 147, P3: var[Z=1][*][*]=148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191 ; When writing the subarray in parallel to a file, the array contents in file are: var[Z=0][*][*]: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, var[Z=1][*][*]: 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110, 111, 112, 113, 114, 115, 116, 117, 118, 119, 120, 121, 122, 123, 124, 125, 126, 127, 128, 129, 130, 131, 132, 133, 134, 135, 136, 137, 138, 139, 140, 141, 142, 143, 144, 145, 146, 147, 148, 149, 150, 151, 152, 153, 154, 155, 156, 157, 158, 159, 160, 161, 162, 163, 164, 165, 166, 167, 168, 169, 170, 171, 172, 173, 174, 175, 176, 177, 178, 179, 180, 181, 182, 183, 184, 185, 186, 187, 188, 189, 190, 191 ; When writing the transposed subarray (XYZ) in parallel to a file, the file contents are: var[Z=0][*][*]= 0, 48, 96, 144, var[Z=1][*][*]= 1, 49, 97, 145, 8, 56, 104, 152, 9, 57, 105, 153, 16, 64, 112, 160, 17, 65, 113, 161, 24, 72, 120, 168, 25, 73, 121, 169, 32, 80, 128, 176, 33, 81, 129, 177, 40, 88, 136, 184, 41, 89, 137, 185, var[Z=2][*][*]= 2, 50, 98, 146, var[Z=3][*][*]= 3, 51, 99, 147, 10, 58, 106, 154, 11, 59, 107, 155, 18, 66, 114, 162, 19, 67, 115, 163, 26, 74, 122, 170, 27, 75, 123, 171, 34, 82, 130, 178, 35, 83, 131, 179, 42, 90, 138, 186, 43, 91, 139, 187, var[Z=4][*][*]= 4, 52, 100, 148, var[Z=5][*][*]= 5, 53, 101, 149, 12, 60, 108, 156, 13, 61, 109, 157, 20, 68, 116, 164, 21, 69, 117, 165, 28, 76, 124, 172, 29, 77, 125, 173, 36, 84, 132, 180, 37, 85, 133, 181, 44, 92, 140, 188, 45, 93, 141, 189, var[Z=6][*][*]= 6, 54, 102, 150, var[Z=7][*][*]= 7, 55, 103, 151, 14, 62, 110, 158, 15, 63, 111, 159, 22, 70, 118, 166, 23, 71, 119, 167, 30, 78, 126, 174, 31, 79, 127, 175, 38, 86, 134, 182, 39, 87, 135, 183, 46, 94, 142, 190, 47, 95, 143, 191 ; ./C/vard_int.c ./CXX/vard_int.cpp ./F77/vard_int.f ./F90/vard_int.f90 These examples show how to use vard APIs to write/read record and fixed-size variables. ./C/bput_varn_uint.c ./F77/bput_varn_int8.f These examples show how to use nonblocking bput_varn APIs ./C/i_varn_int64.c ./F77/i_varn_real.f These examples show how to use nonblocking iput_varn and iget_varn APIs ./C/fill_mode.c ./CXX/fill_mode.cpp ./F77/fill_mode.f ./F90/fill_mode.f90 These examples show how to enable file mode ./C/req_all.c This example show how to use NC_REQ_ALL to flush all pending nonblocking requests without providing the request IDs and status array. ./C/nonblocking_write_in_def.c This example is the same as nonblocking_write.c except all nonblocking write requests (calls to iput and bput) are posted in define mode. After entering to data mode, a single call to ncmpi_wait_all() to flush all pending nonblocking requests. ./burst_buffer/create_open.c This program shows an example of creating and opening a NetCDF file with the burst buffering feature enabled. ./burst_buffer/nonblocking.c This program shows an example of using non-blocking put APIs with the burst buffering feature enabled. ./C/pthread.c This program shows an example of using one-file-per-thread I/O operation when the thread-safe capability is enabled.