Sponsor:
Advanced Scientific Computing Research (ASCR) under the U.S. Department of Energy Office of Science (Office of Science)
Project Team Members:
Northwestern University
The HDF Group
- Quincey Koziol
- Gerd Herber
Argonne National Laboratory
North Carolina State University
- Nagiza Samatova
- Sriram Lakshminarasimhan
Damsel API Documentation
Damsel Library functions fall into the following categories.- Damsel environment functions
- Damsel Data Model functions\a>
- Damsel Container functions
- Damsel Tag functions
- Damsel Entity function
- Damsel Collection function
- Damsel Datatype function
- Damsel trait function
Damsel Environment Functions
There are two functions to initialize and finalize Damsel Library. These functions are prefixed by DMSLlib_.-
damsel_err_t DMSLlib_init
The Damsel environment is initialized by calling the following function.
damsel_err_t DMSLlib_init(damsel_library *lib); Input: *lib - which is of type damsel_library Return Value: damsel_err_t - which is DMSL_OK on success, and in case of error the value will be DMSL_ERROR_GENERAL, etc.
-
damsel_err_t DMSLlib_finalize
The Damsel environment is finalized, i.e. all resource allocated by Damsel internally will be freed once exiting the environment by calling the following function.
damsel_err_t DMSLlib_finalize(damsel_library *lib); Input: *lib - which is of type damsel_library Return Value: damsel_err_t - which is DMSL_OK on success, and in case of error the value will be DMSL_ERROR_GENERAL, etc.
Damsel Model Functions
A Damsel model is a in-memory store that describes the data model from the application perspective. The functions that are used to describe damsel data model are prefixed by DMSLmodel_.DMSLmodel_create
damsel_err_t DMSLmodel_create(damsel_handle_type app_handle_type, damsel_model *model);
DMSLmodel_create is used to create a model.
app_handle_type (IN)
The only supported type at this time is DAMSEL_HANDLE_TYPE_HANDLE64.
model (OUT)
A returned pointer for future reference to the created model.
Return Value:
DMSL_OK on success, and in case of error the value will be DMSL_ERROR_GENERAL.
DMSLmodel_map_tag
damsel_err_t DMSLmodel_map_tag(void * const tag_buffer, damsel_container app_handles, damsel_handle_ptr tag_handle);
DMSLmodel_map_tag is used to map data with a tag to the handles in a container.
tag_buffer (IN)
holds the data values
app_handles (IN)
type is damsel_container and holds application side/in-memory damsel handle of entities
tag_handle (IN)
type is damsel_handle_ptr and holds application-side tag handle whose values are being mapped
Return Value:
damsel_err_t which is DMSL_OK on success, and in case of error the value will be DMSL_ERROR_GENERAL, etc.
DMSLmodel_attach
damsel_err_t DMSLmodel_attach(damsel_model model_id, const char *filename, MPI_Comm comm, damsel_trait_bundle trait); Input: model_id - which is of type damsel_model, i.e. model being attached to file *filename - a const *char, name of the file to which we are attaching comm - which is of type MPI_Comm i.e.MPI communicator trait - which is of type damsel_trait_bundle, NULL is used if there are no traits Return Value: damsel_err_t - which is DMSL_OK on success, and in case of error the value will be DMSL_ERROR_GENERAL, etc.
DMSLmodel_transfer_async
damsel_err_t DMSLmodel_transfer_async( damsel_model model_id, damsel_transfer_type transfer_type, damsel_request_t *transfer_request) Input: damsel_model - model being queried damsel_transfer_type transfer_type - type of transfer: DAMSEL_TRANSFER_TYPE_READ, DAMSEL_TRANSFER_TYPE_WRITE, DAMSEL_TRANSFER_TYPE_REOPEN damsel_request_t *transfer_request - request object for the asynchronous operation Return Value: damsel_err_t - which is DMSL_OK on success, and in case of error the value will be DMSL_ERROR_GENERAL, etc.
DMSLmodel_populate
If a file is opened in read mode, then we can populate the model with information from the file, i.e. get the tuple information in the model description, metadata only, not "raw" data
damsel_err_t DMSLmodel_populate(damsel_model model_id) Input: model_id - which is of type damsel_model, i.e. model being attached to file Return Value: damsel_err_t - which is DMSL_OK on success, and in case of error the value will be DMSL_ERROR_GENERAL, etc.
DMSLmodel_close
The following API is used to close a damsel model
damsel_err_t DMSLmodel_close(damsel_model); Input: model_id - which is of type damsel_model. It is the model being attached to file Return Value: damsel_err_t - which is DMSL_OK on success, and in case of error the value will be DMSL_ERROR_GENERAL, etc.
DMSLmodel_map_handles
The following API is used to map data with a tag to the handles in a container, mappings created with this call (and the wrappers to this call) will exist until the model is closed.
damsel_err_t DMSLmodel_map_handles(damsel_containerapp_container, damsel_container model_container) Input: app_container - which is of type damsel_container, and describes application side in memory handles model_container - which is of type damsel_container, and describes file side handles Return Value: damsel_err_t which is DMSL_OK on success, and in case of error the value will be DMSL_ERROR_GENERAL, etc.
DMSLmodel_map_handles_inventing_file_handles
The following API is used to map data with a tag to the handles in a container, this maps handles from memory side into the file, allowing damsel to choose its own file-side handles.
damsel_err_t DMSLmodel_map_handles_inventing_file_handles(damsel_container app_container) Input: app_container - which is of type damsel_container, and describes application side in memory handles Return Value: damsel_err_t which is DMSL_OK on success, and in case of error the value will be DMSL_ERROR_GENERAL, etc.
DMSLmodel_map_handles_inventing_file_handles_with_permute
The following API is used to maps handles from memory side into the file, allowing damsel to choose its own file-side handles, but with the permutation of app-side data chosen by the supplied callback
damsel_err_t DMSLmodel_map_handles_inventing_file_handles_with_permute(damsel_container app_container, void *permute_callback) Input: app_container - which is of type damsel_container, and describes application side in memory handles permute_callback - Return Value: damsel_err_t which is DMSL_OK on success, and in case of error the value will be DMSL_ERROR_GENERAL, etc.
DMSLmodel_map_single_handle_inventing_file_handle
The following API is used to map a single handle into the file-side handle.
damsel_err_t DMSLmodel_map_single_handle_inventing_file_handle(damsel_model m, damsel_handle h) Input: m - damsel_model h - damsel_handle, which will be mapped to the file side handle Return Value: damsel_err_t which is DMSL_OK on success, and in case of error the value will be DMSL_ERROR_GENERAL, etc.
DMSLmodel_map_global_handles
The following API is used to map app-side handles on calling process rank to the handles on process ranks specified in proc_ranks list.
damsel_err_t DMSLmodel_map_global_handles(damsel_container app_container, int *proc_ranks, damsel_handle_ptr remote_handles ); Input: app_container - which is of type damsel_container, and has application space handles which are being mapped to the handles on remote ranks proc_ranks - which is an array of ranks of processors with remote handles remote_handles - it is a vector of application-space handles on remote processes Return Value: damsel_err_t which is DMSL_OK on success, and in case of error the value will be DMSL_ERROR_GENERAL, etc.
DMSLmodel_wait
The following API is used to wait for an async read/write request to complete.
damsel_err_t DMSLmodel_wait(damsel_request_t transfer_request, damsel_status_t *request_status) Input: transfer_request - which is of type damsel_request_t, request to wait for request_status - status of the request Return Value: damsel_err_t which is DMSL_OK on success, and in case of error the value will be DMSL_ERROR_GENERAL, etc.
DMSLmodel_transfer_sync
The following API is used for synchronous transfer of data to/from file depending on the transfer mode.
damsel_err_t DMSLmodel_transfer_sync(damsel_model model_id, damsel_transfer_type transfer_type); Input: model_id - which is of type damsel_model and describes the model being queried transfer_type - which is of type damsel_transfer_type, and describe type of transfer: DAMSEL_TRANSFER_TYPE_READ, DAMSEL_TRANSFER_TYPE_WRITE, DAMSEL_TRANSFER_TYPE_REOPEN Return Value: damsel_err_t - which is DMSL_OK on success, and in case of error the value will be DMSL_ERROR_GENERAL, etc.
DMSLmodel_start_read
The following API is used to start a read transaction, as in some places, it has been necessary to read some data from the file in order to decide what else should be read from the file.
void DMSLmodel_start_read(damsel_model m) Input: m - which is of type damsel_model, model being queried
DMSLmodel_stop_read
The following API is used to stop a read operation.
void DMSLmodel_stop_read(damsel_model m) Input: m - which is of type damsel_model, model being queried
DMSLmodel_get_container_infos
The following API is used to return information about all containers known in the attached file.
damsel_err_t DMSLmodel_get_container_infos(damsel_model model_id, damsel_container_buf_type *container_buf) Input: model_id - which is of type damsel_model, and describes the model being queried container_buf - which is of type damsel_container_buf_type, and describes the information about a container (i.e. sequence, vector, tree) Return Value: damsel_err_t which is DMSL_OK on success, and in case of error the value will be DMSL_ERROR_GENERAL, etc.
DMSLmodel_get_tag_infos
The following API is used to get tag information in model. tag_buf must be big enough to accomodate as many damsel_tag_buf_type structs as returned by DMSLmodel_get_tuple_count.
damsel_err_t DMSLmodel_get_tag_infos(damsel_model model_id, damsel_tag_buf_type *tag_buf); Input: model_id - which is of type damsel_model, and describes the model being queried tag_buf - which is of type damsel_tag_buf_type, and describes the vector of tag buffers, of length retrieved by DMSLget_info_count Return Value: damsel_err_t which is DMSL_OK on success, and in case of error the value will be DMSL_ERROR_GENERAL, etc.
DMSLmodel_get_entity_infos
The following API is used to populate the supplied buffer with information about entity-definitions. buf must be big enough to hold the number of entity-definitions returned by DMSLentity_get_count().
damsel_err_t DMSLmodel_get_entity_infos(damsel_model m, damsel_entity_buf_type *buf) Input: m - which is of type damsel_model, and describes the model being queried buf - which is of type damsel_entity_buf_type, and describes the information about an entity/sequence of entities Return Value: damsel_err_t which is DMSL_OK on success, and in case of error the value will be DMSL_ERROR_GENERAL, etc.
DMSLmodel_get_entity_count
The following API is used to return the number of entities in a model.
int DMSLmodel_get_entity_count(damsel_model m) Input: m - which is of type damsel_model, and describes the model being queried Return Value: int - number/count of entities created in the model m.
DMSLmodel_get_collection_count
The following API is used to return the number of collections in a model.
int DMSLmodel_get_collection_count(damsel_model model_id) Input: m - which is of type damsel_model, and describes the model being queried Return Value: int - number/count of collections created in the model m.
DMSLmodel_get_collection_infos
The following API is used to return information about all collections known in the attached file.
damsel_err_t DMSLmodel_get_collection_infos(damsel_model model_id, damsel_collection_buf_type *collection_buf) Input: model_id - which is of type damsel_model, and describes the model being queried collection_buf - which is of type damsel_collection_buf_type, and describes the information about a collection Return Value: damsel_err_t which is DMSL_OK on success, and in case of error the value will be DMSL_ERROR_GENERAL, etc.
DMSLmodel_close_collection_infos
The following API is used to close a collection infos structure, so that damsel can release holds on internal memory used in the structure.
damsel_err_t DMSLmodel_close_collection_infos(damsel_model model_id, damsel_collection_buf_type *collection_buf) Input: model_id - which is of type damsel_model, and describes the model being queried collection_buf - which is of type damsel_collection_buf_type, and describes the information about a collection to be closed Return Value: damsel_err_t which is DMSL_OK on success, and in case of error the value will be DMSL_ERROR_GENERAL, etc.
DMSLmodel_get_tuple_count
The following API is used to return the number of containers and tags in a model.
void DMSLmodel_get_tuple_count(damsel_model m, int *container_count, int *tag_count) Input: m - which is of type damsel_model, and describes the model being queried container_count - returns number of containers tag_count - returns number of tags
Damsel Container Functions
A Damsel container is a holder of a set of damsel_handle. These functions are prefixed by DMSLcontainer_.
DMSLcontainer_create_sequence
The container constructor below is used to to create a container with handles of a sequence of contiguous integral values.
damsel_err_t DMSLcontainer_create_sequence(damsel_model model, damsel_handle base, unsigned int quantity, size_t step, damsel_container *container); Input: model - which is of type damsel_model, a sequence of damsel_handles will be added to the damsel model base - which is of type damsel_handle, and describes the start value of damsel_handle in a sequence step - which is of type size_t, and describes the step size between two consecutive damsel_handle values in a sequence *container - which is of type damsel_container, output of the function call. This container will be used while adding tags, etc to the sequence of handles in the container. Return value: damsel_err_t - which is DMSL_OK on success, and in case of error the value will be DMSL_ERROR_GENERAL, etc.
DMSLcontainer_create_vector
The container constructor below is used to to create a vector container with the integral values of damsel_handle in an array.
damsel_err_t DMSLcontainer_create_vector(damsel_model model, damsel_handle *values, unsigned int num, damsel_container *container) Input: model - which is of type damsel_model, a sequence of damsel_handles will be added to the damsel model values - which is of type *damsel_handle, and describes an array of damsel_handle values num - which is of type unsigned int, and describes the number of values *container - which is of type damsel_container, output of the function call. This container will be used while adding tags, etc to the sequence of handles in the container. Return value: damsel_err_t - which is DMSL_OK on success, and in case of error the value will be DMSL_ERROR_GENERAL, etc.
DMSLcontainer_tree_create
DMSLcontainer_tree_add
DMSLcontainer_tree_add_fast
DMSLcontainer_sequence_get_contents
The following function is used to get the contents of damsel container sequence.
damsel_err_t DMSLcontainer_sequence_get_contents(damsel_model model_id, damsel_container seq_container_id, damsel_handle_ptr handle_ptr, size_t *count, size_t *increment Input: model_id - which is of type damsel_model, a sequence of damsel_handles will be added to the damsel model seq_container_id - which is of type damsel_container, and describes the sequence container id handle_ptr - which is of type damsel_handle_ptr, and gives the returned start handle of the container *count - which is of type size_t, and has the number of handles in container *increment - which is of type size_t, and gives the increment in the values Return type: damsel_err_t - which is DMSL_OK on success, and in case of error the value will be DMSL_ERROR_GENERAL, etc.
DMSLcontainer_vector_get_contents
The following functions are used to get the contents of damsel vector container.
damsel_err_t DMSLcontainer_vector_get_contents(damsel_model model_id, damsel_container vec_container_id, damsel_handle_ptr *const handle_ptr_ptr, size_t *count) Input: model_id - which is of type damsel_model, a vector of damsel_handles will be added to the damsel model vec_container_id - which is of type damsel_container, and describes the vector container id handle_ptr_ptr - which is of type damsel_handle_ptr, and gives pointer to pointer to handles in this vector *count - which is of type size_t, and has the number of handles in container Return Value: damsel_err_t - which is DMSL_OK on success, and in case of error the value will be DMSL_ERROR_GENERAL, etc.
DMSLcontainer_handle_at_position
The following functions are used to get the handles value at a specific index.
damsel_handle DMSLcontainer_handle_at_position(damsel_container c, int i) Input: c - which is of type damsel_container, i - which is of type int, and describes the position/index of a damsel handle inside damsel_container Return Value: damsel handle of at position i in container c
DMSLcontainer_count
The following function is used to get the handles count inside a container while reading a damsel file.
int DMSLcontainer_count(damsel_container c) Input: c - which is of type damsel_container, given a damsel_container this function returns the number of handles in the container Return value: i - which is of type int, and gives the number of handles in damsel_container c
DMSLcontainer_get_type
The following API is used to return the type of container (sequence, vector, or tree).
int DMSLcontainer_get_type(damsel_container c) Input: c - which is of type damsel_container, and describes the container for which type is requested Return Value: int - type of container.
DMSLcontainer_take
The following function takes a reference on a container.
damsel_err_t DMSLcontainer_take(damsel_container c); Input: c - which is of type damsel_container Return type: damsel_err_t - which is DMSL_OK on success, and in case of error the value will be DMSL_ERROR_GENERAL, etc.
DMSLcontainer_release
The following function indicates that a reference to the container is no longer used.
damsel_err_t DMSLcontainer_release(damsel_container c) Input: c - which is of type damsel_container, and represents the container that is no longer used Return type: damsel_err_t - which is DMSL_OK on success, and in case of error the value will be DMSL_ERROR_GENERAL, etc.
DMSLcontainer_close
The following function closes the specified container. only containers that have been explicitly created by the application using container_create_* calls should be closed in this way. There maybe needs to be a way for the app to indicate that it has released other containers returned by the library that might be shared elsewhere and so need to not always be released when the application has finished with them. (Need to clean this description, copied from the .h file directly)
damsel_err_t DMSLcontainer_close(damsel_container c) Input: c - which is of type damsel_container, and represents the container to be closed Return Value: damsel_err_t - which is DMSL_OK on success, and in case of error the value will be DMSL_ERROR_GENERAL, etc.
Damsel Tag Functions
One can tag a set of damsel_handles in a container with a name, I/O buffer, and data type of data in the buffer. A tag can be considered as a solution variable, for example, associated to all vertices of a grid. A tag is represented by a name that must be uniquely defined in a model, a data type, and an I/O buffer pointing to the memory space where the data is stored. Damsel tag functions are prefixed by DMSLtag_.
DMSLtag_define
The APIs below are used to define a tag and tag it to the top-level handles in a container.
damsel_err_t DMSLtag_define(damsel_model model, damsel_handle *tag_handle, damsel_data_type type, const void *buffer) Input: model - which is of type damsel_model tag_handle - which is of type damsel_handle, and describes the handle of tag type - which is damsel_data_type, and can be any of the primitive or user defined Damsel data type buffer - data buffer containing data values of type 'type' Return Value: damsel_err_t - which is DMSL_OK on success, and in case of error the value will be DMSL_ERROR_GENERAL, etc.
Damsel Entity Functions
Damsel entity functions are prefixed by DMSLentity_.
DMSLentity_define
The API below is used to define entities. It declares that handles in a handle container are handles to geometric entities.
damsel_err_t DMSLentity_define(damsel_container app_container, damsel_entity_type entity_type, size_t vertices_per_entity, damsel_container vertec_container) Input: app_container - a container with the application-space handles for the entities damsel_entity_type - entity types of entities being defined (DMSL_ENTITY_TYPE_*) vertices_per_entity - number of vertices per entity; this is used when the damsel_entity_type has variable number of vertices vertex_container - container of application-space vertex handles Return Value: damsel_err_t - which is DMSL_OK on success, and in case of error the value will be DMSL_ERROR_GENERAL, etc.
DMSLentity_define_fast
Damsel Collection Functions
Damsel collection functions are prefixed by DMSLcoll_.
DMSLcoll_create
damsel_collection DMSLcoll_create(damsel_model model_id, damsel_handle_ptr app_handle_ptr, damsel_container container_id, damsel_collection_type coll_type)
model_id (IN)
type is damsel_model
app_handle_ptr (IN)
container_id (IN)
type is damsel_container and holds application side/in-memory damsel handle of entities
coll_type (IN)
type is damsel_collection_type
Return Value:
damsel_collection
Damsel Data Types Functions
Below are the primitive data types predefined in Damsel.
DAMSEL_DATA_TYPE_BYTES DAMSEL_DATA_TYPE_INTEGER DAMSEL_DATA_TYPE_INT64 DAMSEL_DATA_TYPE_FLOAT DAMSEL_DATA_TYPE_DOUBLE DAMSEL_DATA_TYPE_HANDLE DAMSEL_DATA_TYPE_SHORTINT
DMSLtype_opaque_create
It creates an opaque type, of specified byte count.
damsel_data_type DMSLtype_opaque_create (size_t bytecount); Input: bytecount - byte count Return Value: damsel_data_type - a damsel data type identifier if successful; otherwise returns a negative value
Note: This must be run on all ranks at once to ensure internal ID numbering matches up across ranks.
DMSLtype_array_create
It creates an array type.
damsel_data_type DMSLtype_array_create ( damsel_data_type base, unsigned ndims, const size_t dims); Input: base - base type for array ndims - number of dimensions for array dims - dims[ndims], size of array's dimensions Return Value: damsel_data_type - a damsel data type identifier if successful; otherwise returns a negative value
DMSLtype_vector_create
It creates a vector data type. A vector contains data elements of damsel_vector_t, which can be an array of any size. Each element in a vector may have different size.
damsel_data_type DMSLtype_vector_create (damsel_data_type base, size_t max_len); Input: base - the base type for the vector type max_len - max length of array type Return Value: damsel_data_type - a damsel data type identifier if successful; otherwise returns a negative value
Note: HDF5 parallel I/O does not support writing variable-length data. For now, it will use fixed length array type of size max_len.
DMSLtype_enum_create
It creates an enum data type. An enum type contains members of damsel_enum_member_t, which defines the <name, value> pairs of enum members.
damsel_data_type DMSLtype_enum_create (unsigned nmembs, char **names, int *values); Input: nmembs - number of members of the enum type names - names of the enum type values - values of the enum type. If values is NULL, enum values start with 0 Return Value: damsel_data_type - a damsel data type identifier if successful; otherwise returns a negative value
DMSLtype_struct_create
It creates a struct data type. A struct type contains members of damsel_struct_member_t, which defines the the members of the struct. A struct member has three fields: name, offset, and type.
damsel_data_type DMSLtype_struct_create (unsigned nmembs, char **names, damsel_data_type *types, size_t *offsets, size_t tsize); Input: nmembs - number of members of the struct type names - names of the struct type types - types of the struct type offsets - offsets of the struct members tsize - total size of the structure type Return Value: damsel_data_type - a damsel data type identifier if successful; otherwise returns a negative value
Damsel Trait Functions
There are four user-level APIs for handling damsel traits.DMSLtrait_create
It creates a damsel trait.
damsel_err_t DMSLtrait_create(damsel_trait_bundle *tbp); Input: tbp - a pointer to damsel trait bundle Return Value: damsel_err_t - returns DMSL_OK if successful
DMSLtrait_release
It release the specified damsel trait.
damsel_err_t DMSLtrait_release(damsel_trait_bundle tb); Input: tbp - damsel trait bundle Return Value: damsel_err_t - returns DMSL_OK if successful
DMSLtrait_put
It puts a key/value pair to the associated damsel trait.
damsel_err_t DMSLtrait_put(damsel_trait_bundle tb, char *key, char *value); Input: tb - damsel trait bundle key - key for a trait value - corresponding value for a key Return Value: damsel_err_t - returns DMSL_OK if successful
Note: key and value will be copied, so they can be released by the caller.
DMSLtrait_get
It gets a key/value pair from the specified damsel trait. The string returned here will be deallocated on release of the trait bundle. NULL can be passed as parameter, in which case NULL will be returned: NULL will be treated as if its an empty trait bundle.
damsel_err_t DMSLtrait_get(damsel_trait_bundle tb, char *key, char **value); Input: tb - damsel trait bundle key - key value to retrive value - pointer to value associated with the key specified Return Value: damsel_err_t - returns DMSL_OK if successful