bfl_util- Big File Utilities
The BFL_UTIL package provides a high-level interface for reading and writing logical files greater than 2GB in size (the typical UNIX limit on physical file size).
A logical "big file" is broken down into multiple physical files, each less than the maximum physical file size:
basePathname_bfl0 basePathname_bfl1 basePathname_bfl2 ... basePathname_bflN
When the size of an individual file reaches the maximum allowable size (a
configurable parameter), the next file in sequence is created. For
example, suppose a logical write to
xyz begins in its last
xyz_bfl22 fills up
before the write is complete, then
xyz_bfl23 is created and
the write continues.
The following code fragment creates a 7GB logical file filled with over a billion "Hello!" records:
#include "bfl_util.h" -- Big file utilities. BigFile file ; int i ; ... bflOpen ("pathname", "-write", &file) ; for (i = 0 ; i < 1*1024*1024*1024 ; i++) bflWrite (file, strlen ("Hello!") + 1, "Hello!", NULL) ; bflClose (file, NULL) ;
and the following code fragment counts the number of records in the file:
char buffer = "Hello!" ; -- Sized to hold a "Hello!" record. int numRecords = 0 ; ... bflOpen ("pathname", "-read", &file) ; while (!bflRead (file, sizeof buffer, buffer, NULL)) numRecords++ ; bflClose (file, NULL) ;
You can seek to an arbitrary position in the logical file as follows:
QuadWord position = qwCreate (0, 1234) ; ... bflSetPos (file, position) ; -- Seek to offset 1234 in file.
Removing a logical file deletes all of its physical files:
bflRemove ("pathname") ;
bflClose()- closes a big file.
bflGetPos()- gets the current I/O position in a big file.
bflOpen()- opens/creates a big file.
bflRead()- reads data from a big file.
bflRemove()- removes a big file.
bflSetPos()- sets the current I/O position in a big file.
bflWrite()- writes data to a big file.