|
|||
| |CI2 Home|Literature|Support|Documentation|Y2K|Order|License| | |||
 |
Chapter 5 PowerFail Protection
This chapter provides a general discussion of the use of the PowerFail Protection functions of C-Index/II. PowerFail Protection is a collection of features that enable the application designer to protect data from abnormal program termination and other data loss problems. This chapter should be read before using these functions. A detailed description of each function is provided in the C-Index/II Reference Guide. The buffer control features are provided to optimize the trade-off between full data security and maximum speed of operation. Changes that are made to C-Index/II files remain in memory buffers. It is possible that buffered changes may never be written to the disk, for example if the power fails, or the computer hardware fails, or the operator simply removes the diskettes before the index file is closed. Depending in part on how many buffers are declared, a small number of changes, or a very large number of changes may exist in memory. The number of buffers is specified using the function cibufnum or in the global variable bcbnum. Regardless of how many buffers exist in memory, however, the failure to write information to disk may cause portions or even the entire index to be unusable. For example, if the "root node", that is, the first place a search begins in the B+Tree, is not written to disk, an entire section of the file could "disappear". The information could be on disk, but it would be inaccessible to the application. To protect against such a case, it would be necessary to write every index file change to disk immediately, thus ensuring maximum data integrity. When a file is opened, the default "Integrity Level" setting of C-Index/II is to always write changes to disk immediately, which of course slows the processing of modifications to the file. In some applications, the penalty of slowness may be an unacceptable trade-off to providing this level of reliability. It would be better to have an in-between level of writing to disk that ensures data integrity in all reasonable cases. C-Index/II provides for these cases with the functions csetfile and cflush. Even when the writing to disk is performed after every change, it is still possible to lose information as a result of hardware failure. In the event that the index file has lost data, C-Index/II provides a way to recover the data that is on the disk using data recovery utility programs and file checking functions, bbuild, ccheck, dbuild, etc. It is important to understand that even with the tools provided in C-Index/II, you must evaluate your requirements for data integrity based on your particular application. For applications that are very critical in their operation, you must devise additional safeguards to those provided by C-Index/II. Fortunately, most applications are not so critical and the functions provided will be more than adequate. The two functions provided for implementing buffer control in your application are:
The csetfile function indicates how C-Index/II is to treat each file when changes are made. The integrity level (also called "security level") may be set in the file after it has been created. The level remains in effect until a new level is specified for the file. Typically the application program would create a new index file, immediately set the integrity level, and leave the setting there. The integrity level will remain in effect through all subsequent open and close operations. It is also possible to change the integrity level depending on conditions in the application.
The integrity levels are as follows:
The cflush function can be used in combination with setting the integrity level. The cflush function writes any changed index information currently in buffer memory to the disk and updates the index header information. Typically an application program may set the integrity level to 1 (to ensure a valid FAT table on the disk) and flush the changed buffers to the disk at times that are most appropriate. This minimizes time consuming disk accesses in those cases where program speed is critical. The csetfile and cflush functions only affect the operation of the system when changes to the file are made. In the event of reading information from the file, the system is always in fully buffered mode. The default Integrity Level is 3. A higher data Integrity Level is available, Level 4. This can be used to determine if a file might be corrupted as the result of abnormal program termination (resulting from power failure, hardware failure, or software failure). A special flag, called the "write interrupt flag", is set in the file to indicate that a write operation has been initiated. If the program fails before completing this task, it leaves the flag set in the file. Subsequent attempts to access this file will result in finding the flag set and will return the WRITEINTR error code. To use this feature, set Integrity Level to 4 after each file open or create:
cc = csetfile(psp, 4);
If a file write operation is interrupted while the file is in Integrity Level 4, subsequent attempts to open, read or write the file will return the WRITEINTR error (-40). The file probably has been corrupted when this occurs. The best course of action if this occurs is to rebuild the file. It is also possible to test the file integrity with the BCHECK utility or bcheck function. BCHECK tests for corruption of single-key information (but not multi-key information) and reset the write interrupt flag if no corruption is found. If corruption is discovered, it will be reported and the file should be rebuilt using REBUILD Utility or the cbuild and ccheck functions. The write interrupt flag can also be reset with the BCLEAR Utility or the clrwrit function. The Image Backup feature protects the file integrity in the event of program interruption while C-Index/II is writing to disk (due to power failure or program failure). Each time C-Index/II modifies a node in the file (or the header) a copy of the original data (the data "image") is saved in a separate file. If the application fails while modifying the file, the condition of the file can be restored using the backup information. For the purposes of this discussion, the file being protected will be referred to as the "main" file and the image backup file will be referred to as the "backup" file. The backup file is a special file that has a different format from a normal file. The header contains a literal copy of the information in the main file. The rest of the file contains backup copies of nodes. The size of the backed up node is slightly larger than a normal node, since it has additional information. The format of the backup node is the imgbcb structure (in the cindexwk.h header file). The application creates the backup file using imgcreate. The file should be opened for SHARED access for multi-user applications, otherwise EXCL is used. Once the backup file has been created, it is open and can be used for the imageback function to start backup. The backup file must be closed with the imgclose function and reopened for subsequent uses with the imgopen function. DO NOT use the bcreate, bopen or bclose functions for processing an image backup file, since this may cause the application to crash. To start the image backup process, the function imageback is called with the parameter structure pointers of the main file and the backup file. The main and backup files must already be open and the main file must have data security set to Integrity Level 4. After calling imageback, a backup of changed nodes or header information will be saved in the backup file. Closing the main file or the image backup file will cancel this backup action. The image backup file must be closed with imgclose. Before each node is written to the image backup file, a call is made to the optional callback function supplied by the application (pointed to by the variable psp->imagebackproc). The definition of the callback function is:
int EDECL backupproc(struct passparm PDECL *psp, struct imgbcb *bcbptr);
This callback can be used by the application to insert information into the imgbcb structure passed to the callback. The fields available for application use are:
void *custinfo1; FILHND custinfo2; int custinfo3; ALLOCHND custinfo4;
When the image backup information is restored, this information can be examined. One use for the information would be to advise the user about what type of process was being performed when the crash occurred. Another usage would be to enter into an application error log about details of the crash (such as date, time, application state, etc.) for purposes of isolating application or environmental problems. The backup process treats the changes as a unit between the call to strtwrit and the call to endwrit. For applications that do not explicitly call strtwrit/endwrit, C-Index/II automatically calls those functions at the beginning and end of each function that performs a write to the file. Many applications will benefit from grouping several C-Index/II function calls together as a unit by calling strtwrit before the group and endwrit after the group. For example, grouping the writes for an accounting transaction containing debits and credits can ensure that the accounts are maintained in balance. Image backup works well for both single-user and multi-user applications. For multi-user usage, a single backup file is shared between all the processes. If a process fails, other processes accessing the file will get the WRITEINTR error on the next attempt to open, read or write the file. The application must be written to gracefully recognize this condition and automatically restore the condition of the file, or advise the user of appropriate actions to take. To restore the main file, the application must call the imagerest function with the psp of the open main file and open backup file. This will automatically copy file changes from the backup file, restoring the file to the condition it was in before the last call to strtwrit. Before each node is restored to the main file, a call is made to another optional callback function supplied by the application (pointed to by the variable psp->imagerestproc). The information in the imgbcb buffer passed to the callback can be examined for information stored at the time image backup occurred. In addition, the application can examine the data in the buffer for accuracy, in the event that the backup file may have been corrupted. If the callback routine returns back any error code other than CCOK, the restore process will be halted and the supplied error code returned from imagerest. For example, if an error is detected in the buffer information, the BADNODE error could be returned. Or the restore process could be halted by returning a PROCINTR (not a good idea, but possible). The definition of the imagerestproc function is:
int EDECL restoreproc(struct passparm PDECL *psp, struct imgbcb *bcbptr);
An example of the steps for restoring the file after a system crash of a single user are as follows:
If the WRITEINTR flag is detected by another process in a multi-user system, the steps of opening the main file and image backup file can be skipped. Normally the last two steps will indicate that the file is OK. If this imposes an unreasonable performance penalty, these steps can be skipped unless other problems are reported using the file after it is restored (such as BADNODE, INCOMPLETE, etc.). The image backup feature imposes a performance penalty for two reasons. The required Integrity Level 4 feature adds two disk writes (to set and clear the write interrupt flag in the header). If the application performs an operation that involves modifying a single node between strtwrit and endwrit, there will be two disk extra writes for the write interrupt flag and an additional write for the backup of the node. This may be up to three times longer than the operation would take without Integrity Level 4 and image backup features. The overhead of the flag writes can be reduced by grouping operations together with strtwrit/endwrit. That way, multiple operations will be performed with just two flag writes, reducing the time spent on flag writes as a percentage of the overall operation. The second performance penalty occurs when each node is modified. There is an additional write of the node to the backup file each time a node is written to the main file, thus doubling the time for node writes. If a node that has been modified is removed from memory (for lack of sufficient buffers) and later in the operation it is modified again, the node will be backed up again. This is because the information indicating the node has been backed up is only maintained in memory as part of the buffer information. This type of extra backup writes can be prevented by ensuring that there are enough buffers. You will have to experiment with your application to find the optimal number. Otherwise, the overhead of the node writes cannot be further minimized. If a bad node is detected in the image backup file while the imagerest function is restoring the file, it will return the BADNODE error code and psp->alterr will be set to the actual invalid node error. Otherwise, psp->alterr will be set to CCOK.
Grouping Multiple C-Index Operations Two internal functions in C-Index/II can be used to group high level C-Index/II operations. When used in combination with Image Backup, the application can control the writing of related changes to the file, and in some cases, speed up processing. The strtwrit function tells C-Index that a set of file modification operations are about to start, and that it should treat multiple C-Index operations as one operation. This modifies the behavior of file buffering and image backup when running in Integrity Level 4. In multi-user operation this also tells C-Index to lock the file for subsequent file modifications, without letting any other process have access. The endwrit function tells C-Index that the set of modifications is complete. This forces buffers to disk in Integrity Level 4, and in multi-user operation the file is unlocked, allowing other users access to the file. It is very important that the strtwrit and endwrit functions be correctly paired in the application. If, for example, the strtwrit is not followed by an endwrit, it is possible for information to be incorrectly written to disk and/or for the file to become locked, preventing access by other users. There are many possible uses for grouping functions with strtwrit and endwrit. It is possible, for example, to lock a record before reading it. This would prevent a "race" condition where two users might be trying to lock and read a single record. Another usage is in combination with the Integrity Level 4 setting. When the endwrit function is called, all pending buffer writes are processed. If your application will always be performing a series of operations in a row, it may be faster to set the Integrity Level to 4 and group the calls between strtwrit and endwrit. The speed improvement is possible because there may be fewer buffer writes. When image backup is activated with the imageback function, the strtwrit/endwrit function pair signals to C-Index the beginning and end of information to be saved in the backup file. Your application can use strtwrit/endwritto ensure that a series of dependent operations are completed. If the program fails, the backup file image can be used to restore the file to a good state. An example of this situation would be in an accounting program where money is moved from one account to another. It would be undesirable to have the program fail after removing money from the original account before depositing it into the new location. By bracketing these two operations together, the application can use the image backup feature to ensure that the file will never be left in an incomplete condition. An important multi-user design consideration when using these functions is that it causes other users to be locked out of the file for a longer period of time. This may not be satisfactory in situations where users expect a fast response to read operations at the same time as long multiple C-Index operations are processed between strtwrit and endwrit. Transaction Logging feature creates a log of all changes to the file. The log can be used to roll back file operations, and also to implement software based file mirroring (see next section). To start logging of a transaction call the transbeg function. Any change to entries in the file will then be recorded in a transaction log until the transend function is called. Any combination of single-key and multi-key operations will be entered into the log. Changes to the semaphore and changes to the custom header info are not logged. The file must be opened with Integrity Level 4 to utilize transaction logging. Each group of operations to be logged are bracketed by calls to transbeg and transend. To test the status of transactions use transtst (for example, to detect failed transactions). An application can enter application specific information into the transaction log using translog. Partially completed transactions and interrupted transactions can be rolled back using transroll. The transroll function has an optional callback function parameter. This callback function can examine log entries, use the progress status report for display purposes, and stop the roll back operation. In the event that a process stops running while a transaction is active (after transbeg and before transend but not in the middle of writing to disk), this condition is detected by an application when performing a record lock set or test (such as msetlock or msetsema), or by calling transtst. If the TRANSINTR error is returned, the application can call the transroll function to reverse the incomplete transaction operations. It is a good idea to call transtst after opening a file to detect incomplete transactions before using the file. In the event that a transaction fails while writing to disk, the WRITEINTR error will be returned by any process trying to write to the file. In this situation the file must first be returned to a stable condition. This can be accomplished by using the REBUILD Utility or rebuild functions, by restoring the original file using the mirror file (if mirroring was used by the file), or by using the image backup (if active at the time of failure). After restoring the file the application can roll back any partially completed transactions using transroll. In the event that the application needs to roll back a transaction before it is complete, the transroll function should be called. The transaction logging functions can also be used to implement software based file mirroring. When specified, all changes made to the file while transaction logging is active will be copied to a second ("mirror") C-Index/II file. This capability is designed so that regardless of when a program interruption would occur, either the original file or the mirror file will be in usable condition without a need to rebuild the file. A second benefit to file mirroring in this manner is that it can be used to guard against database corruption resulting from hardware failure. If the two files are on different disk drives, for example, loss of access to one of the drives will not result in total loss of the data. The primary disadvantages to file mirroring is the doubling of disk space usage (two copies), and halving of performance (from writing to disk twice). To use mirroring requires that the application be modified very carefully. At any point in the code where the file may be modified, transaction logging must be active with the TRANSMIRROR or TRANSALL operation type (TRANSCHANGE does not perform mirroring). The transbeg function must be called outside of any code that has been bracketed by strtwrit/endwrit. If there is any operation that modifies the file without the transbeg having been initiated, the mirror file will not be an exact copy of the primary file and may be corrupted. The transaction processing routines detect that mirroring is active, and automatically log the occurrence of changes to the primary file. When endwrit is called (either by the application or for the last time in a C-Index function), any changes to the primary file are copied to the mirror file and the header of the mirror file is updated. When transend is called the transaction log is cleared. Since clearing the log file takes some time, it may be desirable to group the transbeg/transend functions around small sets of operation, or it may be better for the application to bracket larger groups and have one large clear be performed at one point, rather than many smaller clears. This will depend on the needs of the application. All writes to the primary file are completed before writing to the mirror file. Since transaction processing requires that Integrity Level 4 be set, the application will be informed if an abnormal program stoppage occurred at file open time. If opening the primary file returns the WRITEINTR error, the mirror file should be copied over the primary file. If opening the mirror file returns the WRITEINTR error, the primary file should be copied over the mirror file. Restoring the primary or mirror file in this manner will put the file in the condition before the last strtwrit call (start of a write operation). At this time it is necessary to call transroll to roll back logged transactions (if TRANSALL was used for logging), and to clear the transaction log so that it can be used again. It is necessary to specify to the transroll function the TRANSMIRROR transaction logging code and the mirror file psp so that the rollback operation itself will be mirrored. After the transroll function the transend should be called to complete this process. Other suggested (but not required) steps: make a backup of the corrupted file before restoring it, just in case the "good" file turns out to have problems. And it would be a good idea to run bcheck on the final file to ensure that all is well before restarting access to the file. C-Index/II provides several functions for reconstructing a file that has been damaged ("corrupted"). Your application will need to use these functions together with application specific routines to verify the accuracy of the data after reconstruction. The rebuild functions are: bbuild rebuild with more control over new file (also _bbuild) bcheck check integrity of single-key records (also _bcheck) ccheck check and fix single key record integrity (also _ccheck) clrwrit clear write interrupt flag dbuild rebuild multi-key records dcheck check integrity of multi-key records
The bbuild rebuild function is provided to permit the reconstruction of files that have been damaged by abnormal program termination and by media failure. The bbuild function requires enough disk space for two copies of the file being reconstructed, the original file and the rebuilt copy. If the rebuild operation is being performed on a file because of media failure, the failing disk sector must be re-written first using DISKCOPY, using the COPY command (and selecting the IGNORE options for bad sectors), or using a file recovery routine that does not remove any sectors from the file. Do not use the DOS RECOVER utility or the Norton Utility recovery routines, as this will remove bad sectors from the file and cause the C-Index/II file addressing to be incorrect. The bbuild function does not check to see if each data entry is complete. In the event that only part of the data was written to disk successfully, the data record in the index will be incomplete. The ccheck function must be used to check the data integrity after the rebuild operation is complete. The bbuild function allows you to specify the byte ordering of the old file being reconstructed, and it uses the index information from the new file to control reading of the old file. The _bbuild function has the additional feature of a parameter specifying a callback function in the application program for receiving a status report on how much has been processed. The cbuild function is included with C-Index/II for backwards compatibility, however, it is completely replaced by bbuild which includes many significant improvements. The ccheck function checks the specified index file to see if all the data in the file is complete. If portions of data in the file are missing, the found portions of data are removed from the original file and copied to the error index file in a legal data format (without the missing pieces of data). The application program can then check the data records that are partially recovered for whatever information is usable. If there are no incomplete data records in the test file, the function will return the CCOK condition code. If there are one or more errors, the BAD condition code will be returned and the error index file will contain the partially complete data records. The ccheck function only works with files containing string keys. The function _ccheck allows you to check the rebuilding of files containing non-string keys (int keys, binary keys, segmented keys, etc). The _ccheck function also has the feature of a parameter specifying a callback function in the application program for receiving a status report on how much has been processed. The cbuild and ccheck functions work with the single-key entries that are in an index file. If you are using the multi-key routines, the rebuild process will not ensure that key entries match data entries. This could cause problems with an application program. If a key entry points to a missing data record, for example, the dread function would fail. Conversely, if keys are missing, the dupdate function would fail. Your application will need to handle these conditions by adding another step to the rebuild process. After a low level rebuild using the cbuild and ccheck functions, the program must call the dbuild function. The application first rebuilds the file on the low level using the bbuild and ccheck functions. Then the application opens the "cleaned" file with bopen and creates a new index file using bcreate, specifying the desired byte ordering, index types and alternate roots. These open files are passed to the dbuild function. If the file contains more than one type of record format (uses multiple datalists), it is possible for dbuild to determine which datalist to use via a callback function. To ensure that the file has not been corrupted on a low level (single-key entries), the bcheck function should be used as a part of the applicationâs file protection strategy. The bcheck function tests the contents of an open file for corruption in the index structure. There is no limit on the size of file to be tested. This is a comprehensive test that should detect any type of file structure corruption involving single-key information. It does not check for all possible types of corruption of keys, for corruption of the data portion of the entry, multi-key corruption involving keys that do not match up with records, or records that have been corrupted within the data which would cause a CHKSUMDERR on dread or ddelete. The _bcheck function has the additional feature of a parameter specifying a callback function in the application program for receiving a status report on how much has been processed. To ensure that the file does not contain corrupted multi-key records, the dcheck function should be called after verifying the single-key entries with the bcheck function. Calling the dcheck function will validate that all records have the correct corresponding keys pointing to the record and conversely, all keys point to the correct record. The clrwrit function allows the application to clear the write interrupt error flag. This flag is set when the file has been opened in Integrity Level 4 and a C-Index buffer write operation has been interrupted. The application must first open the file using the _bopen function (to avoid the check of the write interrupt flag performed by a normal open operation) and then call the clrwrit function. This will allow the application to open and write to the file normally. After the write interrupt flag has been cleared, it still be necessary to reconstruct the file (using image restore or rebuild functions). The BCLEAR Utility also will clear the write interrupt flag. Utility Programs for Rebuilding Files There are several utility programs that are provided for evaluating and rebuilding corrupted files: BCHECK Check file integrity using bcheck function. BCLEAR Clear write interrupt flag using clrwrit. REBUILD Rebuild single key indexes using _bbuild and _ccheck. DUMP Display or print physical content of a C-Index/II file. These can be run from the command line. Full source code is provided in the \ci2\util distribution subdirectory. These utilities may be distributed with the application without payment of royalties. Full source code is included and you may choose to incorporate portions of this code directly into your application to make the user interfaces compatible with your other application code. For more information about these and other supplied utility programs, see Chapter 11, "C-Index/II Utility Programs."
C-Index/II Home Pagewww.triosystems.com © Copyright 1996 - 1999 Trio Systems LLC |