Difference between revisions of "Libmu dbm"

From Mailutils
Jump to navigationJump to search
 
(19 intermediate revisions by the same user not shown)
Line 1: Line 1:
 
{{DISPLAYTITLE:libmu_dbm}}
 
{{DISPLAYTITLE:libmu_dbm}}
  
The library <tt>libmu_dbm</tt> provides a uniform and consistent interface to various <tt>DBM</tt>-like database managers.  Presently (as of version 2.99.93), the following backend libraries are supported:
+
The library <tt>libmu_dbm</tt> provides a uniform and consistent interface to various <tt>DBM</tt>-like database managers.  Presently (as of version 2.99.99), the following backend libraries are supported:
  
 
* [http://www.gnu.org/software/gdbm GDBM]
 
* [http://www.gnu.org/software/gdbm GDBM]
 
* [http://www.oracle.com/technetwork/database/berkeleydb/overview/index.html Berkeley DB] (BDB)
 
* [http://www.oracle.com/technetwork/database/berkeleydb/overview/index.html Berkeley DB] (BDB)
 
* Traditional [http://en.wikipedia.org/wiki/Ndbm NDBM]
 
* Traditional [http://en.wikipedia.org/wiki/Ndbm NDBM]
 
+
* [http://fallabs.com/tokyocabinet/ Tokyo Cabinet]
{{Note|The [http://fallabs.com/tokyocabinet/ Tokyo Cabinet] support was available in previous versions. It is not yet re-implemented via <tt>libmu_dbm</tt>, but it will probably be in the future.}}
+
* [http://fallabs.com/kyotocabinet/ Kyoto Cabinet]
  
 
== Header Files and Libraries ==
 
== Header Files and Libraries ==
Line 21: Line 21:
 
A ''database file'' is represented by the <tt>mu_dbm_file_t</tt> type.  It is an opaque type, declared as
 
A ''database file'' is represented by the <tt>mu_dbm_file_t</tt> type.  It is an opaque type, declared as
  
<source lang="C">
+
<syntaxhighlight lang="C">
 
   typedef struct _mu_dbm_file *mu_dbm_file_t;
 
   typedef struct _mu_dbm_file *mu_dbm_file_t;
</source>
+
</syntaxhighlight>
  
 
A ''datum'' is a piece of information stored in the database.  It is declared as:
 
A ''datum'' is a piece of information stored in the database.  It is declared as:
  
<source lang="C">
+
<syntaxhighlight lang="C">
 
   struct mu_dbm_datum
 
   struct mu_dbm_datum
 
   {
 
   {
Line 35: Line 35:
 
     struct mu_dbm_impl *mu_sys;  /* Pointer to implementation */
 
     struct mu_dbm_impl *mu_sys;  /* Pointer to implementation */
 
   };
 
   };
</source>
+
</syntaxhighlight>
  
 
Of all its fields, only <tt>mu_dptr</tt> and <tt>mu_dsize</tt> are of interest for application developers.  The <tt>mu_dptr</tt> points to the actual data and <tt>mu_dsize</tt> keeps the number of bytes pointed to by <tt>mu_dptr</tt>.
 
Of all its fields, only <tt>mu_dptr</tt> and <tt>mu_dsize</tt> are of interest for application developers.  The <tt>mu_dptr</tt> points to the actual data and <tt>mu_dsize</tt> keeps the number of bytes pointed to by <tt>mu_dptr</tt>.
Line 41: Line 41:
 
When initializing an object of <tt>struct mu_dbm_datum</tt> type it is important to fill all the rest of its fields with zeros.  The usual initialization sequence is therefore:
 
When initializing an object of <tt>struct mu_dbm_datum</tt> type it is important to fill all the rest of its fields with zeros.  The usual initialization sequence is therefore:
  
<source lang="C">
+
<syntaxhighlight lang="C">
 
   struct mu_dbm_datum key;
 
   struct mu_dbm_datum key;
  
Line 47: Line 47:
 
   key.mu_dptr = data;
 
   key.mu_dptr = data;
 
   key.mu_dsize = size;
 
   key.mu_dsize = size;
</source>
+
</syntaxhighlight>
  
 
== Global Variables ==
 
== Global Variables ==
<source lang="C">
+
<syntaxhighlight lang="C">
 
   mu_url_t mu_dbm_hint;
 
   mu_url_t mu_dbm_hint;
</source>
+
</syntaxhighlight>
  
 
This variable keeps an [[URL hint]] which is used to supply missing URL parts for <tt>mu_dbm_create_from_url</tt> and <tt>mu_dbm_create</tt> calls (see next section).  By default it contains only a scheme part pointing to the first available '''DBM''' implementation.
 
This variable keeps an [[URL hint]] which is used to supply missing URL parts for <tt>mu_dbm_create_from_url</tt> and <tt>mu_dbm_create</tt> calls (see next section).  By default it contains only a scheme part pointing to the first available '''DBM''' implementation.
 +
 +
This variable is <tt>NULL</tt> before any of the following functions is called: [[#mu_dbm_init|<tt>mu_dbm_init</tt>]], <tt>mu_dbm_get_hint</tt>, [[#mu_dbm_register|<tt>mu_dbm_register</tt>]], [[#mu_dbm_impl_iterator|<tt>mu_dbm_impl_iterator</tt>]], [[#Creating a Database Object|<tt>mu_dbm_create_from_url</tt>]] or [[#Creating a Database Object|<tt>mu_dbm_create</tt>]]. 
 +
 +
<div id="mu_dbm_get_hint"></div>
 +
Therefore it is not recommended to use this variable directly.  The preferred way is to use the the <tt>mu_dbm_get_hint</tt> function instead:
 +
 +
<syntaxhighlight lang="C">
 +
  mu_url_t mu_dbm_get_hint (void);
 +
</syntaxhighlight>
 +
 +
This function makes sure the variable is initialized before returning it.
  
 
== Creating a Database Object ==
 
== Creating a Database Object ==
Line 60: Line 71:
 
A variable of type <tt>mu_dbm_file_t</tt> is created using the following functions:
 
A variable of type <tt>mu_dbm_file_t</tt> is created using the following functions:
  
<source lang="C">
+
<syntaxhighlight lang="C">
   int mu_dbm_create_from_url (mu_url_t url, mu_dbm_file_t *db);
+
   int mu_dbm_create_from_url (mu_url_t url, mu_dbm_file_t *db, int default_safety_flags);
   int mu_dbm_create (char *name, mu_dbm_file_t *db);
+
   int mu_dbm_create (char *name, mu_dbm_file_t *db, int default_safety_flags);
</source>
+
</syntaxhighlight>
  
 
The former function creates a database object from an URL supplied as its first argument.  The latter one creates a database object from a file name or a [[Database URL|URL]] in string form.  On success both functions store a pointer to the initialized <tt>struct _mu_dbm_file</tt> in the memory location pointed to by the '''db''' parameter and return 0.  On error, they return a [[Error Code|Mailutils error code]] and do not touch '''db'''.
 
The former function creates a database object from an URL supplied as its first argument.  The latter one creates a database object from a file name or a [[Database URL|URL]] in string form.  On success both functions store a pointer to the initialized <tt>struct _mu_dbm_file</tt> in the memory location pointed to by the '''db''' parameter and return 0.  On error, they return a [[Error Code|Mailutils error code]] and do not touch '''db'''.
 +
 +
The '''default_safety_flags''' argument specifies the default safety criteria to apply to the database unless explicitly stated in its [[Database_URL#param|URL parameters]]
  
 
== Safety Checking ==
 
== Safety Checking ==
Line 71: Line 84:
 
The following function verifies whether the database file is [[File Safety Checks|safe to use]]:
 
The following function verifies whether the database file is [[File Safety Checks|safe to use]]:
  
<source lang="C">
+
<syntaxhighlight lang="C">
 
   int mu_dbm_safety_check (mu_dbm_file_t db);
 
   int mu_dbm_safety_check (mu_dbm_file_t db);
</source>
+
</syntaxhighlight>
  
 
It returns 0 if the file is OK, or an [[File_Safety_Functions#MU_ERR_PERM|error code]] describing the problem otherwise.
 
It returns 0 if the file is OK, or an [[File_Safety_Functions#MU_ERR_PERM|error code]] describing the problem otherwise.
Line 79: Line 92:
 
There are two ways to set safety criteria.  First, they can be configured using the [[Database URL#param|URL parameters]] when calling <tt>mu_dbm_create_from_url</tt> or <tt>mu_dbm_create</tt>.  Second, they can be set using the following functions:
 
There are two ways to set safety criteria.  First, they can be configured using the [[Database URL#param|URL parameters]] when calling <tt>mu_dbm_create_from_url</tt> or <tt>mu_dbm_create</tt>.  Second, they can be set using the following functions:
  
<source lang="C">
+
<syntaxhighlight lang="C">
 
   int mu_dbm_safety_set_flags (mu_dbm_file_t db, int flags);
 
   int mu_dbm_safety_set_flags (mu_dbm_file_t db, int flags);
 
   int mu_dbm_safety_set_owner (mu_dbm_file_t db, uid_t uid);
 
   int mu_dbm_safety_set_owner (mu_dbm_file_t db, uid_t uid);
</source>
+
</syntaxhighlight>
  
 
The <tt>mu_dbm_safety_set_flags</tt> functions defines the set of safety criteria for this file.  The '''flags''' argument has the same meaning as the [[File_Safety_Functions#MU_FILE_SAFETY|'''mode''']] argument to [[mu_file_safety_check]].  If it contains the [[MU_FILE_SAFETY_OWNER_MISMATCH]] bit, the file owner UID can be set using the <tt>mu_dbm_safety_set_owner</tt> function.
 
The <tt>mu_dbm_safety_set_flags</tt> functions defines the set of safety criteria for this file.  The '''flags''' argument has the same meaning as the [[File_Safety_Functions#MU_FILE_SAFETY|'''mode''']] argument to [[mu_file_safety_check]].  If it contains the [[MU_FILE_SAFETY_OWNER_MISMATCH]] bit, the file owner UID can be set using the <tt>mu_dbm_safety_set_owner</tt> function.
Line 88: Line 101:
 
Whatever way was used to set them, the current safety criteria can be queried using the following two functions:
 
Whatever way was used to set them, the current safety criteria can be queried using the following two functions:
  
<source lang="C">
+
<syntaxhighlight lang="C">
 
   int mu_dbm_safety_get_flags (mu_dbm_file_t db, int *flags);
 
   int mu_dbm_safety_get_flags (mu_dbm_file_t db, int *flags);
 
   int mu_dbm_safety_get_owner (mu_dbm_file_t db, uid_t *uid);
 
   int mu_dbm_safety_get_owner (mu_dbm_file_t db, uid_t *uid);
</source>
+
</syntaxhighlight>
  
 
== Opening a Database ==
 
== Opening a Database ==
Line 97: Line 110:
 
Once a <tt>mu_dbm_file_t</tt> is created, it can be opened:
 
Once a <tt>mu_dbm_file_t</tt> is created, it can be opened:
  
<source lang="C">
+
<syntaxhighlight lang="C">
 
   int mu_dbm_open (mu_dbm_file_t db, int flags, int mode);
 
   int mu_dbm_open (mu_dbm_file_t db, int flags, int mode);
</source>  
+
</syntaxhighlight>  
  
 
The function <tt>mu_dbm_open</tt> opens a database described by '''db'''.  The '''flags''' argument specifies how to open the database.  The valid values (declared in <tt>mailutils/stream.h</tt><ref>http://git.gnu.org.ua/cgit/mailutils.git/tree/include/mailutils/stream.h</ref>) are:
 
The function <tt>mu_dbm_open</tt> opens a database described by '''db'''.  The '''flags''' argument specifies how to open the database.  The valid values (declared in <tt>mailutils/stream.h</tt><ref>http://git.gnu.org.ua/cgit/mailutils.git/tree/include/mailutils/stream.h</ref>) are:
Line 110: Line 123:
 
:Create an empty database and open it for reading and writing.  If the database file already exists, all existing records will be removed from it.
 
:Create an empty database and open it for reading and writing.  If the database file already exists, all existing records will be removed from it.
  
The '''mode''' argument supplies the file mode for newly created files.  Its meaning is the same as in [http://www.manpagez.com/man/2/chmod/ chmod(2)] call.
+
The '''mode''' argument supplies the file mode for newly created files.  Its meaning is the same as in {{man|2|chmod}} call.
  
 
==Database Lookups==
 
==Database Lookups==
<source lang="C">
+
<syntaxhighlight lang="C">
 
   int mu_dbm_fetch (mu_dbm_file_t db, struct mu_dbm_datum const *key,
 
   int mu_dbm_fetch (mu_dbm_file_t db, struct mu_dbm_datum const *key,
 
    struct mu_dbm_datum *ret);
 
    struct mu_dbm_datum *ret);
</source>
+
</syntaxhighlight>
  
 
The <tt>mu_dbm_fetch</tt> functions looks up in the database '''db''' a record whose key matches '''key'''.  If such a record is found, its associated data are stored in '''ret''' and 0 is returned.  If there is no such key in the database, the function returns [[MU_ERR_NOENT]].  If an error occurs, it returns [[MU_ERR_FAILURE]].  The textual error description can then be obtained using [[#mu_dbm_strerror|mu_dbm_strerror]].
 
The <tt>mu_dbm_fetch</tt> functions looks up in the database '''db''' a record whose key matches '''key'''.  If such a record is found, its associated data are stored in '''ret''' and 0 is returned.  If there is no such key in the database, the function returns [[MU_ERR_NOENT]].  If an error occurs, it returns [[MU_ERR_FAILURE]].  The textual error description can then be obtained using [[#mu_dbm_strerror|mu_dbm_strerror]].
Line 149: Line 162:
  
 
==Storing Records==
 
==Storing Records==
<source lang="C">
+
<syntaxhighlight lang="C">
 
   int mu_dbm_store (mu_dbm_file_t db, struct mu_dbm_datum const *key,
 
   int mu_dbm_store (mu_dbm_file_t db, struct mu_dbm_datum const *key,
 
      struct mu_dbm_datum const *content, int replace);
 
      struct mu_dbm_datum const *content, int replace);
</source>
+
</syntaxhighlight>
  
 
The function <tt>mu_dbm_store</tt> inserts or replaces records in the database.
 
The function <tt>mu_dbm_store</tt> inserts or replaces records in the database.
Line 162: Line 175:
  
 
==Deleting a Record==
 
==Deleting a Record==
<source lang="C">
+
<syntaxhighlight lang="C">
 
   int mu_dbm_delete (mu_dbm_file_t db, struct mu_dbm_datum const *key);
 
   int mu_dbm_delete (mu_dbm_file_t db, struct mu_dbm_datum const *key);
</source>
+
</syntaxhighlight>
  
 
This function deletes from the database a record with the given key.  It returns 0 on success, [[MU_ERR_NOENT]], if there is no such key in the database and [[MU_ERR_FAILURE]] on error.  In the latter case the textual description of the error can be obtained using [[#mu_dbm_strerror|mu_dbm_strerror]].
 
This function deletes from the database a record with the given key.  It returns 0 on success, [[MU_ERR_NOENT]], if there is no such key in the database and [[MU_ERR_FAILURE]] on error.  In the latter case the textual description of the error can be obtained using [[#mu_dbm_strerror|mu_dbm_strerror]].
Line 171: Line 184:
 
The following two functions allow for accessing all items in the database:
 
The following two functions allow for accessing all items in the database:
  
<source lang="C">
+
<syntaxhighlight lang="C">
 
   int mu_dbm_firstkey (mu_dbm_file_t db, struct mu_dbm_datum *ret);
 
   int mu_dbm_firstkey (mu_dbm_file_t db, struct mu_dbm_datum *ret);
 
   int mu_dbm_nextkey (mu_dbm_file_t db, struct mu_dbm_datum *ret);
 
   int mu_dbm_nextkey (mu_dbm_file_t db, struct mu_dbm_datum *ret);
</source>
+
</syntaxhighlight>
  
 
The function <tt>mu_dbm_firstkey</tt> initializes <tt>db</tt> for sequential access and stores the first key in the datum pointed to by <tt>ret</tt>.  The function returns <tt>0</tt> on success, [[MU_ERR_NOENT]] if the database is empty and [[MU_ERR_FAILURE]] on error.  
 
The function <tt>mu_dbm_firstkey</tt> initializes <tt>db</tt> for sequential access and stores the first key in the datum pointed to by <tt>ret</tt>.  The function returns <tt>0</tt> on success, [[MU_ERR_NOENT]] if the database is empty and [[MU_ERR_FAILURE]] on error.  
Line 198: Line 211:
 
</syntaxhighlight>
 
</syntaxhighlight>
  
Notice, that sequential access does not imply any particular ordering in which the keys will be visited.  It only guarantees that each key in the database will be visited exactly once, '''provided that no database updates take place within the loop'''.  The latter condition is important.  You cannot store or delete records while doing sequential access, because this will change key ordering and can change the order in which keys are returned.  In other words, the following is wrong:
+
Notice, that sequential access does not imply any particular ordering in which the keys will be visited.  It only guarantees that each key in the database will be visited exactly once, '''provided that no database updates take place within the loop'''.  The latter condition is important.  You cannot store or delete records while doing sequential access, because doing so can change the order in which keys are returned.  In other words, the following is wrong:
  
<source lang="C">
+
<syntaxhighlight lang="C">
 
   struct mu_dbm_datum key;
 
   struct mu_dbm_datum key;
 
   int rc;
 
   int rc;
Line 213: Line 226:
 
         mu_dbm_delete (db, &key);
 
         mu_dbm_delete (db, &key);
 
     }
 
     }
</source>
+
</syntaxhighlight>
  
 
==Additional Functions==
 
==Additional Functions==
Line 219: Line 232:
 
The function <tt>mu_dbm_get_fd</tt> can be used to obtain file descriptors of the underlying database:
 
The function <tt>mu_dbm_get_fd</tt> can be used to obtain file descriptors of the underlying database:
  
<source lang="C">
+
<syntaxhighlight lang="C">
 
   int mu_dbm_get_fd (mu_dbm_file_t db, int *pag, int *dir);
 
   int mu_dbm_get_fd (mu_dbm_file_t db, int *pag, int *dir);
</source>
+
</syntaxhighlight>
 +
 
 +
The function returns two descriptors, because some libraries (namely, NDBM) keep databases in two files: so called ''page'' and ''directory'' files.  If so, the descriptors of these files are returned in the memory locations pointed to by <tt>pag</tt> and <tt>dir</tt>, correspondingly.  If the underlying library stores the database in a single file, the descriptor of this file will be stored in both <tt>pag</tt> and <tt>dir</tt>.  In any case, <tt>dir</tt> can be <tt>NULL</tt> if that information is of no interest for the caller.  However, <tt>pag</tt> can not be <tt>NULL</tt>, otherwise <tt>mu_dbm_get_fd</tt> will return <tt>ENOMEM</tt>.
 +
 
 +
<div id="mu_dbm_init"></div>
 +
The function <tt>mu_dbm_init</tt> initializes the library:
 +
 
 +
<syntaxhighlight lang="C">
 +
  void mu_dbm_init (void);
 +
</syntaxhighlight>
  
The function returns two descriptors, because some libraries (namely, NDBM) keep databases in two files: so called ''page'' and ''directory'' files.  If so, the descriptors of these files are returned in the memory locations pointed to by <tt>pag</tt> and <tt>dir</tt>, correspondingly.  If the underlying library stores the database in a single file, the descriptor of this file will be stored in both <tt>pag</tt> and <tt>dir</tt>.  In any case, <tt>dir</tt> can be <tt>NULL</tt> if that information is of no interest for the called.  However, <tt>pag</tt> can not be <tt>NULL</tt>, otherwise <tt>mu_dbm_get_fd</tt> will return <tt>ENOMEM</tt>.
+
Normally it is not necessary to call this function because it is called automatically by any of the following functions: [[#mu_dbm_get_hint|<tt>mu_dbm_get_hint</tt>]], [[#mu_dbm_register|<tt>mu_dbm_register</tt>]], [[#mu_dbm_impl_iterator|<tt>mu_dbm_impl_iterator</tt>]], [[#Creating a Database Object|<tt>mu_dbm_create_from_url</tt>]] and [[#Creating a Database Object|<tt>mu_dbm_create</tt>]].
  
 
==Return Codes==
 
==Return Codes==
Line 241: Line 263:
 
The functions from <tt>libmu_dbm</tt> return <tt>MU_ERR_FAILURE</tt> if an error occurred in the underlying database implementation.  A textual description of this error can be obtained using the following call:
 
The functions from <tt>libmu_dbm</tt> return <tt>MU_ERR_FAILURE</tt> if an error occurred in the underlying database implementation.  A textual description of this error can be obtained using the following call:
  
<source lang="C">
+
<syntaxhighlight lang="C">
 
   char const *mu_dbm_strerror (mu_dbm_file_t db);
 
   char const *mu_dbm_strerror (mu_dbm_file_t db);
</source>
+
</syntaxhighlight>
  
 
The pointer returned by this function must not be freed or its contents altered by the application.
 
The pointer returned by this function must not be freed or its contents altered by the application.
Line 263: Line 285:
 
An open database is closed using:
 
An open database is closed using:
  
<source lang="C">
+
<syntaxhighlight lang="C">
 
   int mu_dbm_close (mu_dbm_file_t db);
 
   int mu_dbm_close (mu_dbm_file_t db);
</source>
+
</syntaxhighlight>
  
 
Unless intended for a subsequent re-opening, a closed database should be ''destroyed'' via a call to <tt>mu_dbm_destroy</tt>:
 
Unless intended for a subsequent re-opening, a closed database should be ''destroyed'' via a call to <tt>mu_dbm_destroy</tt>:
  
<source lang="C">
+
<syntaxhighlight lang="C">
 
   void mu_dbm_destroy (mu_dbm_file_t *pdb);
 
   void mu_dbm_destroy (mu_dbm_file_t *pdb);
</source>
+
</syntaxhighlight>
  
 
This function reclaims the memory used by the <tt>mu_dbm_file_t</tt> object pointed to by its argument and initializes it to <tt>NULL</tt> before returning.  Calling <tt>mu_dbm_destroy</tt> on an open database is OK: the function will call <tt>mu_dbm_close</tt> prior to freeing the memory.  In fact, it is seldom necessary to call <tt>mu_dbm_close</tt> explicitly.  Instead, it suffices to call <tt>mu_dbm_destroy</tt> once the database is no longer necessary.
 
This function reclaims the memory used by the <tt>mu_dbm_file_t</tt> object pointed to by its argument and initializes it to <tt>NULL</tt> before returning.  Calling <tt>mu_dbm_destroy</tt> on an open database is OK: the function will call <tt>mu_dbm_close</tt> prior to freeing the memory.  In fact, it is seldom necessary to call <tt>mu_dbm_close</tt> explicitly.  Instead, it suffices to call <tt>mu_dbm_destroy</tt> once the database is no longer necessary.
 +
 +
==API for Backend Implementors==
 +
 +
The backend data and methods are defined in the following structure:
 +
 +
<syntaxhighlight lang="C">
 +
  struct mu_dbm_impl
 +
  {
 +
    char *_dbm_name;
 +
    int (*_dbm_file_safety) (mu_dbm_file_t db, int mode, uid_t owner);
 +
    int (*_dbm_get_fd) (mu_dbm_file_t db, int *pag, int *dir);
 +
    int (*_dbm_open) (mu_dbm_file_t db, int flags, int mode);
 +
    int (*_dbm_close) (mu_dbm_file_t db);
 +
    int (*_dbm_fetch) (mu_dbm_file_t db, struct mu_dbm_datum const *key,
 +
                      struct mu_dbm_datum *ret);
 +
    int (*_dbm_store) (mu_dbm_file_t db, struct mu_dbm_datum const *key,
 +
                      struct mu_dbm_datum const *contents, int replace);
 +
    int (*_dbm_delete) (mu_dbm_file_t db,
 +
                        struct mu_dbm_datum const *key);
 +
    int (*_dbm_firstkey) (mu_dbm_file_t db, struct mu_dbm_datum *ret);
 +
    int (*_dbm_nextkey) (mu_dbm_file_t db, struct mu_dbm_datum *ret);
 +
    void (*_dbm_datum_free) (struct mu_dbm_datum *datum);
 +
    char const *(*_dbm_strerror) (mu_dbm_file_t db);
 +
  };
 +
</syntaxhighlight>
 +
 +
The <tt>_dbm_name</tt> method names the backend.  It will be used as a scheme in [[Database URL|URL]]s referring to databases of this type.  The purpose of the remaining members should be pretty obvious.  They correspond to API functions discussed above.  Each API call is guaranteed to do the necessary error checking before calling the corresponding <tt>_dbm_</tt> method.
 +
 +
The structure <tt>_mu_dbm_file</tt> is defined in <tt>mailutils/sys/dbm.h</tt> as follows:
 +
 +
<syntaxhighlight lang="C">
 +
  struct _mu_dbm_file
 +
  {
 +
    char *db_name;                /* Database name */
 +
    void *db_descr;              /* Database descriptor */
 +
    int db_safety_flags;          /* Safety checks */
 +
    uid_t db_owner;              /* Database owner UID */
 +
    struct mu_dbm_impl *db_sys;  /* Pointer to the database implementation */
 +
    union _mu_dbm_errno db_errno; /* Error description for the latest failed call */
 +
  };
 +
</syntaxhighlight>
 +
 +
The members <tt>db_name</tt>, <tt>db_safety_flags</tt>, <tt>db_owner</tt> and <tt>db_sys</tt> are initialized by high level API.  Implementation functions should initialize <tt>db_descr</tt> and <tt>db_errno</tt> as necessary.  The <tt>db_descr</tt> method keeps a pointer to the implementation-specific data describing the database.  For example, in a NDBM implementation, it can hold a pointer to the <tt>DBM</tt> structure for the open database:
 +
 +
<syntaxhighlight lang="C" line="GESHI_NORMAL_LINE_NUMBERS">
 +
static int
 +
_ndbm_open (mu_dbm_file_t db, int flags, int mode)
 +
{
 +
  DBM *dbm;
 +
 +
  /* Open the database and store its handle in db_descr */
 +
  dbm = dbm_open (db->db_name, _mu_flags_to_ndbm (flags), mode);
 +
  if (!dbm)
 +
    return MU_ERR_FAILURE;
 +
  db->db_descr = dbm;
 +
  return 0;
 +
}   
 +
</syntaxhighlight>
 +
 +
The <tt>db_errno</tt> member is intended to keep the latest error value for use by the <tt>_dbm_strerror</tt> method.  To satisfy most implementation, it is able to keep both numeric and generic pointer data:
 +
 +
<syntaxhighlight lang="C">
 +
  union _mu_dbm_errno
 +
  {
 +
    int n;            /* numeric value */
 +
    void *p;          /* pointer to an error object */
 +
  };
 +
</syntaxhighlight>
 +
 +
The implementation methods are supposed to keep its value up to date.  For example, in GDBM implementation, they store the value of [http://www.gnu.org.ua/software/gdbm/manual/html_node/Variables.html <tt>gdbm_errno</tt>] in <tt>db_errno.n</tt>:
 +
 +
<syntaxhighlight lang="C" line="GESHI_NORMAL_LINE_NUMBERS">
 +
static int
 +
_gdbm_store (mu_dbm_file_t db,
 +
            struct mu_dbm_datum const *key,
 +
            struct mu_dbm_datum const *contents,
 +
            int replace)
 +
{
 +
  /* initialize local variables */
 +
  ...
 +
  /* Do the underlying call and return: */
 +
  switch (gdbm_store ((GDBM_FILE)db->db_descr, key_datum, content_datum, replace))
 +
    {
 +
    case 0:
 +
      break;
 +
 +
    case 1:
 +
      return MU_ERR_EXISTS;
 +
 +
    case -1:
 +
      db->db_errno.n = gdbm_errno;
 +
      return MU_ERR_FAILURE;
 +
    }
 +
 +
</syntaxhighlight>
 +
 +
A corresponding <tt>_dbm_strerror</tt> implementation uses the stored value:
 +
 +
<syntaxhighlight lang="C" line="GESHI_NORMAL_LINE_NUMBERS">
 +
static char const *
 +
_gdbm_strerror (mu_dbm_file_t db)
 +
{
 +
  return gdbm_strerror (db->db_errno.n);
 +
}
 +
</syntaxhighlight>
 +
 +
<div id="mu_dbm_register"></div>
 +
A properly initialized <tt>mu_dbm_impl</tt> structure must be ''registered'' with the library by calling the following function:
 +
 +
<syntaxhighlight lang="C">
 +
  int mu_dbm_register (struct mu_dbm_impl *impl);
 +
</syntaxhighlight>
 +
 +
Once done, it becomes avaiable for use by the API.
 +
 +
<div id="mu_dbm_impl_iterator"></div>
 +
Registered implementations can be accessed via an [[iterator]] returned by the <tt>mu_dbm_impl_iterator</tt> function:
 +
 +
<syntaxhighlight lang="C">
 +
  int mu_dbm_impl_iterator (mu_iterator_t *itr);
 +
</syntaxhighlight>
 +
 +
For example, the following code snippet lists avaialble DBM implementations:
 +
 +
<syntaxhighlight lang="C" line="GESHI_NORMAL_LINE_NUMBERS">
 +
  mu_iterator_t itr;
 +
 
 +
  mu_dbm_impl_iterator (&itr);
 +
  for (mu_iterator_first (itr), i = 0; !mu_iterator_is_done (itr);
 +
      mu_iterator_next (itr), i++)
 +
    {
 +
      struct mu_dbm_impl *impl;
 +
 +
      mu_iterator_current (itr, (void**)&impl);
 +
      fprintf (stream, " ");
 +
      fprintf (stream, "%s", impl->_dbm_name);
 +
    }
 +
  fputc ('\n', stream);
 +
  mu_iterator_destroy (&itr);
 +
</syntaxhighlight>
 +
 +
For an example of using the backend implementation API see <tt>libmu_dbm/gdbm.c</tt><ref>http://git.gnu.org.ua/cgit/mailutils.git/tree/libmu_dbm/gdbm.c</ref>.
  
 
==Notes==
 
==Notes==

Latest revision as of 13:33, 16 October 2023


The library libmu_dbm provides a uniform and consistent interface to various DBM-like database managers. Presently (as of version 2.99.99), the following backend libraries are supported:

Header Files and Libraries

The data types and function prototypes of libmu_dbm are declared in mailutils/dbm.h[1]. Internal interfaces, which are of interest for implementors of new backends, are declared in mailutils/sys/dbm.h[2].

File loader arguments for linking programs with libmu_dbm can be obtained running:

 mu ldflags dbm

Data Types

A database file is represented by the mu_dbm_file_t type. It is an opaque type, declared as

  typedef struct _mu_dbm_file *mu_dbm_file_t;

A datum is a piece of information stored in the database. It is declared as:

  struct mu_dbm_datum
  {
    char *mu_dptr;               /* Data pointer */
    size_t mu_dsize;             /* Data size */
    void *mu_data;               /* Implementation-dependent data */
    struct mu_dbm_impl *mu_sys;  /* Pointer to implementation */
  };

Of all its fields, only mu_dptr and mu_dsize are of interest for application developers. The mu_dptr points to the actual data and mu_dsize keeps the number of bytes pointed to by mu_dptr.

When initializing an object of struct mu_dbm_datum type it is important to fill all the rest of its fields with zeros. The usual initialization sequence is therefore:

  struct mu_dbm_datum key;

  memset (&key, 0, sizeof key);
  key.mu_dptr = data;
  key.mu_dsize = size;

Global Variables

  mu_url_t mu_dbm_hint;

This variable keeps an URL hint which is used to supply missing URL parts for mu_dbm_create_from_url and mu_dbm_create calls (see next section). By default it contains only a scheme part pointing to the first available DBM implementation.

This variable is NULL before any of the following functions is called: mu_dbm_init, mu_dbm_get_hint, mu_dbm_register, mu_dbm_impl_iterator, mu_dbm_create_from_url or mu_dbm_create.

Therefore it is not recommended to use this variable directly. The preferred way is to use the the mu_dbm_get_hint function instead:

  mu_url_t mu_dbm_get_hint (void);

This function makes sure the variable is initialized before returning it.

Creating a Database Object

A variable of type mu_dbm_file_t is created using the following functions:

  int mu_dbm_create_from_url (mu_url_t url, mu_dbm_file_t *db, int default_safety_flags);
  int mu_dbm_create (char *name, mu_dbm_file_t *db, int default_safety_flags);

The former function creates a database object from an URL supplied as its first argument. The latter one creates a database object from a file name or a URL in string form. On success both functions store a pointer to the initialized struct _mu_dbm_file in the memory location pointed to by the db parameter and return 0. On error, they return a Mailutils error code and do not touch db.

The default_safety_flags argument specifies the default safety criteria to apply to the database unless explicitly stated in its URL parameters

Safety Checking

The following function verifies whether the database file is safe to use:

  int mu_dbm_safety_check (mu_dbm_file_t db);

It returns 0 if the file is OK, or an error code describing the problem otherwise.

There are two ways to set safety criteria. First, they can be configured using the URL parameters when calling mu_dbm_create_from_url or mu_dbm_create. Second, they can be set using the following functions:

  int mu_dbm_safety_set_flags (mu_dbm_file_t db, int flags);
  int mu_dbm_safety_set_owner (mu_dbm_file_t db, uid_t uid);

The mu_dbm_safety_set_flags functions defines the set of safety criteria for this file. The flags argument has the same meaning as the mode argument to mu_file_safety_check. If it contains the MU_FILE_SAFETY_OWNER_MISMATCH bit, the file owner UID can be set using the mu_dbm_safety_set_owner function.

Whatever way was used to set them, the current safety criteria can be queried using the following two functions:

  int mu_dbm_safety_get_flags (mu_dbm_file_t db, int *flags);
  int mu_dbm_safety_get_owner (mu_dbm_file_t db, uid_t *uid);

Opening a Database

Once a mu_dbm_file_t is created, it can be opened:

  int mu_dbm_open (mu_dbm_file_t db, int flags, int mode);

The function mu_dbm_open opens a database described by db. The flags argument specifies how to open the database. The valid values (declared in mailutils/stream.h[3]) are:

MU_STREAM_READ
Open database for reading only. If the database file does not exist, an error is returned.
MU_STREAM_RDWR
Open database for reading and writing. If the database file does not exist, it is created.
MU_STREAM_CREAT
Create an empty database and open it for reading and writing. If the database file already exists, all existing records will be removed from it.

The mode argument supplies the file mode for newly created files. Its meaning is the same as in chmod(2) call.

Database Lookups

  int mu_dbm_fetch (mu_dbm_file_t db, struct mu_dbm_datum const *key,
		    struct mu_dbm_datum *ret);

The mu_dbm_fetch functions looks up in the database db a record whose key matches key. If such a record is found, its associated data are stored in ret and 0 is returned. If there is no such key in the database, the function returns MU_ERR_NOENT. If an error occurs, it returns MU_ERR_FAILURE. The textual error description can then be obtained using mu_dbm_strerror.

The memory for data returned in ret is allocated by the library. To reclaim this memory, use the mu_dbm_datum_free function. If the ret parameter points to a datum initialized by a previous call to a libmu_dbm function (such as mu_dbm_fetch, mu_dbm_firstkey, etc.) it will be freed automatically. Therefore it is safe not to free existing datum before passing it as a third argument to mu_dbm_fetch.

It is important to notice, however, that the datum pointed to by ret must always be initialized. This means, in practice, that it should either be returned from another libmu_dbm library call or initialized manually. For example:

 1  struct mu_dbm_datum key, content;
 2  int rc;
 3   
 4  /* Initialize search key */
 5  memset (&key, 0, sizeof key);
 6  key.mu_dptr = "user";
 7  key.mu_dsize = 4;
 8  /* Initialize content */
 9  memset (&content, 0, sizeof content);
10  /* Look up the data */
11  rc = mu_dbm_fetch (db, &key, &content);
12  ...
13  /* Find another key: */
14  key.mu_dptr = "hostname";
15  key.mu_dsize = 8;
16  /* It is OK not to re-set content here, because it was initialized
17     by the previous call to mu_dbm_fetch and will be freed automatically. */
18  rc = mu_dbm_fetch (db, &key, &content);
19  ...
20  /* Free the content */
21  mu_dbm_datum_free (&content);

Storing Records

  int mu_dbm_store (mu_dbm_file_t db, struct mu_dbm_datum const *key,
 		    struct mu_dbm_datum const *content, int replace);

The function mu_dbm_store inserts or replaces records in the database.

The parameters key and content supply the record key and associated data. The parameter replace determines behavior of the function if a record with such key already exists in the database. If replace is 1, mu_dbm_store will replace the data part of the existing record with data from content. Otherwise, if replace is 0, the function will leave the record unchanged and return MU_ERR_EXISTS.

The function returns 0 on success and MU_ERR_FAILURE on error. In the latter case the textual description of the error can be obtained using mu_dbm_strerror.

Deleting a Record

  int mu_dbm_delete (mu_dbm_file_t db, struct mu_dbm_datum const *key);

This function deletes from the database a record with the given key. It returns 0 on success, MU_ERR_NOENT, if there is no such key in the database and MU_ERR_FAILURE on error. In the latter case the textual description of the error can be obtained using mu_dbm_strerror.

Sequential Access

The following two functions allow for accessing all items in the database:

  int mu_dbm_firstkey (mu_dbm_file_t db, struct mu_dbm_datum *ret);
  int mu_dbm_nextkey (mu_dbm_file_t db, struct mu_dbm_datum *ret);

The function mu_dbm_firstkey initializes db for sequential access and stores the first key in the datum pointed to by ret. The function returns 0 on success, MU_ERR_NOENT if the database is empty and MU_ERR_FAILURE on error.

The function mu_dbm_nextkey takes a pointer to the datum returned by a prior call to either of the two functions and locates the key immediately following this one. On success, it stores the new key in that same datum and returns 0. It returns MU_ERR_NOENT if all keys in the database has been visited and MU_ERR_FAILURE if a database error occurred.

This function takes care about the memory allocated for ret. This means that it is not necessary to call mu_dbm_datum_free (ret) neither before calling this function nor after it returns MU_ERR_NOENT (although doing so does not constitute an error). The usual sequential access code looks like:

 1  struct mu_dbm_datum key;
 2  int rc;
 3
 4  memset (&key, 0, sizeof key);
 5  for (rc = mu_dbm_firstkey (db, &key); 
 6       rc == 0; 
 7       rc = mu_dbm_nextkey (db, &key))
 8    {
 9      do_something (&key);
10    }
11  if (rc != MU_ERR_NOENT)
12    mu_dbm_datum_free (&key);

Notice, that sequential access does not imply any particular ordering in which the keys will be visited. It only guarantees that each key in the database will be visited exactly once, provided that no database updates take place within the loop. The latter condition is important. You cannot store or delete records while doing sequential access, because doing so can change the order in which keys are returned. In other words, the following is wrong:

  struct mu_dbm_datum key;
  int rc;

  memset (&key, 0, sizeof key);
  for (rc = mu_dbm_firstkey (db, &key); 
       rc == 0; 
       rc = mu_dbm_nextkey (db, &key))
    {
      if (key_matches_something (&key))
        /* This is wrong! */
        mu_dbm_delete (db, &key);
    }

Additional Functions

The function mu_dbm_get_fd can be used to obtain file descriptors of the underlying database:

  int mu_dbm_get_fd (mu_dbm_file_t db, int *pag, int *dir);

The function returns two descriptors, because some libraries (namely, NDBM) keep databases in two files: so called page and directory files. If so, the descriptors of these files are returned in the memory locations pointed to by pag and dir, correspondingly. If the underlying library stores the database in a single file, the descriptor of this file will be stored in both pag and dir. In any case, dir can be NULL if that information is of no interest for the caller. However, pag can not be NULL, otherwise mu_dbm_get_fd will return ENOMEM.

The function mu_dbm_init initializes the library:

  void mu_dbm_init (void);

Normally it is not necessary to call this function because it is called automatically by any of the following functions: mu_dbm_get_hint, mu_dbm_register, mu_dbm_impl_iterator, mu_dbm_create_from_url and mu_dbm_create.

Return Codes

Apart from the return codes discussed above, each libmu_dbm function can return the following:

ENOMEM
Not enough memory for the operation.
EINVAL
The db or some other required argument is NULL.
MU_ERR_NOT_OPEN
Database is not open. This error can be returned by any function taking mu_dbm_database_t as argument, except the following: mu_dbm_create_from_url, mu_dbm_create, mu_dbm_safety_get_owner, mu_dbm_safety_get_flags, mu_dbm_safety_set_owner, mu_dbm_safety_set_flags, mu_dbm_safety_check.
ENOSYS
Function not implemented.

Error Reporting

The functions from libmu_dbm return MU_ERR_FAILURE if an error occurred in the underlying database implementation. A textual description of this error can be obtained using the following call:

  char const *mu_dbm_strerror (mu_dbm_file_t db);

The pointer returned by this function must not be freed or its contents altered by the application.

An example usage:

1   rc = mu_dbm_store (db, &key, &contents, replace);
2   if (rc)
3     mu_error ("cannot store datum: %s",
4	       rc == MU_ERR_FAILURE ? 
5                  mu_dbm_strerror (db) : mu_strerror (rc));

Notice how an appropriate error reporting function is selected in lines 4--5.

Closing the Database

An open database is closed using:

  int mu_dbm_close (mu_dbm_file_t db);

Unless intended for a subsequent re-opening, a closed database should be destroyed via a call to mu_dbm_destroy:

  void mu_dbm_destroy (mu_dbm_file_t *pdb);

This function reclaims the memory used by the mu_dbm_file_t object pointed to by its argument and initializes it to NULL before returning. Calling mu_dbm_destroy on an open database is OK: the function will call mu_dbm_close prior to freeing the memory. In fact, it is seldom necessary to call mu_dbm_close explicitly. Instead, it suffices to call mu_dbm_destroy once the database is no longer necessary.

API for Backend Implementors

The backend data and methods are defined in the following structure:

  struct mu_dbm_impl
  {
    char *_dbm_name;
    int (*_dbm_file_safety) (mu_dbm_file_t db, int mode, uid_t owner);
    int (*_dbm_get_fd) (mu_dbm_file_t db, int *pag, int *dir);
    int (*_dbm_open) (mu_dbm_file_t db, int flags, int mode);
    int (*_dbm_close) (mu_dbm_file_t db);
    int (*_dbm_fetch) (mu_dbm_file_t db, struct mu_dbm_datum const *key,
                       struct mu_dbm_datum *ret);
    int (*_dbm_store) (mu_dbm_file_t db, struct mu_dbm_datum const *key,
                       struct mu_dbm_datum const *contents, int replace);
    int (*_dbm_delete) (mu_dbm_file_t db,
                        struct mu_dbm_datum const *key);
    int (*_dbm_firstkey) (mu_dbm_file_t db, struct mu_dbm_datum *ret);
    int (*_dbm_nextkey) (mu_dbm_file_t db, struct mu_dbm_datum *ret);
    void (*_dbm_datum_free) (struct mu_dbm_datum *datum);
    char const *(*_dbm_strerror) (mu_dbm_file_t db);
  };

The _dbm_name method names the backend. It will be used as a scheme in URLs referring to databases of this type. The purpose of the remaining members should be pretty obvious. They correspond to API functions discussed above. Each API call is guaranteed to do the necessary error checking before calling the corresponding _dbm_ method.

The structure _mu_dbm_file is defined in mailutils/sys/dbm.h as follows:

  struct _mu_dbm_file
  {
    char *db_name;                /* Database name */
    void *db_descr;               /* Database descriptor */
    int db_safety_flags;          /* Safety checks */
    uid_t db_owner;               /* Database owner UID */
    struct mu_dbm_impl *db_sys;   /* Pointer to the database implementation */
    union _mu_dbm_errno db_errno; /* Error description for the latest failed call */
  };

The members db_name, db_safety_flags, db_owner and db_sys are initialized by high level API. Implementation functions should initialize db_descr and db_errno as necessary. The db_descr method keeps a pointer to the implementation-specific data describing the database. For example, in a NDBM implementation, it can hold a pointer to the DBM structure for the open database:

 1static int
 2_ndbm_open (mu_dbm_file_t db, int flags, int mode)
 3{
 4  DBM *dbm;
 5
 6  /* Open the database and store its handle in db_descr */
 7  dbm = dbm_open (db->db_name, _mu_flags_to_ndbm (flags), mode);
 8  if (!dbm)
 9    return MU_ERR_FAILURE;
10  db->db_descr = dbm;
11  return 0;
12}

The db_errno member is intended to keep the latest error value for use by the _dbm_strerror method. To satisfy most implementation, it is able to keep both numeric and generic pointer data:

  union _mu_dbm_errno
  {
    int n;            /* numeric value */
    void *p;          /* pointer to an error object */
  };

The implementation methods are supposed to keep its value up to date. For example, in GDBM implementation, they store the value of gdbm_errno in db_errno.n:

 1static int
 2_gdbm_store (mu_dbm_file_t db,
 3             struct mu_dbm_datum const *key,
 4             struct mu_dbm_datum const *contents,
 5             int replace)
 6{
 7  /* initialize local variables */
 8  ...
 9  /* Do the underlying call and return: */
10  switch (gdbm_store ((GDBM_FILE)db->db_descr, key_datum, content_datum, replace))
11    {
12    case 0:
13      break;
14
15    case 1:
16      return MU_ERR_EXISTS;
17
18    case -1:
19      db->db_errno.n = gdbm_errno;
20      return MU_ERR_FAILURE;
21    }
22}

A corresponding _dbm_strerror implementation uses the stored value:

1static char const *
2_gdbm_strerror (mu_dbm_file_t db)
3{
4  return gdbm_strerror (db->db_errno.n);
5}

A properly initialized mu_dbm_impl structure must be registered with the library by calling the following function:

  int mu_dbm_register (struct mu_dbm_impl *impl);

Once done, it becomes avaiable for use by the API.

Registered implementations can be accessed via an iterator returned by the mu_dbm_impl_iterator function:

  int mu_dbm_impl_iterator (mu_iterator_t *itr);

For example, the following code snippet lists avaialble DBM implementations:

 1  mu_iterator_t itr;
 2  
 3  mu_dbm_impl_iterator (&itr);
 4  for (mu_iterator_first (itr), i = 0; !mu_iterator_is_done (itr);
 5       mu_iterator_next (itr), i++)
 6    {
 7      struct mu_dbm_impl *impl;
 8
 9      mu_iterator_current (itr, (void**)&impl);
10      fprintf (stream, " ");
11      fprintf (stream, "%s", impl->_dbm_name);
12    }
13  fputc ('\n', stream);
14  mu_iterator_destroy (&itr);

For an example of using the backend implementation API see libmu_dbm/gdbm.c[4].

Notes