Tags: fdatasync

fdatasync()????????????????

By MD on Mar 4, 2008 | In db4o, ??? | Send feedback »

???????????????????????????????????????????????????????db4o???????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????

?????????????????????????????????????????????????????????????????

POSIX???OS???fsync()??????fdatasync()???????????????????????????????????????????????????????????????????????fsync()???????????fdatasync()??????????10-20%??????????????

????????db4o?????????db4o?IoAdapter?????????I/O??????????????????JNI?POSIX?fdatasync()????????????

?????????????IoAdapter??????????????????????

Code:

#include <jni.h>
#include "LinuxIoAdapter.h"
#include <stdio.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
 
/*
 * Class:     LinuxIoAdapter
 * Method:    openFile
 * Signature: (Ljava/lang/String;ZJ)J
 */
JNIEXPORT jint JNICALL Java_LinuxIoAdapter_openFile
  (JNIEnv *env, jclass jc, jstring path, jboolean lockfile, jlong initiallength)
{
    printf("openFile\n");
    const jbyte *filename = (*env)->GetStringUTFChars(env, path, NULL);
 
    if(filename == NULL){
        return 0; // OutOfMemoryError or NULL value
    }
 
    int fd = open(filename, O_RDWR|O_CREAT, 0644);
 
    if(fd < 0){
        printf("open failed\n");
    }
 
    (*env)->ReleaseStringChars(env, path, NULL);
 
    return fd;
 
}
 
/*
 * Class:     LinuxIoAdapter
 * Method:    closeFile
 * Signature: (J)V
 */
JNIEXPORT void JNICALL Java_LinuxIoAdapter_closeFile
  (JNIEnv * env, jclass jc, jint handle)
{
    printf("closeFile\n");
    if(close(handle) < 0){
        printf("close failed\n");
    }
 
}
 
/*
 * Class:     LinuxIoAdapter
 * Method:    getLength
 * Signature: (J)J
 */
JNIEXPORT jlong JNICALL Java_LinuxIoAdapter_getLength
  (JNIEnv *env, jclass jc, jint handle)
{
    struct stat st;
    fstat(handle, &st);
    return st.st_size;
}
 
/*
 * Class:     LinuxIoAdapter
 * Method:    read
 * Signature: (J[BI)I
 */
JNIEXPORT jint JNICALL Java_LinuxIoAdapter_read
  (JNIEnv *env, jclass jc, jint handle, jbyteArray bytes, jint length)
{
    
    jbyte buf[length];
 
    ssize_t n = read(handle, &buf, length);
 
    if(n != length)
        printf("read failed: %i->%i", length, n);
 
    (*env)->SetByteArrayRegion(env, bytes, 0, length, buf);
 
    return n;
}
 
/*
 * Class:     LinuxIoAdapter
 * Method:    seek
 * Signature: (JJ)V
 */
JNIEXPORT void JNICALL Java_LinuxIoAdapter_seek
  (JNIEnv *env, jclass jc, jint handle, jlong pos)
{
 
    lseek(handle, pos, SEEK_SET);
}
 
/*
 * Class:     LinuxIoAdapter
 * Method:    sync
 * Signature: (J)V
 */
JNIEXPORT void JNICALL Java_LinuxIoAdapter_sync
  (JNIEnv *env, jclass jc, jint handle)
{
    fdatasync(handle);
    //fsync(handle);
}
 
/*
 * Class:     LinuxIoAdapter
 * Method:    write
 * Signature: (J[BI)V
 */
JNIEXPORT void JNICALL Java_LinuxIoAdapter_write
  (JNIEnv *env, jclass jc, jint handle, jbyteArray bytes, jint length)
{
    jbyte buf[length];
 
    (*env)->GetByteArrayRegion(env, bytes, 0, length, buf);
 
    ssize_t n = write(handle, &buf, length);
 
    if(n != length)
        printf("read failed: %i->%i", length, n);
}
Tags: fdatasync, fsync, ioadapter, jni, posix

fdatasync() makes db4o 10-20% faster

By MD on Mar 3, 2008 | In db4o | Send feedback »

I have investigated how much fdatasync() makes db4o faster on Linux. It showed 10-20% faster performance in commit.

This benchmarking was done on Linux2.6, write cache off, ext3 file system, with db4o 6.4.

fdatasync() adapter
---------- 3: insert ----------
Used Memory: 273600/5177344
total commit cost: 33816 ms
average commit cost: 135 ms
Execution Time: 36625ms
Used Memory: 530904/5177344
---------- END ----------

---------- 9: update ----------
Used Memory: 289840/5177344
total commit cost: 38886 ms
average commit cost: 155 ms
Execution Time: 45488ms
Used Memory: 348824/5177344
---------- END ----------

---------- 11: delete ----------
Used Memory: 349088/5177344
total commit cost: 49090 ms
average commit cost: 196 ms
Execution Time: 52867ms
Used Memory: 576280/5177344
---------- END ----------

fsync adapter(the default one on Java)
---------- 3: insert ----------
Used Memory: 354584/5177344
total commit cost: 38190 ms
average commit cost: 152 ms
Execution Time: 41294ms
Used Memory: 659360/5177344
---------- END ----------

---------- 9: update ----------
Used Memory: 372136/5177344
total commit cost: 45015 ms
average commit cost: 180 ms
Execution Time: 50755ms
Used Memory: 428448/5177344
---------- END ----------

---------- 11: delete ----------
Used Memory: 428712/5177344
total commit cost: 49602 ms
average commit cost: 198 ms
Execution Time: 52775ms
Used Memory: 651192/5177344
---------- END ----------

The performance improvements depend on, how frequently user data is committed, and the implementation of fdatasync call. But even in the worst case, it is as fast as the default one.

BTW, fdatasync have triggered an inconsistent state in fread&fwrite. As people suggest, I should have avoided to mix POSIX(direct) and Standard C(buffered).

Tags: c, fdatasync, fsync, posix

Another skeptical guy

By MD on Mar 1, 2008 | In Durability | Send feedback »

Proposal for "proper" durable fsync() and fdatasync()

Jamie made a proposal in linux kernel newsgroup.

As I wrote about it in my latest column, it is getting hot. If you are serious about durability, let's make some noise together!

Tags: fdatasync, fsync, linux