Note: This is a beta release of Red Hat Bugzilla 5.0. The data contained within is a snapshot of the live data so any changes you make will not be reflected in the production Bugzilla. Also email is disabled so feel free to test any aspect of the site that you want. File any problems you find or give feedback here.
Bug 162416 - GDBM database dump/restore utility to migrate files between architectures
Summary: GDBM database dump/restore utility to migrate files between architectures
Keywords:
Status: CLOSED UPSTREAM
Alias: None
Product: Fedora
Classification: Fedora
Component: gdbm
Version: rawhide
Hardware: x86_64
OS: Linux
medium
medium
Target Milestone: ---
Assignee: Marek Skalický
QA Contact: Fedora Extras Quality Assurance
URL:
Whiteboard:
Depends On:
Blocks:
TreeView+ depends on / blocked
 
Reported: 2005-07-04 11:31 UTC by Matthias Saou
Modified: 2017-05-16 12:07 UTC (History)
5 users (show)

Fixed In Version:
Doc Type: Enhancement
Doc Text:
Clone Of:
Environment:
Last Closed: 2017-05-16 12:07:38 UTC


Attachments (Terms of Use)
Output from strace dumpgdbm (deleted)
2005-07-04 13:28 UTC, Matthias Saou
no flags Details

Description Matthias Saou 2005-07-04 11:31:10 UTC
Description of problem:
PHP recompiled with GDBM support didin't seem to work on RHEL4 x86_64, and
trying to access the GDBM files by other means seems to fail all the time.

Version-Release number of selected component (if applicable):
gdbm-1.8.0-24.x86_64

How reproducible:
Always.

Steps to Reproduce:
1. Download and compile dumpgdbm from http://krabulator.free.fr/devel/dumpgdbm.html
2. Run dumpgdbm on a test GDBM file, it segfaults
3. Now compile dumpgdbm on i386, install gdbm.i386, readline.i386 and
libtermcap.i386 (all provided in RHEL4 x86_64), run that dumpgdbm on the same
GDBM file, it works
  
Actual results:
Access to GDBM files from x86_64 applications doesn't work, from i386 it does.

Expected results:
GDBM file access should work natively from x86_64 applications.

Additional info:
I've also tried installing the latest FC Development x86_64 package, same
result. I also tried with gdbm 1.8.3 (since all RH/FC products are still at
1.8.0) and again, same result.

Comment 1 Matthias Saou 2005-07-04 13:26:44 UTC
Some more information on the problem :

- With testgdbm, on a file that works fine on an i386 host :
./testgdbm -g ~/stats.db
gdbm_open failed, File seek error

- Strace of dumpgdbm on that same file :
"lseek(3, 35184372097024, SEEK_SET)      = -1 EINVAL (Invalid argument)" (see
the attached strace.txt file)

So it seems to be some kind of file seek error, for whatever that might mean.

Comment 2 Matthias Saou 2005-07-04 13:28:17 UTC
Created attachment 116328 [details]
Output from strace dumpgdbm

Comment 3 Matthias Saou 2005-07-04 13:54:44 UTC
More digging into the issue :
- Recompile the "testgdbm" program with "make progs" in the gdbm sources.
- On the x86_64 machine, run "testgdbm -g test.gdbm" and insert 2 entries.
- Copy the file over to the i386 machine, run "testgdbm -g test.gdbm", it seems
to work, but the database doesn't contain the inserted entries, it only contains
one empty key/value pair.

The other way around :
Doing the exact same operation but inserting on i386 and trying to read on
x86_64 results in testgdbm's "gdbm_open failed, File seek error" error message.

So it seems that GDBM files created on i386 and x86_64 are incompatible. This is
probably not normal, as endianess is the same, so sompatibility could probably
be achieved.

Comment 4 Bill Nottingham 2005-07-05 00:52:38 UTC
gdbm databases are, afaik, wordsize specific. Cc'ing Nalin, who I believe looked
at this once.

Comment 5 Matthias Saou 2005-07-05 09:46:06 UTC
Unfortunately (for me), you are right. I got this reply from the current
maintainer :

"Of course it works.  You're accessing a 32bit GDBM file with a 32bit library.
You can not, however, access a 32bit GDBM file with a *64bit* library.
GDBM files are not portable between different architectures."

So, I guess I could nevertheless change this bug to an RFE, to include some kind
of dump/restore utility that would allow moving GDBM files across architectures?
I'm pretty sure I won't be the last to move GDBM files from i386 to x86_64
servers...

The utility that did it for me was gdbmtext :
http://starynkevitch.net/Basile/gdbmtext.c

Which requires the following change to not segfault in some common cases when
importing :

--- gdbmtext.c.orig     2005-07-04 17:15:06.963692448 +0200
+++ gdbmtext.c  2005-07-04 19:10:02.505410768 +0200
@@ -71,6 +71,7 @@ indatum (char **ppc)
       xbuf[1] = pc[2];
       xbuf[2] = xbuf[3] = 0;
       *(wp++) = (char) strtol (xbuf, 0, 16);
+      pc += 2;
     } else
       *(wp++) = *pc;
   };

Comment 6 Matthias Saou 2005-11-07 11:45:49 UTC
Anyone willing to review that gdbmtext.c piece of code and get it included
either in upstream gdbm or at least in the Fedora/RH packages? Some basic
dump/restore functionnality is something really valuable when the format is
binary incompatible across supported architectures...

Changing this to a RFE while I'm at it.

Comment 7 Paul Nasrat 2006-08-08 22:29:22 UTC
Move to Fedora

Comment 8 Matthias Saou 2007-02-05 13:59:00 UTC
Is anyone still interested in providing a gdbm "migration tool"?

Comment 9 Red Hat Bugzilla 2007-08-21 05:20:29 UTC
User pnasrat@redhat.com's account has been closed

Comment 10 Ondrej Dvoracek 2007-09-05 22:51:27 UTC
Hi,
I am new owner of this bug. 
I will have a look at the gdbmtext.c and will decide whether we should try to
add this package to the distribution.

Cheers Ondrej

Comment 11 Sergio Monteiro Basto 2010-01-19 22:17:12 UTC
(In reply to comment #8)
> Is anyone still interested in providing a gdbm "migration tool"?    

Hi , I am facing this kind of problem , but I need a dbm which is not architecture dependent , because I need read on 32 and 64-bit platforms , 

Any suggestion ? 
Thanks,

Comment 12 Karel Klíč 2010-03-02 13:30:24 UTC
Hi Sergio,
please ask on a public mailing list, only very few people read Bugzilla entries.

Comment 13 Fedora Admin XMLRPC Client 2011-01-07 14:10:37 UTC
This package has changed ownership in the Fedora Package Database.  Reassigning to the new owner of this component.

Comment 14 Honza Horak 2011-10-05 11:32:45 UTC
Note, that a new gdbm-1.9.1 (already in Rawhide) provides different magic values for 32 and 64 bits, so we can discover what system the file was created on if we use this new version. Even though it is not a solution, I hope it can help at least a bit.

Comment 15 Fedora Admin XMLRPC Client 2014-09-11 15:13:42 UTC
This package has changed ownership in the Fedora Package Database.  Reassigning to the new owner of this component.

Comment 16 Sergio Monteiro Basto 2015-12-03 01:33:38 UTC
Some weeks ago I found that sqlite3 is a very good alternative and also seems that is platform independent [1] 

[1] http://stackoverflow.com/questions/15024658/are-sqlite3-databases-are-platform-independent

Comment 17 Honza Horak 2015-12-09 17:18:43 UTC
Well, sqlite3 might be suitable alternative in some cases, but it is generally very different product.

Comment 18 Marek Skalický 2017-05-16 12:07:38 UTC
Gdbm in latest versions contains gdbm_dump (and gdbmexport) tool which allows exporting of database to Flat Format (see gdbm doc).

Also it is possible to install gdbm.i686 on 64-bit systems to support reading of gdbm databases created without largefiles support.

Feel free to reopen. Or for new features create bugs in upstream tracker - https://puszcza.gnu.org.ua/bugs/?group=gdbm


Note You need to log in before you can comment on or make changes to this bug.