[leveldb] LevelDB files are corrupt, How can we repair them?

[Junaid Farooq]

Hi everyone,

We have been using a filesystem build in Golang, seaweedfs
<http://github.com/chrislusf/seaweedfs>, and our own disk's file system was
XFS,

a few days back we got an error as

[ 42.236828] XFS (sda5): xfs_do_force_shutdown(0x8) called from line 118
of file /build/linux-lts-wily-JUyKGw/linux-lts-wily-4.2.0/fs/xfs/
xfs_bmap_util.c. Return address = 0xffffffffc02e35e4
[ 42.237271] XFS (sda5): Corruption of in-memory data detected. Shutting
down filesystem
[ 42.237320] XFS (sda5): Please umount the filesystem and rectify the
problem(s)

so, we mounted and unmounted the disk and then It needs cleaning, so we
repaired it with
xfs_repair -L /dev/sda5

there was nothing in lost+found folder but after that, our seaweed master
server started to crash, As We asked a question to Chris Lu, who made the
seaweedFS, he mentioned the files are corrupt, in filer, actually leveldb
files. the error was something like this

I1101 12:42:19 31159 volume.go:110] loading index file /storage/1082.idx readonly false
F1101 12:42:19 31159 filer_server.go:53] Can not start filer in dir /storage/filer : leveldb/storage: corrupted or incomplete meta file
goroutine 21 [running]:
github.com/chrislusf/seaweedfs/weed/glog.stacks(0x1214600, 0x0, 0x0, 0x0)
/root/go/src/github.com/chrislusf/seaweedfs/weed/glog/glog.go:767 +0xb8
github.com/chrislusf/seaweedfs/weed/glog.(*loggingT).output(0x11f59e0, 0xc800000003, 0xc8200faf00, 0x11cb17f, 0xf, 0x35, 0x0)
/root/go/src/github.com/chrislusf/seaweedfs/weed/glog/glog.go:718 +0x259
github.com/chrislusf/seaweedfs/weed/glog.(*loggingT).printf(0x11f59e0, 0xc800000003, 0xe57d80, 0x22, 0xc82004dd10, 0x2, 0x2)
/root/go/src/github.com/chrislusf/seaweedfs/weed/glog/glog.go:656 +0x1d4
github.com/chrislusf/seaweedfs/weed/glog.Fatalf(0xe57d80, 0x22, 0xc82004dd10, 0x2, 0x2)
/root/go/src/github.com/chrislusf/seaweedfs/weed/glog/glog.go:1149 +0x5d
github.com/chrislusf/seaweedfs/weed/server.NewFilerServer(0xc8200659e0, 0xd8b9b0, 0x7, 0x22b8, 0xc8200eb740, 0x13, 0xc82018d280, 0xe, 0x0, 0x0, ...)


after that, we used a tool to repair leveldb files,
https://github.com/rchunping/leveldb-tools, It repaired the files without
any error, and seaweedfs Master started with filer, with no errors like the
mentioned above, But as when everything is working fine, filer should
respond on port `8888`, but when we access port 8888, it gave error like
this,

017/11/02 01:50:41 http: panic serving 39.36.53.157:46124: leveldb: internal key "\x00\x01d,", len=4: invalid length
goroutine 2498 [running]:
net/http.(*conn).serve.func1(0xcd3aaf8080)
/usr/lib/go/src/net/http/server.go:1389 +0xc1
panic(0xaf99a0, 0xcfefca8210)
/usr/lib/go/src/runtime/panic.go:426 +0x4e9
github.com/syndtr/goleveldb/leveldb.internalKey.assert(0xc8712930ed, 0x4, 0x20)
/root/go/src/github.com/syndtr/goleveldb/leveldb/key.go:115 +0x22c
github.com/syndtr/goleveldb/leveldb.internalKey.ukey(0xc8712930ed, 0x4, 0x20, 0x0, 0x0, 0x0)
/root/go/src/github.com/syndtr/goleveldb/leveldb/key.go:120 +0x46
github.com/syndtr/goleveldb/leveldb.(*iComparer).Compare(0xc852e100a0, 0xc8712930ed, 0x4, 0x20, 0xcfefca81d0, 0xc, 0xc, 0xc850f5cf60)
/root/go/src/github.com/syndtr/goleveldb/leveldb/comparer.go:36 +0x4a
github.com/syndtr/goleveldb/leveldb/table.(*block).seek.func1(0x1, 0x8cce81)
/root/go/src/github.com/syndtr/goleveldb/leveldb/table/reader.go:68 +0x278
sort.Search(0x3, 0xcfefc44bd0, 0xe8)
/usr/lib/go/src/sort/search.go:66 +0x52
github.com/syndtr/goleveldb/leveldb/table.(*block).seek(0xc87231c1c0, 0x7f8204327100, 0xc852e100a0, 0x0, 0x3, 0xcfefca81d0, 0xc, 0xc, 0xcdcd40, 0x443815, ...)
/root/go/src/github.com/syndtr/goleveldb/leveldb/table/reader.go:69 +0xa9
github.com/syndtr/goleveldb/leveldb/table.(*blockIter).Seek(0xcfefb9e5a0, 0xcfefca81d0, 0xc, 0xc, 0x0)
/root/go/src/github.com/syndtr/goleveldb/leveldb/table/reader.go:229 +0x10e
github.com/syndtr/goleveldb/leveldb/table.(*Reader).newBlockIter(0xc8387461a0, 0xc87231c1c0, 0x7f81f57e7168, 0xcb79534140, 0xc87026d3b0, 0x1, 0x0)
/root/go/src/github.com/syndtr/goleveldb/leveldb/table/reader.go:736 +0x1db
github.com/syndtr/goleveldb/leveldb/table.(*Reader).NewIterator(0xc8387461a0, 0xc87026d3b0, 0x0, 0x0, 0x0)
/root/go/src/github.com/syndtr/goleveldb/leveldb/table/reader.go:804 +0x356
github.com/syndtr/goleveldb/leveldb.(*tOps).newIterator(0xc86b6ad350, 0xc85214bf90, 0xc87026d3b0, 0x0, 0x0, 0x0)
/root/go/src/github.com/syndtr/goleveldb/leveldb/table.go:411 +0x1a5
github.com/syndtr/goleveldb/leveldb.(*version).getIterators(0xc844da5cc0, 0xc87026d3b0, 0x0, 0x0, 0x0, 0x0)
/root/go/src/github.com/syndtr/goleveldb/leveldb/version.go:253 +0x1a7
github.com/syndtr/goleveldb/leveldb.(*DB).newRawIterator(0xc8201f4780, 0x0, 0x0, 0x0, 0x0, 0xc87026d3b0, 0x0, 0x0, 0x0)
/root/go/src/github.com/syndtr/goleveldb/leveldb/db_iter.go:41 +0xdc
github.com/syndtr/goleveldb/leveldb.(*DB).newIterator(0xc8201f4780, 0x0, 0x0, 0x0, 0x0, 0x16571aa28, 0xcfefc45550, 0x0, 0x4ee651)
/root/go/src/github.com/syndtr/goleveldb/leveldb/db_iter.go:79 +0x229
github.com/syndtr/goleveldb/leveldb.(*DB).NewIterator(0xc8201f4780, 0xcfefc45550, 0x0, 0x0, 0x0)
/root/go/src/github.com/syndtr/goleveldb/leveldb/db.go:882 +0x205
github.com/chrislusf/seaweedfs/weed/filer/embedded_filer.(*FileListInLevelDb).ListFiles(0xc82009c138, 0xcf00000000, 0x0, 0x0, 0x64, 0x0, 0x0, 0x0)
/root/go/src/github.com/chrislusf/seaweedfs/weed/filer/embedded_filer/files_in_leveldb.go:64 +0x48c
github.com/chrislusf/seaweedfs/weed/filer/embedded_filer.(*FilerEmbedded).ListFiles(0xc849337300, 0xcfefca8144, 0x1, 0x0, 0x0, 0x64, 0x0, 0x0, 0x0, 0x0, ...)
/root/go/src/github.com/chrislusf/seaweedfs/weed/filer/embedded_filer/filer_embedded.go:63 +0xf1
github.com/chrislusf/seaweedfs/weed/server.(*FilerServer).listDirectoryHandler(0xc8201f6000, 0x7f81e2155190, 0xcbbe8221a0, 0xc820156380)
/root/go/src/github.com/chrislusf/seaweedfs/weed/server/filer_server_handlers_read.go:40 +0x6b1
github.com/chrislusf/seaweedfs/weed/server.(*FilerServer).GetOrHeadHandler(0xc8201f6000, 0x7f81e2155190, 0xcbbe8221a0, 0xc820156380, 0xffffffffffffff01)
/root/go/src/github.com/chrislusf/seaweedfs/weed/server/filer_server_handlers_read.go:50 +0x150
github.com/chrislusf/seaweedfs/weed/server.(*FilerServer).filerHandler(0xc8201f6000, 0x7f81e2155190, 0xcbbe8221a0, 0xc820156380)
/root/go/src/github.com/chrislusf/seaweedfs/weed/server/filer_server_handlers.go:10 +0xf6
github.com/chrislusf/seaweedfs/weed/server.(*FilerServer).(github.com/chrislusf/seaweedfs/weed/server.filerHandler)-fm(0x7f81e2155190, 0xcbbe8221a0, 0xc820156380)
/root/go/src/github.com/chrislusf/seaweedfs/weed/server/filer_server.go:60 +0x3e
net/http.HandlerFunc.ServeHTTP(0xc82ddf86d0, 0x7f81e2155190, 0xcbbe8221a0, 0xc820156380)
/usr/lib/go/src/net/http/server.go:1618 +0x3a
net/http.(*ServeMux).ServeHTTP(0xc82017eb40, 0x7f81e2155190, 0xcbbe8221a0, 0xc820156380)
/usr/lib/go/src/net/http/server.go:1910 +0x17d
net/http.serverHandler.ServeHTTP(0xc852de9f00, 0x7f81e2155190, 0xcbbe8221a0, 0xc820156380)
/usr/lib/go/src/net/http/server.go:2081 +0x19e
net/http.(*conn).serve(0xcd3aaf8080)
/usr/lib/go/src/net/http/server.go:1472 +0xf2e
created by net/http.(*Server).Serve
/usr/lib/go/src/net/http/server.go:2137 +0x44e
2017/11/02 01:51:12 http: panic serving 39.36.53.157:46128: leveldb: internal key "\x00\x01d,", len=4: invalid length
goroutine 2467 [running]:
net/http.(*conn).serve.func1(0xc921cf6780)
/usr/lib/go/src/net/http/server.go:1389 +0xc1
panic(0xaf99a0, 0xcf62fea290)
/usr/lib/go/src/runtime/panic.go:426 +0x4e9
github.com/syndtr/goleveldb/leveldb.internalKey.assert(0xc8712930ed, 0x4, 0x20)
/root/go/src/github.com/syndtr/goleveldb/leveldb/key.go:115 +0x22c
github.com/syndtr/goleveldb/leveldb.internalKey.ukey(0xc8712930ed, 0x4, 0x20, 0x0, 0x0, 0x0)
/root/go/src/github.com/syndtr/goleveldb/leveldb/key.go:120 +0x46
github.com/syndtr/goleveldb/leveldb.(*iComparer).Compare(0xc852e100a0, 0xc8712930ed, 0x4, 0x20, 0xcf62fea250, 0xc, 0xc, 0x0)
/root/go/src/github.com/syndtr/goleveldb/leveldb/comparer.go:36 +0x4a
github.com/syndtr/goleveldb/leveldb/table.(*block).seek.func1(0x1, 0x8cce81)
/root/go/src/github.com/syndtr/goleveldb/leveldb/table/reader.go:68 +0x278
sort.Search(0x3, 0xca5cc78bd0, 0xe8)
/usr/lib/go/src/sort/search.go:66 +0x52
github.com/syndtr/goleveldb/leveldb/table.(*block).seek(0xc87231c1c0, 0x7f8204327100, 0xc852e100a0, 0x0, 0x3, 0xcf62fea250, 0xc, 0xc, 0xcdcd40, 0x443815, ...)
/root/go/src/github.com/syndtr/goleveldb/leveldb/table/reader.go:69 +0xa9
github.com/syndtr/goleveldb/leveldb/table.(*blockIter).Seek(0xce24069b30, 0xcf62fea250, 0xc, 0xc, 0x0)
/root/go/src/github.com/syndtr/goleveldb/leveldb/table/reader.go:229 +0x10e
github.com/syndtr/goleveldb/leveldb/table.(*Reader).newBlockIter(0xc8387461a0, 0xc87231c1c0, 0x7f81f57e7168, 0xcb795cc1f8, 0xc87024edb0, 0x1, 0x0)
/root/go/src/github.com/syndtr/goleveldb/leveldb/table/reader.go:736 +0x1db
github.com/syndtr/goleveldb/leveldb/table.(*Reader).NewIterator(0xc8387461a0, 0xc87024edb0, 0x0, 0x0, 0x0)
/root/go/src/github.com/syndtr/goleveldb/leveldb/table/reader.go:804 +0x356
github.com/syndtr/goleveldb/leveldb.(*tOps).newIterator(0xc86b6ad350, 0xc85214bf90, 0xc87024edb0, 0x0, 0x0, 0x0)
/root/go/src/github.com/syndtr/goleveldb/leveldb/table.go:411 +0x1a5
github.com/syndtr/goleveldb/leveldb.(*version).getIterators(0xc844da5cc0, 0xc87024edb0, 0x0, 0x0, 0x0, 0x0)
/root/go/src/github.com/syndtr/goleveldb/leveldb/version.go:253 +0x1a7
github.com/syndtr/goleveldb/leveldb.(*DB).newRawIterator(0xc8201f4780, 0x0, 0x0, 0x0, 0x0, 0xc87024edb0, 0x0, 0x0, 0x0)
/root/go/src/github.com/syndtr/goleveldb/leveldb/db_iter.go:41 +0xdc
github.com/syndtr/goleveldb/leveldb.(*DB).newIterator(0xc8201f4780, 0x0, 0x0, 0x0, 0x0, 0x16571aa28, 0xca5cc79550, 0x0, 0x4ee651)
/root/go/src/github.com/syndtr/goleveldb/leveldb/db_iter.go:79 +0x229
github.com/syndtr/goleveldb/leveldb.(*DB).NewIterator(0xc8201f4780, 0xca5cc79550, 0x0, 0x0, 0x0)
/root/go/src/github.com/syndtr/goleveldb/leveldb/db.go:882 +0x205
github.com/chrislusf/seaweedfs/weed/filer/embedded_filer.(*FileListInLevelDb).ListFiles(0xc82009c138, 0xcf00000000, 0x0, 0x0, 0x64, 0x0, 0x0, 0x0)
/root/go/src/github.com/chrislusf/seaweedfs/weed/filer/embedded_filer/files_in_leveldb.go:64 +0x48c
github.com/chrislusf/seaweedfs/weed/filer/embedded_filer.(*FilerEmbedded).ListFiles(0xc849337300, 0xcf62fea1c4, 0x1, 0x0, 0x0, 0x64, 0x0, 0x0, 0x0, 0x0, ...)
/root/go/src/github.com/chrislusf/seaweedfs/weed/filer/embedded_filer/filer_embedded.go:63 +0xf1
github.com/chrislusf/seaweedfs/weed/server.(*FilerServer).listDirectoryHandler(0xc8201f6000, 0x7f81e2155190, 0xcdf1bf21a0, 0xc82018c8c0)
/root/go/src/github.com/chrislusf/seaweedfs/weed/server/filer_server_handlers_read.go:40 +0x6b1
github.com/chrislusf/seaweedfs/weed/server.(*FilerServer).GetOrHeadHandler(0xc8201f6000, 0x7f81e2155190, 0xcdf1bf21a0, 0xc82018c8c0, 0xffffffffffffff01)
/root/go/src/github.com/chrislusf/seaweedfs/weed/server/filer_server_handlers_read.go:50 +0x150
github.com/chrislusf/seaweedfs/weed/server.(*FilerServer).filerHandler(0xc8201f6000, 0x7f81e2155190, 0xcdf1bf21a0, 0xc82018c8c0)
/root/go/src/github.com/chrislusf/seaweedfs/weed/server/filer_server_handlers.go:10 +0xf6
github.com/chrislusf/seaweedfs/weed/server.(*FilerServer).(github.com/chrislusf/seaweedfs/weed/server.filerHandler)-fm(0x7f81e2155190, 0xcdf1bf21a0, 0xc82018c8c0)
/root/go/src/github.com/chrislusf/seaweedfs/weed/server/filer_server.go:60 +0x3e
net/http.HandlerFunc.ServeHTTP(0xc82ddf86d0, 0x7f81e2155190, 0xcdf1bf21a0, 0xc82018c8c0)
/usr/lib/go/src/net/http/server.go:1618 +0x3a
net/http.(*ServeMux).ServeHTTP(0xc82017eb40, 0x7f81e2155190, 0xcdf1bf21a0, 0xc82018c8c0)
/usr/lib/go/src/net/http/server.go:1910 +0x17d
net/http.serverHandler.ServeHTTP(0xc852de9f00, 0x7f81e2155190, 0xcdf1bf21a0, 0xc82018c8c0)
/usr/lib/go/src/net/http/server.go:2081 +0x19e
net/http.(*conn).serve(0xc921cf6780)
/usr/lib/go/src/net/http/server.go:1472 +0xf2e
created by net/http.(*Server).Serve
/usr/lib/go/src/net/http/server.go:2137 +0x44e
2017/11/02 01:52:23 http: panic serving 39.36.53.157:46142: leveldb: internal key "\x00\x01d,", len=4: invalid length
goroutine 2456 [running]:
net/http.(*conn).serve.func1(0xc97d935b80)
/usr/lib/go/src/net/http/server.go:1389 +0xc1
panic(0xaf99a0, 0xce2f857050)
/usr/lib/go/src/runtime/panic.go:426 +0x4e9
github.com/syndtr/goleveldb/leveldb.internalKey.assert(0xc8712930ed, 0x4, 0x20)
/root/go/src/github.com/syndtr/goleveldb/leveldb/key.go:115 +0x22c
github.com/syndtr/goleveldb/leveldb.internalKey.ukey(0xc8712930ed, 0x4, 0x20, 0x0, 0x0, 0x0)
/root/go/src/github.com/syndtr/goleveldb/leveldb/key.go:120 +0x46
github.com/syndtr/goleveldb/leveldb.(*iComparer).Compare(0xc852e100a0, 0xc8712930ed, 0x4, 0x20, 0xce2f857010, 0xc, 0xc, 0x0)
/root/go/src/github.com/syndtr/goleveldb/leveldb/comparer.go:36 +0x4a
github.com/syndtr/goleveldb/leveldb/table.(*block).seek.func1(0x1, 0x8cce81)
/root/go/src/github.com/syndtr/goleveldb/leveldb/table/reader.go:68 +0x278
sort.Search(0x3, 0xca5cc78bd0, 0xe8)
/usr/lib/go/src/sort/search.go:66 +0x52
github.com/syndtr/goleveldb/leveldb/table.(*block).seek(0xc87231c1c0, 0x7f8204327100, 0xc852e100a0, 0x0, 0x3, 0xce2f857010, 0xc, 0xc, 0xcdcd40, 0x443815, ...)
/root/go/src/github.com/syndtr/goleveldb/leveldb/table/reader.go:69 +0xa9
github.com/syndtr/goleveldb/leveldb/table.(*blockIter).Seek(0xce24069ef0, 0xce2f857010, 0xc, 0xc, 0x0)
/root/go/src/github.com/syndtr/goleveldb/leveldb/table/reader.go:229 +0x10e
github.com/syndtr/goleveldb/leveldb/table.(*Reader).newBlockIter(0xc8387461a0, 0xc87231c1c0, 0x7f81f57e7168, 0xcb795cc478, 0xc8701be090, 0x1, 0x0)
/root/go/src/github.com/syndtr/goleveldb/leveldb/table/reader.go:736 +0x1db
github.com/syndtr/goleveldb/leveldb/table.(*Reader).NewIterator(0xc8387461a0, 0xc8701be090, 0x0, 0x0, 0x0)
/root/go/src/github.com/syndtr/goleveldb/leveldb/table/reader.go:804 +0x356
github.com/syndtr/goleveldb/leveldb.(*tOps).newIterator(0xc86b6ad350, 0xc85214bf90, 0xc8701be090, 0x0, 0x0, 0x0)
/root/go/src/github.com/syndtr/goleveldb/leveldb/table.go:411 +0x1a5
github.com/syndtr/goleveldb/leveldb.(*version).getIterators(0xc844da5cc0, 0xc8701be090, 0x0, 0x0, 0x0, 0x0)
/root/go/src/github.com/syndtr/goleveldb/leveldb/version.go:253 +0x1a7
github.com/syndtr/goleveldb/leveldb.(*DB).newRawIterator(0xc8201f4780, 0x0, 0x0, 0x0, 0x0, 0xc8701be090, 0x0, 0x0, 0x0)
/root/go/src/github.com/syndtr/goleveldb/leveldb/db_iter.go:41 +0xdc
github.com/syndtr/goleveldb/leveldb.(*DB).newIterator(0xc8201f4780, 0x0, 0x0, 0x0, 0x0, 0x16571aa28, 0xca5cc79550, 0x0, 0x4ee651)
/root/go/src/github.com/syndtr/goleveldb/leveldb/db_iter.go:79 +0x229
github.com/syndtr/goleveldb/leveldb.(*DB).NewIterator(0xc8201f4780, 0xca5cc79550, 0x0, 0x0, 0x0)
/root/go/src/github.com/syndtr/goleveldb/leveldb/db.go:882 +0x205
github.com/chrislusf/seaweedfs/weed/filer/embedded_filer.(*FileListInLevelDb).ListFiles(0xc82009c138, 0xce00000000, 0x0, 0x0, 0x64, 0x0, 0x0, 0x0)
/root/go/src/github.com/chrislusf/seaweedfs/weed/filer/embedded_filer/files_in_leveldb.go:64 +0x48c
github.com/chrislusf/seaweedfs/weed/filer/embedded_filer.(*FilerEmbedded).ListFiles(0xc849337300, 0xce2f856f64, 0x1, 0x0, 0x0, 0x64, 0x0, 0x0, 0x0, 0x0, ...)
/root/go/src/github.com/chrislusf/seaweedfs/weed/filer/embedded_filer/filer_embedded.go:63 +0xf1
github.com/chrislusf/seaweedfs/weed/server.(*FilerServer).listDirectoryHandler(0xc8201f6000, 0x7f81e2155190, 0xcc30caf520, 0xcfef0d87e0)
/root/go/src/github.com/chrislusf/seaweedfs/weed/server/filer_server_handlers_read.go:40 +0x6b1
github.com/chrislusf/seaweedfs/weed/server.(*FilerServer).GetOrHeadHandler(0xc8201f6000, 0x7f81e2155190, 0xcc30caf520, 0xcfef0d87e0, 0xffffffffffffff01)
/root/go/src/github.com/chrislusf/seaweedfs/weed/server/filer_server_handlers_read.go:50 +0x150
github.com/chrislusf/seaweedfs/weed/server.(*FilerServer).filerHandler(0xc8201f6000, 0x7f81e2155190, 0xcc30caf520, 0xcfef0d87e0)
/root/go/src/github.com/chrislusf/seaweedfs/weed/server/filer_server_handlers.go:10 +0xf6
github.com/chrislusf/seaweedfs/weed/server.(*FilerServer).(github.com/chrislusf/seaweedfs/weed/server.filerHandler)-fm(0x7f81e2155190, 0xcc30caf520, 0xcfef0d87e0)
/root/go/src/github.com/chrislusf/seaweedfs/weed/server/filer_server.go:60 +0x3e
net/http.HandlerFunc.ServeHTTP(0xc82ddf86d0, 0x7f81e2155190, 0xcc30caf520, 0xcfef0d87e0)
/usr/lib/go/src/net/http/server.go:1618 +0x3a
net/http.(*ServeMux).ServeHTTP(0xc82017eb40, 0x7f81e2155190, 0xcc30caf520, 0xcfef0d87e0)
/usr/lib/go/src/net/http/server.go:1910 +0x17d
net/http.serverHandler.ServeHTTP(0xc852de9f00, 0x7f81e2155190, 0xcc30caf520, 0xcfef0d87e0)
/usr/lib/go/src/net/http/server.go:2081 +0x19e
net/http.(*conn).serve(0xc97d935b80)
/usr/lib/go/src/net/http/server.go:1472 +0xf2e
created by net/http.(*Server).Serve
/usr/lib/go/src/net/http/server.go:2137 +0x44


So Chris says , the leveldb file seems corrupted.

My question is:

Is there any way to repair those corrupted files, As I have used the above
tool, but I made little bit difference only, are there any good tools which
can repair those corrupted files, or we are shooting in the dark and it not
even possible.

I don't have any knowledge about Golang or leveldb, So pardon me if the
question seems silly to all of you. Purpose of posting here is: If you can
see the error and suggest about , how to recover that. :)

['Chris Mumford' via leveldb]

For this I would use the latest release of the official leveldb library,
and write a small C/C++ application to do the repair. The problem with
Python/Go implementations is that they may not use the leveldb library, may
not be the current release of the library, or may have their options
hard-coded. Also, make sure to set the paranoid_checks option to true when
calling RepairDB. I would also set the info_log option so that you can
write out detailed status of the repaired db.

I'm not very familiar with RepairDB, but I don't see how it would restore
deleted records intentionally. It does look for manifests (which reference
the active tables). If the corruption somehow deleted the most recent
manifest then the most recent (highest numbered) one will be used and then
you'll be using the old tables during the restoration.

[Junaid Farooq]

Hello, Thanks for your reply, I tried that in a few days back,
#python

import leveldb;
leveldb.RepairDB("path")

I did recover but it really recovered everything else. but not repaired
actually.

I understand what you are saying at the moment for c++ application, But is
there anyway, or any option in repair, to say not recover just repair?

Post by 'Chris Mumford' via leveldb
For this I would use the latest release of the official leveldb library,
and write a small C/C++ application to do the repair. The problem with
Python/Go implementations is that they may not use the leveldb library, may
not be the current release of the library, or may have their options
hard-coded. Also, make sure to set the paranoid_checks option to true when
calling RepairDB. I would also set the info_log option so that you can
write out detailed status of the repaired db.
I'm not very familiar with RepairDB, but I don't see how it would restore
deleted records intentionally. It does look for manifests (which reference
the active tables). If the corruption somehow deleted the most recent
manifest then the most recent (highest numbered) one will be used and then
you'll be using the old tables during the restoration.