Repository navigation
Random segfault in kh_get_off #132
Description
I have a pretty involved setup that causes nodegit to segfault. It happens when I use https://github.com/msiebuhr/slint to lint the staged contents of a git repository backed by https://github.com/papandreou/fsplusgit which uses https://github.com/papandreou/gitfakefs which again uses nodegit :). I've tried to create a test case with fewer moving parts, but I've failed so far, so now I'm reporting it in case it points to something obvious in the nodegit code base.
Here's the gdb backtrace:
(gdb) run
Starting program: /home/andreas/.local/bin/node /home/andreas/.local/bin/slint --git slint.json
warning: no loadable sections found in added symbol-file system-supplied DSO at 0x7ffff7ffa000
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[New Thread 0x7ffff7ff7700 (LWP 7354)]
[New Thread 0x7ffff5e75700 (LWP 7355)]
[New Thread 0x7ffff5674700 (LWP 7356)]
[New Thread 0x7ffff4e73700 (LWP 7357)]
[New Thread 0x7fffeffff700 (LWP 7358)]
Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7ffff4e73700 (LWP 7357)]
0x00007ffff66e6e0e in kh_get_off (h=0x0, key=983856) at /home/andreas/work/slint/node_modules/fsplusgit/node_modules/gitfakefs/node_modules/nodegit-papandreou/vendor/libgit2/src/pack.h:65
65 GIT__USE_OFFMAP;
(gdb) backtrace
#0 0x00007ffff66e6e0e in kh_get_off (h=0x0, key=983856) at /home/andreas/work/slint/node_modules/fsplusgit/node_modules/gitfakefs/node_modules/nodegit-papandreou/vendor/libgit2/src/pack.h:65
#1 0x00007ffff66e7d32 in cache_get (cache=0x7fffd80092b0, offset=983856)
at /home/andreas/work/slint/node_modules/fsplusgit/node_modules/gitfakefs/node_modules/nodegit-papandreou/vendor/libgit2/src/pack.c:109
#2 0x00007ffff66e89b4 in packfile_unpack_delta (obj=0x7ffff4e72cf0, p=0x7fffd8009240, w_curs=0x7ffff4e72c88, curpos=0x7ffff4e72c90, delta_size=23, delta_type=GIT_OBJ_OFS_DELTA, obj_offset=984012)
at /home/andreas/work/slint/node_modules/fsplusgit/node_modules/gitfakefs/node_modules/nodegit-papandreou/vendor/libgit2/src/pack.c:483
#3 0x00007ffff66e8bc2 in git_packfile_unpack (obj=0x7ffff4e72cf0, p=0x7fffd8009240, obj_offset=0x7ffff4e72d10)
at /home/andreas/work/slint/node_modules/fsplusgit/node_modules/gitfakefs/node_modules/nodegit-papandreou/vendor/libgit2/src/pack.c:555
#4 0x00007ffff671b8a2 in pack_backend__read (buffer_p=0x7ffff4e72d90, len_p=0x7ffff4e72d98, type_p=0x7ffff4e72da0, backend=0x7fffd8000ed0, oid=0xeaae50)
at /home/andreas/work/slint/node_modules/fsplusgit/node_modules/gitfakefs/node_modules/nodegit-papandreou/vendor/libgit2/src/odb_pack.c:393
#5 0x00007ffff6724844 in git_odb_read (out=0x7ffff4e72e00, db=0x7fffd8000900, id=0xeaae50)
at /home/andreas/work/slint/node_modules/fsplusgit/node_modules/gitfakefs/node_modules/nodegit-papandreou/vendor/libgit2/src/odb.c:646
#6 0x00007ffff6711fc1 in git_object_lookup_prefix (object_out=0xebd5c0, repo=0x7fffe8000970, id=0xeaae50, len=40, type=GIT_OBJ_BLOB)
at /home/andreas/work/slint/node_modules/fsplusgit/node_modules/gitfakefs/node_modules/nodegit-papandreou/vendor/libgit2/src/object.c:177
#7 0x00007ffff67120d5 in git_object_lookup (object_out=0xebd5c0, repo=0x7fffe8000970, id=0xeaae50, type=GIT_OBJ_BLOB)
at /home/andreas/work/slint/node_modules/fsplusgit/node_modules/gitfakefs/node_modules/nodegit-papandreou/vendor/libgit2/src/object.c:211
#8 0x00007ffff69978e0 in GitRepo::GetBlobWork(uv_work_s*) () from /home/andreas/work/slint/node_modules/fsplusgit/node_modules/gitfakefs/node_modules/nodegit-papandreou/build/Release/nodegit.node
#9 0x00000000006ddebd in worker (arg=arg@entry=0x0) at ../deps/uv/src/unix/threadpool.c:74
#10 0x00000000006d3a0f in uv__thread_start (ctx_v=<optimized out>) at ../deps/uv/src/uv-common.c:322
#11 0x00007ffff6f9af8e in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
#12 0x00007ffff6cc4a0d in clone () from /lib/x86_64-linux-gnu/libc.so.6
(gdb) up
#1 0x00007ffff66e7d32 in cache_get (cache=0x7fffd80092b0, offset=983856)
at /home/andreas/work/slint/node_modules/fsplusgit/node_modules/gitfakefs/node_modules/nodegit-papandreou/vendor/libgit2/src/pack.c:109
109 k = kh_get(off, cache->entries, offset);
(gdb) up
#2 0x00007ffff66e89b4 in packfile_unpack_delta (obj=0x7ffff4e72cf0, p=0x7fffd8009240, w_curs=0x7ffff4e72c88, curpos=0x7ffff4e72c90, delta_size=23, delta_type=GIT_OBJ_OFS_DELTA, obj_offset=984012)
at /home/andreas/work/slint/node_modules/fsplusgit/node_modules/gitfakefs/node_modules/nodegit-papandreou/vendor/libgit2/src/pack.c:483
483 if ((cached = cache_get(&p->bases, base_offset)) != NULL) {
(gdb) up
#3 0x00007ffff66e8bc2 in git_packfile_unpack (obj=0x7ffff4e72cf0, p=0x7fffd8009240, obj_offset=0x7ffff4e72d10)
at /home/andreas/work/slint/node_modules/fsplusgit/node_modules/gitfakefs/node_modules/nodegit-papandreou/vendor/libgit2/src/pack.c:555
555 error = packfile_unpack_delta(
(gdb) up
#4 0x00007ffff671b8a2 in pack_backend__read (buffer_p=0x7ffff4e72d90, len_p=0x7ffff4e72d98, type_p=0x7ffff4e72da0, backend=0x7fffd8000ed0, oid=0xeaae50)
at /home/andreas/work/slint/node_modules/fsplusgit/node_modules/gitfakefs/node_modules/nodegit-papandreou/vendor/libgit2/src/odb_pack.c:393
393 if ((error = pack_entry_find(&e, (struct pack_backend *)backend, oid)) < 0 ||
(gdb) up
#5 0x00007ffff6724844 in git_odb_read (out=0x7ffff4e72e00, db=0x7fffd8000900, id=0xeaae50)
at /home/andreas/work/slint/node_modules/fsplusgit/node_modules/gitfakefs/node_modules/nodegit-papandreou/vendor/libgit2/src/odb.c:646
646 error = b->read(&raw.data, &raw.len, &raw.type, b, id);
(gdb) up
#6 0x00007ffff6711fc1 in git_object_lookup_prefix (object_out=0xebd5c0, repo=0x7fffe8000970, id=0xeaae50, len=40, type=GIT_OBJ_BLOB)
at /home/andreas/work/slint/node_modules/fsplusgit/node_modules/gitfakefs/node_modules/nodegit-papandreou/vendor/libgit2/src/object.c:177
177 error = git_odb_read(&odb_obj, odb, id);
(gdb) up
#7 0x00007ffff67120d5 in git_object_lookup (object_out=0xebd5c0, repo=0x7fffe8000970, id=0xeaae50, type=GIT_OBJ_BLOB)
at /home/andreas/work/slint/node_modules/fsplusgit/node_modules/gitfakefs/node_modules/nodegit-papandreou/vendor/libgit2/src/object.c:211
211 return git_object_lookup_prefix(object_out, repo, id, GIT_OID_HEXSZ, type);
(gdb) up
#8 0x00007ffff69978e0 in GitRepo::GetBlobWork(uv_work_s*) () from /home/andreas/work/slint/node_modules/fsplusgit/node_modules/gitfakefs/node_modules/nodegit-papandreou/build/Release/nodegit.node
(gdb) up
#9 0x00000000006ddebd in worker (arg=arg@entry=0x0) at ../deps/uv/src/unix/threadpool.c:74
74 w->work(w);
(gdb) up
#10 0x00000000006d3a0f in uv__thread_start (ctx_v=<optimized out>) at ../deps/uv/src/uv-common.c:322
322 entry(arg);
(gdb) up
#11 0x00007ffff6f9af8e in start_thread () from /lib/x86_64-linux-gnu/libpthread.so.0
(gdb) up
#12 0x00007ffff6cc4a0d in clone () from /lib/x86_64-linux-gnu/libc.so.6
(gdb)
Let me know if there's anything else I can do to help narrow it down?
Activity
tbranyen commentedon Jul 3, 2014
Hey @papandreou is slint still interested in using NodeGit? It's possible the segfault issue is resolved with the latest code changes, I'd like to confirm.
papandreou commentedon Jul 7, 2014
@tbranyen Thanks for following up. I just recreated the failing scenario and tried updating to 0.1.4, and it still failed. Then I tried updating to the newest commit on master and linking everything up, but that made all my tests fail due to API changes? Seems like
Repositoryno longer has anopenIndexmethod?papandreou commentedon Jul 15, 2014
@tbranyen I can't figure out if it's a regression that
Repository.prototype.openIndexis missing on master. It's still mentioned in the docs http://www.nodegit.org/nodegit/#Repo-openIndex -- should I create an issue, or has it moved elsewhere?tbranyen commentedon Jul 15, 2014
Create an issue, it isn't added yet. I can try and add it tonight.
papandreou commentedon Jul 15, 2014
Sounds great. Created the issue here: #197
tbranyen commentedon Jul 19, 2014
Closed via #197
papandreou commentedon Jul 19, 2014
I'm afraid this issue persists, even on master. I'm digging further -- it seems to be concurrency related, eg. one of the errors that I see intermittently always happens in the 67th concurrently running operation.
Still no luck creating a simple test case, though. I'll dig further and hopefully create a new issue.
tbranyen commentedon Jul 19, 2014
@papandreou this appears while cloning right?
papandreou commentedon Jul 19, 2014
No, it happens while running node-glob on a virtual fs module backed by nodegit (papandreou/node-gitfakefs) on a git repo with many files and directories. So many concurrent tree object read operations and a few blob read operations (symlinks).
tbranyen commentedon Jul 19, 2014
It'd be amazing if we could isolate this segfault in a unit test.
tbranyen commentedon Oct 24, 2014
@papandreou I'm going to have to close this since we've made some radical changes to the library, and I don't want to keep lingering segfault issues that we cannot test or track.
papandreou commentedon Oct 24, 2014
Fair enough.