Skip to content

Fix problems with mrb_open() if occurs out of memory#4250

Closed
dearblue wants to merge 18 commits into
mruby:masterfrom
dearblue:fix-mrb_open-with-nomem
Closed

Fix problems with mrb_open() if occurs out of memory#4250
dearblue wants to merge 18 commits into
mruby:masterfrom
dearblue:fix-mrb_open-with-nomem

Conversation

@dearblue

Copy link
Copy Markdown
Contributor

(This comment is a copied from #4239)

If out of memory occurs with mrb_open(), the process will be crashed by SIGABRT or SIGSEGV.

This behavior is undesirable for me as I try to generate multiple mrb_state.

This patch set corrects problems that occurred by simulating out of memory.

Thanks.

Potential issue

If mrb_XXX_gem_final() is called while initialization processing is not completed, problems may arise there.

Add internal functions (not `static`):

  * `mrb_raise_nomemory()`
  * `mrb_core_init_abort()`
As a result of this change, no backtrace information is set
for NoMemoryError (`mrb->nomem_err`).

Detailes:

When generating a backtrace, called `mrb_intern_lit()`,
`mrb_str_new_cstr()` and `mrb_obj_iv_set()` function with
`exc_debug_info()` function in `src/error.c`.

If a `NoMemoryError` exception occurs at this time,
the `exc_debug_info()` function will be called again,
and in the same way `NoMemoryError` exception raised will result
in an infinite loop to occurs stack overflow (and SIGSEGV).
The `mrb_str_pool()` function has a path to call `malloc()` twice.

If occurs `NoMemoryError` exception in second `malloc()`,
first `malloc()` pointer is not freed.
 1. `e = mrb_obj_alloc(...)`
 2. `e->stack = mrb->c->stack` (`mrb->c->stack` is anywhere in the range `stbase...stend`)
 3. And raised exception by `mrb_malloc()`!
 4. `mrb_free(e->stack)` by GC part (wrong free)
@matz

matz commented Jun 5, 2020

Copy link
Copy Markdown
Member

merged.

@matz matz closed this Jun 5, 2020
@dearblue dearblue deleted the fix-mrb_open-with-nomem branch June 5, 2020 14:55
dearblue added a commit to dearblue/mruby that referenced this pull request Nov 8, 2020
If no gem was specified, it was warning because `init_mrbgems()` was not used.

ref: mruby#4250
dearblue added a commit to dearblue/mruby that referenced this pull request Jan 10, 2021
Previously I used the `RData` object to avoid a memory leak in `mrb_irep` if `src/load.c` failed.
ref: mruby#4250
commit: f1523d2

Considering that the `RProc` object will be created in the subsequent process, it is preferable to create the `RProc` object from the beginning.
Along with this, the inside of `read_irep()` is replaced with the processing centered on the `RProc` object.

The global function that returns the `mrb_irep` pointer is still provided for compatibility.
dearblue added a commit to dearblue/mruby that referenced this pull request Jan 10, 2021
Previously I used the `RData` object to avoid a memory leak in `mrb_irep` if `src/load.c` failed.
ref: mruby#4250
commit: f1523d2

Considering that the `RProc` object will be created in the subsequent process, it is preferable to create the `RProc` object from the beginning.
Along with this, the inside of `read_irep()` is replaced with the processing centered on the `RProc` object.

The global function that returns the `mrb_irep` pointer is still provided for compatibility.
dearblue added a commit to dearblue/mruby that referenced this pull request Nov 20, 2022
- In case of `NoMemoryError` exceptions, the error message is now printed directly.
- Replaced `mrb_p()` used by mruby#4250 with `mrb_print_error()`.

  ref. squashed commit f1523d2
  ref. subcommit da7d7f8
  ref. subcommit d9c7b6b
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants