Skip to content

Use strict writers to avoid leaking memory#1410

Merged
garyb merged 1 commit into
masterfrom
1297-writer-leak
Aug 23, 2015
Merged

Use strict writers to avoid leaking memory#1410
garyb merged 1 commit into
masterfrom
1297-writer-leak

Conversation

@garyb

@garyb garyb commented Aug 22, 2015

Copy link
Copy Markdown
Member

Part of #1297, there may be a round 2 yet.

An unscientific test of compiling SlamData from scratch and eyeballing the process monitor with these changes applied reduces the peak memory usage from ~12.5gb to ~2.5gb. Also compilation time is reduced from ~68s to ~37s.

💥💥💥

@garyb

garyb commented Aug 23, 2015

Copy link
Copy Markdown
Member Author

Alright, culprit 2 was 6d3a777, so this probably is as good as it gets as it makes that Writer strict too.

We're still using 10x as much as memory as the pre-warnings version though, I'm sure there's room for some other improvements. Maybe some use of $! with tell?

@paf31

paf31 commented Aug 23, 2015

Copy link
Copy Markdown
Contributor

👍 Amazing.

garyb added a commit that referenced this pull request Aug 23, 2015
Use strict writers to avoid leaking memory
@garyb garyb merged commit 825e109 into master Aug 23, 2015
@garyb garyb deleted the 1297-writer-leak branch August 23, 2015 10:18
@jdegoes

jdegoes commented Aug 23, 2015

Copy link
Copy Markdown

👍 👍 🎆 💥 🎆 👍 👍

@jdegoes

jdegoes commented Aug 23, 2015

Copy link
Copy Markdown

10x down, 10x more to go. 😉

@garyb

garyb commented Aug 23, 2015

Copy link
Copy Markdown
Member Author

I hope so! 😉 I think SlamData's Travis should start working again at least, even with just this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants