Skip to content

resolve issue 28 (parallelize by sample) - #33

Merged
josenavas merged 9 commits into
biocore:masterfrom
ekopylova:issue_28
Jul 30, 2016
Merged

josenavas merged 9 commits into
biocore:masterfrom
ekopylova:issue_28

Conversation

@ekopylova

@ekopylova ekopylova commented May 28, 2016 •

Copy link
Copy Markdown
Contributor

Parallelize deblur by sample.

Click needs to be development version since the option to hide click options (e.g. workflow_parallel via @deblur_cmds.command(hidden=True)) will only be available in version 7.0 [see merged PR].

Example usage (works without issues):

parallel_deblur workflow --seqs-fp seqs.fna --output-dir seqs_deblur --ref-fp 70_otus.fasta --ref-db-fp 70_otus -O 4

Unit test (not working currently, even with 1 thread):

python test_parallel_deblur.py

The issue arises during trimming reads to specified length. The output file is empty which causes errors downstream (e.g. Could not run SortMeRNA).

Comment thread deblur/parallel_deblur.py
----------
params: dict
parameter settings to pass to deblur
fps: tuple

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just curious, why a tuple instead of passing them in individually?

@ekopylova ekopylova Jun 28, 2016 •

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you wrote that code see this PR and it was because the input filename and output directory were being stored as a list of tuples at line 151 prior to mapping to functor. Do you want me to change this?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No strong feelings

@wasade

wasade commented Jun 27, 2016

Copy link
Copy Markdown
Member

few comments.

Travis is reporting a single failure with a bfillings import, and also a few flake8 issues

@amnona

amnona commented Jun 27, 2016

Copy link
Copy Markdown
Contributor

We removed all the bfillings function calls so can remove the bfillings
dependency?
On Jun 27, 2016 8:25 AM, "Daniel McDonald" notifications@github.com wrote:

few comments.

Travis is reporting a single failure with a bfillings import, and also a
few flake8 issues

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
#33 (comment), or mute
the thread
https://github.com/notifications/unsubscribe/AFkA8ktCQuCH1Va-KqAm0u6DADXD08Efks5qP-twgaJpZM4IpFi8
.

@josenavas
josenavas merged commit 5e68ecc into biocore:master Jul 30, 2016
@ekopylova ekopylova mentioned this pull request Aug 11, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants