Skip to content

Add ai methods that sort threads based on content similarity w/ given…#1391

Merged
OpenSourceSimon merged 3 commits intoelebumm:masterfrom
dmoralis:add_ai_similarity
Dec 19, 2022
Merged

Add ai methods that sort threads based on content similarity w/ given…#1391
OpenSourceSimon merged 3 commits intoelebumm:masterfrom
dmoralis:add_ai_similarity

Conversation

@dmoralis
Copy link
Copy Markdown
Contributor

Description

Hi, I added a new feature to your project which refers to sorting threads based on their similarity to the keywords given by the user. The way this is achieved is through a HuggingFace library model, which creates an embedding from the title and content of each thread and compares it to the embedding of each keyword. Thus, for each thread the overall similarity to the keywords is summed up, through which the objects of type "Submission" are sorted and then given to the function "get_subreddit_undone" for further checks (allow_nsfw, already done etc). Finally in case that all the threads are used up then for each different date filter "VALID_TIME_FILTERS" applied the threads are re-sorted based on their similarity to the keywords. I am open to your opinion about this feature and sorry about any possible mistakes, this is my first contribution to a project.

Issue Fixes

None

Checklist:

  • I am pushing changes to the develop branch
  • I am using the recommended development environment
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have formatted and linted my code using python-black and pylint
  • I have cleaned up unnecessary files
  • My changes generate no new warnings
  • My changes follow the existing code-style
  • My changes are relevant to the project

How to test -> Run the main.py file

Copy link
Copy Markdown
Collaborator

@OpenSourceSimon OpenSourceSimon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will need to test this but thanks! Good functionality!

Comment thread video_creation/data/videos.json Outdated
@OpenSourceSimon OpenSourceSimon added the enhancement New feature or request label Dec 17, 2022
@OpenSourceSimon OpenSourceSimon added this to the 2.5 milestone Dec 17, 2022
Copy link
Copy Markdown
Collaborator

@OpenSourceSimon OpenSourceSimon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested and works! Thank you for your PR

@dmoralis
Copy link
Copy Markdown
Contributor Author

No problem :) Keep doing a great job

@OpenSourceSimon OpenSourceSimon merged commit 3380d69 into elebumm:master Dec 19, 2022
OpenSourceSimon added a commit that referenced this pull request Dec 19, 2022
…with given keywords by dmoralis (#1391)"

This reverts commit 3380d69.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants