Skip to content
This repository has been archived by the owner on Oct 2, 2020. It is now read-only.

Smart Duplication Detection #8

Open
sr229 opened this issue Dec 22, 2019 · 4 comments
Open

Smart Duplication Detection #8

sr229 opened this issue Dec 22, 2019 · 4 comments
Labels
enhancement New feature or request
Projects
FVP
Milestone
FVP

Comments

@sr229
Copy link
Contributor

sr229 commented Dec 22, 2019

We can use a AI Service for this.

We train a model everytime a new image is uploaded, but its only scheduled per 2 hours to train from the entire collection or just what got uploaded new.

  • Post exists:
    • block/warn user.
@sr229 sr229 added the enhancement New feature or request label Dec 22, 2019
@sr229 sr229 added this to the FVP milestone Dec 22, 2019
@sr229 sr229 mentioned this issue Dec 22, 2019
Open
@sr229 sr229 added this to To do in FVP via automation Dec 22, 2019
@DonaldTsang
Copy link

Also what kind of hashing will you use? There are many like pHash, aHash, dHash and wHash (with varying resolutions)

@sr229
Copy link
Contributor Author

sr229 commented Apr 25, 2020

I don't think Hashing can solve duplication since anyone can just waifu2x or change the format and it'll result in a different hashing.

@DonaldTsang
Copy link

No, pHash and the like hashes the LOOKS of an image, not the data.
https://en.wikipedia.org/wiki/Perceptual_hashing

@sr229
Copy link
Contributor Author

sr229 commented Apr 25, 2020

I'll see what I can do to adapt the hashing standard, but this is on hold for now since its in the FVP log.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
enhancement New feature or request
Projects
FVP
  
To do
Milestone
FVP
Development

No branches or pull requests
2 participants
@DonaldTsang @sr229