[woller]

I'm noticing a lot of posts with duplicated content within itself recently.
Eg. recent ursecretgirlfriend post has many videos 3+ times duplicated in its vid folder. This increases the download size by at least factor 3.

I'd like to recommend to have a "duplicated" content voting button, so others can warn other users, when posts contain duplicated content, maybe even have some penalty for the user uploading it (during a contest).  I have nothing against multiple posts sharing the same content, but duplicated files within are a no-go.

I usually skip these posts, as removing the duplicate videos/pics is a lot of work. I'd like to now before, if posts contain duplicated content.
Maybe it's also possible to have the website check posts for duplicated content, like comparing file sizes. You recently introduced a download size display, this would be also cool to have a detector for duplicate files.

Thanks,. still loving the forum!

[Lucifer NightStar]

I have an idea how to tackle this.

Give me some time and I'll implement it.

[woller]

thanks, looking forward to your implementation.

usually I find duplicates by file size, but also video duration can help.

unfortunately there are also duplicates with different file size and slightly different duration, probably can only use some AI to detect these.

[wackytobbacy]

99% of the time I do not post with duplicates as i know how annoying it would be if i upload them, I have only done one post with duplicates because there were no alternatives but i warned about it in the subject. I thought it was my ursecretgirlfriend post that you mentioned but it appears it was not mine. Mistakes do happen especially with the larger mega but scanning the thumbnails is usually enough to pick it up.

[woller]

It was one of yours, but not the only one. A few months ago this wasn't that big of a problem with duplicates, but since the contest went live there were so many posts. Also BigTitsAreNice and other posted stuff with duplicated content.

I'd hope that at least the people could do a check before uploading to remove duplicates, as said before, often these files have exactly the same size or same video duration.
I have seen a few posts recently were the downloads were 30+ gig, where I already had all of the vids in a 10gig download before. It's very cumbersome to check each download that I'm interested in first in the vids by thumbnail, which could be done before uploading them.
I'd give extra likes/points if I knew the uploader cleaned the uploads before, but anyway. Just wanted to note that its becoming a problem/more often the case that we have uploads with 2-3x the size which contain a lot of dupes.

[wackytobbacy]

Let me know which posts these are and i will have them removed and replaced with ones without duplicates.

[BigTitsAreNice]

I promise, no bad intention to get more likes or points. Mega has problem, when I rename files, sometimes it keeps copies, sometimes it just adds new ones automagically

Also, I don’t usually make these myself. I find, clean up, then use a bot to make two links from my Mega, and then share.

I will try to do better next time. Thank you kha for understanding.

[woller]

Thanks guys. I know it's not intentionally, that you post duplicates. But still I'd appreciate if these could be checked.
I will collect some links, it's hard backwards as there are a lot of post in recent time, but I will post here when I find new ones.

Here are examples from BigTitsAreNice

RE: Recent problem with duplicated content.

Registered Members Only

You need to be a registered member to see more on RE: Recent problem with duplicated content.
Login or Sign up to get access to a huge variety of top quality leaks.

RE: Recent problem with duplicated content.

Registered Members Only

You need to be a registered member to see more on RE: Recent problem with duplicated content.
Login or Sign up to get access to a huge variety of top quality leaks.

Here from wackytobbacy

RE: Recent problem with duplicated content.

Registered Members Only

You need to be a registered member to see more on RE: Recent problem with duplicated content.
Login or Sign up to get access to a huge variety of top quality leaks.

RE: Recent problem with duplicated content.

Registered Members Only

You need to be a registered member to see more on RE: Recent problem with duplicated content.
Login or Sign up to get access to a huge variety of top quality leaks.

[woller]

wackytobbacy: Here's a big one with also a lot of duplicates. Did not even download the 200+ gig, as it will take very long because of my limited download and then I will need to sort them out.

RE: Recent problem with duplicated content.

Registered Members Only

You need to be a registered member to see more on RE: Recent problem with duplicated content.
Login or Sign up to get access to a huge variety of top quality leaks.

[Veronica88]

Hey all
I love all your uploads, but I definitely also see the problem with duplicates. I do know about the mega problem too, so would definitely be great to have some sort of ai or other to check it out.

[John Boy]

Ya I’ve noticed a lot of duplicate content as well. Some of them lead to broken links so I think it would not be a bad idea to employ some sort of ai api to track any duplicates.

[PrestonW]

I was disappointed when I used credits to view a link that was supposedly more leaks than another due to its larger file size just for it to be duplicated content- finding a way to alert people would be great

[woller]

Here's more recently from wackytobbacy with a lot of duplicates inside.

RE: Recent problem with duplicated content.

Registered Members Only

You need to be a registered member to see more on RE: Recent problem with duplicated content.
Login or Sign up to get access to a huge variety of top quality leaks.

RE: Recent problem with duplicated content.

Registered Members Only

You need to be a registered member to see more on RE: Recent problem with duplicated content.
Login or Sign up to get access to a huge variety of top quality leaks.

I also found a lot of posts by Le-Boom and ProfessorMega with duplicated stuff.
It seems all uploaders have these now

Le-Boom:

RE: Recent problem with duplicated content.

Registered Members Only

You need to be a registered member to see more on RE: Recent problem with duplicated content.
Login or Sign up to get access to a huge variety of top quality leaks.

RE: Recent problem with duplicated content.

Registered Members Only

You need to be a registered member to see more on RE: Recent problem with duplicated content.
Login or Sign up to get access to a huge variety of top quality leaks.

[Le-Boom]

One of those links is the creator making images that look exactly the same. I gain absolutely 0 by duplicating things to make the folder bigger. Nothing. Mega's asinine file version thing is bugged and will make duplicates when we upload stuff.

[woller]

As said, I don't think it'smainly the uploader's fault, that there is duplicated content.
Often these uploads are from other sources, where they either don't care or extra duplicate content for more size.
But still I'd love if these duplicates could/would be cleaned before.

Here's another example from BigTitsAreNice

RE: Recent problem with duplicated content.

Registered Members Only

You need to be a registered member to see more on RE: Recent problem with duplicated content.
Login or Sign up to get access to a huge variety of top quality leaks.

You can see in the video folder, that there are a lot of exact copies of the same file. (Sort by duration or file size working here too)

[xOrion]

I simply run a script checking for duplicate hash files before uploading. Filters most if not all.

[woller]

Here's another one with a lot of duplicated stuff by Le-Boom, the folder is ~140g, I'd say almost any video is there twice (in different file sizes though, but can be easily checked by file duration/thumbnails)

RE: Recent problem with duplicated content.

Registered Members Only

You need to be a registered member to see more on RE: Recent problem with duplicated content.
Login or Sign up to get access to a huge variety of top quality leaks.

This one is a bit harder to spot since it has slightly different durations of the videos, though some videos are 3x there which inflates it to 86g+ (should be around 30-40 without duplicates)

RE: Recent problem with duplicated content.

Registered Members Only

You need to be a registered member to see more on RE: Recent problem with duplicated content.
Login or Sign up to get access to a huge variety of top quality leaks.