Site problems? ddos?

Something not working on my site? post it here!

Moderator: crazyankan

User avatar
crazyankan
Crazy Editor!
Posts: 1040
Joined: Sun Nov 27, 2005 7:30 pm
Location: Tiphares/Sweden

Site problems? ddos?

Post by crazyankan » Fri Feb 11, 2011 12:55 am

Any news about this ^Ripper?
Image

User avatar
^Ripper
Site Admin
Posts: 939
Joined: Thu Nov 17, 2005 2:00 pm
Location: The Netherlands
Contact:

Re: Site problems? ddos?

Post by ^Ripper » Fri Feb 11, 2011 6:24 pm

The simple version:

In 2004 (more or less) i had a bittorrent tracker running on my site.
.com/Tracker/announce.php was the url.
Then i got kicked and i set up my own tracker.

Google this and it all becomes clear: rippersanime.com AND /Tracker/announce.php
Some idiots who run trackers or indexing sites put every known tracker in the .torrent files they serve.
Doesn't matter what torrent, my old tracker url is in there.

So basically i get insane amounts of traffic from users downloading these torrents and trying to connect to my non existing tracker.

Now if this old tracker was on a subdomain like my private one was, then removing the dns record would do the trick.
However the old one was on my primary domain as a php script.

So lots of traffic on a shared hosting equals screaming hosting people wanting to kick you off.

404 Document Not Found | 1063694 hits |99.9 %|349.59 MB
The 1 million hits in <24 hours

I am moving the site to my vps hosting and hopefully i can get this friggin iptables to work filtering the requests.

Oh the website will go down later today for a complete reinstall of the vps :/

User avatar
moooV
Tipharean
Posts: 899
Joined: Tue Nov 21, 2006 2:41 pm
Location: Tokyo

Re: Site problems? ddos?

Post by moooV » Fri Feb 11, 2011 6:38 pm

Now it makes sense.
ac8dad43d497508fe83d143ee096c252

User avatar
moooV
Tipharean
Posts: 899
Joined: Tue Nov 21, 2006 2:41 pm
Location: Tokyo

Re: Site problems? ddos?

Post by moooV » Sat Feb 12, 2011 4:40 pm

The forum gets unable to connect to once in several minutes.
ac8dad43d497508fe83d143ee096c252

User avatar
^Ripper
Site Admin
Posts: 939
Joined: Thu Nov 17, 2005 2:00 pm
Location: The Netherlands
Contact:

Re: Site problems? ddos?

Post by ^Ripper » Sat Feb 12, 2011 5:02 pm

That would be me rebooting the vps a lot i guess.
Since all the stock VPS images lack additional modules for iptables i had to come up with something to block all that traffic.
I don't have the knowledge to recompile the kernel with string support for iptables.

[edit]Don't TRY the bad url...you WILL get IP banned![/edit]

RtrentC
Posts: 1
Joined: Sat Dec 04, 2010 8:45 am

Re: Site problems? ddos?

Post by RtrentC » Sun Feb 13, 2011 8:48 pm

As long as we use http://www.rippersanime.com/ that won't be a problem. Only the tracker url that was causing you problems will get a ban. Correct.

User avatar
^Ripper
Site Admin
Posts: 939
Joined: Thu Nov 17, 2005 2:00 pm
Location: The Netherlands
Contact:

Re: Site problems? ddos?

Post by ^Ripper » Tue Feb 15, 2011 12:41 am

You're not getting banned anymore.

Well iptables was a bust.
fail2ban to drop all traffic when someone touches a file.

Result: low traffic, very high system resource usage to the point a forced 12/24 hour reboot is mandatory.

But the solution is simple: Give them a PROPER fake response!:

d8:intervali604800e12:min intervali604800e8:retry ini604800e8:completei0e10:incompletei0e5:peers0:10:tracker id5:91919e

This along with a lighttpd rewrite rule that serves Tracker/announce.php a static txt file outside of PHP.
url.rewrite-if-not-file = ("^/Tracker/announce.php(.*)$" => "/Tracker/announce.txt")

Basically put, after they read the url, they have to wait 7 days before attempting to access the url again :P
I could have gone higher, but this value is comfortable for both utorrent and Vuze.
Other trackers ignore, but still use a 60 minute timeout (fucking bitcomet among ohers)

Its fun to watch the logs for system resource usage.
Both bandwidth and especially system usage plummeted. (0.00 / 0.10 / 0.23 instead of something like 0.60 / 0.80 / 1.10)

...learning linux the hard way :P

User avatar
gilsand
GIB
Posts: 211
Joined: Fri May 14, 2010 2:16 pm
Location: Southern California

Re: Site problems? ddos?

Post by gilsand » Tue Feb 15, 2011 1:20 am

^Ripper wrote: ...learning linux the hard way :P
Wow, talk about bad memories...
Back when dinosaurs walked the earth, I used to work here.
http://postgroup.com/ndex.php?option=co ... &Itemid=85
and at the time they were the only Hollywood studio to have their own
Sun Solarius Super Compuer. I trained as a DBM, and still remember
Mnemonics that I used.
IPCONFIG DASH ARP :geek:
I still have my Red Hat Server gathering dust.
I must say, well done!
For to win one hundred victories in one hundred battles is not the acme of skill.
To subdue the enemy without fighting is the acme of skill.
Sun Tzu

User avatar
moooV
Tipharean
Posts: 899
Joined: Tue Nov 21, 2006 2:41 pm
Location: Tokyo

Re: Site problems? ddos?

Post by moooV » Wed Mar 09, 2011 7:35 pm

What happened this time? The site was down for three days!
ac8dad43d497508fe83d143ee096c252

User avatar
^Ripper
Site Admin
Posts: 939
Joined: Thu Nov 17, 2005 2:00 pm
Location: The Netherlands
Contact:

Re: Site problems? ddos?

Post by ^Ripper » Thu Mar 10, 2011 3:53 am

The fun never ends...

Had to do a fresh install because for some reason sendmail/qmail was being a bitch, restarting its service every 5 minutes.
Google for a solution...useless.
Linux is fun...when it works.

And then the raid array blew a disk, so the hoster had to shutdown the entire node to have it rebuild the array.

And then 3 days passed...

User avatar
moooV
Tipharean
Posts: 899
Joined: Tue Nov 21, 2006 2:41 pm
Location: Tokyo

Re: Site problems? ddos?

Post by moooV » Thu Mar 10, 2011 4:58 am

3 days of downtime should make you a huge discount if they've signed an SLA agreement with you.
ac8dad43d497508fe83d143ee096c252

User avatar
moooV
Tipharean
Posts: 899
Joined: Tue Nov 21, 2006 2:41 pm
Location: Tokyo

Re: Site problems? ddos?

Post by moooV » Wed Mar 23, 2011 7:13 pm

What's happening?

The site has been down for several hours and now I can't login from my main browser (Firefox). I've managed to log in by some miracle using google chrome, but in FF I constantly get this:
You exceeded the maximum allowed number of login attempts. In addition to your username and password you now also have to solve the CAPTCHA below.
Exceeded the maximum number of login attempts. It was since the FIRST time I tried to log in. Of course, solving the captcha and trying to log in again doesn't work at all.

I've deleted all cookies associated with rippersanime in FF, but still no progress.

WTF is going on?

UPD.
I've reset my password and changed email - that worked. Unexplainable.
ac8dad43d497508fe83d143ee096c252

User avatar
^Ripper
Site Admin
Posts: 939
Joined: Thu Nov 17, 2005 2:00 pm
Location: The Netherlands
Contact:

Re: Site problems? ddos?

Post by ^Ripper » Thu Mar 24, 2011 4:54 am

Not really sure why the forum went all crazy.
Maybe the sessions table of phpbb got corrupted.
I checked but all tables are fine.
I did do a clear cache/sessions to to make sure.
Site-Traffic.png
Site-Load.png
Site-Memory.png
There is a gap where the vps went down.
Possible the entire node, or just my vps crashed for some reason.
Since i'm on a unmanaged vps now, i lack the comfort of having other people monitor the hosting.
Maybe i should write something on my home server to ping the hosting and reboot when its down.

Still, backup's are made every 6 hours to my home server.
So if anything fucks up, i can restore it.
I have an undo level of 30 days :P
All hail Goodsync

User avatar
fargred
Posts: 21
Joined: Tue Jul 27, 2010 2:57 am

Re: Site problems? ddos?

Post by fargred » Fri Apr 15, 2011 6:53 am

I've posted on GUNNM imageboard a message, and image appears at home page as a new on that board. But when i follow the link of this thumb on home page, it leads on a board as it was before i post. There's no my post. However, if you go to page 1 then back to page 0, it would be as it must be.
Deckman 100 – OH! You still have solenoid quench gun!!
6 – Shut up and give me my clothes or i'll f##k you right here!

Deckman 100 – Buf, mafta, i'fe feen bwinwin cfothes fo you!…*splusht-splusht*
11 & 12 - Oh, no. He spoiled another one =_=

User avatar
^Ripper
Site Admin
Posts: 939
Joined: Thu Nov 17, 2005 2:00 pm
Location: The Netherlands
Contact:

Re: Site problems? ddos?

Post by ^Ripper » Sat Apr 16, 2011 1:16 am

Hmmm well first of all the latest post No.635 was posted outside of the thread No.632.
Matter of not hitting the reply button and then post a new image.
This is a common mistake :)

However the other problem you describe sounds like a caching problem.
There is a possibility that the pages are displayed cached and thus you see an earlier version of the page when you post something.
A forced reload of the page (F5) fixes this.
Possibly i have caching enabled on those folders, I'll check if it somehow got enabled due to server switching.

Also a general note about Firefox 4.0/Nightly builds and the font looking like rubbish on my pages.
The blinking of the text from greyscale rendering to subpixel rendering after 2 seconds of scrolling.

I filed a bug report and its a genuine bug...
Lets hop they fix it soon!

User avatar
fargred
Posts: 21
Joined: Tue Jul 27, 2010 2:57 am

Re: Site problems? ddos?

Post by fargred » Sat Apr 16, 2011 5:13 am

^Ripper wrote:Matter of not hitting the reply button and then post a new image.
But i did. Now it looks normal, thanks!
Deckman 100 – OH! You still have solenoid quench gun!!
6 – Shut up and give me my clothes or i'll f##k you right here!

Deckman 100 – Buf, mafta, i'fe feen bwinwin cfothes fo you!…*splusht-splusht*
11 & 12 - Oh, no. He spoiled another one =_=

User avatar
^Ripper
Site Admin
Posts: 939
Joined: Thu Nov 17, 2005 2:00 pm
Location: The Netherlands
Contact:

Re: Site problems? ddos?

Post by ^Ripper » Sat Apr 16, 2011 7:51 pm

There WAS a caching problem.
Put a set of extra rules in the config and its working now.
Subtle difference in /MusicBoard/ and /MusicBoard/index.php while they are serving the same file.

Apache is much easier with rewrite rules.

partaload
Posts: 3
Joined: Thu Jun 09, 2011 3:33 pm

Re: Site problems? ddos?

Post by partaload » Thu Jun 09, 2011 3:59 pm

Hello!
I arrived to this forum googling a way to get rid of a lot of "/tracker/announce.php" traffic I'm receiving (without having a tracker).

I would recommend you to give a try to a free service which gives you CDN and anti-webspam services for free. It can saves you loads of traffic. Also is the best way I know to stop "botnets", "webspammers" and the like. It is http://www.cloudflare.com and it's really easy to implement.

Regards.

User avatar
^Ripper
Site Admin
Posts: 939
Joined: Thu Nov 17, 2005 2:00 pm
Location: The Netherlands
Contact:

Re: Site problems? ddos?

Post by ^Ripper » Thu Jun 09, 2011 8:10 pm

Just read their faq and indeed it looks interesting in dealing with spam/hacks/idiot trackers/etc
I will register an account there an look around.
Not sure if it will be compatible with my site since i disabled most of the caching while they cache a lot.

Going to look into it further.
Thanks for the link.

Also the fix above for tracker traffic works like a charm.
You may need to have an Apache .htaccess ruleset instead of the lighttpd one.
But basically putting that text string into a /announce.php file does the trick.
Delivering the file without having it being parsed by PHP is icing on the cake :)

You can also make a /scrape.php file.
Didn't mention it before, but here it is.

Code: Select all

d5:filesd20:12345678901234567890d8:completei0e10:downloadedi0e10:incompletei0ee5:flagsd20:min_request_intervali604800eee
Technically you need to make a parsable php file that puts the 20 byte request hash into the response (replace 12345678901234567890).
I have code here, buts its not worthy to post atm.
Not posting potentially insecure code snippets :P

partaload
Posts: 3
Joined: Thu Jun 09, 2011 3:33 pm

Re: Site problems? ddos?

Post by partaload » Thu Jun 09, 2011 9:16 pm

Cloudshare works like a CDN (Content Delivery Network).
This kind of service uses to be a pay service (like CloudFront, from Amazon, and some other).
I met them looking for a solution to stop chinese webspam in a Dolphin portal I'm running, I decided to test it and it works like a charm. Not only I'm just removing one or zero spam users a day (when before I had to remove over 10-15 chinese fake spam users/day), but also it has improved speed of the whole website by a factor of 45%. The have several datacenters distributed all over the world, so they serve the static contents of your site from the datacenter geographically closer to the visitor.

The also give you analytics and control over IPs to be banned, etc. They use the Honey Pot project (among other techinques to stop spamers, robots, etc.)

Now, about the tracker problem I have in this concrete website (I'm not using cloudshare with it because another reasons), I just see in my Apache logs requests to "/tracker/announce.php?<params>". I've not seen any "/tracker/scrape.php" request. Anyway, I just created another fake scrape.php following your instructions. The traffic still remains... don't know how long it will last.

Regards!

User avatar
^Ripper
Site Admin
Posts: 939
Joined: Thu Nov 17, 2005 2:00 pm
Location: The Netherlands
Contact:

Re: Site problems? ddos?

Post by ^Ripper » Thu Jun 09, 2011 10:24 pm

Ok on apache you need something like this:

Create a text file called "htaccess" (without the quotes) and insert this code:

Code: Select all

RewriteEngine On
Options +FollowSymlinks +SymlinksIfOwnerMatch
RewriteCond %{REQUEST_URI} ^/Tracker.*$ [NC]
RewriteRule ^(.*)$ /Fake_Tracker_Response.txt [L]

<Files "Fake_Tracker_Response.txt">
ForceType text/plain
</Files>
 
Change "/Tracker" part if needed to whatever torrent url you wish to block. [NC] means not case sensitive match.

Create a text file called "Fake_Tracker_Response.txt" and insert this code:

Code: Select all

d8:intervali604800e12:min intervali604800e8:retry ini604800e8:completei0e10:incompletei0e5:peers0:10:tracker id5:91919e
One line, no returns!

Upload both the "htaccess" and "Fake_Tracker_Response.txt" to your root folder.
Rename the "htaccess" file into ".htaccess".
Access your: domain.com/Tracker/announce.php
Access your: domain.com/Tracker/12345678
If you don't get a blank error 500 page, both instances should give you the contents of the "Fake_Tracker_Response.txt" file.
Otherwise remove the .htaccess if you are unable to view your website properly.
You might need to set your ftp client to display hidden files!

Now i haven't tested the above code, since i don't run Apache anymore.
But it should work.
This is the easiest way and resource friendly, but doesn't cover SCRAPE requests.

Scrape traffic is low but existing.
If you want to add a separate filter for scrape [ CHANGED CODE UNDERNEATH on 11-6-2011 ]
Create a text file called "htaccess" (without the quotes) and insert this code:

Code: Select all

RewriteEngine On
Options +FollowSymlinks +SymlinksIfOwnerMatch
RewriteCond %{REQUEST_URI} ^/Tracker/announce.*$ [NC]
RewriteRule ^(.*)$ /Fake_Tracker_Response_Announce.txt [L]
RewriteCond %{REQUEST_URI} ^/Tracker/scrape.*$ [NC]
RewriteRule ^(.*)$ /Fake_Tracker_Response_Scrape.php [L]

<Files "Fake_Tracker_Response_Announce.txt">
ForceType text/plain
</Files>
 
Create a text file called "Fake_Tracker_Response_Announce.txt" and insert this code:

Code: Select all

d8:intervali604800e12:min intervali604800e8:retry ini604800e8:completei0e10:incompletei0e5:peers0:10:tracker id5:91919e
One line, no returns!

Create a text file called "Fake_Tracker_Response_Scrape.php" and insert this code:

Code: Select all

<?php
$hash = isset($_GET["info_hash"]) ? $_GET["info_hash"] : "";
$hash = rawurldecode($hash);
if (strlen($hash) == 20) {
    echo ("d5:filesd20:".$hash."d8:completei0e10:downloadedi0e10:incompletei0ee5:flagsd20:min_request_intervali604800eee");
}
else {
    echo ("d5:filesd20:12345678901234567890d8:completei0e10:downloadedi0e10:incompletei0ee5:flagsd20:min_request_intervali604800eee");
}
?>
Upload all 3 files to your root folder.
Rename the "htaccess" file into ".htaccess".

This will provide a PROPER fake response for scrape requests to also obey the 7 day timeout.
Since there is a small php script that needs to get parsed, server load will increase slightly to moderate depending on the amount of traffic.

USE THIS ON YOUR OWN RISK!

partaload
Posts: 3
Joined: Thu Jun 09, 2011 3:33 pm

Re: Site problems? ddos?

Post by partaload » Fri Jun 10, 2011 10:47 am

It worked, it was a matter of time that traffic started to decrease.

Thankyou very much for your detailed explanations!

Regards.

User avatar
^Ripper
Site Admin
Posts: 939
Joined: Thu Nov 17, 2005 2:00 pm
Location: The Netherlands
Contact:

Re: Site problems? ddos?

Post by ^Ripper » Fri Jun 10, 2011 4:43 pm

Good to hear it worked.

If you want to check it for yourself:
Install a bittorrent client.
Download a random torrent.
Add your "tracker url" to the tracker list (easy on utorrent)

domain.com/Tracker/announce.php

See how after you connected, you get a 7 day timeout :)
Initial connections are no problem, but reconnection every 60 seconds x 1000+ users is.

User avatar
moooV
Tipharean
Posts: 899
Joined: Tue Nov 21, 2006 2:41 pm
Location: Tokyo

Re: Site problems? ddos?

Post by moooV » Thu Jan 05, 2012 5:29 pm

What has happened this time?
ac8dad43d497508fe83d143ee096c252

User avatar
^Ripper
Site Admin
Posts: 939
Joined: Thu Nov 17, 2005 2:00 pm
Location: The Netherlands
Contact:

Re: Site problems? ddos?

Post by ^Ripper » Thu Jan 05, 2012 6:01 pm

Datacenter (instead of host) kicked my vps for violating their TOS...
Didn't get any specifics, didn't bother to ask...
I got relocated...

If this keeps on happening i'll put the forum on a subdomain and on a different *safe* hosting...

The torrent crap from above might be the reason tho.
If that keeps on happening i'll move to my backup domain.
rippersanime.info
The only way for sure to get rid of that traffic.

Post Reply