The Glade 4.0 • View topic - Glade 4.0, two years in

View unanswered posts | View active topics

Board index » The Glade » General

All times are UTC - 6 hours [ DST ]

Glade 4.0, two years in

Page 2 of 2

[ 42 posts ]

Go to page Previous 1, 2

Print view

Previous topic | Next topic

Author

Message

Taskiss

Post subject:

Posted: Tue Aug 30, 2011 1:32 pm

Joined: Fri Feb 05, 2010 11:59 am
Posts: 3879
Location: 63368

Somebody doing a net backup?

What's the size of the glade these days?

_________________
In time, this too shall pass.

Top

Mookhow

Post subject: Re: Glade 4.0, two years in

Posted: Tue Aug 30, 2011 1:38 pm

God of the IRC

Joined: Wed Sep 02, 2009 7:35 pm
Posts: 3041
Location: The United States of DESU

The Glade database backup is 51 megabytes.

I dunno, I'm not particularly worried about it. 24 gigabytes out of 650 isn't really a problem, just an unusual occurence. If it continues, I'll have Stathol see if he can figure out where the network traffic is coming from.

_________________

Top

Stathol

Post subject: Re: Glade 4.0, two years in

Posted: Tue Aug 30, 2011 2:07 pm

Lean, Mean, Googling Machine

Joined: Thu Sep 03, 2009 9:35 am
Posts: 2903
Location: Maze of twisty little passages, all alike

Hmmm....

Code:

root@gladerebooted:~# vnstat --days

 venet0  /  daily

         day         rx      |     tx      |    total    |   avg. rate
     ------------------------+-------------+-------------+---------------
      08/01/11     35.40 MiB |  189.63 MiB |  225.03 MiB |   21.34 kbit/s
      08/02/11     78.91 MiB |  220.10 MiB |  299.01 MiB |   28.35 kbit/s
      08/03/11    114.79 MiB |  278.86 MiB |  393.64 MiB |   37.32 kbit/s
      08/04/11     35.38 MiB |  183.67 MiB |  219.04 MiB |   20.77 kbit/s
      08/05/11     55.48 MiB |    1.36 GiB |    1.42 GiB |  137.39 kbit/s
      08/06/11     70.40 MiB |    3.00 GiB |    3.07 GiB |  298.18 kbit/s
      08/07/11     62.40 MiB |    2.55 GiB |    2.61 GiB |  253.39 kbit/s
      08/08/11     31.51 MiB |  179.50 MiB |  211.01 MiB |   20.01 kbit/s
      08/09/11     36.88 MiB |  239.11 MiB |  275.99 MiB |   26.17 kbit/s
      08/10/11     39.67 MiB |  206.70 MiB |  246.38 MiB |   23.36 kbit/s
      08/11/11     37.69 MiB |  185.96 MiB |  223.65 MiB |   21.21 kbit/s
      08/12/11     74.46 MiB |    2.93 GiB |    3.01 GiB |  292.00 kbit/s
      08/13/11    156.53 MiB |    8.12 GiB |    8.27 GiB |  803.18 kbit/s
      08/14/11     22.48 MiB |  125.44 MiB |  147.92 MiB |   14.02 kbit/s
      08/15/11     32.25 MiB |  168.63 MiB |  200.88 MiB |   19.05 kbit/s
      08/16/11     39.84 MiB |  180.65 MiB |  220.49 MiB |   20.91 kbit/s
      08/17/11     34.22 MiB |  192.08 MiB |  226.30 MiB |   21.46 kbit/s
      08/18/11     29.54 MiB |  168.60 MiB |  198.14 MiB |   18.79 kbit/s
      08/19/11     34.63 MiB |  189.23 MiB |  223.87 MiB |   21.23 kbit/s
      08/20/11     34.77 MiB |  952.49 MiB |  987.26 MiB |   93.61 kbit/s
      08/21/11     24.12 MiB |  394.88 MiB |  419.00 MiB |   39.73 kbit/s
      08/22/11     40.33 MiB |  211.73 MiB |  252.06 MiB |   23.90 kbit/s
      08/23/11     30.42 MiB |  153.99 MiB |  184.42 MiB |   17.49 kbit/s
      08/24/11     32.98 MiB |  162.32 MiB |  195.31 MiB |   18.52 kbit/s
      08/25/11     44.34 MiB |  289.48 MiB |  333.82 MiB |   31.65 kbit/s
      08/26/11     23.29 MiB |  128.84 MiB |  152.12 MiB |   14.42 kbit/s
      08/27/11     17.48 MiB |   87.34 MiB |  104.82 MiB |    9.94 kbit/s
      08/28/11     13.98 MiB |   81.54 MiB |   95.52 MiB |    9.06 kbit/s
      08/29/11     26.19 MiB |  135.25 MiB |  161.44 MiB |   15.31 kbit/s
      08/30/11     14.62 MiB |   79.83 MiB |   94.45 MiB |   16.56 kbit/s
     ------------------------+-------------+-------------+---------------
     estimated        25 MiB |     146 MiB |     171 MiB |

Unfortunately, vnstat, atsar, etc. are a monthly log rotation, so I can't compare this to the previous month to figure out what's going on. You can see that the bulk of the traffic is on the weekend, but it's not immediately clear to me exactly what's going on here. Even for a Saturday, 8 GiB tx on a single day seems ... out of proportion to our user base. Even if 50 people browsed the Glade that day, they'd have to average fetching 200MiB of data apiece.

I'll take a closer look at the Apache logs and see if I can make any sense of it. In any case, I think I'll leave

_________________
Sail forth! steer for the deep waters only!
Reckless, O soul, exploring, I with thee, and thou with me;
For we are bound where mariner has not yet dared to go,
And we will risk the ship, ourselves and all.

Top

Midgen

Post subject: Re: Glade 4.0, two years in

Posted: Tue Aug 30, 2011 2:10 pm

Joined: Thu Sep 03, 2009 3:08 am
Posts: 6465
Location: The Lab

Stathol wrote:

In any case, I think I'll leave

lol wut?

Top

Stathol

Post subject: Re: Glade 4.0, two years in

Posted: Tue Aug 30, 2011 2:29 pm

Lean, Mean, Googling Machine

Joined: Thu Sep 03, 2009 9:35 am
Posts: 2903
Location: Maze of twisty little passages, all alike

Whoops.

That should have been "I think I'll leave iftop or something running this weekend". :p

Top

Stathol

Post subject: Re: Glade 4.0, two years in

Posted: Tue Aug 30, 2011 2:33 pm

Lean, Mean, Googling Machine

Joined: Thu Sep 03, 2009 9:35 am
Posts: 2903
Location: Maze of twisty little passages, all alike

Okay, so...yeah...there's definitely something weird going on here.

*processing*

Top

Raltar

Post subject:

Posted: Wed Aug 31, 2011 5:52 am

I am here, click me!

Joined: Wed Sep 02, 2009 8:00 pm
Posts: 3676

I find that really odd since the weekend seems to be the slowest time for posting on the Glade as it is.

_________________
Los Angeles Kings 2014 Stanley Cup Champions

"I love this **** team right here."
-Jonathan Quick

Top

LadyKate

Post subject: Re: Glade 4.0, two years in

Posted: Wed Aug 31, 2011 7:12 am

Home of the Whopper

Joined: Thu Sep 03, 2009 8:51 am
Posts: 6098

Midgen wrote:

Stathol wrote:

In any case, I think I'll leave

lol wut?

rofl!!

_________________
"Therefore do not worry about tomorrow, for tomorrow will worry about itself. Each day has enough trouble of its own." Jesus of Nazareth

Top

Dash

Post subject:

Posted: Wed Aug 31, 2011 7:47 am

Site Admin

Joined: Mon Jun 01, 2009 7:54 am
Posts: 2369

Things are running smoothly which is all I ever need, thanks Mook!

_________________
“Strong people are harder to kill than weak people, and more useful in general”. - Mark Rippetoe

Top

Mookhow

Post subject: Re: Glade 4.0, two years in

Posted: Wed Aug 31, 2011 10:11 am

God of the IRC

Joined: Wed Sep 02, 2009 7:35 pm
Posts: 3041
Location: The United States of DESU

Stathol looked into the bandwidth thing. It wasn't something anyone on the Glade did. It seems to have stopped, but if it happens again I'll probably block the offending IP address.

_________________

Top

Lex Luthor

Post subject: Re: Glade 4.0, two years in

Posted: Wed Aug 31, 2011 10:12 am

Joined: Thu Sep 03, 2009 10:03 am
Posts: 4922

Mookhow wrote:

Stathol looked into the bandwidth thing. It wasn't something anyone on the Glade did. It seems to have stopped, but if it happens again I'll probably block the offending IP address.

You mean I won't be able to use the Glade as a proxy to torrent pr0n? Damn.

Top

Mookhow

Post subject: Re: Glade 4.0, two years in

Posted: Wed Aug 31, 2011 10:19 am

God of the IRC

Joined: Wed Sep 02, 2009 7:35 pm
Posts: 3041
Location: The United States of DESU

Lex Luthor wrote:

Mookhow wrote:

Stathol looked into the bandwidth thing. It wasn't something anyone on the Glade did. It seems to have stopped, but if it happens again I'll probably block the offending IP address.

You mean I won't be able to use the Glade as a proxy to torrent pr0n? Damn.

... No, you won't.

_________________

Top

TheRiov

Post subject:

Posted: Wed Aug 31, 2011 10:57 am

Rihannsu Commander

Joined: Thu Sep 03, 2009 9:31 am
Posts: 4709
Location: Cincinnati OH

What were they doing?

Top

Mookhow

Post subject: Re: Glade 4.0, two years in

Posted: Wed Aug 31, 2011 11:05 am

God of the IRC

Joined: Wed Sep 02, 2009 7:35 pm
Posts: 3041
Location: The United States of DESU

Stathol can probably explain it better, but apparently some company, that supposedly is supposed to help websites get better positioning in search engines, mistakenly latched onto the Glade and was spamming us with requests. Something like 300k requests for a handful of random threads over the course of a couple of weeks. It appears to have stopped on 8/25.

_________________

Top

Stathol

Post subject: Re: Glade 4.0, two years in

Posted: Wed Aug 31, 2011 3:24 pm

Lean, Mean, Googling Machine

Joined: Thu Sep 03, 2009 9:35 am
Posts: 2903
Location: Maze of twisty little passages, all alike

Pretty much that.

We had a single IP address that generated some odd 321k requests between Aug. 5 and Aug. 25. At ~60kB per response (in this case), that accounts for all of the extraneous bandwidth this month.

After a bit of sleuthing and some more analysis, it appears to me that a particular SEO (Search Engine Optimization) firm's automated script ran amok, causing it to infinitely loop over the first page of the first thread following thread #1000 in the first forum of the board. This firm basically sells blog/fora/twitter/etc. linkspamming services, so ... yeah. Shady people, as SEO asshats usually are. The "attacks" likely stopped abruptly on the 25th because someone on their side discovered the bug. That's just a guess based on its behavior and the wares they're hawking, but IMHO this is probably attributable to stupidity rather than malice.

Regardless, if I see any more traffic from them, I'll block the IP outright. At some point I may also add some additional general-purpose anti-(non-distrubted)DoS filters, but that takes a little more planning than I can lend right at the moment. To be honest, this particular situation is pretty much a non-starter. 18GB and 320k requests sounds like a lot, but it's really not. Even on the worst day, our CPU utilization was basically 0. And seeing as we're at a whopping 4% of our bandwidth cap with the excess traffic....yeah. Obviously, they shouldn't be spamming other peoples' web servers, even if by accident. But ultimately it's just a weird thing that happened, and not really of any particular consequence.

Edit: That said, I will probably report these guys to Google because seriously, **** SEOs. You can put whatever window dressing you want on it, but linkspamming is linkspamming.

Top

Lex Luthor

Post subject:

Posted: Wed Aug 31, 2011 5:22 pm

Joined: Thu Sep 03, 2009 10:03 am
Posts: 4922

I have mixed thoughts about SEO. I think for the most part, it has helped sites become more accessible to the people who want to find them. However, it has also been abused a lot. I'd wager that historically it has done more good than harm, but since the popular search engines are now extremely effective and sophisticated (relative to before)... yeah **** SEO.

Top

Raell

Post subject:

Posted: Thu Sep 01, 2011 5:21 pm

Solo Hero

Joined: Wed Sep 02, 2009 9:32 pm
Posts: 3874
Location: Clarkston, Mi

Only thing I would change... Make it so the Glade isn't blocked from my work. Since you guys can't do anything about that, I won't *****.

_________________
Raell Kromwell

Top

Page 2 of 2

[ 42 posts ]

Go to page Previous 1, 2

Board index » The Glade » General

All times are UTC - 6 hours [ DST ]

Who is online

Users browsing this forum: No registered users and 143 guests

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum