You are destroying my server Ausgamers FIX the server FFS.

Staff send me an email at 14:50 yesterday saying that DICE wants logs from my server to identify an issue. Server starts constantly dropping all players 20 minutes earlier.

Last time DICE asked for logs from my server they turned on enhanced debugging which did the similar things... random disconnects.

Any staff reading this please if you have enabled extra debugging for DICE, please turn it off.


http://imageshack.com/a/img835/9397/q3gc.png

I have had players getting kicked from about 11am yesterday and again today but so far it only seems to be happening on 1 server but the other 2 have not been populated enough to tell :(

Well it just happened again at 17:23:29 :(

#2 server is also having the same drama it just kicked everyone at 18:50:34

19:56
Just got the server to number 4 in AUS too....

Update: server that had crashed 8 times just crashed twice more. Total of 10 crashes. This has been my servers very own personal record.

Hello Eorl,

Did you find anything?

I have had to warn players off using the server. We would appreciate if you could let us know when the issue is identified/resolved so we can let them know its ok to start using the servers.

"Hi there,
The issue affecting your server is much more complicated than we had hoped, and we are working with DICE to figure out the best solution possible."

This is a quote from my ticket so the good news is they are working on it the bad news is it involves dice :(.

It was nice to get an updated and be informed of some progress thanks AusGamers I appreciate it :)

Yes 8-10 times for our server today.

Are we looking to set a record here? See how many times the server kicks everyone.

The record stands at 36+ in one day.

We are doing alot better atm with only 12 each day. (every 2 hours to the minute)

Been happening everyday,every 2 hours for the last 8 days........

Issue still happening. Can you please keep investigating urgently.

Issue 10:25:58

Looking at server stats for other GSP's this issue is not happening with them.

and there it goes again 9:47WST....

Strangely enough other instances on the same hardware node are up and operating just fine, remind's me of um two years ago.

Next week aint good enough AusGamers, It has been 1 1/2 Days since i first reported.

If it is DDoS, you have not mitigated it as it is still constantly happening.

It is Not the UDP issue.

No one can play on your servers. Pretty soon no one will want to.

Just look at it.....

http://imageshack.com/a/img843/2896/60ti.png

Yeap one of my servers has crashed 5 times already. How is Gameserver and hypernia for reliability? Looks like im just about to swap over.

Testing one of them now. Have exactly the same idea. This BS has been going on for almost 2 weeks now, not just the past few days.
If they look at the tickets I have logged it will show them just that.

This is crap, All 3 servers unplayable. Start playing then everyone is dropped.
Where's the support?

They say they have mitigated the DDoS, if that's the case what the F..K is it.

An Phenom600, if it is affecting other AUS servers why is this not showing in the GameTracker graphs of the top servers in AUS.

My server
http://cache.www.gametracker.com/images/graphs/server_players.php?GSID=3672654&start=-1w&request=02933170555162544

Other AUS servers (Service provided by Internode)
http://cache.www.gametracker.com/images/graphs/server_players.php?GSID=3688328&start=-1w&request=043958039622052114http://cache.www.gametracker.com/images/graphs/server_players.php?GSID=3687633&start=-1w&request=04113891727533936http://cache.www.gametracker.com/images/graphs/server_players.php?GSID=4022202&start=-1w&request=09221908639239728http://cache.www.gametracker.com/images/graphs/server_players.php?GSID=4078451&start=-1w&request=08692762187647468http://cache.www.gametracker.com/images/graphs/server_players.php?GSID=3687626&start=-1w&request=016633657022095483

This issue is only affecting AusGamer Servers.

While it's sad to hear the issues occurring, I'm kinda glad we aren't the only ones. For some reason however, the Games.on.net servers seem to be going like troopers (Gameservers). That's whats making me want to switch over.

While it's sad to hear the issues occurring, I'm kinda glad we aren't the only ones. For some reason however, the Games.on.net servers seem to be going like troopers (Gameservers). That's whats making me want to switch over.

Did and thought the same thing and am now asking them for my money back and telling them to jam it where the sun dont shine. Its worse!

One thing that Games.on.net have in there favor however is if any kind of deal was struck it would involve using the pipenetwork and that is exactly what appears to be the problem. The servers are starved for bandwidth.

Still happening and we are kept in the dark.

What is happening?

LOL Thanks for that..... Put a smile on my face....

Same here 20:52:40 by my clock.

I could have posted literally hundreds of these over this weekend, but here's another to give you an idea of how close together they are.

[20:36:12 62] [FailLog] Type:BLAZE_DISCONNECT, UTC:20140420_12:36:12, Server:"[FPSG] 24/7 Metro Madness 1500 Tickets! (NO EXPLOSIVES)", Map:Operation Metro 2014, Mode:Conquest Large, Round:1/2, Players:64/19/0, Uptime:03:02:36, Details:"BF4,Ausgamers,FPSG,www.fpsgamerz.com,Australia,OC/au,,"

Lag in game prior to anything dying in the arse....

Guess i spoke to soon......... FFS

Morning AusGamers Staff.

I hope u are fully refreshed after the long week end.

Can we please have a answer as to what is happening with our servers.

game hosts with IP's of
27.50.71.x
173.199.80.x
59.167.241
58.6.24.x
108.61.227.x
look to have not experienced the issue this week end as gametracker showed player game times of over 2.5 to 4 hours when randomly checking, indicating that they either experienced minimal outages or non at all.

edit
In case your wondering..... Its is still happening

http://2.images.au.gamecreate.com/admin/RenderImage.aspx?entity=9720341&class=1&sample=300&domain=6108&features=63

We were number 4 in AUS based on Gametracker stats when this began, now we are number 11.

This issue needs to be addressed today please.

And still Happening.......

Well I think you still have some work to do everyone just got kicked from #1 server 19:24:53 :(

Servers still going down tonight.

Have to say since i've been paying a little more attention, games.on.net has had 1056+ slots online all night, how the f*** does that work... mind you I'm not really sure how it's profitable for GSPs in AU when you have that happening.

Thanks Malthius,

Would have been nice to have this earlier on the weekend.

A simple post stating that DDoS was still being observed and that u guys were working on it would have been appreciated. Would have saved a lot of frustration and time.

There's nothing mentioned about this on the "AusGamers Server Rental Status" page about the problems or a time frame to resolution.

Eorl said Saturday morning that AusGamers had mitigated the DDoS attacks, then we heard nothing and our servers were still unusable.

Any update today as to the DDoS attacks?

This server #1 AUS Infantry-Wars Locker No Explosives, player times of 2h43m.... mine within the hour, every hour, all players dropped.

AusGamers u need to investigate this better or upgrade ur gateway kit for better DDoS protection.

To add more pain, Server patch R32 is being deployed, though you would never know it from AusGamers. Ever other GSP provides information about server updated though..

http://status.hypernia.com/battlefield-4-bf4-ranked-server-r32-patch-released/

The silence is deafening..

So where are you going to host them Phenom?

Every time the server starts to get going it fricken emptied, 20-30 minutes all dropped. Maybe one round thirty are able to finish.

games.on.net servers, player up times 4-5 hours. They get to have many rounds before dropping. What are they doing different?

AusGamers give them a call and ask......!

Do something .... say something... It is now 7 days since we have not been able to utilise our servers.

The AusGamers brand is going to crap.

This is becoming extremely painful.

One server was up and running for 2+ hours then just as i joined... blank screen and then my desktop.

Even the dedicated regulars are struggling to want rejoin our servers after they drop all players. I don't blame them.

The only good thing i can see out of this is that Gameservers and Multiplay hosted servers seem to be doing much better at mitigating the DDoS attacks.
All the server that have moved up in the ranks belong to these two hosters.

Looks to be where to move hosting to.....

Good to see some things aren't changing.

Our server dropped all players about 2 hours ago.

Also AusGamers I just checked... You have reneged on the agreement that we had where you asked me not to move or change my 70 player subscription as i can only run 62 players due to CPU LAG and you would give me credit/extra time. You have been billing me consistently every month on the same day since the 14th Feb.

I still cant run the server at full compliment without CPU LAG. You have been made aware of this......

Hi,

This is done. The calculation I've used is:

Price difference = $6 per month.

A 64 player server is $64 per month, assuming 28 day month, = $2.29 per day.

$6 = 2.62 days of credit.

To be safe, I've credited your account with 6 days, so your 70 slot
server will renew on Feb 14.

Regards,
.


On Wed Jan 29 17:25:35 2014, wrote:
> Hi Brendan,
>
> That's fine with me I only run 58 players plus 2 commanders and 1
> spectator anyway.
> I appreciate this.
>
>
>
> On 29/01/2014 4:37 PM, AusGamers Support wrote:
>> Hi,
>>
>> It is technically possible, but would require some effort on your part
>> to compensate for a potential problem on our part, so I'd rather you
>> continue to run the server with a cap of 64 players for now and I'll
>> credit your server for the difference in cost between 64 and 70.
>>
>> Let me know if that is OK.
>>
>> Regards,
>>
>>
>> On Tue Jan 28 09:25:36 2014, wrote:
>>> User IP: 172.20.187.144
>>> Forwarded IP: 110.175.159.136
>>> User Port: 55545
>>> User Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0)
>>> Gecko/20100101 Firefox/24.0
>>>
>>> ------------
>>>
>>> Hello,
>>>
>>> I see no real improvement in server performance and you have not
>>> contacted me back with regards to all the extra logging DICE performed
>>> as to their findings.
>>>
>>> As i still cannot run the server without the full compliment of
>>> players i would like to drop the slot count down to 64 from 70 as i am
>>> essentially paying $6 for nothing.
>>>
>>> I do not want the server moved to an other host just drop the player
>>> slots to 64. If this cannot be done please let me know.
>>>
>>> Regards
>>>
>>>

edit: Now that i can, i have dropped the server down to 64 players.

HEHE network is s***. Thats a joke. They should be honest and say what I know to be a fact due to having ties with hypernia a few years back and say that the cost of data in australia is so high and we are tight arsed.

All overseas GSP view it in the same light, "To expensive to run in Australia due to data cost"

Look up a single GSP, I'll use i3d for this example. They are using dedicatedgaming who advertises traffic via the USA, this is so they can utilise blacklotus for filtering to save on AU traffic (good idea) and claim to have NSFocus devices locally to take care of the AU based traffic.

i3d pays them under $200 AUD a month for a single server, in my case it was a xeon 1230v2 that had 6x 64 slot instances running on it and two other games, so for that node alone they are getting paid ~$700 a month without taking into account the other games running on it. Data usage is approximately 5TB a month which is included in the cost of the server.

The slot costs for them in my example are around ~$107, so we will make the total outlay $310 (rounded) bucks a month on $700 in income, this is of course disregarding that i3d has their own pop which would cost them a tidy little sum of money each month. They don't pay AU taxes on the money either, they aren't based in AU, like most of the other GSP's, only ausgamers is.

The AU market is rather small, you have GON with an endless supply of slots, 1200+ at last count which in reality is taking away potential profit for the GSPs.

So what I don't see happening is them investing heavily in hardware or infrastructure, especially the capacity to be able to handle the attacks nor the extremely expensive gear to actually mitigate the attacks. I also don't expect them to be spending much time on the customer service side, after all staff need to be paid and we aren't talking much money here at all.

Apart from GS (internode) AG (bigpond) they are all customers of dedicatedgaming or serversaus, leaving them with very little control over the network they are operating off. I know for a fact several support tickets were lodged through to both of these providers and nothing improved for us.

Having said all of that, the best placed GSP in AU to deal with these attacks should be Ausgamers, granted they are backed by telstra, who has the capacity and I know provides DDoS mitigation services. So the question is really why? and personally I believe it's a lack of interest due to the costs and little in return, the closure of GA in a way confirms telstras lack of interest.

server just kicked everyone again at 11:50:30, 9 days of this s*** is really starting to piss me off how about some feedback on what the bloody hell is going on and whether or not you can fix it, The damage this is doing cannot be payed back but so far you owe us 10 days on 3 servers.

#2 server kicks everyone at 13:09:39 :(

Our new Gameserver server has not skipped a beat.

We have a lot of work to do to get it repopulated and its rank up to 4 again however no drops since I bought it online yesterday 16:00.

What are you going to do today AusGamers to stop other people leaving?

Send me a bill for the servers i havent used for a month.

Don't waste your time on the reasons why, just tell us how it's going to be fixed....

Just happened on all our servers, a bunch of Gameserver servers went down aswell even Games.on.line.

DDoS back with a vengance.

Just happened on all our servers, a bunch of Gameserver servers went down aswell even Games.on.line.

DDoS back with a vengance.

AG instances haven't stopped going down, even my clan which are/were metro whores aren't bothering to seed our AG instance and are pretty much over it all.

GS instances have always gone down, I think you just had a pretty good run last night, or the last two nights, however long it was.

What are you hoping to achieve out of comparing your stuff with gameservers and why can't you liaise directly with them?

http://www.gametracker.com/server_info/108.61.227.201:30080/ (it was ranked 3rd or 4th until it was DDoSed to s*** the last two weeks which was confirmed through support tickets)

I had sent you a support request a month (or more) ago about moving an instance ranked 13th in the world over, never received a response, which is rather common theme that people have been complaining about for a long time now.

Either way, GS, i3d, insert GSP X here have just as many issues, and I can post up log files all day of that if required but it's not going to get us anywhere.

Can you do me a favour, bring up a dedicated server via mammoth media in that new location you showed me. I will provide you with a 1RU one if required, heck even pay for the transit costs, then spin up a few instances and I will populate them.

I would like you to then monitor network traffic using a utility such as wireshark, then come back here and post your findings.

Surprise.....

UTC:20140430_09:15:02, Server:"[FPSG] 24/7 Metro Madness 1500 Tickets! (NO EXPLOSIVES)", Map:Operation Metro 2014, Mode:Conquest Large, Round:1/2, Players:64/14/0, Uptime:1.02:13:22, Details:"BF4,Ausgamers,FPSG,www.fpsgamerz.com,Australia,OC/au,,"


So we filled it up and it lasted an entire hour....

What are you hoping to achieve out of comparing your stuff with gameservers and why can't you liaise directly with them?

I want to know whether the complaints about our servers being less stable overall than gameservers service are factual.

They are our main competitor, so I doubt that they'd be willing to share that information, and frankly I doubt that they have the level of detail that I get from my bespoke log parsing solution.

I'm bored of not knowing exactly where the problem is. If gameservers servers in AU are approximately as stable as ours that indicates one thing. If our servers are less stable, that indicates something else. I want facts, not opinions. With facts I can put more pressure on DICE to resolve the issue.

I want to know whether the complaints about our servers being less stable overall than gameservers service are factual.

They are our main competitor, so I doubt that they'd be willing to share that information, and frankly I doubt that they have the level of detail that I get from my bespoke log parsing solution.

I'm bored of not knowing exactly where the problem is. If gameservers servers in AU are approximately as stable as ours that indicates one thing. If our servers are less stable, that indicates something else. I want facts, not opinions. With facts I can put more pressure on DICE to resolve the issue.

I've been rather vocal about the other GSP's so you don't have to look very far for information.

Gameservers - I can tell you now that the experience in game is much better, i.e. no rubber banding and the only time there has been there was packet loss on the iinet network which I reported and after another nudge they got it fixed. I believe they've recently replaced some hardware (server wise)

Gameservers - In all cases bar one my server has crashed due to a DDoS attack, this was confirmed via a support ticket each time. So yes Brendan single instances can go down without impacting on the rest of the instances on the same node. Due to this you can see my server ranking taking a nose dive.

I can confirm this is also the case with NFO servers, where I have access to view the interface traffic, network graphs and also confirmed via their automated DDoS alerting system and via support. Difference is they get attacked and don't go down.

As for GS being competition, that's only because you've allowed that to happen....

I've never seen a DDOS against our network that has taken down less than all of the servers that were reporting to DICE at the time. This may be a function of different network architecture, but it isn't possible with our network architecture.

Dremark: the last time we had an issue with your server disconnecting regularly it was a network interface issue. I can't find the same thing in the logs now (previously we'd see a drop/dump at (e.g.) 2:11, then 3:11, then 5:11. I'm going through in more detail and will ticket the issue. I was advised that the next step is replacing the motherboard on that hardware, so I'll keep you up to date via support on where we are.

So you're basically saying that you run a different network architecture to NFO, iinet/internode, servers australia and dedicated gaming? I haven't worked at Telstra for 12 years but I can tell you now that every network I've seen follows the same set of architecture principals/ topology. There is of course many other variables that come into play such as scale/dimensioning but the underlying principals remain the same.

Fact - The attacks we've captured through other providers align with the outages of our AG instances, That's to say when I am alerted to attacks on my NFO equipment my AG instances have gone offline at the same time or within a few minutes of the report. When I've been notified of attacks on my GS instances via the support ticket process, my AG instances have gone offline, rinse and repeat for i3d, hypernia and multiplay.

Granted the outages could have been related to a wider issue within AU, we ran up the NFO instances in the USA to test this exact theory, and guess what... as above and this time I was able to capture the attack types. This dedicated server is in the first one to two hops from AU, in fact from within AU it can be as little as 5-6 hops from a residential user and this was specifically chosen to be as close to home as possible. For the record our Lockers instance which is ranked 13th in the world and restricted to AU players only, remained up 99% of the time in the first few weeks and once the attacks were sorted it's remained up 100% apart from the forced reboots due to the DICE patches, I can list several other maps that have performed the same.

Fact - I've more than proven that an individual instance can go down, even on our dedicated at NFO a single instance (initially) was able to be dropped, sometimes two of them would go down it really just depended on how large the attack was and how long it went for.

Fact - I was surprised at how often the EA backend would drop players, however lately it has been pretty good and I'm sure 50-80ms of latency and 2-4 hops on top of what our NFO server has wouldn't be the cause of all these issues.

Fact - There is most definitely issues with the game, there's no doubting that, however what I noticed on a xeon 1270v3 with 32GB ram, 1Gbit network interface and a 240GB SSD is that when I ran 600+ slots, the CPU wasn't anywhere near pegged, however the game did become sluggish, the network interface wasn't even budging above 3% utilisation and having crap loads of memory was pointless as the game barely used it.

When I reduced the slot count, the game seemed to operate better, so there appears to be some kind of bottle neck, however it doesn't show in any of the resource graphs, from CPU, Network, Memory or DISK.

Brendan, we've had to deal with this for 3x years now, possibly longer and I would say much longer than anyone else posting in these threads. We have absolutely no where else to go within AU and our only option is to continue on with servers based out of the USA. Could you please stop being so stubborn and give what I have suggested a go? I am willing to cover the costs of it even though this is something that could potentially benefit everyone.

203.46.105.24:20400 and aging at 20:41:55. Cant even get a round in.
No players are joining anymore.

Ausgamers this is serious without stable gaming servers player wont join.
Without players no point in paying for servers!

Malthius
"As for the rest of it - I'll have a longer response soon, I can't give a quick response that does justice to the complaints."


I have canceled my other server. Before I leave, Can you please provide the longer response.

I would be willing too, however not without a explanation as to what is happening to the current infrastructure.