Author Topic: Datacenter issue - July 23, 2019  (Read 2526 times)

Offline Chris Bergin

Datacenter issue - July 23, 2019
« on: 07/24/2019 01:07 am »
So we were down for about 20 minutes due to a datacenter at the forum host going down.

"Our networking team has identified an issue with networking in our ------ regions that is currently causing Droplet connectivity issues and event processing errors. We will post an update as additional information is available."

The forum is back, but it will be a bit shakey while they fully fix it.
« Last Edit: 07/24/2019 01:21 am by Chris Bergin »
Support NSF via L2 -- Help improve NSF -- Site Rules/Feedback/Updates
**Not a L2 member? Whitelist this forum in your adblocker to support the site and ensure full functionality.**

Offline Zed_Noir

  • Senior Member
  • *****
  • Posts: 4261
  • Canada
  • Liked: 1267
  • Likes Given: 1132
Re: Datacenter issue - July 23, 2019
« Reply #1 on: 07/24/2019 01:17 am »
So we were down for about 20 minutes due to a datacenter at the forum host going down.

"Our networking team has identified an issue with networking in our ------ regions that is currently causing Droplet connectivity issues and event processing errors. We will post an update as additional information is available."

The forum is back, but it will be a bit shakey while they fully fix it.

Being getting 522 time out error messages in the last little while. Messages indicated problem connecting with the NSF server.  :(

Note - why is the thread title got "May 23, 2019" in it? ::)
« Last Edit: 07/24/2019 01:21 am by Chris Bergin »

Offline Chris Bergin

Re: Datacenter issue - July 23, 2019
« Reply #2 on: 07/24/2019 01:22 am »
So we were down for about 20 minutes due to a datacenter at the forum host going down.

"Our networking team has identified an issue with networking in our ------ regions that is currently causing Droplet connectivity issues and event processing errors. We will post an update as additional information is available."

The forum is back, but it will be a bit shakey while they fully fix it.

Being getting 522 time out error messages in the last little while. Messages indicated problem connecting with the NSF server.  :(

Note - why is the thread title got "May 23, 2019" in it? ::)

Thread title is because I'm an idiot ;)

Yeah, as noted, they got the sites back, but not to full speed it seems. Just got another e-mail saying resolved and monitoring, so it should improve now!

Support NSF via L2 -- Help improve NSF -- Site Rules/Feedback/Updates
**Not a L2 member? Whitelist this forum in your adblocker to support the site and ensure full functionality.**

Offline Chris Bergin

Re: Datacenter issue - July 23, 2019
« Reply #3 on: 07/24/2019 01:48 am »
Still seeing some 522 timeout errors, so clearly still not 100 percent fixed. Will update when they update.
Support NSF via L2 -- Help improve NSF -- Site Rules/Feedback/Updates
**Not a L2 member? Whitelist this forum in your adblocker to support the site and ensure full functionality.**

Online catdlr

  • Member
  • Senior Member
  • *****
  • Posts: 6846
  • Enthusiast since the Redstones
  • Marina del Rey, California, USA
  • Liked: 3463
  • Likes Given: 2741
Re: Datacenter issue - July 23, 2019
« Reply #4 on: 07/24/2019 04:15 am »
Got a 502right now:

Cloudflare Ray ID: 4fb2f93d9b7053d6 Your IP: 104.174.123.107

Ray ID: 4fb2f93d9b7053d6 2019-07-24 03:58:35 UTC

Response time on the entire site is very slow (10 - 15 sec)
« Last Edit: 07/24/2019 04:16 am by catdlr »
Tony De La Rosa

Online catdlr

  • Member
  • Senior Member
  • *****
  • Posts: 6846
  • Enthusiast since the Redstones
  • Marina del Rey, California, USA
  • Liked: 3463
  • Likes Given: 2741
Re: Datacenter issue - July 23, 2019
« Reply #5 on: 07/24/2019 04:39 am »
another batch;

Error 522 Ray ID: 4fb32f797ef0d37a 2019-07-24 04:35:37 UTC
Connection timed out

Error 522 Ray ID: 4fb332c95f05d37a 2019-07-24 04:37:52 UTC
Connection timed out

I would say I'm erroring about 1out of every 10-15 threads.  The response is very good except when I get the Timeout.
« Last Edit: 07/24/2019 04:40 am by catdlr »
Tony De La Rosa

Offline Chris Bergin

Re: Datacenter issue - July 23, 2019
« Reply #6 on: 07/24/2019 08:14 am »
Really should be OK now:

DO earlier: "This network issue resurfaced and our Network Engineers have identified the issue and are working to fix this. We will share the details of the issue and progress on fixing the issue here :

Once this issue is resolved, we believe the 522 error should also get resolved. We apolagize for the inconvenience caused in this case. "

An hour ago:
"Incident resolved
Our engineering team has resolved the issue with networking in our ______ regions. Droplet connectivity issues and event processing errors should now be operating normally."
« Last Edit: 07/24/2019 08:15 am by Chris Bergin »
Support NSF via L2 -- Help improve NSF -- Site Rules/Feedback/Updates
**Not a L2 member? Whitelist this forum in your adblocker to support the site and ensure full functionality.**

Offline mtakala24

Re: Datacenter issue - July 23, 2019
« Reply #7 on: 07/24/2019 01:03 pm »
Getting errors again.
4fb6169a28ac5b43

Offline RocketLover0119

  • Senior Member
  • *****
  • Posts: 2871
  • Space Geek
  • Tampa, Florida
  • Liked: 6710
  • Likes Given: 1606
Re: Datacenter issue - July 23, 2019
« Reply #8 on: 07/24/2019 01:11 pm »
Had issues getting on this morning, kept getting 522's.
"The Starship has landed"

Offline Chris Bergin

Re: Datacenter issue - July 23, 2019
« Reply #9 on: 07/24/2019 01:17 pm »
Yeah, that 522 error was fixed about 8am UTC. Looks they just patched the entire fix just now with some bad gateways. Should be fine now!
Support NSF via L2 -- Help improve NSF -- Site Rules/Feedback/Updates
**Not a L2 member? Whitelist this forum in your adblocker to support the site and ensure full functionality.**

Tags:
 

Advertisement NovaTech
Advertisement SkyTale Software GmbH
Advertisement Northrop Grumman
Advertisement
Advertisement Brady Kenniston
Advertisement NextSpaceflight
Advertisement Nathan Barker Photography
0