COMMUNITY - FORUMS - GENERAL DISCUSSION
3/12/2020 9:18:11 PMPublic [NDA] Testing of things: Part 3 Redux Return of the tests!
User Portrait

Cyneric

  • Member
  • Rank 11
  • 365 Posts
  • November 2015

Hey Folks, how goes, it's your boy Cyn here back with a third too cool for school thread. I'm about to head out for my big move to Japan here so I won't be checking this thread too much after I post.

As always I advocate for public testing to reduce issues and delays that could otherwise be avoided, lessened, minimized, or to increase consumer confidence and let devs shift the blame to testers not catching something vs the devs not catching something, which sounds like semantics but imo it makes all the difference. I don't particularly care who tests, how they draw testers into the lot, but we do have nda forums and now with the 10k QA stuff no longer being NDA, what a cool use of that forum stuff if they used it for testing as they shift gears into bigger and better features. It's a pipeline that needs building and it sorely has to be made because I think without it that things are only going to get worse.

With public testing, you'll know if something works to the best of your ability before it's fully released to the public in what could be a haphazard state, or to prevent last minute unexpected delays that shake consumer confidence. You'll also gain insights into things like how does the general public use the systems, are they using it in a way that we didn't expect/intend, and if so how does that change xyz about what outcomes we want for said system, does it break the system, is this what we want the system to represent given that's how players use it. These are things that aren't very easy for staff to test for one simple reason that any coder can tell you, you don't test your own stuff, because you're too in the weeds and understand too deeply how the systems work. You need someone who has no familiarity with the system to just waddle in, do some dumb nonsense and see if it breaks, see if it meets your target outcomes and goals of the system. If you as the coder intuitively know how something should work, then you'll generally speaking execute in that very narrow way, but that isn't how the general public use technology, if it was then tech support hotlines wouldn't exist because everyone would use the software exactly as intended with no deviation from the norm. That's clearly not something that happens and people do weird af stuff with programs, systems and oh so much more.

I think they need a private test for the public before launching to the public, if for no other reason than to sort out last minute kinks before it goes live. When we asked for no dates, we meant we wanted the software fully and totally done, 100% baked and on the shelf ready to deploy with no delays before you announce to us that something is ready. IF that isn't possible then I think that's a reason to have public testing, I think sbs needs more validation in their system and public nda testing could be a good validation step to ensure that things are working correctly, as clearly we're still seeing delays with their private internal testing that isn't meeting the standards that we as customers should be expecting certainly with their promises to do and be better.

If you've enjoyed this cool hot take for millenials, may I interest you in the prequels of this trilogy, perhaps we may return with a part 4 to this story? Who knows the suspense is real boys and girls.

Testing DSS Part 1

Testing DSS Part 2

3/12/2020 9:29:01 PM #1

Obento

  • Member
  • Rank 9
  • 26 Posts
  • January 2019

Have a safe flight Cyn, whole heartily agree with this post.

Friend Code: 5AE86A

3/12/2020 9:42:59 PM #2
User Portrait

CocoaSip

  • Member
  • Rank 12
  • 19 Posts
  • August 2018

Have a safe flight and stay clear of the Corona Virus. But I agree with you and always have on each of these posts. Maybe now they will take your idea into consideration.

3/13/2020 1:11:50 AM #3
User Portrait

Marovec

  • Member
  • Rank 10
  • 1434 Posts
  • January 2017

Sooo...back when SoE launched the first time, and they had to do all that "our bad, we messed up" stuff, didn't they acknowledge that they did not test sufficiently, and planned on utilizing their community for testing prior to relaunching, so this thing didn't happen again?

Didn't they, during the 10k stream, say that they were going to do "some public testing with some of our moderators and administrators and people who are under NDA, just to confirm we had an opportunity to get it in public hands and do some real live testing on it, before we go live again"?

Quick answer, they did, I checked...listened to the part where it was said multiple times to make sure I quoted it right

So...now, come launch time, not only didn't they schedule a legitimate read only period of a few days just to make sure, not only did they not use their A1/A2/XA NDA groups for large scale public testing, they didn't even do whatever "moderator/administrator public testing" they said they were going to do a week ago.

I honestly don't know how to react to this. I have no words to express the abject failure in this area - not only just the simple failure of yet again another launch, but the failure to, again, do what they told the community they were going to do.

Wow...just...wow.

Imgur

3/13/2020 1:40:06 AM #4

Damocles316

  • Member
  • Rank 7
  • 27 Posts
  • August 2018

I wish I was surprised.

I wish I was disappointed.

But I had honestly forgotten that today the auction was supposed to start, because I automatically filter out SBS promises.

3/13/2020 2:05:02 AM #5
User Portrait

Roarer

  • Member
  • Rank 10
  • 588 Posts
  • March 2016

Posted By Marovec - Thu Mar 12 2020

...

Didn't they, during the 10k stream, say that they were going to do "some public testing with some of our moderators and administrators and people who are under NDA, just to confirm we had an opportunity to get it in public hands and do some real live testing on it, before we go live again"?

>... So...now, come launch time, not only didn't they schedule a legitimate read only period of a few days just to make sure, not only did they not use their A1/A2/XA NDA groups for large scale public testing, they didn't even do whatever "moderator/administrator public testing" they said they were going to do a week ago.

...

We need a SBS respond to this.

Never argue with an idiot, cuz he will drag you down to his level and beat you with experience.

Vice mayor of Lux Verloren

3/13/2020 2:08:01 AM #6

Poldano

  • Member
  • Rank 7
  • 2007 Posts
  • September 2015

Posted By Roarer at 7:05 PM - Thu Mar 12 2020

Posted By Marovec - Thu Mar 12 2020

...

Didn't they, during the 10k stream, say that they were going to do "some public testing with some of our moderators and administrators and people who are under NDA, just to confirm we had an opportunity to get it in public hands and do some real live testing on it, before we go live again"?

>... So...now, come launch time, not only didn't they schedule a legitimate read only period of a few days just to make sure, not only did they not use their A1/A2/XA NDA groups for large scale public testing, they didn't even do whatever "moderator/administrator public testing" they said they were going to do a week ago.

...

We need a SBS respond to this.

Um, why?

I figure they are going to screw up things like this until they happen by chance upon something that works. That's because I don't believe there is any certain way to make sure ahead of time that some critical thing has not been glossed-over, overlooked, or outright forgotten. Think of it as a corollary to Murphy's Law.

3/13/2020 2:18:10 AM #7
User Portrait

Roarer

  • Member
  • Rank 10
  • 588 Posts
  • March 2016

@ Poldano

Cuz it is a very obvious inconsistency , and this directly address a concern of the community.

Never argue with an idiot, cuz he will drag you down to his level and beat you with experience.

Vice mayor of Lux Verloren

3/13/2020 2:21:14 AM #8
User Portrait

Marovec

  • Member
  • Rank 10
  • 1434 Posts
  • January 2017

You know what makes a big difference to making sure software works ahead of time?

Testing.

You know what has repeatedly failed to identify problems, that have later been revealed by the public accessing their site on a large scale?

Their internal testing.

Do you know what they said they would do before this release, that they didn't do...yet found issues within 30 minutes of actually being given access?

Public testing.

Imagine how many more issues would have been identified if they did even small scale public testing for a couple days, or large scale A1/A2/XA testing...

They said they were going to do public testing prior to going live. They did not do public testing prior to going live.

Sure...plans change, I get that.

So, they either had no intention of actually doing the public testing they said they would do, or someone legitimately said "nah, we don't need public testing" as a conscious choice.

One option indicates deception, the other option indicates incompetence.

They can respond. They can not respond. Doesn't matter...

They have proven, yet again, that we simply have no reason to believe anything will happen the way they say it will - even things 100% within their control, like the decision to do public testing.

That's fine. That's their choice.

I have no sympathy - this is basically comedy at this point.

Imgur

3/13/2020 2:44:08 AM #9
User Portrait

Roarer

  • Member
  • Rank 10
  • 588 Posts
  • March 2016

Posted By Marovec - Thu Mar 12 2020

...

Sure...plans change, I get that.

So, they either had no intention of actually doing the public testing they said they would do, or someone legitimately said "nah, we don't need public testing" as a conscious choice.

One option indicates deception, the other option indicates incompetence.

They can respond. They can not respond. Doesn't matter...

They have proven, yet again, that we simply have no reason to believe anything will happen the way they say it will - even things 100% within their control, like the decision to do public testing.

...

I understand where you are coming from, but I would withhold judgement for now. Hopefully, we will get more info soon.

Never argue with an idiot, cuz he will drag you down to his level and beat you with experience.

Vice mayor of Lux Verloren

3/13/2020 2:59:58 AM #10
User Portrait

Trug

  • Subject Expert
  • Rank 10
  • 243 Posts
  • May 2016

Posted By Marovec at 9:11 PM - Thu Mar 12 2020

So...now, come launch time, not only didn't they schedule a legitimate read only period of a few days just to make sure, not only did they not use their A1/A2/XA NDA groups for large scale public testing, they didn't even do whatever "moderator/administrator public testing" they said they were going to do a week ago.

They did actually do the mod testing. The post in Discord says "Delaying SoE in order for time to resolve some issues our mod community identified".

Can certainly still argue that they did this a day late and a dollar short, but it's dishonest to claim that they didn't do it at all.

3/13/2020 3:17:59 AM #11
User Portrait

Brynath

  • Member
  • Rank 10
  • 189 Posts
  • May 2016

Edit

Not worth leaving up.

COMMUNITY - FORUMS - GENERAL DISCUSSION