COMMUNITY - FORUMS - GENERAL DISCUSSION
The need for a NDA testing Pipeline

Hey It's your boy Cyn back at it again with another thread! Hitting you with the hot takes here.

So if you remember last time Testing DSS Then you know I'm really adamant about player testing and from the dev's own admission this time they didn't catch the crazy bug interplay in their testing environment, but only noticed it when it went into their live environment. At this point I don't want an apology or anything of the story, people mess up, coding is tough, I get it. You're not going to hear scorn from me about that topic, I couldn't care less about code not working right. What I care about and the only thing I care about at this point is that they make a testing pipeline to get the nda folks involved to make sure that bugs like these now and in the future are caught.

They need to make this pipeline anyway, and at least to me it's clear that they could really use it now, as each mess up really hurts consumer confidence. I had folks ready to spend money on auction, like lots of money, at least a couple counties worth and now they are 100% scared off of doing that. Getting people interested in this game with each and every public failure is really hurting perception of the devs and I do truly believe they can do this but if they keep having these failures it's really going to hurt in the long run. I don't mind waiting or pushing back this or other events to make sure the pipeline is done. That's what I want for the New year is a firm and real commitment by the devs to create a testing pipeline using whatever NDA testers that they want to ensure that at the very least if bugs happen then it's more than just the devs that didn't catch them, that they tried everything and things went horrible anyway. At the very least it would mean people who are trying to get others to spend money on CoE have a way of explaining why these failures happened in a real and legitimate way then just constant excuses trying to keep someone from jumping ship.

I don't need gifts, or apologies, or we'll do better. What I need is the devs to say we're making a pipeline and we realize that we need one for our testing and for the good of the community. Anything less than that and I'm not particularly interested because I think the lesson here from this and the mess up with DSS MUST be we need a testing pipeline now.

That's my hot take folks, love it or leave it, there you go.

...
12/28/2019 4:30:02 PM #1

I agree. Especially for large projects/events like SoE that are big additions to the website they require some form of testing and there are lovely groups at the disposal waiting for testing of really well, anything. It would be wonderful to help however possible to make the launching of projects/events successful.


12/28/2019 4:42:33 PM #2

I agree as well.

Particularly for things that aren't time sensitive like personas, or things that are going to have a large impact like this Settlers of Elyria event and like how we tested DSS.

If it's something like another lost vault/searing plague event, I'm fine either way tbh, but unless it's time sensitive then again, I don't see why not.


12/28/2019 4:46:03 PM #3

Couldn't agree more, I really feel that The devs could /should be a lot more open about about the current state and open up testing to catch issues early on.


12/28/2019 5:06:00 PM #4

100% agreed


12/28/2019 5:08:54 PM #5

Can't argue with this idea.


12/28/2019 5:21:08 PM #6

Supported for the reasons cited.


Imgur

12/28/2019 5:25:46 PM #7

Agree. Been said a dozen times, but maybe as these things seem to keep happening, it will get some more traction.


Imgur

12/28/2019 5:37:08 PM #8

Exactly, but many other times the studio has been given good advice by the community and it has been ignored.

If this is not ignored (as I suspect it will) then the "testers" should be critical thinkers that are not afraid to point something out when it is wrong and not the current echo chamber that is around the studio now.

12/28/2019 5:37:43 PM #9

They’ve already got NDA groups built in. It only makes sense to utilize the resources they have.

12/28/2019 5:39:33 PM #10

The fact we need to even make these posts saddens me greatly. I recruited several people during the plague event who saw what has happened since and basically have told me not to even bring the game up in conversation to them because of how 'shady' it sounds.

I mean, current policies are literally scaring people away. Something needs to change.


12/28/2019 5:44:59 PM #11

hmm perhaps the issue isn't that the dev's disagree with an NDA testing pipeline, but the time it takes to develop multiple iterations per content release and they still don't have the dev staff to keep up with multiple versions?

All while not loosing community confidence by extending the work cycles thus prolonging the game further ?

Thought: I wonder what the overall costs to outsource a testing environment would be? that way anything involving player testing and real world money could have temp contractors to gather information and work with the dev team on prioritization and speed up development by adding more throughput that can be handled at once.

  • Much in the same way you have pay as you go elastic server compute time.

That might also let the devs know if the cost outweighs the need for a permanent team fixture or if content updates can be "agile",


12/28/2019 5:48:17 PM #12

Agree 100%. Users are so much better at breaking systems than the devs that make them.


Tir Slaes

12/28/2019 6:09:32 PM #13

Posted By Noslim at 11:37 AM - Sat Dec 28 2019

Exactly, but many other times the studio has been given good advice by the community and it has been ignored.

If this is not ignored (as I suspect it will) then the "testers" should be critical thinkers that are not afraid to point something out when it is wrong and not the current echo chamber that is around the studio now.

My personal feeling would be: The entire point of testing something is to see if you can hack, glitch, or break it.

I think the echo chamber exists in a setting where we are not meant to tamper with or break anything - so people are generally positive. I presume if we were truly playtesting... the gloves would come off.

Also - I generally agree with this thread. I thought that was the entire point of the NDA focus group? To get our hands on stuff first, and tinker around to make sure it can handle reality beyond our group.


12/28/2019 6:09:36 PM #14

While I'm not currently under NDA, I'm sure there are a solid core of us who would sign one to help move this forward, either to join the current NDA crowd, or be a modified second-tier to help add burden to stress-testing after it works with a small crowd. Those of us who are already committed to this world, and this game, want more people to want to join us, let us help make sure they see it as a positive place.


The road less traveled means I've more to make from it.

12/28/2019 6:10:44 PM #15

This is still a good thing and still needs to happen.

...