COMMUNITY - FORUMS - GENERAL DISCUSSION
How involved should devs be with the game?
+1

I was browsing the EVE subreddit yesterday and I saw a post in there by Jester, who was on the CSM, a community liaison between the playerbase and the CCP (the devs). He was finally released from his NDA and was giving out some fascinating information on the inside workings of things that had happened with the game. I wanted to share some of his comments because I think they really speak to the right and wrong ways to run an MMO. I highly recommend people go read the entire thread, even if you've never even heard of EVE before, to get a good view of what makes a game community start to die.

Here is a link to the whole thread: https://www.reddit.com/r/Eve/comments/bo4zed/iwasonevescsm8iwasspacefamousforalong/

"See, here's the thing: CCP devs are not allowed to play the game in leadership roles of large null-sec alliances. So they don't really understand how those alliances work. And they simply don't have a good feel for the control that operates in those alliances.

Every CCP dev I've ever talked to who seems to have good in-game knowledge has it in areas that are fundamentally single-player sorts of things: industry and mining and missions, stuff like that. But when I would bring up things that are commonplace to EVE players, things like alarm clock ops and the logistics of moving ships around on a large scale and storage of thousands of modules or anything else that involved large scale operations, they had no idea.

I've often told the story about how I'd show devs how my hangar was broken up into a few dozen containers with one for ammo, one for mods, one for drones, one for drugs, etc. The devs I showed this to were, to a person, shocked that this was the case. It just never occurred to any of them the scale that things in EVE happen in.

So the answer to your question is "no", devs that I talked to never played above what the average EVE player would consider a casual level."

To give a little backstory here, in the very early days of EVE, long before I started playing, there was a scandal in which a dev was part of one of the biggest alliances in game and was actively supporting them through their position. After that, devs were, understandably, prohibited from holding leadership positions, but this also meant that they had essentially no insight into the high level parts of their game. This combined with way CSM members were selected meant the devs had huge blindspots when getting feedback on the game.

"This is a really good question. The short answer is that CCP devs as a whole pay a lot more attention to perceived experts on the CSM than they do on the general member. Captain Kirk on Star Trek once joked that after the Organian Peace Treaty, the Organians would give a planet to whomever developed it the most efficiently, "and whatever else you can say about the Klingons... they are most efficient!"

In the same way, null-sec CSM members tend to be perceived experts in their fields and they're definitely efficient about making their desires known to the devs. And since they represent a lot of players, those desires get a broad audience.

This desire to only work with experts applies to every CSM dev and the more active the dev, the more they displayed it. Fozzie is notorious for this (or was, when I was on the CSM). At one point, I was making a point about how HACs were used in small-gang battles and how they should be changed, and he wouldn't engage with me on the topic until after I'd answered a question: "Well, which HACs do you think are the ones that are most used?" It was kind of infuriating to have these check-points thrown at us. To be fair, I was able to answer to the question to Fozzie's satisfaction and after that, he agreed to talk with me about it, but only after that.

I saw the same thing happen with the null-sec CSMs, notably mynnna and PGL, both of whom would often have these sorts of questions thrown at them before their positions were taken seriously. Since they were able to do so, they got a serious hearing. Korvin, on the other hand, tended not to get serious hearings both because he couldn't answer these questions and because his ideas tended to be more outlandish and disorganized."

CCP relied on "experts" that only had a limited field of knowledge about the game and thus made poor decisions that have really impacted the health of EVE.

I bring all this up because I wanted to start a discussion on the best ways to make sure devs have sufficient knowledge of their own product without having conflicts of interest because they play their own game. How do we solve the riddle of making sure that devs know all the issues that come with playing this sort of game at a high level without compromising the game itself?

Edit: I tried formatting the quotes as normal and they decided not to work, so I've simply added quotation marks. I realize the formatting isn't great, but you should be able to get the gist.


5/15/2019 8:39:37 PM #1
+5

I honestly think having them not play is just flat-out the wrong decision and not many studios do this.

I'm not really sure how a conflict of interest even matters unless the management strucure of the company is so loose that a solo dev can on a whim make changes to the game solo w/o any red flags. i.e. if a dev is able to change the live production version of the game w/o it being apart of the current iteration plan w/o raising a red flag or at least logs being generated to somewhere, that company might need to restructure it's release pipeline.

I know if I went in and altered my companies code-base that wasn't apart of the current iteration plan I would get a message the same day asking why I was touching that system when we are working on X.


Crimson County

5/15/2019 10:25:36 PM #2
+0

The only way that I can think of that would give them first-hand experience without compromising the integrity of the game is to have an independent testbed where the devs can play the game and compete against one another. If they need to test at scale, either run large free-to-play events that players outside of the studio can participate in or offer access to it as a free-to-play demo version that is either level capped or wiped regularly. The latter has an added benefit of serving as a kind of ideas lab where community involvement can lead to new and interesting ideas feeding back into the main game as well as a great way for prospective players to get some exposure to the game before deciding to take the full plunge.

The key is for devs to simulate a similar experience but independent of the main game.


5/15/2019 10:30:43 PM #3
+3

My view is that SBS should be as engaged with the game as they wish to be. As long as an employee is not using GM tools to gain an advantage, there should be literally no cap on what they can do in the game - although they ought shy away from revealing story elements they only know because of their work.

Further, SBS should be playing things normal players cannot when it suits the story.

King of the Yoru leading an invasion of Aleysia? Go for it. I'd much rather a Dev playing a key character than an NPC.


Link to my story

5/15/2019 11:57:52 PM #4
+15

It's always an interesting questions, and in a lot of ways how much control your game dev team has over the world-state is a deciding factor on how much interactions your devs should have with the world, if you ask me. In a static game world, a dev from your content team just shouldn't be playing with players - they know way too much about things that can affect your play experience directly. In a world where you physically place every enemy, you'll never be surprised by what's around the corner and just that fact alone will influence other players near you. For this reason, we had pretty strict ideas of what was "right" for a dev to do on Auto Assault. Generally if you played at all, you had to play in a way that would never draw attention to yourself, which meant casual play, essentially.

Likewise, the dev's intended roll in ongoing content plays into what a dev should and shouldn't do in your game world. In both Rift and another project I was working on called Revival, the devs were meant to actively play the parts of key characters in the world, not as players, but as intelligent NPCs. Sort of like what Lady Grace is suggesting with her Yoru king idea. The "Live team" was as much a troupe of performers as they were a group of developers. This meant a high degree of intended interaction with players, but never on equal footing. In fact in both games, originally, playing as a player was off the table for the team. I don't know if that carried through after launch for Rift since we made some pretty dramatic changes in its last year of pre-launch dev that threw that plan out the window, and I never played it as a player after launch, though.

However, with a few exceptions, neither constraint or concern really applies all that much to Chronicles of Elyria. One the one hand, the content team has almost no responsibility for the state of the world - that's managed actively by the game systems and those game systems operate on so many inputs that any single player's actions, not matter how informed, aren't going significantly disruptive. And on the other, our team (again barring a few exceptions) won't be playing key roles as non-players in the world. We put so much stock into our focus on AI because the intent is for the game to not require that sort of activity from the team. We won't have a troupe of performers because we've built the game to not need one. And with the story engine not only generating new activities for everyone, but also deciding when and how those activities are introduced and executed in the world, the team isn't really involved and doesn't really have any more insight into "what will happen next" than any other player.

There are some exceptions - anyone working on the 10 year story will have some idea of the sorts of signs and portents that could precede major events for example - and so at least some of us won't be perfectly free no matter what, but in theory our group of devs should have more freedom since the "safe decision space of play" is much broader for us.

Now, whether that means we'll allow everyone to do everything might not have anything to do with what's safe to do. There are business and community reasons to potentially dive in or stay out of the pool completely. But, mechanically, most of the reasons you'd want to keep your devs out of the pool don't really exist in CoE.

Hope that helps! :)


  • Snipehunter
5/16/2019 12:20:48 AM #5
+0

Snipe's post is encouraging. As long as those devs who are aware of the secrets in CoE are not exploiting that knowledge, it shouldn't impact anyone.

As for how SBS gets valid, reliable data and measures on their game at the highest level, one of the most straight forward answers would be through metrics at the farthest end of the spectrum: who has the highest fame? who has the most murders? who has perma-died the most on the same soul? who are the greatest weaponsmiths?

Ask these people about their respective areas of familiarity and you're bound to come up with some educational patterns. You'll still miss out on some more nuanced and interpersonal questions, but its a start.

5/16/2019 12:49:23 AM #6
+2

Supposedly some of the SBS devs will be playing on a count level, so they will at least get some experience on the economically competitive level. But I don't think feature exposure is the sole problem the EVE developers have, but also feedback exposure.

As it stands the "expert" system that EVE uses is honestly a step above listening to whatever "the loudest" or "most influential" members of the community have to say on any social media as its hard to measure importance and priority for suggestions made "in the moment" so to speak. In addition there's a lot of noise involved with social media feedback, and potentially important issues that don't affect too many just get drowned. Still EVE's "expert" feedback system have quite apparent shortcomings, hence why their devs probably feel the need to use "check-point questions", but any shortcomings should be addressed by making a better system, not going back down a step.

CoE do have an opportunity to be more uniquely feedback driven than most games depending on how they design their research system, making them less reliant on individual "experts". My hope is that they'll allow for customizable research nodes (suggestions that can be "voted on" by continuously researching them) from within the game in addition to any of the research nodes initially seeded by the developers. That way any lingering issues in areas overlooked by the majority may still eventually rise to the top if those interested in changing something continue to put research into their "suggestion".

Feedback through the in-game research system would be far less noisy and more constructive, especially if the nodes description could be community edited (similarly to a wiki), and since the players will actually have to put time and resources into a topic it will allow the devs to have an actual metric in regards to the dedication and popularity behind any given issue, whether the issue have instant popularity or gains strength over time.

5/16/2019 3:26:32 AM #7
+1

Interesting read. I think Caspian mentioned on discord that he had a Count title in some NA Kingdom that was gifted to him?

I feel like devs should be allowed to play the game, but maybe not at kingdom level? Monarchs and Dukes really have a lot of influence on the shape of the world and its politics so having people from the game-company filling these spots is going to stir up so much drama... Even if they don't do anything wrong people are always going to be suspicious and that's understandable.

There's probably no perfect solution for this issue, you definitely want devs to play their own game (not in a testing environment, like, as entertainment); if they don't they will never really understand what's good or bad about it. Good communication between Dukes/Kings and SBS should be key here. Hopefully they can listen to everyone's opinions and take good decisions from that. Because it didn't work for CCP doesn't mean it won't with SBS, let's be a little optimistic here. :-)


5/16/2019 4:35:42 AM #8
-19

SBS employee's with INTEGRITY?

TROLOLOLOLOLOOLOLOLOLOLOLOLOLOOLOLOL

After the "pump and dump" hype for funding over DSS, I seriously doubt anyone believe's these Devs won't abuse their position. It already happens to people the Mods don't like. Shamefully so.

I fully expect SBS devs to cheat if they play and would be shocked otherwise.


ERROR: Integrity.EXE is missing.

5/16/2019 5:27:35 AM #9
+2

Posted By ShadowTani at 8:49 PM - Wed May 15 2019

Supposedly some of the SBS devs will be playing on a count level, so they will at least get some experience on the economically competitive level. But I don't think feature exposure is the sole problem the EVE developers have, but also feedback exposure.

As it stands the "expert" system that EVE uses is honestly a step above listening to whatever "the loudest" or "most influential" members of the community have to say on any social media as its hard to measure importance and priority for suggestions made "in the moment" so to speak. In addition there's a lot of noise involved with social media feedback, and potentially important issues that don't affect too many just get drowned. Still EVE's "expert" feedback system have quite apparent shortcomings, hence why their devs probably feel the need to use "check-point questions", but any shortcomings should be addressed by making a better system, not going back down a step.

CoE do have an opportunity to be more uniquely feedback driven than most games depending on how they design their research system, making them less reliant on individual "experts". My hope is that they'll allow for customizable research nodes (suggestions that can be "voted on" by continuously researching them) from within the game in addition to any of the research nodes initially seeded by the developers. That way any lingering issues in areas overlooked by the majority may still eventually rise to the top if those interested in changing something continue to put research into their "suggestion".

Feedback through the in-game research system would be far less noisy and more constructive, especially if the nodes description could be community edited (similarly to a wiki), and since the players will actually have to put time and resources into a topic it will allow the devs to have an actual metric in regards to the dedication and popularity behind any given issue, whether the issue have instant popularity or gains strength over time.

I should have included more context with this quote. It was the best I could find but it didn't really convey the point across as well as I had hoped. The problem with the EVE devs listening to the CSM was that the CSM members were elected by the playerbase at large. While this sounds good at first blush, what happened routinely was that the large null security blocs would vote their own candidates in to try and skew any possible changes towards their own playstyle. People that lived in my area (FW/lowsec) were lucky if we had even one rep that would be knowledgeable about our playstyle, versus the maybe 6 or 7 from nullsec that would inevitably get voted in.

The other problem was that, because this was essentially a popularity contest, people would get elected that honestly didn't have a clue about things like large scale PVP or alliance wide logistics. Now if the devs had firsthand knowledge of the game, they would be able to figure out in short order that a lot of these guys were completely full of crap. But since they didn't, we'd frequently get harebrained ideas thrown out during CSM sessions.

For my part, I think Casp playing at a count level is fine, although I won't deny it makes me somewhat nervous. I think that as long as SBS shadows the monarchs/dukes, and solicits feedback from all of them (lets be honest, people will be biased so you need as many viewpoints as possible), then it should probably work out fine.


5/16/2019 5:36:26 PM #10
+2

Posted By Takeda_Shinukage at 4:39 PM - Wed May 15 2019

I honestly think having them not play is just flat-out the wrong decision and not many studios do this.

I'm not really sure how a conflict of interest even matters unless the management strucure of the company is so loose that a solo dev can on a whim make changes to the game solo w/o any red flags. i.e. if a dev is able to change the live production version of the game w/o it being apart of the current iteration plan w/o raising a red flag or at least logs being generated to somewhere, that company might need to restructure it's release pipeline.

I know if I went in and altered my companies code-base that wasn't apart of the current iteration plan I would get a message the same day asking why I was touching that system when we are working on X.

As an EVE player, I can give some insight into how they could impact a situation, whether consciously or unconsciously. The economy is (almost) 100% player created and driven. Every ship you board is made by players, almost every module you equip on that ship is made by players. Large scale alliance operations could mean fielding hundreds or thousands of these ships fit certain ways at a time. Knowing ahead of time that a ship may be impacted by an upcoming change, or a decision being made to alter the rarity of some component of manufacturing any piece of the ship or equipment could cause drastic changes to availability or financial suitability for various roles. Further, a change to any number of other mechanics, such as simply moving the more valuable moons around gives many opportunities for conflict, which can be unfair if one group or another has advance knowledge. More than one member of the CSM has been banned in the past for abusing knowledge they had because of their role on that council, and devs are simply going to have even more access to information.

I do a lot of roleplaying, online and in person, and one thing I try to stress to people when talking about that topic is that not metagaming can be impossible in a lot of situations. You simply can't unknow something, and that extends to a more mechanical system, like an MMO. If you know this change is coming down the line, how do you behave? Obviously moving to take advantage of a situation you have advance knowledge of, for being a dev, is unethical. Specifically avoiding the area of interest also means intentionally removing the possibility that your alliance might have just happened into an advantageous position on their own, which is unfair to your teammates. In the end, the only reasonable solution is to prevent the devs from being in a situation to impact any of this, in the first place. This means, by neccessity, a dev may have an idea of what it's like to be a line-member in a large alliance, but having to intentionally distance themselves from the behind the scenes leadership process can mean losing some idea of how these higher level things work as the game advances and tactics evolve.


Knight Chancellor of Ironholde - Servant of Serverus, Lord of Ironholde - Darkholm Dutchy - Kingdom of Blackheart

5/16/2019 6:29:38 PM #11
+0

very interesting to read, i wonder where this convo is going, saved it in favourites :P


Arbiter of the Elyrian Marketplace...…………. Athorias#7637

5/16/2019 8:24:58 PM #12
+0

From a development point of view this is my view:

The SBS devs could follow somebody who streams via twitch or whatever (private) channel and use some extended logging for that specific player with their logging tools to see what is happening under the hood in case something goes wrong.

Then its vital to follow a few type of players who are in different area's within the game: kings, counts, mayors, bloodliners and also regular elyrians. Adventuring people, explorers, builders and farmers. Just select a limited amount of players who are willing to help on this.

Time box when you are following them so that you limit the spent resources here. This is an excellent way of doing a "sprint review" and see if the implemented changes have the expected effect and you can get direct response from end users. It makes your product stronger. This is the most "Agile" way i think. Involve end users early. Use direct communication (speech, visuals/camera's) over indirect communication (boards, bug report software, mail). 80% of human communication is non-verbal/not written you miss that in the indirect communication channels. Fail fast, fix it even faster. There are always issues you wont find in a stubbed environment, you need real user data, real user interaction and the real production systems being in place to find out what goes wrong. If you have been testing already in a lower environment, what would make you sure you will find other things in production if you'd follow the same patterns to test it?

Having employees in "high" ingame roles themselves have the benefits that you see stuff first hand. But the big negative effect is that as a human you are unable to not influence the game of other players. Thus having direct or indirect impact on what happens in the world you create. Intentional and good impact, but without any doubt also unintentional and negative impact. And that is something to keep in mind. The negative and unintentional impact will if detected be remembered the most and are a danger to the view of your playerbase towards your trustworthiness as a dev crew.

By following end-users and monitoring the behavior "under the hood" you are not intentionally or unintentionally affecting the impact on the game world. But you have the chance to involve the user early and have real user experience as close as you can be, without pushing the buttons yourself.


5/16/2019 9:04:35 PM #13
+1

I think so long as devs aren't finding unique items/features etc and leaving that to players there isn't much problem here. Caspian invading a neighboring count would be one thing, Caspian being the first to "discover" a new continent is something entirely different.

It's important that SBS employees act with a certain level of discretion and restraint to preserve the game for the rest of us.


Friend Code: 1C7762

5/16/2019 10:46:11 PM #14
+0

I will echo Goros' comment that it is impossible to un-know something, and that can direct behavior even on subconscious levels.

My greatest lingering concern is that too much freedom allows a scenario where some dude in the COE Art Department overhearing another SBS developer saying "yeah I moved the highest concentration of gold deposit to under this mountain", and even though he's not part of the gold deposit development per se, the Art Department guy now knows, simply by virtue of his work environment, about gold deposits and all kinds of things he couldn't possibly know otherwise.

It doesn't have to be intentionally malicious or exploitative to still be a problem.

5/17/2019 12:18:58 AM #15
+12

There's a common idiom in the security world called "Security through obscurity." The idea is that you obfuscate or hide the inner workings of a system to the degree that, unless someone has insider knowledge of how a system works, it's practically impossible to exploit.

Generally speaking, security through obscurity is a bad thing however and is no longer relied upon in truly secure systems.

It's much better to build a system such that even if someone has the source code to both ends of your connection, it still wouldn't matter. In this case, knowledge is far less powerful as the systems themselves prevent you from doing anything about them, even knowing the inner workings.

There's a comparison here to the way most MMOs are designed and built. The key takeaway being that, with CoE being skill-based, requiring character knowledge to be able to unlock certain types of progression, and with the world being procedurally generated and none of the developers really having access to or knowledge of when/where/how events are going to unfold, the game depends very little on obscurity.

We too must spend the time/energy going around learning what we need to, and we too must figure out where to look. Likewise, we too must be skilled in combat, etc. and simply having intimate knowledge of how combat works doesn't buy us the hand-eye coordination or timing required to be good in combat.

Not to say developers participating in a competitive, multiplayer games is never a problem, or ever in CoE - in fact, the 10 year story is the one area where the developers are forbidden from participating, but in general, CoE isn't really the type of game that would make it hard for us to play with other players.

It's sort of like if we made an online chess game - just because we made the game doesn't mean we'd have any inherent advantage in playing chess.


Log in to post