Good topic.

All of the map data available during D&SS will be collected and analyzed by a whole list of folks in various groups for various purposes. That will have an impact on the site.

If the studio is able and willing to provide the map data in an approved and consolidated format, it would definitely simplify any actions taken to reduce the impact of scraping the site.

Question from me not knowing any better.

What would stop these folks from downloading the map they are interested in and then just scraping it there and then on their own equipment?

Posted By DracoKalen at 08:39 AM - Thu May 30 2019

Question from me not knowing any better.

What would stop these folks from downloading the map they are interested in and then just scraping it there and then on their own equipment?

It wouldn't, that's kinda OPs point. One of the problems SBS has been facing recently is DDOS attacks, where someone is using a large number of computers to continually access the site to slow it down or crash it (using "bots"). The situation the OP is worried about is those people and their bots continuously accessing the site to pull all the gigabytes of data to their computer at the expense of regular users like you and me being unable to access the site (because of the bandwidth being taken up by the bots). This is what the OP meant by "scraping". His solution is to let everyone download that data themselves from a source that isn't on the site, to avoid that kind of traffic.

Posted By DracoKalen at 4:39 PM - Thu May 30 2019

What would stop these folks from downloading the map they are interested in and then just scraping it there and then on their own equipment?

The issue would be that large scores of people would download the entire map in an automated fashion. This is what I mean by scraping.

The only thing that holds people back is the threat of account suspension. This will not scare off tech-savy people, who can hide themselves well. In the end, they will be able to find the answer some statistical questions at the cost of server resources.

A torrent with raw data would provide answers to these questions to everyone, while not pressuring the server.

--Edit: Essentially what BioRules posted. Thanks!

I think this is especially necessary if SBS is not providing extremely robust searching and filtering. If a mayor can't search for all settlements in X or Y biomes with a sustainability of at least Z and the executioner listed as a profession, the impact on the site is going to be tremendous both from scraping and from thousands of people clicking settlement after settlement! This is an opportunity to let the community build the tools needed, as long as you can provide the data in a torrent!

OK. Thanks!

I think the original idea was to give the maps with this kind detail (see image below). Never in pixel accurate detail like examples of map H. Or has this been changed?

Anyway, I think maps should be revealed looking like the one above. This should prevent pixel accurate resource etc gathering from maps?

How about settlements? Give the data for us in JSON-format which includes same data as UI provides. No hidden data. Those who want and can, can create whatever they want from the data, but the same data is available for everyone if they click through the settlements via SBS's UI. SBS UI should include slider and selector filters to filter settlements with user given filter options. This would help people to find settlements suitable for their needs without clicking through everything or relying other players to create filterable maps for them. Map filtering shouldn't be too hard to implement, perhaps half a day coding.

I think the key thing here is NOT revealing the map in details like example map H.

Revealing high detail maps would also hurt the game a lot since revealing resources etc in pixels would mean that kings/dukes/everyone have satellite imagery of their enemies backyard which reveals their resources, map elevation, whatever, basicly almost everything. Would this be fair? I don't think so.

There are other things as well which speak against high detail maps as well.

SBS - please don't reveal high detail maps! Even if you want to show us how amazing your maps are and the water flows etc.

Edit: removed web traffic stuff... Shouldn't be a problem.

I think you're forgetting that although we may be initially presented with something like the above, the GUI will allow drilling down to Duchy and County levels.

With just shy of 2000 (is it still 1728?) counties per 6 kingdom map there is a lot of data contained therein.

Even though the data presented is likely to be pictographic it's data nonetheless.

Cartography profession...

.... and it's gone.

Good topic - valid concern.

Would it be smart to offload the entire public map-viewing endeavor to a separate invite-only domain? Separate from the actually picking of settlements?

On topic:

A torrent may not be considered accessible for every part of the community.

Detail-wise: I think at the county-level you may be conceptually overestimating the amount of exacting data. I think relative amounts may be compared map-wide, but nothing like pixel countable data.

So inferences of relative amounts of resources, land and relative proximity, but nothing future game-breaking or cartography breaking.

Well to minimize time and effort and bandwidth. I was hoping you could just download smaller chunks.

For instance: As a mayor I know what server I'm on so I don't need to download all of the server maps

I know what kingdom I'm going to so I only need that kingdom

I know what duchy I'm going to...at least I'm 99% sure. There's only one other area in another duchy I would consider.

So, what would be optimum is to start on a linked web page where I could select server, that would drop down kingdoms (text). Select the kingdom I'm going with. Then select duchy and click "download" for the map

That way I'm only downloading the piece I want, and later after I've gone through the map, if I can't find a settlement to my liking I can download another duchy. But that is still way less bandwidth and time.

The next question is, if we're still in the dark on DSS timing in general, how much longer would it take to build the infrastructure for this torrent tool.

Posted By Lord_Greystoke at 09:55 AM - Thu May 30 2019

Cartography profession...

.... and it's gone.

That’s awfully dramatic, and it’s not a problem that is created by this topic.

All I want is the data that was shown to us in the previous shiny: major industry, sustainability, professions, etc. Nothing that they are not already planning on providing.

I appreciate all the replies!

Some post mention how the data, the information will not be vast. Keep in mind that I'm not just talking about the duchy, county and settlement information but also the map itself.

Think about distances from settlements to one another, Proximities to rivers, distances between settlements and specific kingdom borders, beach length, et cetera. This is all valuable information that can be derived once the maps have been scraped.

Posted By Xarkfleur at 6:21 PM - Thu May 30 2019

A torrent may not be considered accessible for every part of the community.

A guide on how to download this torrent would be nice. Can be explained in a few sentences.

Posted By DracoKalen at 6:38 PM - Thu May 30 2019

Well to minimize time and effort and bandwidth. I was hoping you could just download smaller chunks.

Depending on how the files are laid out in the torrent, you can choose not to download everything but only a small segment.

Integrating it into the website and/or creating data segments would be far more time consuming than a torrent with raw data. I understand how your solution would help most people a lot, but I would expect it to lead to significantly more development time.

Keep it simple stupid.

Posted By Vucar at 6:58 PM - Thu May 30 2019

The next question is, if we're still in the dark on DSS timing in general, how much longer would it take to build the infrastructure for this torrent tool.

Fair question. If the data is present as files, it takes only a minute to convert it to a torrent.

The map tiles will probably already be files as this integrates nicely with websites and tiled map systems in particular. But I could be wrong on this.

The data files might take some effort to create, as this information is probably stored in a database. Virtually all databases have export capabilities so I cannot imagine this being significantly time consuming.

But this is all speculation.