Hi everyone, Calandiel here.

It's been a while since the last dev diary on itch.io so I thought I'd write an update for this part of our community. This time you can also see the update on YouTube, thanks to Solitarian from Discord, who streamed and recorded the test build. It's a bit long, however, so the more standard, text-based dev diary is still here.

Anyhow, onto the rest of the diary.

The population tab of the tile inspector is functioning again. The game spawns up to 64 tribes at game start, each (usually) between 10 and 50 people. It also creates unique cultures and religions for them.

The tile inspector was expanded by a few new tabs. One of them is the buildings tab. It contains a list of all buildings within the tile, with an entry for every structure. In the future, the UI will be modified so that buildings of the same type form a collapsible list, for easier interaction. The word "building" here is used very liberally. It could be as simple as a piece of cleared land with a designated purpose.

Buildings can have multiple purposes but they all keep track of their inventories, employees and residents. Inventories in particular are kept in great detail, with every building having its own capacity (for both volume and weight), so that expanding cities will be forced to expand their granaries and warehouses to meet the needs of growing populations, especially in areas with winters and perishables. Buildings also keep track of their owners, differentiating between property of the state ("state" is also used very liberally in this context and may be replaced by a different term later in the future) and that of private characters.

Trade goods in SotE have detailed information attached to them, describing their properties for a variety of use cases. Every item has an icon, description, size (volume) and weight. Edible items also detail their nutritional value, which differentiates between calories, water, proteins, fats, minerals and vitamins.

There are also two new mapmodes. Local plants and local animals. They show what types of trade goods can be foraged from tiles, with each unique color representing one unique trade good. There is exactly one such foragable plant-based item and one animal-based item per tile. This may sound limiting but settlements have a "catchment radius" of up to 5 tiles, meaning that they send out foragers up to five tiles away from the center of the settlement. This means that any randomly selected settlement is very likely to have access to all foragable trade goods of a given biome and biogeographic real, as long as its entire catchment area is composed of relatively similar tiles. If a settlement is on a tile boundary, some trade goods may be "missing", but we find this limitation to be desirable as it adds to strategic considerations of settlement placement.

 These local plants and animals can be viewed in the tile inspector, in the ecology tab.

Populations can be viewed in greater detail now, with a separate list of characters. SotE's population model keeps track of every person as a separate agent (with a caveat that each agent represents more than one person in a world with realistic population figures).

Every character stores data describing it's physical and nutritional attributes. They also have personalities, skills and physical attributes but the UI for them isn't ready yet. Nutrition is satisfied by consumption of trade goods from stockpiles the agent has access to (such as foodstuffs stored in the household).

You may also notice that "ACTIVITY" has a value of "SLEEPING" on the above screenshot. The game keeps track of timezones and advances time on ticks shorter than a day. Each hourly tick, characters reconsider their activities, based on their personality, material conditions and relationships with other characters. Lazy characters, for example, work less. Unemployed characters try to find some kind of work. Sleepy characters go to bed and rest until the next day. At the moment, available activities are sleeping, working and leisure but the list will be greatly expanded in the future.

As can be seen on the above screenshot, the game keeps track of ticks shorter than an hour. Careful scheduling of calculations ensures that it doesn't become a computational bottleneck. A lot of calculations only happen at specific local hours or on hourly basis, with only the absolutely necessary ones (like movement of agents on the strategic map) running every tick.

Buildings can serve as sources of employment for characters ("residents" on the above screenshot should really spell "workers", "employees" or something else along those lines but the UI hasn't been made context aware just yet and that same table is used for residents of households). This simplifies calculations for less important characters as they can be processed in bulk and without checking for all possible actions on every tick (it's relatively safe to assume that outside of exceptional circumstances a person working as a hunter won't ponder whether or not they should spend the next three hours trying to be a blacksmith or begging on the streets).

For gathering (and hunting) camps in particular, the job involves going out on foraging trips, usually outside of settlements central tile. The building selects a random tile within settlements catchment area and sends a foraging party, which then returns with foraged goods. Foraging rates depend on a variety of things, such as depletion of local resources, tile properties, attributes of the foraged plant and of foragers (the game performs calculations for spotting, climbing, digging, picking and processing of local plants to figure out the final yield per hour per forager) and so on.

Some buildings are used as households. When characters sleep, they do it in their household, unless stated otherwise or unless they're homeless. Characters also consume households stockpiles first, before they look for resources elsewhere. Households will attempt to keep a few months worth of preserved food in climates with winters, lengthy droughts or other phenomena that could make food a temporarily scarce sight.

Other buildings are used as warehouses or granaries. Spoilage and overall deterioration will be simulated in SotE, taking into account things like temperature and humidity. Different climates and trade goods will necessitate different storage methods.

Lastly, modding of data files the game uses was expanded to accomodate new structs. Races, plants and animals are moddable in the folder with raws, with tools as simple as notepad. The syntax of the current modding language is the same as in Dwarf Fortress, which makes all syntax highlighters for DF also compatible with SotE.

As for work done since the release of Solitarians video, that wasn't covered above, it's been mostly focused on fixing the bottlenecks found in the test build. Especially the bottleneck with the maximum game speed.

SotEs networking ran on something called a deterministic lockstep. It functions by sending inputs to all clients connected to a server and then having all clients simulate the game locally given inputs from all other clients, so that the game doesn't have to synchronize all the gigabytes of the game state -- instead, it only synchronizes the inputs. It was trivial to implement but it had one small issue. When I increased number of ticks from 1 per day to 24 per day and then to 240 per day, it introduced delays caused by the fact that inputs can only travel so quickly over the wire.

Normally, you'd fix it by introducing a playout buffer (this is what I believe Paradox does). It works by having the server send the commands back with a slight delay, while also receiving inputs from clients at all times. That way, ping is less of an issue, but it does add some input lag. That input lag depends on the size of the buffer. When the buffer is small (both in terms of in-game ticks and real-life time), say, 3 ticks and 100ms, it doesn't cause problems. However, as you decrease the real-life time used to play back the buffer or as you increase the number of ticks covered by it, you start running into issues with game speed and input lag respectively. I had to re-implement it so that instead of relying on deterministic lockstep it relies on game state synchronization. I will tackle the issue with the size of the game state by not sending data that players agent doesn't have access to nor data that the player doesn't request in the UI.

To wrap it up (and summarize). The basic behaviors of agents (such as foraging and consumption) are now functioning and performance bottlenecks exposed in the latest test build are fixed so the focus for the coming week(-s) will shift towards population dynamics. In particular, pregnancies and death. After that, attention will be given to migrations of foraging bands and a new test build will be uploaded.