The Data Map
Data Map Segments
Data Map Specks
Laying Out your Data
Skipped Bytes
The Data Flush Mechanism
The Error Detection and Correction Mechanism
Putting It All Together
This overview describes the mechanisms Dasyne uses to synchronize data, and defines the terms you'll see in the rest of the manual. If you're just starting with Dasyne, it's a good idea to read this section first. Dasyne's underlying mechanisms are relatively simple, and once you understand how Dasyne works you'll be able to configure your networking for optimal performance.
The data map is an array of bytes. It contains all data that needs to be synchronized between the server and clients. The server will create a data map, and clients will create data maps of the same size when they connect.
Dasyne uses special SyncData variables that automatically synchronize data between the server and clients. Each SyncData variable contains a pointer to a location in the data map. When you retrieve the variable's contents, it accesses the data map, extracts the bytes, and uses them to construct the variable, which is then returned.
Whenever a SyncData variable is created, its pointer is set to the next available free space in the data map. This means that the order in which you declare variables will affect the data map region each variable gets, which makes it very important to declare variables in exactly the same order on both the client and server machines. Otherwise, variable A on the server might correspond to a different variable on the client. Then whenever the server updates the data map location for variable A, you'll be wondering why the client's A variable isn't changing. (And why some other variable is getting A's values)
The data map is split into distinct areas called segments. Segments range from 1 to 30 kilobytes in size, and are used to detect synchronization errors in the data. Each segment will compute a 4-byte CRC value for its data. The client will periodically send its segment's CRC values to the server. If they don't match, the server compresses the mismatched segments and sends them over.
Each segment is further split into small groups of bytes, called specks. A speck is the smallest amount of data the server will send when it's updating the data map. When a byte in a speck is updated, we know that the rest of the speck was also updated. Specks are needed for a few reasons:
- We want to make sure that the entire SyncData variable is updated. If the server should update one byte in the variable, but not have enough bandwidth for the rest, the variable will now contain garbage. This means that the number of bytes in a speck must be greater than or equal to the size of the largest variable you want to synchronize. (Otherwise the variable won't fit)
- The server will only send the parts of the data map that changed since the last update. So, it needs to tell the client which parts it's sending. Dasyne does this with a two-byte value that gives each speck's index in the segment. If we were using a speck size of one, we'd have to send two bytes of index for every one byte of data, which is very inefficient.
This business with the indexes sounds wasteful, but it can actually save you bandwidth. Because we only have to update the data that changed, the amount of index data we send may be less than the data we don't have to update. Also, all update messages are compressed, and sequential indexes tend to compress rather well.
As a general rule, you'll want to declare frequently-changing variables at the same time, so that they have adjacent locations in the data map. For example, imagine you're designing a first-person shooter. Each player has a 2-byte x and y variable, a 1-byte speed variable, and a 1-byte hitpoint variable. You've set your speck size to be four bytes.
You would probably want to declare the x, y, direction and speed variables together. If a speck contains an x and a y coordinate, the server will update y whenever it updates x, which it probably would have had to do anyways. But if that speck contains an x coordinate and a hit point value, the player's hit points are needlessly updated whenever their x-coordinate changes.
After x, y, speed and direction variables have been declared for all players, you would then declare variables that don't change very frequently. Player hitpoints, the weapon they're wielding, the number of kills they've achieved, etc.
Dasyne automatically arranges variables so that no variable overlaps onto two specks. With a speck size of four, if you declare a two-byte x coordinate, followed by a 1-byte direction value, there will only be one free byte left in the speck. If you should declare another 2-byte coordinate, Dasyne will detect that a speck overlap would occur. It skips to the next speck, and places your variable there.
Skipped bytes consume RAM, and can reduce the efficiency of the networking. The DasyneServer has an unusedBytes function you can use to tell how many bytes were skipped in the data map. Ideally, this should be zero. If a lot of bytes are skipped, you might want to re-arrange the order you're declaring the synchronized variables, or alter your speck size.
Whenever the server changes one of its synchronized variables, the speck containing that variable is marked as changed. The server will periodically send these changed specks to all clients. This is called a data flush, and is used to send recent changes to all clients. You can configure the number of data flushes that you want the server to send, typically around 10 times per second.
Data flushes are unreliable - if a change wasn't sent because of network congestion or wasn't received correctly by the client, so be it. Data flushes are best sent over UDP, which reduces lag as we aren't wasting time resending stale changes.
But hold on - what if a change isn't received by the client, and that variable is no longer being altered? The server only sends out specks that changed since the last update. The client will be stuck with an incorrect value that isn't being updated!
The Error Detection and Correction Mechanism
A second mechanism is also synchronizing the data map; the Error Detection and Correction mechanism. (EDC) As mentioned earlier, the data map is split into segments. Clients will periodically send EDC queries to the server, which contain the CRC values for each segment.
If the server detects a mismatch, the entire segment is compressed and sent to the client.
When clients modify the data map the changes are temporary, and will be undone after a few seconds unless confirmed by the server. This temporary modification allows the player to see the changes without having to wait for the server to send them. The basic Dasyne usage goes something like this:
-Player hits the forward key
-Client modifies the player's coordinates in the SyncData variable (temporary)
-Client sends the forward command to the server
-Server receives the command, modifies the coordinates
-The new coordinates are sent to all clients in the next data flush
-The original client receives the coordinates, confirming the change
Currently, sending the player's command to the server is best done with the sendUDP function.
Now that you've read up on data maps, segments and specks, you're ready to write a networked game with Dasyne. Congratulations!