How to send 1.2 million pixels

The Pixenomics board size is 1200 pixels by 1000 pixels with an entire size of 1.2 million pixels. The data associated with that is the color (6 hex characters), the cost (4 characters) and the owner (up to 6 characters). This means roughly 16 characters per pixel.

The first thing we changed is how this data is represented. First of all, all the data is converted to hex. Second we looked at how we could make the values smaller. The cost is stored in cents but only in increments of 10 cents. Therefore we can chop off a character by storing the cost in amounts of 10 cents. The owner ID was decided to be 4 characters so the largest value would be FFFF in hexadecimal or 65,535.

Originally the pixel data was sent with it’s location so that the pixels that didn’t have any data associated with it, weren’t sent at all and therefore saved space. But this introduced even more data to send with every pixel. The location would at most be 4 characters for the x postion, a 1 character delimiter, and another 4 characters for the y position.

We changed this format so the pixel data is sent in the order that it appears in so the pixel location can be pulled out. This means that we have to send a character as a delimiter to indicate the absence of a pixel and will at the very least be 1.2 million bytes of data (or 1.2 megabytes).

However, this format is perfect for compression because it can just say “use this delimiter for the next million characters” and we end up with a tiny amount of data to send.

But we want the data to be up to date and grabbing the entire board every couple of seconds is a very inefficient way to update the board. To solve this we introduced logs. Whenever a change is made, a log is stored with a timestamp. So the client can just ask the server “What has changed since the last time I checked?” and it can either return nothing or the logs which tells us everything we need to know to keep the board up to date.

There are some cases where this works against us. After every cycle there are lots of changes so the log data might end up being larger than the board itself. So in these cases we just tell the client: “you’re better off grabbing the entire board than reconstructing the many many changes”.


Blog comments powered by Disqus