Category: Techniques

How to measure the area of a circle

How to measure the area of a circle

Odd ShapesWell if you remember maths (or math for US readers), you’ll know that the area of a circle is pi X r X r where r is the radius and pi is that 3.14159 number.

But what if you are given a weird shape instead of a circle? Like some of those in the picture.

There’s a method called Monte Carlo because of the casino there. In effect you simulate throwing darts at the shape (say printed out and stuck on the wall). You throw thousands or millions of darts and count how many hit the particular shape and how many darts were thrown in total. You only count those that hit the rectangular paper; those that miss it don’t count.

Then you divide the number of hits by the number thrown, multiply it by the width and height of the picture (which is rectangular or square) and that is a close approximation to the area. In more advanced ,maths/math, this method is sometimes used to calculate the integral of a function. The integral is the area under the function. Some complicated functions are particularly hard to integrate, so using a Monte Carlo method can solve it for you.

I’ll publish a C program to run Monte Carlo on an image in a day or two.

An alternative way of measuring the area is by using recursion. I discussed it in this article Using recursive fill to count maps.

How many riffles are needed to shuffle a deck?

How many riffles are needed to shuffle a deck?

card riffle photoIt fascinates me because 52! is such a large number.  Here it is in full 8.0658* 10^67 or 80,658,175,170,943,878,571,660, 636,856,403,766,975,289,505,440,883, 277,824,000,000,000,000. That is the possible number of ways of shuffling a pack of cards.

It means that when you shuffle a deck of cards, it’s possible that you are the first person on Earth to ever get that particular arrangement. It’s the kind of fact that amazes me. Another one is that it takes a very long time for particles emitted from the centre of the sun to reach the surface and blast into space. On the order of many many years. (Thousands of years!)

Playing cards have only been around maybe 500 years as we know it (52 card deck) though date back to 9th century China for their invention. If there had been a billion shuffles each day during that 500 years, that’s only 1.8 x 10^14 shuffles. That is a minuscule fraction of the possible number of arrangements so the chances are that any shuffled arrangement is new is pretty high.

It’s accepted that seven is the number of riffles needed to perfectly shuffle a pack of cards. A riffle is where you split the deck in two and then merge the two halves back into one deck as in the photo I took.

I proved this once by writing a program to simulate riffles  and looking how far cards have moved after seven. In fact a card at the top of the deck moved to the bottom after only six riffles. I’ll try and write that in C and will publish it here in a day or two.

Other shuffling techniques like smooshing (spreading out all the cards on the table with their backs face up and then pushing them together) are nowhere near as efficient. It’s estimated it can take thousands of smooshes to properly shuffle a pack. It’s not easy to simulate, though one of these days I’ll have a go and see if I can come up with a more accurate estimation.

How to install SDL2 in Visual Studio

How to install SDL2 in Visual Studio

Visual Studio IDEThis is the first of a number of longer-piece game related tutorials. You’ll see I’ve added a tutorials link to the top menu. That page will grow as I add tutorials, as each is added to it .

You’ll see I use the terms SDL and SDL2 mostly interchangeably. SDL is the name of the library but we don’t want the older SDL1 instead we want SDL2 which seems more or less permanently at version 2.0.12.

I’ve left the Visual Studio version off as the process is mostly the same whether it’s Visual Studio 2017, 2019 or future versions. Screenshots are from Visual Studio 2019.

You don’t have to do this on Linux as it takes three or four sudo apt installs to add the various SDL2 dev modules in, but with Windows you need to configure Visual Studio and it can be a somewhat confusing process if you are new to Visual Studio.

Also you need to download and fetch the various files. This is made slightly more complicated because there are 32-bit and 64-bit versions and you want to keep them both so you can switch between the two.

Here are the various steps we have to go through.

  1. Download the various files and unzip them
  2. Setup include and lib paths in Visual Studio
  3. Add the lib files into Visual Studio.
  4. Copy the dlls into the folder where the game will run.
  5. Compile and run it.

What you are downloading are basically three types of files.

  1. Header files., This is files like sdl.h.  Your program will #include these.
  2. Lib files. This is the bulk of the SDL code.
  3. Dll files (Dynamic Link Liobraries). Needed for runtime.

Download the various files and unzip them

The SDL website is libsdl.org and if you click the SDL Releases in the bottom right it will take you to the SDL downloads page. WE don’t need the source code (you are welcome to download it and take a look but it’s not needed to use SDL2.

We do however need the development libraries. These include the runtime binaries so we don’t need to download those. as well. Just the one file SDL2-devel-2.0.12-VC.zip

I suggest you create a folder SDL or SDL2 on your drive. My C: drive is for Windows so I use d: \SDL2

If you unzip the files into there you’ll end up with three folders and five .txt files. Just under 7 MB in total. Other than docs there are include and lib.  The lib folder is further split into x64 (64-bit) and x86 (32-bit) . It also includes the runtime dlls. These will have to be in the path for your program but we’ll leave that until later.

Sounds, Images and Truetype fonts

As well as these, you are probably going to want image file support, sounds and possibly truetype font support in your program. These are separate files in the SDL projects folder.

  1. Image file support. Download the development library file from the sdl_image page. As before you want development library file. SDL2_image-devel-2.0.5-VC.zip.
  2. Sounds file support. Download the development library from the sdl mixer page. It’s SDL2_mixer-devel-2.0.4-VC.zip.
  3. ttf file support. Once again a development library downloaded from the SDL_ttf project page. It’s SDL2_ttf-devel-2.0.15-VC.zip.

All three files have similar structure to the SDL2 dev library.  Unzip the include files into the SDL include folder and the lib files into the relevant X86 and X64 lib sub-folders. So all your SDL files that you need are in the same include and lib sub-folders.

I suggest you extract the file folders one by one, do the three include files first then the x64 files and then the x86. Do not get x86 and x64 mixed up. The only way to tell them apart is by size and it’s not always an accurate way.  Don’t worry if it complains about overwriting zlib1.dll. There’s a copy in both the images and sounds zip files.

Setup include and lib paths in Visual Studio

This can be a bit complicated, just follow these instructions exactly.

To configure a C/C++ project in Visual Studio, you have to specify where the compiler gets its include files, where it finds its lib files and which lib files you want to link to.

Solution ExplorerI’ve created a blank C++ project called sdltest in VS 2019.  Now I actually want it to be a C project so just rename the main file sdltest.cpp to sdltest.c. You need to delete all of the C++ code in that file as well.  After renaming the Solution Explorer should look something like this. We will have a sdltest program to run later so just save this for now. You can get the file from GitHub and overwrite sdltest.c.

Now click Project on the top menu then sdltest properties at the foot of the menu.  You should see this form (below).  This is how you specify properties for your project in Visual Studio.

You’ll see I have selected VC++ Directories on the left. This is where you specify some of the directories (folders and directories mean the same thing BTW) .

But the Platforms pull down probably shows Win32 on yours. Change it to All Platforms. Visual Studio lets you specify configurations for all things or for x86 or x64 separately. We’ll use the same include folders for both x86 and x64 but we’ll specify the paths to the lib files individually as the x64 lib files are in the x64 sub-folder and the x86 files in the x86 folder.

Property Pages
To specify the path click on Include Directories, you’ll see a down arrow appear on the right.  Click it and you’ll see <Edit…> appear, click it and a form like this below will popup.
Visual studio folder editClick on the blue area in the form and you’ll be able to paste or type in the path or click the … button to get a file browser appear. Type in, paste or select the folder then press Ok.  You should now see your path in the folders.

Here I typed in d:\SDL\Include. Be careful that you don’t get rid of $(VC_includePath);$(WindowsSDK_IncludePath); in the include path as I did as your program won’t compile!

folder paths

We now have to do the same for the lib paths.  But first we must change the Platform to specify x86 or x64.

If you change it, a popup will appear asking if you want to save your changes. Click the Yes button.

Confusingly the platform choices on mine are Win32 and x64, but Win32 is the same as x86.

You’ll see that the include path you added shows up in the x86/Win32 platforms because we changed it for all platforms.

Now add the path for Library Directories. Click the down arrow then <edit…> and put in the full path to the folder that matches the Platform. x86 for Win32 platform, x64 for x64 platform. After you’ve entered it will show up in the directories.

Visual studio all paths

If you want both x64 and x86 then change the platform and re-enter it. Don’t forget to save!

Add the lib files into Visual Studio

The last configuration to do is specify the lib files that are needed. We’ve specified the paths for include and lib files but the compiler linker doesn’t know what lib files to link.

As linking depends on 32-bit or 64-bit we have to specify this twice as we did for the lib paths. It’s in a different place in the property pages. Click Linker then Input.

Visual Studio Linker Configuration It’s the top line (Additional Dependencies) that we need to work with. Click into it to get the down arrow then click that and the <edit…> as before.

You’ll have noticed that it comes pre-populated with all the various library .lib files.  We’ll be adding some more. The ones we need are

SDl2.lib SDL2_mixer.lib SDL2_ttf.lib SDL2_image.lib and SDL2main.lib

Add these into the edit box one by one and press return after each one.

After you’ve added them and pressed Ok, you’ll see them in a list. Something like this though I’ve not added SDL2_ttf.lib in to it yet.

As before repeat for both x86 and x64.

 

We’re now ready to compile. Only we need a program to do that. I’m not going to list the whole sdltest as it’s 135 lines but you can download the VS project in the file sdltest.zip from GitHub. It should compile with no errors. If you get errors, please recheck the include folders and lib folders and make sure you have configured them correctly.

So it compiles, but it won’t run. If you look in the Debug folder under the x64 (or Win32 if you built that) , you’ll see a whole lot of files. but only sdltest.exe is important. You can delete the rest. Leave sdltest.pdb if you wish to debug.

We have to

Copy the dlls into the folder where the game will run

That folder is this the \sdltest\x64\Debug folder. We need several .dll files from the same lib folder that holds the x64 libs. (Again if you are on Win32 you need dlls from the x86 lib folder).

What files do we need?

Just SDL2.dll. If we were using images we’d also need sdl2_image.dll and zlib1.dll. We don’t currently need the SDL2_mixer.dll or the SDL2_ttf.dll but if you ever use sounds or Truetype then you’ll need those. For sounds you’ll also need some of the lib*.dll files such as libogg-0.dll or libvorbis-0.dll. For image we might need in the future libjpeg-9.dll (if we ever use jpg files).

So you’ve compiled it and should see something like this when you run sdltest.exe. It doesn’t do much except draw coloured rectangles. Press the esc key to close it. On my POC it draws 100,000 coloured rectangles each 120 x 120 pixels in about a second. That’s pretty fast!

The heart of the program is this function:

void DrawRandomRectangle() {
	char buff[20];
	SDL_Rect rect;
	SDL_SetRenderDrawColor(renderer, Random(256) - 1, Random(256) - 1, Random(256) - 1, 255);
	rect.h = 120;// Random(100) + 20;
	rect.w = 120;// Random(100) + 20;
	rect.y = Random(HEIGHT - rect.h - 1);
	rect.x = Random(WIDTH - rect.w - 1);
	SDL_RenderFillRect(renderer, &rect);

	rectCount++;
	if (rectCount % 100000 == 0) {
		SDL_RenderPresent(renderer);
		stopTimer(&s);
		sprintf_s(buff, sizeof(buff), "%10.6f", getElapsedTime(&s));
		SetCaption(buff);
		startTimer(&s);
	}
}

Uncomment the two lines with 120; // Random(100) + 20; to have it draw random sized rectangles. The figure in the caption is how long it takes to draw 100,000 rectangles.

Completely off-topic but how to tether an iPhone to Windows 10

Completely off-topic but how to tether an iPhone to Windows 10

A PC tethered by two phonesThanks to an apparent hardware failure in my internet modem (power light remains stubbornly off), I found myself without internet except on my iPhone which has 20 GB of 4G data allowance.

Normally in a month I use less that 0.25 GB of the 20 GB $G monthly allowance data because of fast WiFi at home and at work but I do use it for phones and messages.  It’s convenient on rare occasions when I’m out though to have internet access ability on a phone.

Unfortunately for reasons best known to Microsoft or Apple or both, iPhones no longer seem to tether with Windows 10.  They did for Windows 7 and it’s a real PITA.

But there is a way to manage it. Just use an Android phone as well. If the Android phone had mobile data then no problem but even if it doesn’t, it can tether to my iPhone over WiFi. And my PC can tether to the Android phone using a USB cable and that is how this is being brought to you.

How I track ideas

How I track ideas

About twelve years ago I came across WOAS (= Wiki on a Stick). It was a single HTML file notebook with built in JavaScript and you opened it in a browser. It showed a simple interface and you could switch it into edit mode and start adding in text in markdown format. It was very easy to create links to pages in the document and add new pages.

And it was all saved in the document. Also very easy to find the text. It had one excellent feature which I miss, You could have a page self-encrypt using AES-256. Great for holding passwords etc.

I used WOAS to deign software and had at least one very detailed design with several hundred pages each with links to other pages. Sadly WOAS was eventually killed by the browsers themselves, preventing files writing to disk. It’s not the only example of a single-file Wiki but they all suffered the same problem. I think someone came up with a Java shim that did the file writing but it was never the same.

Now I use WikidPad, it’s written in Python and I think just runbs on Windows, (though you wouldn’t know it) which works like WOAS but internally it uses SQLite to hold the Wiki pages.  You can even use their website WikidPad but you have to register first.  That picture is a screenshot of WikidPad.

An example of a game idea

An example of a game idea

Sometimes I wake up in the middle of the night and my brain is bursting with ideas. I’ve just finished reading Shaman by Kim Stanley Robinson that’s set 30,000 years ago and came up with ideas for a tribes game very loosely based on that (Absolutely nothing to do with the plot of the book!) .  I opened up my iPhone around 3:00 a.m. and these are the notes I wrote down in the Notes app.

I opened my email (Gmail) which picks up notes from my phone automatically, so I have the notes now here in my PC.  They are as I wrote them so they are quite rough.

“Tribe game. All players start with a small village and 50 tribes people.
Can relocate, hunt, plant etc. Can raid other tribes. Can split tribes spread over land. Have children. Found religion. Use animals. New options as tribe progresses. Discover and trade gold, furs. Tribes can move to other places or settle and form villages.
Raids build up resentment between tribes. Enough resentment leads to war. But can reduce resentment by gifts of women, food, furs etc st festivals.
Can do crafts, paint, culture, spirituality, warfare, building. Each has 1-10 levels with names. Orders focus on an area.
Better food means stronger, smarter.
Periodic festivals. Tribes get together and share attributes.
Run for 1,000 years. 100 turns x 10 years. “

And that’s how they go into WikidPad. Then I start fleshing them out into a fuller multi-player game.  Of course I come up with lots of ideas and few will make it into an actual game creation but its always a good idea to keep them. In the future it may yet become a game…

And how was the book? Very good.

Geomorphing for the masses

Geomorphing for the masses

Geomorphic village by davesmapper.comThis is very clever. The website Davesmapper.com lets you create different type of geomorphic terrain maps, whether it be dungeons, city or village or even the interior of a spaceship.  You get a choice of tile sets according to what type of terrain you’ve picked. If you are artistic, you can upload your own designs. This mixing and mashing of tile sets can lead to some very odd looking cities with each tile a different style!

The view options controls lets you change how many tiles wide and high your map is, but watch out, the tiles are quite big so go above say 16 x 16 and you start to generate maps that are too big to export.  The village at the top of the post was 2100 x 2100 pixels when exported.

The terrain maps generated are very nice. If you think these look a bit rough, it’s probably my fault. To speed up this website all images are both shrunk to typically 350-450 pixels wide and compressed and some images are possibly not quite as nice looking as their uncompressed originals.

This city below shows you the type of thing with two greenish tiles and a bluish one. You can untick the tile sets that you don’t want.

Geomorphic city from davesmapper.com

Geomorphic means it’s made of tiles that are almost always square and they fit together so that doors or passageways always join up.

Back in my D & D (Dungeons and Dragons, not drunk and disorderly!) dungeon mastering days (about 1979) I use to have a set of physical geomorph tiles. They were great but sadly lost along with my D & D toolbox that had all my figures.

There are some computer games where you might use geomorphic graphics; perhaps a procedurally-generated dungeon crawler.

MonoGame – How to make a clickable button

MonoGame – How to make a clickable button

Clear Button with one card clickedI’m using this in Android games but the principle applies to any MonoGame game. Here clickable and touchable mean the same.

You need three things to make something clickable.

  1. The area. For simplicity sake this will always be rectangular.
  2. The Action. When you touch (or click it), you want it to run code.
  3. An id of sorts. If you have multiple buttons etc, you meed to distinguish them.

We’ll use an int for 3. Use const ints to give it a name rather than being a magic number.

 

When you click whatever, the Event should include the ide so let’s define the event Type first.

public class ButtonClickedEventArgs : EventArgs
{
    public int Id { get; set; }
}

Not rocket science is it!

Now lets define a ClickButton class.

public class ClickButton
{
    public int Id { get; set; }
    public Rectangle TouchArea { get; set; }
    public Action<object, ButtonClickedEventArgs> AnAction { get; set; } = null;

    public ClickButton(int id)
    {
        Id = id;
        TouchArea = new Rectangle(0, 0, 0, 0);
    }
}

Again nothing too complicated. There’s an int Id which which is set in the Constructor, a Rectangle TouchArea and an Action which is defined to include our ButtonClickedEventArgs.

Here;’s some example code. I’ve defined a graphic for a Clear button in my game and this is loaded as usual in the LoadContent().  I’ve also defined the button in Class declaration.

ClickButton ClearButton;

In LoadControl(), this is also initialised. Here’s it’s two line. It could be done in one, by altering the constuctor.

ClearButton = new ClickButton(1);  // 1 is the id
ClearButton.AnAction += ClearCards;

ClearCards is a method that does something when the button is cleared. This matches our Event handler as it has the usal two parameters for an Event except the args is our ButtonClickedEventArgs.

private void ClearCards(object sender, ButtonClickedEventArgs args)  {
   // all the code here, not shown
}

Where is the TouchArea defined?

So are we all setup ready to rock’n’roll? Well not quite. We haven’t said exactly where the TouchArea is. That of course depends on where we draw the button. In my game, the ClearButton moves across. It’s absent when all six cards are present but as soon as one has been touched and a back card touched to move it, the Clear Card appears. In the top image you can see it to the right of the five top cards. I had clicked the seven of clubs when it was in the top row then the first back on the top row of the three rows.

For my next move I clicked the ten of diamonds and the second back on that first row where the ten of diamonds no wit. It now looks like this and the Clear Button has moved over.  If I click the Clear button it will move the seven clubs and ten diamonds back to the top row and replace them in the first row with backs.

Second clickTo set the TouchArea of the ClearButton, I do it in the Draw method. It’s as simple as this:

if (hand.Count < 6) // Draw ClearButton on end
  {
    spriteBatch.Draw(clearButton, new Rectangle(x, y, cardWidth, cardHeight), 
        new Rectangle(0, 0, cardWidth, cardHeight), Color.White);
    ClearButton.TouchArea = new Rectangle(x, y, cardWidth, cardHeight);
  }

There’s actually a small bug visible. When you click a top row card, the game highlights it by reducing its opacity to 0.5. If you look at the ten of diamonds, you can see it still has that reduced opacity; it should have been restored to 1.0 when it was moved to the first row. The same is true in the top picture for the seven clubs! Easily overlooked but just a one-line fix.

Some Touchy Code

The very last bit is to fire touch detection and this is done in the Update() method. When you touch the screen, calling Touchpanel.GetState() returns a collection of touches. This is my code. It cycles through the touches collection, gets the X, Y position of each touch and adjusts it to match the virtual screen coordinates. (Drawing to a virtual screen means this looks the same on every Android irrespective of its physical screen sizes and resolution).

touchState = TouchPanel.GetState();
foreach (var touch in touchState)
{
   if (touch.State == TouchLocationState.Pressed)
    {
         var x = touch.Position.X / xRatio; // adjust for virtual screen
         var y = touch.Position.Y / yRatio;
         CheckClickedTop(x, y, hand);
         if (pickedCard == null) return; // no point checking further...
         for (var i = 0; i < 3; i++)
            CheckClickedBottom(x, y, commonCards[i], true);
            if (hand.Count == 6) continue; // All top cards so no ClearButton visible
            // Check if Clear button
            if (ClearButton.TouchArea.Contains(x, y))
              {
                  ClearButton.AnAction(ClearButton, new ButtonClickedEventArgs() { Id = ClearButton.Id }); 
              }
      }
}

The top cards are held in a class called hand and the bottom three rows are held in CommonCards[3] each of which is a hand. The CheckClicked.. methods cycle through the cards in the hand comparing coordinates to see which card if at all has been clicked. If it has it calls the AnAction Event for the one clicked. The very last if shows how this works to check if the ClearButton was clicked. I’ll simplify this code by moving it into a ClickButton method.

In praise of Hyper-V

In praise of Hyper-V

Hyper-V Manager screen I’ve been using virtual machines for years. Originally I started with VirtualBox, the free VM manager from Oracle. I’m not sure why I switched to Hyper-V, about five or six years ago but I’ve been on Hyper-V since then. You need to be on Windows Pro and have at least 8 GB though the more RAM the better. I have 64 GB and the most I’ve ever had in use at one time is 29 GB. I always try to keep RAM use below 50% as there’s less disk swapping.

VMs are an excellent way to try out other Operating systems, install software and venture outside the comfort zone of a Windows PC and Windows 10.  Many of the screenshots published here in earlier posts have come from Ubuntu 18.04 LTS running in Hyper-V. It takes a minute to fire it up and connect then login.

I learnt to use Linux that way, both via the Ubuntu GUI and terminal commands and much of my 2nd Ebook has been tested on a Raspberry Pi OS running in a VM. Of course it lacks the hardware of the real Pi, so I have to test programs on both, but it’s quite a bit quicker doing a screen grab using the commercial Snagit on my Windows 10 PC.

I can do screen grabs on a PI using scrot, but then I have to upload the image using WinSCP or Putty. It’s not the end of the world but when you are doing a lot, having the image in the Snagit editor ready to copy/paste is a time saver. Also in a similar way, make sure you can do copy/paste via the clipboard and can resize the guest OS (as it’s called). It makes a difference.

I have mine configured so its uses RDP and you can just see the corner of the RDP bar in the Ubuntu image below where I’ve just launched Visual Studio Code.

Ubuntu 18.04 LTS running in an RDP session on Windows 10 in Hyper-V

From a terminal I did ifconfig and got

eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 172.21.212.121  netmask 255.255.240.0  broadcast 172.21.223.255
        inet6 fe80::2b71:6d6b:9a09:9aae  prefixlen 64  scopeid 0x20
        ether 00:15:5d:38:01:03  txqueuelen 1000  (Ethernet)
        RX packets 47317  bytes 69248755 (69.2 MB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 27622  bytes 1791175 (1.7 MB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

and on Windows

C:\Users\David>ping 172.21.212.121

Pinging 172.21.212.121 with 32 bytes of data:
Reply from 172.21.212.121: bytes=32 time<1ms TTL=64
Reply from 172.21.212.121: bytes=32 time<1ms TTL=64
Reply from 172.21.212.121: bytes=32 time<1ms TTL=64
Reply from 172.21.212.121: bytes=32 time<1ms TTL=64

This makes it very handy for testing network servers or if you do web development running a web server locally.

How to use indexed sequential files

How to use indexed sequential files

Files
Image by Pontep Luangon from Pixabay

This was a big thing long before PCs existed but you don’t see it so much nowadays.  Say we have a lot of static data, perhaps text strings which can vary in length from a few bytes to hundreds of bytes long. What is an efficient method to store them, i.e. in both space and time?

The answer is ISAM (Indexed Sequential Access Method)  which sounds more complicated than it is. We will use two binary files- one is the data file and one is the index file. The data file holds the raw text strings. We’ll write the raw strings using the C file write. At the same time as write them, we also build up an array of structs. This struct will look something like this:

typedef struct {
  int id;
  unsigned long offset;
  unsigned long length;
} indexRec;

The id could be a char * string. It’s just a way to search for and find the data.

As each text string is written to the data file we populate a struct with the file offset and length. Eventually after all strings are written you can write the arrays of structs to the  index file.

Then anytime you wish to read text strings back, load the structs into an array in ram and scan for the matching id then call fgetpos with offset (this moves the file pointer to the specified byte offset) and read in the length number of bytes from the data file.

An even simpler approach

If you just use an index number then you can scrap the id field and instead store the index file as a collection of offset and length fields. To read string #100 just do a fgetpos to byte at 100 * 8 (four for the size of offset plus four for the size of length)  in the index file, read the eight bytes for offset and length then do a fgetpos in the data file to the offset and read in length bytes as before. Very efficient and very fast. Just two reads in two files and one of those is only eight bytes.

If you wish to change the text strings in the data file, use the same method to retrieve them. If the new string is smaller then just write it and change the length field of the index record and rewrite it. If the new string is longer than the old, write it on to the end of the data file and change the index offset and length fields and rewrite them in the index file.

Both of these edit methods (shorter or longer string edits) will ‘lose’ bytes in the data file. When replacing with a shorter string and adjusting the length, the extra bytes that were in the previous string are no longer referenced. With a longer edit the entire previous string is no longer referenced.

Reclaim lost bytes

However it’s very easy to compact the data file and reclaim these lost bytes. Just process the index file and read each string and write them on the end of a new initially-empty file.  You also change each index file record to have the new offset and length of each string in the new data file. Once it’s rewritten just delete or backup the old data file and rename the new data file to be the one used from now on. Depending on how often the data file is edited, you will need to “compact it” every few days or weeks. But compacting is a pretty quick process, even for mega byte or larger sized files.

I used this technique albeit in Turbo Pascal not C to store game data for postal games I wrote back in the late 1980s. Games that are still run today but on the internet, not by post. For example in a map location there can be several parties of adventurers. The file record for that location has a file pointer to the start of a chain of party file ptr records. Each index file of adventurer parties has a binary file offset to the party in a binary file and a file ptr (or -1 if the end) of the next party’s file pointer. Like a pointer linked list but using file pointers instead of real pointers. Similarly the map location has a file pointer to a dungeon if one exists in this location (or -1 if it doesn’t).

This technique kept the map file fairly small but there were lots of binary file pairs for parties, dungeon, characters (in parties), and towns and shops in towns. Who needed a database? I didn’t… (That’s cos databases didn’t really exist back in the day. Had they existed I might have used them… maybe).

The minus one problem

In the struct definition, you’ll notice that I use unsigned longs for file pointers and length. There is no -1 for unsigned longs. It’s 4294967295. Using this as an end of chain pointer is ok because it is never going to be used as an offset. If I had used signed numbers then I could have used -1 but remember when I wrote this, it ran on a 16 bit computer so I used 65535 instead of -1. I could easily get binary files 40 Kb or 50 KB in size, so a signed number would have overflowed after 32767 bytes.