Category: Techniques

In praise of Hyper-V

In praise of Hyper-V

Hyper-V Manager screen I’ve been using virtual machines for years. Originally I started with VirtualBox, the free VM manager from Oracle. I’m not sure why I switched to Hyper-V, about five or six years ago but I’ve been on Hyper-V since then. You need to be on Windows Pro and have at least 8 GB though the more RAM the better. I have 64 GB and the most I’ve ever had in use at one time is 29 GB. I always try to keep RAM use below 50% as there’s less disk swapping.

VMs are an excellent way to try out other Operating systems, install software and venture outside the comfort zone of a Windows PC and Windows 10.  Many of the screenshots published here in earlier posts have come from Ubuntu 18.04 LTS running in Hyper-V. It takes a minute to fire it up and connect then login.

I learnt to use Linux that way, both via the Ubuntu GUI and terminal commands and much of my 2nd Ebook has been tested on a Raspberry Pi OS running in a VM. Of course it lacks the hardware of the real Pi, so I have to test programs on both, but it’s quite a bit quicker doing a screen grab using the commercial Snagit on my Windows 10 PC.

I can do screen grabs on a PI using scrot, but then I have to upload the image using WinSCP or Putty. It’s not the end of the world but when you are doing a lot, having the image in the Snagit editor ready to copy/paste is a time saver. Also in a similar way, make sure you can do copy/paste via the clipboard and can resize the guest OS (as it’s called). It makes a difference.

I have mine configured so its uses RDP and you can just see the corner of the RDP bar in the Ubuntu image below where I’ve just launched Visual Studio Code.

Ubuntu 18.04 LTS running in an RDP session on Windows 10 in Hyper-V

From a terminal I did ifconfig and got

eth0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 172.21.212.121  netmask 255.255.240.0  broadcast 172.21.223.255
        inet6 fe80::2b71:6d6b:9a09:9aae  prefixlen 64  scopeid 0x20
        ether 00:15:5d:38:01:03  txqueuelen 1000  (Ethernet)
        RX packets 47317  bytes 69248755 (69.2 MB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 27622  bytes 1791175 (1.7 MB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

and on Windows

C:\Users\David>ping 172.21.212.121

Pinging 172.21.212.121 with 32 bytes of data:
Reply from 172.21.212.121: bytes=32 time<1ms TTL=64
Reply from 172.21.212.121: bytes=32 time<1ms TTL=64
Reply from 172.21.212.121: bytes=32 time<1ms TTL=64
Reply from 172.21.212.121: bytes=32 time<1ms TTL=64

This makes it very handy for testing network servers or if you do web development running a web server locally.

How to use indexed sequential files

How to use indexed sequential files

Files
Image by Pontep Luangon from Pixabay

This was a big thing long before PCs existed but you don’t see it so much nowadays.  Say we have a lot of static data, perhaps text strings which can vary in length from a few bytes to hundreds of bytes long. What is an efficient method to store them, i.e. in both space and time?

The answer is ISAM (Indexed Sequential Access Method)  which sounds more complicated than it is. We will use two binary files- one is the data file and one is the index file. The data file holds the raw text strings. We’ll write the raw strings using the C file write. At the same time as write them, we also build up an array of structs. This struct will look something like this:

typedef struct {
  int id;
  unsigned long offset;
  unsigned long length;
} indexRec;

The id could be a char * string. It’s just a way to search for and find the data.

As each text string is written to the data file we populate a struct with the file offset and length. Eventually after all strings are written you can write the arrays of structs to the  index file.

Then anytime you wish to read text strings back, load the structs into an array in ram and scan for the matching id then call fgetpos with offset (this moves the file pointer to the specified byte offset) and read in the length number of bytes from the data file.

An even simpler approach

If you just use an index number then you can scrap the id field and instead store the index file as a collection of offset and length fields. To read string #100 just do a fgetpos to byte at 100 * 8 (four for the size of offset plus four for the size of length)  in the index file, read the eight bytes for offset and length then do a fgetpos in the data file to the offset and read in length bytes as before. Very efficient and very fast. Just two reads in two files and one of those is only eight bytes.

If you wish to change the text strings in the data file, use the same method to retrieve them. If the new string is smaller then just write it and change the length field of the index record and rewrite it. If the new string is longer than the old, write it on to the end of the data file and change the index offset and length fields and rewrite them in the index file.

Both of these edit methods (shorter or longer string edits) will ‘lose’ bytes in the data file. When replacing with a shorter string and adjusting the length, the extra bytes that were in the previous string are no longer referenced. With a longer edit the entire previous string is no longer referenced.

Reclaim lost bytes

However it’s very easy to compact the data file and reclaim these lost bytes. Just process the index file and read each string and write them on the end of a new initially-empty file.  You also change each index file record to have the new offset and length of each string in the new data file. Once it’s rewritten just delete or backup the old data file and rename the new data file to be the one used from now on. Depending on how often the data file is edited, you will need to “compact it” every few days or weeks. But compacting is a pretty quick process, even for mega byte or larger sized files.

I used this technique albeit in Turbo Pascal not C to store game data for postal games I wrote back in the late 1980s. Games that are still run today but on the internet, not by post. For example in a map location there can be several parties of adventurers. The file record for that location has a file pointer to the start of a chain of party file ptr records. Each index file of adventurer parties has a binary file offset to the party in a binary file and a file ptr (or -1 if the end) of the next party’s file pointer. Like a pointer linked list but using file pointers instead of real pointers. Similarly the map location has a file pointer to a dungeon if one exists in this location (or -1 if it doesn’t).

This technique kept the map file fairly small but there were lots of binary file pairs for parties, dungeon, characters (in parties), and towns and shops in towns. Who needed a database? I didn’t… (That’s cos databases didn’t really exist back in the day. Had they existed I might have used them… maybe).

The minus one problem

In the struct definition, you’ll notice that I use unsigned longs for file pointers and length. There is no -1 for unsigned longs. It’s 4294967295. Using this as an end of chain pointer is ok because it is never going to be used as an offset. If I had used signed numbers then I could have used -1 but remember when I wrote this, it ran on a 16 bit computer so I used 65535 instead of -1. I could easily get binary files 40 Kb or 50 KB in size, so a signed number would have overflowed after 32767 bytes.

How to use some advanced data structures in C

How to use some advanced data structures in C

Linked Lists
From Wikimedia Commons

In my eBook, I used only structs and arrays. In fact the commonest “structure” was an array of structs. That was used for bullets, asteroids and aliens ships.The benefits of that are it’s easy to process; I started out by using it as an array and eventually switched to using a pointer to each struct “walking” the pointer through the array, jumping it if you like from struct to struct. Very simple and ideal for a game like asteroids.

You don’t need complexity because that slows things down and the only constraint in the design was making sure I could run everything at 60 frames per second..

C doesn’t come with anything more complex, yet computer science has determined many many advanced data structures that have their uses.

If you were programming a dungeon exploration game, an array might not be the best way to store the dungeon. You’d have multiple levels where each level consist mainly of rooms and corridors. How might you store that? Also there’d be monsters, treasures, traps, transporters and so on in the dungeon.  How would you traverse this structure, guaranteeing there’s a path so that every room can be reached?

Coirridor
Image by Rudy and Peter Skitterians from Pixabay

You might use an array to hold each level but how would you connect levels? Thinking about it I probably wouldn’t. I’d have staircases connecting corridors, traps that let you fall through to a room or corridor below, a transporter that lets you appear in a room or corridor above.

C is flexible enough to support a structure like this but you would have to code all the storage yourself. One way round it is to find an open source library like SGLIB. One reason I chose this is because it lets you create container data structures.

Sglib is fairly low level. It includes code for sorting arrays, for manipulating singe, double and sorted linked lists, red-black trees and hashed containers. As they say “A hashed container is a table of fixed size containing another (we say base) container in each cell. Once an object is going to be inserted into the hashed container, the hash function is used to determine the cell where it belongs and then the object is inserted into the base container stored in this cell. The base container is usually a list, however it can be a sorted list, double linked list or a red-black tree as well. Sglib’s hashed container is parameterised by the name of the base container.”

There are examples provided in the documentation as how to use each type of data structure.

 

What is the best way to include text?

What is the best way to include text?

Words
Image by Free-Photos from Pixabay

One of the games I’m working on needs to include a 1.9 MB text file (200,000 words). Now it’s not that big a deal but using 7zip I can compress it down to 400 KB and that’s 20% of the uncompressed size. So I’m thinking of including some C# code to decompress it.

But I did wonder if there was any kind of built in decompression, so you could add it to say Resources and have it decompressed automatically when read. I haven’t seen anything like that but it would be cool if if existed.

 

Very nice guide to Threads and Processes

Very nice guide to Threads and Processes

Concurrency vs Paralleism b y Backblaze
Concurrency vs parallism by Backblaze.

In my experience, if you ask most programmers to explain the difference between processes and threads they will struggle a bit. Even trickier is the difference between concurrency and parallelism. I wasn’t 100% sure myself.  Do you know the difference between multi-threading and multiprocessing?

This article on the Backblaze website is one of the nicer explanations of threads vs processes and I would recommend it to anyone. Backblaze is a backup solution that uploads your files (and subsequent changes) to their site. When I first moved into my home I used Backblaze but had to abandon it (not their fault!) because my internet was abysmally slow. It told me it would take a year to do the first upload!

Although in C, multi-threading/multi-processing is not that common, it is in other languages and with the trend towards more and more cores (my CPU is five years old and has 6 cores or 12 with hyper-threading), so writing software that only uses one thread is wasting a lot of processing power.

Improving on C strings

Improving on C strings

T
Image by Anja🤗#helpinghands #solidarity#stays healthy🙏 from Pixabay

Although C has many useful features, Text or string handling is not one of them. Unless you are writing text adventures, this is probably not a big deal and even games like Asteroids do write text messages, show the score etc.

Givn C’s age, it’s not surprising that people have tried to improve on C’s strings (char *) and one that I came across is the Better string library. The library is stand alone, portable and known to work with gcc/g++, MSVC++, Intel C++, WATCOM C/C++, Turbo C, Borland C++, IBM’s native CC compiler on Windows, Linux and Mac OS X. If you are bothered by buffer overflow type bugs, this is one area that BSL can definitely improve on. Note although the main documentation is just one text file, it is a very long file!

I haven’t tested it with clang but if it works with gcc the odds are it will work with clang.

I’ve added it to the C code links page.

 

My different programming style from before

My different programming style from before

Software
Image by Gerd Altmann from Pixabay

Next year I will have been developing software as a job for 40 years. I graduated in mid 1981 and three months later started my first job in October 1981. To say that things have changed enormously would be an understatement. Back then there was no internet so information was limited to three sources: Books, magazines and what you learnt yourself.

Along the way I have used commercially a large number of programming languages: BASIC (8k Basic on Apple II and ACT Sirius- a precursor to the IBM PC), CBasic on Dec Rainbow, Z80 assembler for home computers (MSX, Zx Spectrum and Amstrad CPC 464), 6502 for CBM Vic-20 and CBM-64 (and also one project for the rarely heard of CBM-16) then Turbo Pascal on CP/M and MsDos machines, Turbo C++, Ada, Delphi (4,5,7,9 and XE7) , SQL, HTML, PHP, CSS,Java, C and C# (with Xamarin for mobile). As well as the ebook (C) I’m working on two side projects (C#and MonoGame).

Back in the day pre-internet when I wanted to do graphics, I had to figure them out myself. Things like Turbo Pascal came with thick manuals that you more or less learnt, To draw rectangles on the CBM-64 in hires mode. I figured out how to draw them and finished a game in 6 weeks. Recently I’ve been doing the same using MonoGame for mobile game development. I did a quick search of Google, found a library in SourceForge or GitHub, downloaded it and fitted it in. No big manuals to learn, just knowing how to find stuff on the web, make sure the licence allows its use and I can understand its source code enough to use it.

Intellisense also helps. No more knowing that such and such a function has three parameters, you just type in the name and it shows the parameters and their types. Is it more productive than before the web? I think so but there’s also more distractions.

Do I miss those days when things were a lot simpler? A little but I know I’d miss being able to find things with Google, on StackOverflow, even in Wikipedia etc.  Things can be a lot more complicated but I think I’m way more productive.

Using Excel for level design

Using Excel for level design

Excel level spreadsheet for AsteroidsIn the asteroids game and shortly in my MatchThree game, I’ll be creating a level data array of struct. This has a struct for each level containing a count of particular features for that level.

This is the struct and array for a level in asteroids.

struct level {
	int nums[4]; // how many of each size of asteroid
	int aliens; // how many aliens
	float factor; // from 1.0 to 1.5 - multiply asteroid speed by this
};

struct level levels[50];

This is the first 3 levels and level 50.

#include "levels.h"

struct level levels[50] = {
{ .factor = (float)1,.aliens = 1,.nums = { 0,0,3,3 } }, // Level 1
{ .factor = (float)1,.aliens = 0,.nums = { 0,1,3,3 } }, // Level 2
{ .factor = (float)1,.aliens = 0,.nums = { 0,1,3,3 } }, // Level 3
..
{ .factor = (float)1.5,.aliens = 3,.nums = { 4,4,5,5 } } // Level 50

I didn’t type any of this in. Instead, I created that spreadsheet above. It’s easy to work out difficulty levels in column H.  The formula that calculates this is ins this on row 5. If you don’t know Excel, the $ in the factors means that as you copy this into successive row, it keeps the $ row value constant.

=(B5*B$4)+(C5*C$4)+(D5*D$4)+(E5*E$4)+(F5*F$4)+(G5*G$4)   - Row 5

=(B6*B$4)+(C6*C$4)+(D6*D$4)+(E6*E$4)+(F6*F$4)+(G6*G$4) - Row 6

So you can see its multiplying the values in rows 5 6 etc by the values in row 4.  Having put that in place I could tinker with the values in rows 5,6 etc to make the difficulty level increase roughly at the same pace. The difficulty level for level 50 is 77.5.

This is what the Excel formula looks like to generate the C code, it’s in cell M5 in the spreadsheet and then copied and pasted down.

="{.factor =(float)"&G5&", .aliens="&F5&", .nums = {"&B5&","&C5&","&D5&","&E5&"}}, // Level "&A5

and this is what it looks like. C code that can be copied and pasted directly. It even includes the comment for the level number!

{.factor =(float)1, .aliens=0, .nums = {0,0,3,3}}, // Level 1

Creating a spreadsheet and C code from it this way saved a lot of typing but let me quantify the numerical difficulty which increases from 17.5 on level 1 to 77.5 on level 50. There’s no meaning to this value, it’s just a calculation.

How to stop access to Global variables in C

How to stop access to Global variables in C

Links
Image by PIRO4D from Pixabay

If you have an application made up of multiple source and header files, it’s possible that you use global variables in those source files. Remember, these are variables declared outside of any function.

Now you may not know it, but by default, those variables are visible to other files through what is known as external linkage.

Slap extern on the definition and the compiler and linker will happily use it. For instance in the Asteroids game, there is an extern int variable used in the file asteroids.c.

// external variables
extern int errorCount;

This is declared in the source file lib.c and when all the modules are linked together the linker sees all the compiled symbols and figures out which refers to which.

Making your program robust

If you don’t want global variables in one file to be accessed from another, just add the keyword static in front of the declaration like this.

static int errorCount = 0;
Now then, even though you’ve got the extern, when you compile it you’ll get an error. This is what Visual C++ says.

Severity Code Description Project File Line Suppression State
Error LNK2001 unresolved external symbol _errorCount asteroids D:\writing\Amazon EBooks\GamesProgrammer\publishing\Learn C Games Programming\Windows\Code\asteroids_ch48\asteroids\asteroids.obj 1 
Error LNK1120 1 unresolved externals asteroids D:\writing\Amazon EBooks\GamesProgrammer\publishing\Learn C Games Programming\Windows\Code\asteroids_ch48\asteroids\Debug\asteroids.exe 1

So obviously only do this with global variables that are not going top be referred to. In this case we do refer to errorCount in Asteroids.c so adding the static would make no sense.

Do you declare variables at the top of functions

Do you declare variables at the top of functions

Matrix code
Image by Pete Linforth from Pixabay

This is for C99 and later. Prior to this, in function you had to declare all the variables that would be used in that function before any code. Since C99, you can declare them nearer to where they are used.

This function is from the Asteroids game. The declaration of the two variables hx,hy could have been moved to just before the do-loop. They can’t be declared inside the do-loop because they are used after it.

// doHyperJump() find empty space on screen fo PlayerShip
void CheckJump() {
	int hx, hy;

	if (jumpFlag && SDL_GetTicks() - jumpTimer > 3000) { 
		jumpTimer = SDL_GetTicks();
		jumpFlag = 0;
		do {
			hx = 65 + Random(SCREENWIDTH - 130);
			hy = 65 + Random(SCREENHEIGHT - 130);
			if (IsEmptySpace(hx, hy)) break;
		} while (1);
		Player.x = (float)hx;
		Player.y = (float)hy;
		Player.vx = 0;
		Player.vy = 0;
	}
}

This is one of those cases where it doesn’t really matter. But generally I will declare them close to where they are first used and I almost always declare the for-loop variable in the for-loop. Only if it’s needed afterwards do I declare it after, but that is a slightly iffy use unless you want to check the value it had when it exited the loop.

Remember the scope of the variable i.e. the part of the function in which the variable can be accessed starts just after it is declared. By keeping declarations near the code, you reduce the scope and this can help robustness.