Category: Tools

Hex Editor for Raspberry-Pi

Hex Editor for Raspberry-Pi

Hex Editor Raspberry PiSometimes you just need to view or maybe a binary file and here I’ve done that on the compiled file for asteroids.

I can’t recommend doing that, but I wanted to try hexedit on the Raspberry Pi. I ran it against the asteroids compiled program and found the text.

If you look I’ve changed the d in starfield (first line of text) to an e. Then I saved it. The program, not unreasonably crashed when run with an error in the errolog.txt file: Couldn’t open images/starfiele.png

You install hexedit with

sudo apt install hexedit

and run with the file you want to view/edit.

hexedit asteroids

To view all the command press F1. It displays the man page for hexedit. Have fun!

Binary editors – Useful tools

Binary editors – Useful tools

HxD - Binary editorAt some time or other you are going to need a binary editor, to let you look in files and see what they contain.

One I can recommend is HxD which is shown here.

This not only lets you look at the contents of a binary file (in hex and decoded as text)  but you can change them.

It also includes file tools so you can split files, combine them (i.e. append) and securely wipe them. And very handy, the ability to compare two binary files, shown below.

It also lets you export binary files as data  for any of these languages (Pascal,C C#, Java, VisualBasic.net, HTML, rich text and some other formats). Here’s what the top file looks like exported into C.

/* D:\development\pivot\pivot\Debug\l.key (11/05/2020 12:03:32)
StartOffset(h): 00000000, EndOffset(h): 0000003F, Length(h): 00000040 */

unsigned char rawData[64] = {
0x18, 0x3C, 0x2B, 0x11, 0x24, 0x1E, 0x17, 0x26, 0x1C, 0x15, 0x04, 0x19,
0x23, 0x28, 0x08, 0x07, 0x35, 0x0F, 0x34, 0x37, 0x32, 0x05, 0x20, 0x27,
0x3E, 0x0A, 0x3B, 0x2E, 0x1F, 0x29, 0x21, 0x25, 0x2F, 0x14, 0x2D, 0x1A,
0x1D, 0x0C, 0x33, 0x01, 0x39, 0x2A, 0x1B, 0x00, 0x36, 0x06, 0x22, 0x31,
0x38, 0x3A, 0x3D, 0x2C, 0x16, 0x0D, 0x03, 0x00, 0x09, 0x12, 0x13, 0x02,
0x0B, 0x0E, 0x30, 0x10
};

In truth there are many binary file editors. You can find them under hex editors as well. But this is a particularly nice one; recommended.

New version of Visual Studio Code

New version of Visual Studio Code

Update to Visual Studio codeNormally I wouldn’t mention it, as there’s nothing really outstanding about the update (accessibility improvements, Timeline view, Better quick open for files etc.) You can read all about it here.

But the online documentation has improved and more importantly there’s a new tutorial using C++ on Linux. I could have done with this a few weeks back; everything there I had to figure out for myself! In our case I’ve used clang not gcc and C not C++ but those are very minor differences.

But it does have better explanations for many of the fields in the various .json files. Also the changes to tasks.json to allow multiple files. I just added them in based on gcc.

 

Install Clang and Visual Studio Code on Ubuntu

Install Clang and Visual Studio Code on Ubuntu

Visual Studio CodeClang is very easy to install. Open a terminal and issue this command:

sudo apt install clang

It takes a little bit more for Visual Studio Code (aka VS Code). Start on the download page of VS Code. Just google for download Visual Studio Code or click here. Click the Deb rectangle and pick the Open with Software install (default) an that will install it.

After that is installed, run VS Code (open a terminal and type code) and click the extensions icon (5th down on the left) and type in C++ in the search box. The first result is by Microsoft so select it and click install.

Now I created a Projects folder under my home, so click on open Folder in the File menu and select Projects. You can create individual folders for applications under that. If you look closely you can see Asteroids, AsteroidsDND and Examples with Asteroids being open and containing two .c files.

To compile, make sure the open tab is the file you wish to compile then click Terminal on the top menu and Run Build Task…  You’ll see the compiled programs on the left under the Asteroids folder (decnot and ex1).

There’s a bit more for configuring Builds and Debugging but I’ll cover that another time.

 

 

How I write my ebooks

How I write my ebooks

Screenshot of JutohMy secret: is I use a piece of software called Jutoh from a Scottish software house. It costs £35 and if you are writing a book or e-book I highly recommend it.

Note I have no connection with them except as a very satisfied customer.  

It’s all the little things, inserting pictures, tables, handling source code that make it particularly good for non-fiction books.

It also includes formatting for various programming languages and is cross-platform with versions for Linux, Mac and Windows.

The current version of the Linux e-book has 55144 words, (307407 characters). It takes about 10 seconds to compile that into a .mobi (Amazon) or .epub file with all sorts of warnings.

Originally I started writing the first e-book in a text editor then imported it to MS-Word. But Word is not exactly great software for publishing non-fiction as it’s not desktop publishing. Actually you don’t need desktop publishing per se because Kindle reflows the text, but having pictures, source code and tables makes it more complicated.

I looked about, found Jutoh and consider myself very fortunate; it makes life much easier. I’ve also had excellent support from Jutoh’s creator Dr. Julian Smart who is also well known as a creator of wxWidgets.

Writing an e-book is a multi-pass project that takes me roughly 3-4 months part-time.

  • Pass 1. Develop the software and e-book structure i.e. chapters.
  • Pass 2. Write the chapters.
  • Pass 3. Look for typos. For now this is just me, but eventually I hope to be able to afford a technical editor.
  • Pass 4. Tidy up the layout. This is probably the slowest phase. Because of the images, tables and shortish code listings, it might look neat and tidy at one size, but if you change the viewing font size, it looks er not quite so neat! I can’t see any way round that.

A friend drew the original book cover but I’ve decided to pay for the cover for the Linux version.

How to view disassembled C/C++ code in Visual Studio Code

How to view disassembled C/C++ code in Visual Studio Code

Disassembled C source codeThere are four steps to do this. That’s assuming that you have successfully compiled your C or C++ program and have the source file handy.

  1. Install Disassembly Explorer extension into Visual Studio Code.
  2. Set a configuration setting.
  3. Run a single line command in terminal.
  4. Select a command in the Visual Studio Code command palette.

Install Disassembly Explorer.

In Visual Studio Code just switch to the extension tab and search for Dissassembly Explorer and click the install button.

Disassembly Explorer

 

 

 

Next is configuration

View the installed extensions and click the cogwheel for Disassembly Explorer then click Extension Settings at the bottom of the popup menu. A Settings tab will open, click Edit in settings.json.

You should see a few settings but only this one matters: “compilerexplorer.compilationDirectory”: “${workspaceFolder}/.vscode” . That path is where the compile etc. .json files are located.

Now open a terminal

I’ve assumed that you are running on Linux or Mac. If you are on Windows, you can also use Visual Studio Community Edition which includes disassembly features and why I’m not mentioning it here.

In the folder with your source files issue this command in the terminal.

clang -g -o asteroids.S -S asteroids.c

If OTOH you are running gcc then just substitute gcc for clang.

Finally, back in Visual Studio Code select your source file as the open tab then press F1 and in the menu that appears type in enough of Disassembly Explorer: Show until you can select it and a second tab should open with the disassembly. Click on a function in the source tab and it will jump to the correct place in the disassembled code.

Visual Studio Vs Visual Studio Code?

Visual Studio Vs Visual Studio Code?

I wrote the first book on Windows and used Visual Studio Community Edition 2017 (VS). For Linux I’m using Visual Studio Code (VSC) but if you are working on Windows you have the choice. So which is better suited to you?

Disassembly of OutputDebugString program

Given that my full time job involved Visual Studio Professional, it wasn’t a difficult choice for me, though having used VSC on Linux, I’m now warming to it.

I found the configuration messy but if you stick at it, you get there. It’s a fairly simple product but once you get into all the configuration and extensions, there’s a lot more to it.

VS (now at 2019 version) is an excellent tool and the navigation features and debugging are better than VSC. You can also view disassembly of C code as the screenshot shows.

But if you are also working on Linux or Mac and using VSC, it might be easier or at least more consistent using it for all platforms. Mind you there’s also the question of MSVC vs Clang to sort as well.

VS has extensions but only 3271 currently compared to 16934 for VSC and many of the VS extensions are trial while it appears that all of VSC are free. There’s also nearly ten times as many programming language extensions for VSC (3427) compared to VS (351).

So there’s no outright clear winner here. YMMV as the saying goes. (Your mileage may vary). Here for your delectation is that disassembly including the original C lines. Don’t worry, I won’t publish too many of these!

int main()
{
00E31700  push        ebp  
00E31701  mov         ebp,esp  
00E31703  sub         esp,0CCh  
00E31709  push        ebx  
00E3170A  push        esi  
00E3170B  push        edi  
00E3170C  lea         edi,[ebp-0CCh]  
00E31712  mov         ecx,33h  
00E31717  mov         eax,0CCCCCCCCh  
00E3171C  rep stos    dword ptr es:[edi]  
00E3171E  mov         ecx,offset _1EF31893_ods@c (0E3C00Ch)  
00E31723  call        @__CheckForDebuggerJustMyCode@4 (0E3120Dh)  
    wchar_t * text=L"Hello World!\n";
00E31728  mov         dword ptr [text],offset string L"Hello World!\n" (0E37B30h)  
    OutputDebugString(text);
00E3172F  mov         esi,esp  
00E31731  mov         eax,dword ptr [text]  
00E31734  push        eax  
00E31735  call        dword ptr [__imp__OutputDebugStringW@4 (0E3B000h)]  
00E3173B  cmp         esi,esp  
00E3173D  call        __RTC_CheckEsp (0E31217h)  
}
00E31742  xor         eax,eax  
00E31744  pop         edi  
00E31745  pop         esi  
00E31746  pop         ebx  
00E31747  add         esp,0CCh  
00E3174D  cmp         ebp,esp  
00E3174F  call        __RTC_CheckEsp (0E31217h)  
00E31754  mov         esp,ebp  
00E31756  pop         ebp  
00E31757  ret  
Logging on Windows – OutputDebugString

Logging on Windows – OutputDebugString

Shows OutputDebugString being calld in the debuggerIn the post about rsyslog three days ago, I explained how to log from Linux programs using the rsyslog daemon.

It’s slightly different in Windows. There’s a built in function called OutputDebugString(LPCWSTR str) that you can call from anywhere in your program. It dumps the string str into the Output window if you are debugging it in Visual Studio.

If you are running this outside of a debugger, the output is lost unless you can capture it with a suitable utility. DebugView from SysInternals.com (it redirects to Microsoft) is one such utility. That’s a screenshot of it below.

Showing DebugView in actionJust run DebugView and leave it there. It might catch other stuff from Windows, but when you run your program from the command line or double click on it, it will execute quickly and you’ll see any strings captured like this one.

 

This is the program that I ran. In Release it compiles to a 9 KB exe! Because OutPutDebugString needs a LPCWSTR  (Long Pointer to a WideString), I declared the text as wchar_t.

#include <Windows.h>

int main()
{
    wchar_t * text=L"Hello World!\n";
    OutputDebugString(text);
}

At work I developed a very large program that only worked running on another computer. I used OutputDebugString extensively and without it, debugging would have been much harder.

Using rsyslog to log

Using rsyslog to log

logging
Image by OpenClipart-Vectors from Pixabay

I’m a great fan of logging; it helps you find out what’s happening in a program even if you can’t debug it. Linux, well Ubuntu has a service (sorry Daemon!)  called rsyslog that logs messages from all sorts of processes and it’s quite easy to use in our programs as well.

If we don’t do the next steps all the log messages will go into a file called syslog in /var/log and as this gets a lot of stuff it can grow reasonably quickly. But if you do the following steps, all logged output will instead go into

/var/log/asteroids.log.

From a terminal type the following command:

sudo gedit /etc/rsyslog.d/30-debugging.conf

Type this in and then save it (click the Save button) and close gedit.

if $programname == 'asteroids' then /var/log/asteroids.log
& ~

Now run this command:

sudo service rsyslog restart

It should return immediately and means that any logging to syslog from a program called asteroids will from now on be redirected to /var/log/asteroids.log.

Using rsyslog in a C program

Here’s a short C Program that puts a message in the asteroids file.

#include <syslog.h>
#include <stdio.h>

int main() {
    openlog("asteroids",LOG_CONS | LOG_PID | LOG_NDELAY, LOG_USER);    
    syslog(LOG_INFO,"Test Message %d",1);
    closelog();
}

In a typical program you’d put the openlog statement in main() near the start and the closelog() near the end of main, and uses as many syslog() calls throughout your program as you need.

Now what I suggest you do is open another terminal and run this command before you run your main program. This will just sit there printing out each message line as it appears in asteroids.log. You can stop tail by hitting ctrl-c.

tail -f /var/log/asteroids.log

H/T to this StackExchange answer from almost 8 years ago and answer by giuspen.

A tale of two diffs

A tale of two diffs

Screenshot of devart code compareI’ve used diff and merge tools since the year dot. They let you compare two files and see on what lines they differ. You can also copy individual or blocks of lines from one to the other; that’s the merge. My all-time favourite was the commercial Araxis Merge which did a three way comparison and could be controlled by COM. I did this to compare two code bases.

Ten years earlier code for Base Metals (commodities trading) had been forked from Oil Trading software. There were ten years of changes to both sets of code and my job was to merge them back into one code base. I wrote a program to walk Araxis Merge through all folders of both code sets and out-put the differences, Then I could start merging where they differed. It still took about two months but I couldn’t have done it as quick without Araxis.

However most of us don’t have access to Enterprise tools, so here are two alternatives. The first is Devart’s Code Compare (that’s the image above) and the second is a free Extension for Visual Studio Code called Diff & Merge. I’ve added both to the Links to C utilities page.