42 Astounding Scripts

If you’re in the habit of transferring BASIC files from old computers to modern ones, you might discover strange characters in the files. They’re like a strange combination of text and binary. This is not a compiled program, however. In old-school BASIC, there was a difference between compilation and tokenization.

Compilation converted BASIC code to machine code, and compiled files would usually be stored with a file extension indicating that the code should be run directly rather than interpreted. Often, this extension was some variation of “.BIN”. On personal computers at least, compilation usually required third-party software to convert the BASIC statements to machine code.

Compiled programs were no longer BASIC. They were machine code. They couldn’t be listed or edited, at least in BASIC. It generally wasn’t possible to convert a compiled BASIC program back to the original BASIC. Like any other compiled software, if you wanted to recompile it you needed to keep the source code on hand.

The default BASIC on these personal computers were also not saved as straight ASCII. While saving to text was usually an option, the default for most BASICs was to tokenize programs both in memory and on disk. Tokenized files were usually saved with some variation of a .BAS file extension—often the very same extension used for straight ASCII, non-tokenized files. Whether a file was tokenized or non-tokenized was a bit in the directory listing for that file, not in the file itself.

Unlike compiling a program, which translates code statements and functions into machine language, a tokenized BASIC program is still BASIC. In the computers I used, tokenization was mostly, if not completely, a one-to-one translation of BASIC statement/function to the one- or two-byte token for that statement or function. This saved space on the system. Both disk space and RAM were limited on older personal computers. But it also made it much easier for the system to run the code on the fly—interpret it—and made the interpretation much faster.

Without tokenization, the difference between RESET and RESTORE in the Radio Shack Color Computer’s Extended Color BASIC, for example, won’t show up until comparing the fourth character. With tokenization, the difference shows up on comparing the first character—9D is the tokenization for RESET and 8F is the tokenization for RESTORE. Nor is there any reason to scan for the end of the statement or function. Each statement or function is at most two bytes.

Why? Where do these results come from?

On the Facebook CoCo group, someone recently asked about how I handled bit-checking in a question I asked on the Retrocomputing StackExchange about the joystick. They didn’t phrase it as a question about bits because they didn’t know that their question was about bits.

It’s not surprising. I don’t know about other computers, but Radio Shack tended to hide the bitwise nature of their 8-bit computers. The only BASIC functions that worked with bits were the logical operators AND, OR, and NOT¹, and they weren’t explained as bitwise. The first two are given a page in Getting Started with Extended Color BASIC, page 78, where they’re explained as if they’re the English words AND and OR.²

But these operators do not behave as the English words do, or as the explanation implies they do. Here’s an example. Go into your CoCo or an emulator such as Xroar, and type:

The first should give you 1 and the second should give you 7. In most modern programming languages, both would give you true, which, depending on the language, might be a boolean value or it might be the integer 1. It might even be one of the two values, the 3 or the 5, because of short-circuit evaluation.

How does Extended Color BASIC arrive by its return values? The answer is that AND and OR are bitwise operations. These are 8-bit computers. They describe their numbers in 8 bits³. Here are the bits for the numbers 3 and 5:

The logical operators do not compare bytes. They compare individual bits at matching bit locations. What AND does is, if the bit in the same location in each byte is 1, the bit in the resulting byte is 1. If either of the two bits are zero, the bit in the result for that location is zero. What OR does is return 1 if either of the bits are 1, and zero only if both bits are zero. So here’s what AND and OR do when comparing 3 and 5:

At the end of my post on the first version of cocobin, I wrote:

It’s also likely that some binaries were provided as decimal instead of hexadecimal numbers.

And only a few weeks later, here I am. I found a really nice Galaxian/Space Invaders-style game by Rodger Smith in the February 1985 Hot CoCo.¹

He used decimal numbers for his DATA statements, so I added that feature to cocobin. If a file or BASIC program contains decimal rather than hexadecimal numbers, add the option --decimal to the cocobin command line.

Also as expected, adding this feature also highlighted another common custom of the era: the DATA items often included a marker to denote the end of the data. Most commonly, as I recall, this was the word “END”, or (for machine code) a negative 1. Smith used the number 999. So I’ve added the ability to recognize “END”, “999”, or “-1” at the end of a line of DATA in a BASIC program. If the program sees any of those key words at the end of a DATA line, it assumes that that is the end of the data to be read.

Either of those keywords not at the end of a DATA line will still be seen as an error, since none of them represent valid POKEable numbers.

The script now accepts the following arguments:

Also, while this is not a change, I did finally verify that the script works when used with multiple files. Charles Husak’s “The Little Runner” from the March 1984 Rainbow uses three BASIC programs to POKE the binary into memory. This command line worked to create a working binary from those three files:

Interesting little game!

A quick note that I’ve just got the update of Safari from 14.0.2 to 14.0.3, and Quick Actions that replace text inside of Safari text fields are now working again.

I’ve removed the “Copy to Clipboard” workaround (so that I no longer erase text previously copied to the clipboard) from the three scripts that I use most often in Safari, and am having no problem editing and modifying text inside of text fields.

I was getting worried that this was a permanent change, and am very happy that it turned out to really be a bug and not deliberate. It’s more than a little disappointing that the bug didn’t get caught before it went out—as both a programmer and a writer, I find Services in Safari text fields extraordinarily useful. I use it often enough that I noticed it within a few hours of performing the update. In fact, I’m using it right now. When I’m done writing this short note, I’ll run my Markdown to Blog HTML service on the text to convert simplified Markdown to the simplified HTML I use for posting.

I don’t use all of my Services scripts on a daily basis, but I do use at least one of them every day.

This is with Safari on Catalina 10.15.7. I should be updating to Big Sur soon, for varying definitions of “soon”, but would expect (or hope) that it’s either also been fixed or was never a problem on Big Sur.