Hello again, neccy!

It’s been a while since I posted about neccy, my toy 48k ZX Spectrum emulator. It’s probably a good idea to remind ourselves where we were.

My next milestone is to get Sinclair BASIC working. After that, who knows - I’ll probably try and go for sound and then onto trying to run a game.

It’s a long journey ahead.

And here’s how it looked:

And this is what things look like today

A lot has changed since I last talked about this project!

Sinclair BASIC

One of the things I got working pretty soon after the last post was Sinclair BASIC. I had a few bugs in the Z80 emulation that caused basic to go wonky. The main thing that I got snagged on was that the offset to the IX/IY register is signed. I was treating it as unsigned, so in my emulation the offset was always positive.

I also had issues in the IO Read routines; I was assuming that no input was a zero byte (0x00), but in the Spectrum the value for unset is 0xFF. Somewhat counter-intuitive to those that are used to a 0 bit being “unset” and a 1 bit being “set”. It makes sense when you get closer to the hardware.

Speaking of hardware, I bought The ZX Spectrum ULA by Chris Smith - it’s a fantastic deconstruction of the ZX Spectrum hardware and has been invaluable for understanding many of the internal details of the system.

With all that working, Sinclair BASIC was up and running!

One thing that stood out immediately, however, was that the keyboard mapping is so very different to how modern computers work. This is true on the 128K Spectrum, but on the 48K model it’s even more extreme. BASIC commands are entered using a sequence of modifiers and keys that emit a full command.

As a result, programming on the spectrum using my laptop feels very cumbersome.

There’s a few QOL things I could do, such as supporting macros that would emulate the key sequences for backspace, but I haven’t done those yet.

It’s worth noting that pretty much all emulators suffer from this, as it’s down to the underlying system and not the emulators themselves. Either way, my career as a Sinclair BASIC programmer is very slow off the ground!

SNA loading & Z80 speed

Soon after getting BASIC up and running, I fel confident and added the ability to load a SNA file. These files are pretty simple, being essentially a copy of the registers and a dump of the RAM.

I was elated to see that Horace Goes Skiing just worked. There were no issues to speak of and I could play the game (badly) on my keyboard.

Shortly after, I got the CPU emulation running at full speed. I am running at exactly 3.5Mhz, so not quite the speed of a real speccy, but its good enough.

This threw up a few timing issues that caused me to implement clock timings properly. On the Z80, an instruction takes several cycles (T-States) to execute, for neccy I execute the full instruction at the first state and then basically do nothing for the rest. This is far from being perfect, but it’s enough for now. At some point in time I want to go back and emulate the Z80 with cycle accuracy, but that’s another story/project.

Using the SNA loading and improved timings I started trying to load some games. Pretty much all of them crashed, had glitches or relied on Z80 instructions I’d not implemented yet. I got so far as getting JetPac running, but was unplayable due to bugs.

I also added Kempston Joystick emulation at this stage as it was easy to do and allowed me to play games using my cursor keys.

Sound’s awful

It’s true, neccy does sound awful. It’s also true that I’ve had an awful time trying to get sound working.

As a bit of fun, I thought I’d add support for the 48K’s “beeper”, a one-bit audio signal. Easy, right?

I began by following javidx9’s audio synth series on YouTube and plugging in the extension to the olcPixelGameEngine. It was at this point that I realised I know nothing about audio programming. Every sound that came out of the sound code was nothing but pop and crackles. It sounded terrible.

The audio extension to the olcPixelGameEngine calls my code back at a regular interval (I was using 22050Hz), essentially asking for a sample value at this point (-1.0f to +1.0f). I tried various things but nothing I did resulting in anything that even resembled the audio I was expecting.

Days passed with various aborted attempts at getting it to work and I just gave up and moved onto something else.

Goodbye olcPixelGameEngine

… well, sort of.

At this point I decided to move away from olcPixelGameEngine in favour of SDL2. There was one, sepcific reason to this, and it’s because I wanted to use dear ImGui. My hand-rolled GUI looked super retro and had a charm to it, but I kept finding it frustrating to work with when doing something new. I decided that I needed a new GUI system, and I didn’t want to write one.

I had a brief look at geting dear ImGui working with the olcPixelGameEngine but I decided that SDL2 would be better for me and would allow me to use SDL’s audio capabilities as well.

The thing with olcPixelGameEngine is that I really like the simplicity of it. It’s easy to work with; the abstractions over the various bits you need to do are intuitive and easy to work with - so I wanted to keep it.

What I ended up doing was to remove the olcPixelGameEngine implementation code, but maintain its interfaces - or at least the bits I was using. I ended up with a lightweight SDL2 SDLPixelGameEngine; I even kept the olc namespaces.

Unfortunately I lost a lot of the useful bits of code from PGE, such as the text output to a sprite, but I was going to move onto using dear ImGui so it wasn’t a huge loss.

dear ImGui

Armed with an SDL2 powered codebase, I was free to integrate dear ImGui into my code. A couple of issues cropped up, mostly around integrating the rendering systems but I managed to work around those pretty easily in the end.

The particular issue I had was having the neccy display render into a dear ImGui window. I was rendering to an SDL_Surface and had to figure out how to get dear ImGui to show it. In the end, I had to make sure the the SDL_Surface was associated with an OpenGL texture and hat I was refreshing that texture when things changed. After that, I could use ImGui::Image to show it.

Moving to dear ImGui gave me immediate benefits. I could have separate (movable) windows for all the things I needed debug displays of. The Z80 state, the disassembly window, the audio (beeper) signal and a bunch of other stuff.

Suddenly, adding a new debug window because effortless and made working on the emulator fun (again).

A huge boost came in the way that the author of dear ImGui had already built and released a memory viewer/editor for the library. It took minutes to integrate and offered me something far better than I had.

Z80 bugs everywhere

My Z80 had a boatload of bugs in it, here’s some examples of games that are buggy.

Jetpac was bugged up; it’s there - but not quite. Lots of issues.

I really needed to iron out these bugs, so my attention turned to zexdoc/zexall.

zexdoc (and zexall) is a program that you run on Z80 powered machine. It essentially runs a ‘family’ of instructions and generates a Crc32 of the output (registers, flags & whatnot). At the end of each family it compares the result to a known Crc32 obtained from a real Z80-based computer. I needed to run zexdoc, but all I could find were TAP files of it for a spectrum and the original CP/M-based program.

TAP loading

As naive as I was, I opted for the TAP version of zexdoc (more on this later).

I set about figuring out how TAP files worked. Luckily, it’s quite simple and well documented.

Loading from TAP format is essentially simulating the pulse signals that a real tape would play to the Spectrum’s audio input. The loading is performed by the ROM itself.

Just loading the TAP format teased out several Z80 bugs. Finally, I got zexdoc running and… lots of tests failed.

zexdoc / zexall

zexdoc takes ages to run, so I added the ability to overclock the emulator. This proved handy as it let me run things faster, getting to the errors quicker.

Now began what I can only refer to as a slow grind. Finding errors and fixing them, one by one. Some errors literally made no sense. I looked at specs, undocumented opcode details, even other emulator source and couldn’t find what was wrong in some of the basic instructions.

Finally, I gave in and ran zexdoc on Fuse and found out we had the same set of errors!

I was both pleased but frustrated. I still had bugs - games still had issues, so where were they? I couldn’t rely on zexdoc to help me anymore, as many of the tests ‘failed’ but ‘passed’ in that they matched an established emulator. I needed a way to see the wood for the trees.

floooh to the rescue

I found a blog post by a chap called Andre Weissflog, that explained how he got zexdoc running in his rust Z80 emulation. Rather than using the Spectrum version of zexdoc, he was using the original CP/M version. Luckily for us, zexdoc relies on two CP/M OS calls - and these are very simple text output calls that we can trap and use to grab the output.

Better yet, the CP/M version of zexdoc run on the ’naked’ Z80 and need nothing but an attached amount of emulated RAM. I was able to get a test up and running without having to simulate anything else of the Spectrum - no ULA, no screen output, nothing. As I didn’t care about timings either, I could make my CPU run at max speed, even skipping the emulated clock ‘wait’ timings I had to put in to get things running at comparable cycle timings to a real Z80.

Armed with this, I was able to rip through and fix a raft of Z80 bugs pretty quickly. It even forced me to implement a bunch of undocumented Z80 instructions and handle the flags correctly (including bit 3 and 5, the X/Y flag).

Current state of the Z80

I’ve fixed all but one failing zexdoc test:

ld <bcdexya>,<bcdexya>........  ERROR **** crc expected:478ba36b found:8088c9d9

zexall fails 5 tests:

bit n,(<ix,iy>+1).............  ERROR **** crc expected:83534ee1 found:9581a6ba
bit n,<b,c,d,e,h,l,(hl),a>....  ERROR **** crc expected:5e020e98 found:e6624aeb
ld <bcdexya>,<bcdexya>........  ERROR **** crc expected:478ba36b found:8088c9d9
ldd<r> (2)....................  ERROR **** crc expected:39dd3de1 found:405ca1c1
ldi<r> (1)....................  ERROR **** crc expected:f782b0d1 found:f531e964

I have spent hours - no, days - hunting down the ld bug and have had no joy. Out of them all, that’s the one that’s killing me most.

Current state of neccy

Games like Jetpac play now (even though I suck at it):

Many games, such as Uridium and Return of the Jedi either outright crash or never get past a specific point, like something doesn’t happen that they game expects.

I think that the issues are either in the Z80 itself (the ld bug?) or are down to expectations on hardware timings that I simply don’t handle yet.

For example, my emulation does all the Z80 work in one tick and lies dormant for the rest; as a result lots of data movement around the bus happens instantly instead of over a series of cycles. I don’t emulate the memory contention of the ULA in the ‘slow RAM’. I don’t emulate the ULA fetching bytes of screen RAM. I don’t emulate any floating bus values that many games use for timing to screen syncs. I’m convinced that my interrupt handling has bugs or quirks. My beeper audio is better, but still has issues because I still have variable frame times. I haven’t even looked at 128K stuff, like banked RAM or AY sound chips. Hell, I don’t even support TZX yet.

I still have a lot to do on this project.

With that in mind I decided to take a break from neccy for a bit. It was mostly down to complete burnout trying to trace the Z80 bugs. The ld bug in particular became days of grind that ended up in an all out death crawl - it completely killed the joy of working on this project. At least, for a bit it did.

Real hardware

Fortunately at the time of this Z80 emulation fatigue, a good friend of mine turned up one day with two Spectrums he found in his loft. He knew about my emulator project and asked if I wanted them - of course!

With this, I decided to get them restored by the fantastic Mutant Caterpillar - Ian & Alex did a fantastic job on the restoration, even doing the various magic mods that allows the Spectrums to work better with modern TVS. I bought a Zipstick and a divmmc future to enable loading from SD cards.

Now I can play Uridum!

Next steps

One of the reasons for me starting neccy was to have a pop at an emulator; but the other - more personal reason is that I wanted to have a go at making games for a system I owned when I was a kid. That was true back when I started it, and it’s even more true now.

I’ve been following Dean Belfield’s recent Spectrum coding and he’s really inspired me to have a go.

The next steps for me are exploring the various options for coding on a 30 year old computer in 2020.

See you next time. I’ll try not to leave it so long ;)