Though the hardware for the project is complete, there's a lot of software left to do. And, I suppose I'll likely tweak the hardware a bit from time to time as well. I'll update things on this page.
My favorite piece of test gear is a Logic Analyzer. My old HP 1631D was critical in getting Magic-1 up and running. It was ancient (mid '80s), big, loud and heavy. But, it helped me find both hardware and software bugs that I don't think I could have found without it.
About a year ago I asked a friend at work to borrow an old HP 3.5" floppy drive to connect to the 1631D so I could save my configurations. He mentioned this to another co-worker at Google who then gave me his old HP 16500 (he'd upgraded to a newer model). It's a fantastic machine - you can see it in action here:
Shortly after I did that video, I got exceedingly busy at work (and removed the Wiznet device - it was a little unstable and I didn't have the time to track down the problems). However, over the 4th of July break I set the new logic analyzer up again and did something I've wanted to for ages: write an Inverse Assembler for Magic-1. An inverse assembler is what we usually call a disassembler in software-land, and it interacts with the logic analyzer to display the mnemonic names of the instructions as they are shown in a state listing trace. Here's a screenshot of a state display of Magic-1 running the old Dhrystone benchmark:
At line 319, it shows the beginning of the execution of a 16-bit load into register A (with an 0x01fc offset from DP - the global data pointer). This particular instruction takes 7 cycles to execute, which you can see in the following lines. Lines 320 and 321 show the operand fetching of the DP offset (the 0x01FC). The aluop at line 322 is the add of the 0x01FC displacement to the base address in register DP. Lines 323 and 324 show the actual load of the data (as two 8-bit loads). Those loads go into the MDR, so the aluop at line 325 is the write of the data to register A.
This makes analyzing a state trace so much easier. All the info was there before, but I had to manually convert the hex data into instructions.
A while back I ran across a slick replay of history: http://olduse.net is replaying Usenet posts - delayed by 30 years. I spent some time reading - though it won't start looking like I remember for 6 or 7 years. However, I did notice quite a bit of chatter on "net.games" about the game of "rogue". I remember rogue quite well - though mostly I played rogue offshoots such as Larn and (on the TRS-80 various rogue-like games such as "Temple of Apshai" and "Hellfire Warrior".
Anyway, years ago when I first got Minix running well I tried to compile rogue - but it was much too large for Magic-1's 64K code limit. Reading the olduse.net posts, though, it became clear that rogue was running on machines 30 years ago that were similar to Magic-1's capabilities. Last weekend I made another attempt to port rogue. It worked:
It ended up taking most of an evening to get it running. First, many minor changes to the source to bring massage the ancient C dialect to something that lcc would accept. After getting it cleanly compiled, however, it was still much too large. The first problem was a subtle one. I looked in detail at what code was being pulled in by the linker and saw about 12K of floating point support. The odd thing, though, was there were were no float or doubles declared in any of the source files. After considerable detective work, I discovered one place in the code that used "3.0" as a literal instead of "3". Once that was removed, we were only about 2K bytes too large. One of the more interesting things about rogue is that it was played on non-graphic terminals. At the time, there were many models and manufacturers of terminals and they each had their own proprietary command language to do direct cursor addressing (which is necessary for a game like rogue). The authors of rogue wrote an abstraction library called "curses" to deal with this. That library was widely adopted and extended for many other uses - including the classic text editor vi (and later vim).
Anyway, today there is much more standardization around terminal control. So, I simply eliminated much of the curses layer and hard-coded my version of rogue to work only with vt100/ansi terminals (which is pretty much all that's left now). That was enough - rogue works. Telnet into Magic-1 and give it shot. Type "?" for help, and see http://www.roguelikedevelopment.org/archive/index.php for more details. Oh, and expect to see this a lot:
I keep promising myself that I'm done with Magic-1, and then I decide to do "just one more thing." Today's thing is to add a native ethernet adapter - this WizNet830MJ. Dave Conroy brought this little gizmo to my attention, which he's using in his latest cool project
I'm generally happy with Magic-1's existing internet connection, which sends all packets through one of its serial ports to another computer which uses its native interface to connect to the world. It works, but it does add a lot of overhead to the process. This is particularly a problem for telnet sessions, which wrap a large packet around each small set of characters transmitted. My goal here is to support telnet through the Wiznet device, which will let network users experience roughly the same interactive performance that I do when using my VT-100 on the dedicated serial port.
It may take me a while to get around to completing the proper interrupt-driven device driver and then splicing it into the Minix TCP/IP stack, but for a few days I'll leave a little test program running which does a simple response to any HTTP request:
There's been nothing new on the Magic-1 front for a long time - and a big part of the reason is that I've been very busy at work. The result of all that work was announced a few weeks ago at Google I/O 2010: a Just-In-Time Compiler for Android systems that will part of the "Froyo" release. Here's a link to the talk that Ben Cheng and I gave to describe the JIT:
Gave a talk on Magic-1 at the San Francisco ACM chapter meeting. Fun stuff - it was held on the old Oak Room at HP (the same room I got my HP new employee orientation in 25 years ago). Lots of interested folks and good questions.
Had a great time at the 2009 Maker Faire - spent two full days on my feet talking continually. My table was next to Steve Chamberlin's great BMOW project, and I was happy to finally meet Steve and check out BMOW. Here's a picture of my setup just before the show opened:
This was the fourth time I've displayed Magic-1 at a public show, and I continue to be surprised at the number of people who have heard of the project and understand what it's all about. You can also see one of my Nixie tube clocks and a dekatron spinner on the table. Those drew considerable interest from the crowd as well. All-in-all, a fun weekend capped off with a Maker Faire Editor's Choice award.
I was too busy to take many pictures, but the folks from IDG News Service (part of the PC World magazine organization) did a video interview and published it on the web:
I haven't made any significant changes to Magic-1 in quite a while. It's pretty much complete now. About the only thing left for me to do is release all of the sofware I've written for it as open source. I just need to find some time to review all of the other open source code I've incorporated into the project to make sure I'm respecting their licenses. After that, I'll push everything into the open - including the build environment in case anyone wants to compile and run their own programs on Magic-1.
WooHoo! - Found and fixed a long-standing bug today. About two months ago I was making some minor changes to the kernel makefile and discovered that if I rearranged the link order the kernel would hang during boot-up. I suspected a linker bug in bss fixups, but after lots of debugging convinced myself that the linker was ok. Kernel hangs can be very difficult to debug - especially during the boot process, so I simply reverted to the older makefile and hoped I would eventually run into a more clear-cut (and easier to debug) failure.
That failure showed up after a minor change Friday night. Instead of hanging, the kernel aborted with an unexpected trap - and I could consistently reproduce it. It took most of the weekend to narrow it down, but I finally tracked the problem down to a mistake I made about 2 years ago when allocating kernel task stacks. One of the tasks stacks overlapped a temp disk buffer that was only used during boot-up. That's why there was never a problem once the system was fully booted up.
Although I've been quite busy with the new job (I'm on Google's Android team - working on the Dalvik VM), I continue to fiddle with Magic-1 from time to time. Over the Christmas break I did quite a bit - mostly performance tuning and recovering from a failed hard drive (one of my 1994 20MB HP Kittyhawk drives). A few years ago I got a handful of Kittyhawks, but I've been losing them at the rate of one every six months or so. For that reason, I've decided to just use a Compact Flash drive for day-to-day operations.
On the performance tuning front I added some features to the kernel to profile system tasks as well as user programs. The good news, I guess, is that there isn't much more performance to get without replacing LCC with an optimizing compiler. I was able to get some noticable speedups by rewriting 16x16 multiplication and 32-bit shifts (which mostly affected floating point speed). I also hand-coded some assembly replacements for key portions of the IDE hard drive driver. I've been toying with the idea of writing a Magic-1 target for the LLVM optimizing compiler framework. Done properly, that could give a pretty dramatic speedup - and I've been wanting an excuse to play with LLVM. Perhaps later.
Today I decided to clean up one of the grungier bits of Magic-1: the boot mechanism. Currently, to boot Minix I have to first boot into my original OS/Monitor, and then load the Minix kernel, servers and init processes. The ugly part of this is that to try out a new kernel or server component, I have to first transfer it to the old OS/Monitor filesystem. I need something like Linux's GRUB or Lilo, and a prerequisite for that is stripped-down, minimal standalone code that understands my modified version of the Minix FS. There is existing code for Minix which is close, but I also wanted it to work in a cross-development environment, so the code has to understand big/little endian issues. Also, my file system uses 2K blocks instead of Minix's 1K blocks. Anyway, the code seems to work. Next up - a simple boot command interpreter/shell. I'm back to work tommorow, but perhaps I can get this done over the New Year's break. It will be nice to have a flexible boot loader.
A little more than 24 years ago, I began my computer programming career in Hewlett-Packard's compiler lab in Cupertino, CA. I started off doing code generation for the then-new PA-RISC architecture, and over the years was lucky enough to get to dabble in Millicode (run-time support libraries written in assembly code), debuggers, dynamic translation systems, mechanisms to optimize programs while they ran and even participate in design of IA64 (Itanium). I left HP during the height of the dot-com craze to join Transmeta (which based it's x86-compatible microprocessor on dynamic translation). I returned to HP in 2002, ending up in HP Labs in Palo Alto (with my coolest title ever: Senior Research Scientist).
My last day at HP was Friday, and I leave with lots of great memories.
On Monday, I start at Google. I don't know the exact details of what I'll be doing there, but I can only assume that the bright folks there have realized that the future of massively powerful data centers belongs to wire-wrapped TTL. I'll be sure to bring my wire-wrap gun.
Heard from Aidil Jazmi, who built the fantastic clone of Magic-1 while a student at Universiti Teknologi Petronas in Malaysia. Aidil is off in the working world now, but found some time to fire up his Magic-1 clone (after finding a ground wire that had jiggled loose) and sent me a picture of it running my original Magic-1 Monitor/OS. I can recognize the program menu in the terminal session on that enormous TV/monitor.
I'm extremely impressed with Aidil's work. It was hard enough for me to get Magic-1 working and I presumably knew at least what I intended the design to do. Aidil managed to get his clone working using only my (sometimes incomplete/buggy) documentation. Amazing. Here's his project web page.
Back from VCF - had a great time. There were lots of excellent exhibits, and most of all I got a chance to chat with lots of interesting people. Here's a picture of Magic-1 on display:
It was running Minix with two sessions (one on each of the two terminals). It also turns out that my project got mentioned on a few technical websites and blogs, resulting in a nice flurry of traffic. When I got back home I put Magic-1 back on line, but decided to disable the web server to allow better response times for the folks who manage to telnet in. I'll have it resume serving pages when the storm blows over.
I've got some catch-up to do around the house, so it may be a week or two before I get around to completing the demand paging. However, I am anxious to do it while everything is still fresh in my mind. After that, it will be about time to call this project complete.
Magic-1 will be on display at the Vintage Computer Festival this coming weekend at the Computer History Museum in Mountain View, CA. I put Magic-1 on display at VCF 8 two years ago, and it took the best of show award. This year I'll be showing off Magic-1's new port of the Minix operating system. (A running diary of my efforts to port Minix to Magic-1 can be found here).
Last weekend's Maker Faire went well - I was pretty much continuously on my feet and talking with people for two days. I was surprised at the number of people who understood what Magic-1 was, and didn't think it strange that someone would build something like this. I didn't think to bring a camera, but a few folks have posted pictures and blogged about it:
More pictures on Flikr:
A lot of the people who stopped to talk about Magic-1 told me of their early computing experiences, and the machines they worked on or built back in the 70's and 80's. Several of them I recognized as active participants of the old Homebrew Computer Club from which Apple Computer arose. That was pretty cool....
Thanks to everyone who dropped by, and to Ken Sumrall - who took over my table several times each day to give me a break.
I've pulled Magic-1 offline to clean it up and get everything ready for this weekend's Maker Faire. Ken Sumrall lent me an old HP green-screen terminal, and it looks really good. I'm always a little nervous whenever I move the machine or pull cards, but it seemed to survive the move to the kitchen table and a good cleaning just fine. While I had things apart, I put in what I expect to be the oscillators for the final clock values - 3.67 Mhz and 4.09 Mhz. I had been running at 4.2 Mhz, but noticed some occasional I/O errors. 4.09 is fast enough. I'll use the 3.67 as a conservative fallback in case I start suspecting problems at 4.09.
I'm not sure what the Maker Faire is going to be like, but it sounds like fun. Maia is excited about going - not to see her nerdy father, or course - but it looks like there are going to be a lot of craft activities for her.
Anyway, besides taking Magic-1, I think I'll also bring the original memory card to show an example of wire-wrap. For that matter, I should also bring the wire-wrap gun, wire, a pile of chips, etc. in case anyone is interested in how it all went together. Also, might as well bring my combo Nixie tube/Dekatron tube clock (probably better stop there - too much stuff already).
Because I'm not going to be able to get the Minix port finished in time for the Maker Faire, I decided I needed to do something new for Magic-1 to show off. So, I've been working on getting the curses and termcap libraries from Minix ported over to my hack OS. Along the way I've had to add some new system calls, but it's starting to look pretty good. I've replaced the old 2-D Tic-Tac-Toe game with one that uses curses to draw the game board on the screen rather than just scroll (curses & termcap enable programs to directly place the cursor on the screen at x,y coordinates on a terminal screen). Right now I'm assuming that the telnet window used to access Magic-1 supports xterm cursor control codes. I've still got some bugs to work out, though. Tic-Tac-Toe works fine, but I'm having some problems with a curses version of Conway's Life program. This should open up a new set of game porting possibilities, and perhaps I'll even try the port of the vi clone.
It looks like my internet connection problems are finally over. A couple of weeks ago I started noticing hangs of Magic-1's web server. It was strange - other programs worked just fine and I could ping the machine. I went through my standard diagnostic actions (reboot, run verification tests, reseat all the chips, etc.), but found no obvious problem. Anyway, to make a long story short, it turns out that my DSL modem was failing. I'm running with a borrowed one right now, and everything is working again.
This cost me enough time, though, that I probably won't be able to complete the Minix port in time for the Maker Faire. I'll see how far I can get. I'll record my progress over on the Minix Port page.
I've taken a bit of a break with this project lately, but hope to restart the Minix port soon. Magic-1 is set to be on display at the Maker Faire on May 19-20 at the San Mateo, CA fairgrounds. Details here. I had hoped to complete the Minix port in time for this event, but I doubt that will happen - I've been much too busy at home and work, and am having difficulty finding time for my projects.
Overall, Magic-1 seems to be running pretty well. A few months ago I finally got around to seeing just how far i could push the clock. I ordered some custom oscillators from Digikey at 4.1 Mhz, 4.2Mhz, 4.3 Mhz and 4.4 Mhz. The verdict: Magic-1 runs at 4.2 Mhz, but not at 4.3 Mhz. So, since January I've been running Magic-1 at 4.2 Mhz. That's going to be the final top speed.
As far as stability goes, I have been having some difficulty lately with the web server. I'm not yet sure, though, that it's a Magic-1 problem or just a home networking problem. I had one case in which I believed that Magic-1 was broken, but later discovered that my problems went away after cycling power on my router. Not helping matters has been my home DSL service - which has been very poor lately. The latest theory from the phone company is that my house is just a little too far away from the switch to support the speeds I'm trying to get, so we're going to try downgrading my service to see if that clears up the problem.
For the Minix port effort, I have spent a little time cleaning up my development environment. I'm doing the work using a Linux virtual machine on my Windows XP host laptop. Previously I was using Fedora Core 5 with a Parallels Virtual Machine. I've now rolled to Ubuntu 7.04 under VMWare Workstation. Both worked, but VMWare has some especially nice features. In particular, cursor handling and the ability to generate images that can be played using their free VMPlayer.
Finally, I decided to write a Magic-1 functional simulator for the Minix port effort. The time I spend writing it will be quickly made up during the port. It will be much easier to debug Minix on Magic-1 in simulation than on the real hardware.
The big news on the Magic-1 front comes from Aidil Jazmi of Universiti Teknologi Petronas. As part of his final year project, Aidil built a fantastic working clone of Magic-1. This is a really impressive accomplishment. it's tough enough for me to understand my own incomplete documentation, but Aidil was able to fill in the gaps. Not only does it work, but it also looks really fantastic. Aidil built a clear enclosure that really shows off his work.
Fantastic stuff, and it gets better: one of Aidil's fellow students cloned Andrew Holme's Mark 1 FORTH computer.
Check out the UTP website: http://www.geocities.com/homemadecpu/.
In other news, I've done a bit of work on the Minix file system. At the moment, Magic-1's primary hard drive is now partitioned into a Magic-1 partition and a Minix partition, and the Minix partition has been initialized with a skeleton file system. Next up, putting together a utility to copy files to and from that Minix file system.
I've been running Magic-1 at 4.09 Mhz for the last week or so, and it seems rock solid. I still don't know how much faster it will go. I tried 4.5 Mhz, and it failed - the my limit is somewhere between 4.09 and 4.5. I have noticed that my heat-related instability has gotten a little worse (which shouldn't surprise me given the clock boost). I've replaced my interior fan with a larger one, so we'll see if that helps matters.
In other news, I've added a diary page to chronicle my efforts to port Minix to Magic-1. I expect that to be a fairly long project. Not only is it somewhat of a challenge, but I remain pretty busy with work, home and other commitments. Once again, I'll make progress a little bit at a time.
WooHoo! - I just fixed a silly coding mistake in my OS/Monitor, and it appears that Magic-1 is up and running solidly at 4 Mhz (and might even go faster). After I rebuilt the memory card earlier this year to shave 40ns off of my memory access time, I had hoped I could get Magic-1 to my original design goal of 4 Mhz. However, I seemed to get fairly consistent I/O errors when I went past 3.75 Mhz.
Now I know why.
Sometime in the past I put in some debugging code intended to force a clock-related failure when accessing a slow device, and I simply forgot to remove it. I vaguely recall doing that in an effort to confirm what I believed my critical path to be. In short, the page in device space (0xF800..0xFFFF) that is mapped to my IDE interface, UARTS, realtime clock, etc. is supposed to have a bit cleared in its page table entry that forces the insertion of wait states on any memory access to that page. However, I had some start-up debugging code enabled that set this bit. No wonder I got I/O failures - I'm now surprised that I was able to get up to 3.75 Mhz.
Anyway, we'll see how stable it really is. The memory access path was my critical path, but there are several others that should start coming into play around 4 Mhz.
In other news, I'm continuing to slowly work towards Minix. My main effort now is a rewrite of the ROM-based boot loader to make it flexible enough to load either my current OS/Monitor or a Minix boot loader. It's a slow process getting this working. I have to burn an EPROM for each new attempt, and the start-up code is easily the most complicated in the entire system. It's quite tricky to make the several required transitions between paging on, paging off, running from device-space ROM and main SRAM. I've had to resort to front-panel switches for much of the debugging, as the most important code happens before the machine is in a state in which the UARTs are functional.
I'm slowly working my way to Minix. Sometime soon I'll go ahead and put together a separate page to record the progress of my porting efforts. At the moment, I'm able to compile all of the kernel, mm, fs, init and most of libc. However, I am cheating a bit - I'm still claiming in the config file that I'm an x86 processor. The goal for this stage is to simply pound on my compiler, assembler and linker to make sure it's in good shape.
Next up I'll need to start writing the machine specific bits of Minix for Magic-1. I've got a pretty good idea of what I need to do, but I'm doing one more reading of the book before I start coding. The tricky part is start-up and the lowest-level code that will handle interrupts and transitions into the kernel. I won't be starting from scratch, though - I based my current OS/monitor on what I thought I'd need to do for Minix so much of the hard work is already done.
Anyway, this is going to take a while - I'm keeping pretty busy on the work and home fronts so there isn't much time left over for my silly projects.
Well, just for jollies I tried running the old Whetstone benchmark on Magic-1 to see how my floating point performance is. I knew it would be really, really poor - and it is. I had to do a bit of modification to the benchmark source because my score was so low that it didn't register in the normal "Millions of Whetstones per Second" metric. As I predicted, it would be better for me to report Magic-1 in the "Dozens of Floating Point Operations per second".
Anyway, I estimate that a full run of the benchmark will take somewhat more than 20 hours. My shortened run ended up at around 0.000134 MWIPS, or 11.2 DWIPS.
I'll probably go ahead and do the full 20-hour run this weekend and officially submit my results :-).
Whew - finally! For the last couple of weeks I've been continuing to work on my compiler toolchain. I ran through a fair number of bugs dealing with my linker and fix-ups generated by the assembler - but the most difficult problems were bugs in the floating point support. Happily, floating point finally seems pretty solid (the last bug took me a long time to find...). Anyway, Magic-1 now supports both single (32-bit) and double precision (64-bit) floating point code via software emulation.
It isn't going to set any speed records, though. Sometime soon I'll try to get one of the floating point benchmarks running to measure it, but I expect we'll end up reporting Magic-1's performance in the "DFLOPS" range (i.e. "dozens" of floating point operations per second). My first significant floating point test program is now available on Magic-1's menu - the old "Phases of the Moon" calculator. Telnet over and give it a shot.
Anyway, I've got a bit more testing and tuning to go, but the compiler, assembler, linker, ar, nlist and ranlib seem solid enough to begin the Minix port in earnest.
Although I haven't updated this page lately, there's actually been quite a lot going on with the project. First, a milestone of sorts has been hit: more than 1,000 people have telnetted into Magic-1 and left a guestbook message. I've had the guestbook program running for about two years now, but am still amazed at how often people telnet in (and from all over the world). When more than 70 or so messages are left on Magic-1, I move them to the guestbook page on this website. Scroll through them here: http://www.homebrewcpu.com/guestbook.htm.
The biggest Magic-1 news is something I hope to talk about soon, and has nothing to do with me. Let's just say that Magic-1 is no longer a unique machine...
Now, what I've been doing lately is working on the compiler tool chain - I'm getting serious about porting Minix to Magic-1. I've done significant tuning of my C compiler's code generation, improving performance by 5-10% and reducing code size by somewhat less. I've also finally enhanced my assembler to produce relocatable object file. I ended up deciding to use a variant of the old BSD a.out format. My plan was to port an existing linker, but in the end I decided it was easier to just write my own. So I did. The old BSD linkers were close to what I needed, but I kept running into trouble because of the cross development environment. I'm running on x86 (little endian) and linking big endian object files. Further, my pointer size is 16 bits, vs 32-bits for the host system. Anyway, my linker is up and running. I made no attempt to make it efficient, as for now I'm intending it to only be a cross linker. Once I get Minix up and running, though, I'll want to have a native linker - and that time I'll probably to back to the BSD 4.3 linker sources.
I did manage to just port (with minor modifications) ar, ranlib and nm.
Anyway, things are looking pretty good. I've still got quite a bit of cleanup of my development environment to do, but soon I expect to start putting together a Minix kernel libc. Oh, and I finally moved back to Linux for development. One twist, though - I'm doing my development in a virtual instance of Linux running on my Windows XP laptop. I've been using both VMWare and Parallels, and both seem to work just fine. It's great to be able to get useful work done again without having to reboot to get into Linux.
On the hardware front, Magic-1 has been rock solid. I haven't had any heat problems, but that's largely because it's been pretty cool on the coast. I'm still not happy with the cooling situation, but I probably won't do anything about it until it gets to be more of a problem.
I found a bit of time to work on Magic-1 over the last few days, so I added a software feature that I've been wanting for a long time: network time sync. One of my minor irritations in life is that I hate setting clocks. I figure it's the clock's job to tell me what time it is - not the other way around. Anyway, Magic-1 was never very good at keeping accurate time. The way it keeps time is that on boot-up it reads the real-time clock chip. From then on, it uses the heartbeat interrupt to update current time. Occasionally, though, one of these interrupts gets dropped on the floor. The result is that over a period of weeks, Magic-1 will lose minutes.
Now, that won't be a problem. I've modified Magic-1's web server code to periodically open a connect to the government's time server (time-a.nist.gov) and fetch the correct time. This value will then be used to update both the real-time clock chip and Magic-1's current time counter. I'm not trying to get super-accurate time here - just within a second or two.
In other news, I've done more experimentation with my fans and heat problem. I was surprised to find that replacing the HEPA filter with just a simple foam one did not make much difference in the temperature - just a single degree. I then tried a different, more open, foam filter and got 3 degrees. Of course, that filter is open enough that it lets a bit of dust through.
I think my next experiment is going to be to use the open foam filter, but try to address the dust problem by using a HEPA room air filter near Magic-1. It would probably do me some good to breath in less dust and dog dander, so perhaps the combination of cleaner room air in general with the simple foam filter will be good enough.
Magic-1 continues to run rock solid at 3.75 Mhz with the new (and last - I hope) wiring modifications. There is only one hardware item left that causes me some minor concern: Magic-1 can't stand the heat. It is located in the master bedroom of my house (which, because I live in a very mild coastal climate, has no air conditioning). Sometimes, though, the temperature in the room gets up to 80 F. When this happens, Magic-1 starts failing. A typical heat-related failure is when running the Original Adventure program and it aborts at start-up with a vocabulary initialization failure. Once the room cools down in the evening, though, correct behavior resumes.
So, over the last few weeks I've been trying to characterize the problem. I have two identical temperature gauges. One is sitting on top of the card cage in the enclosure, and one is just outside of Magic-1 on my desk. What I've found is that the interior temperature of Magic-1's enclosure normally runs 10 degrees Fahrenheit hotter than room temperature. Further, it seems to fail when the inside enclosure temperature hits about 87 degrees.
Now, I have a somewhat unusual cooling setup - which contributes to the problem. Besides having no air conditioning in my house, I also have a significant dust/dog dander problem. The other computers in my house quickly become massively clogged with dust and dog hair. I didn't want this to happen to Magic-1 - particularly with the nice window in the top. So, I pull cooling air into the case through a HEPA air filter. I have a secondary fan inside the case to simply circulate the air, and the fan for the power supply pushes air out of the case. The power supply is also a high efficiency one so the fan barely spins.
Anyway, my next experiment was to see how much of an effect the filtering mechanism introduced. I removed the air filter and found that the case interior temperature dropped by about 3.5 degrees. In other words, rather than being 10 degrees above room temperature, it was 6.5 degrees above. This seem just about enough to avoid heat-related problems for all but the hottest days. However, removing the filter will quickly gunk up the interior.
At the moment I think I have two choices. First, I could cut a hole for another fan/filter combination. I don't much like that choice, though. What I think I'll try first is to replace the HEPA filter with a simple foam one. It won't do as good a filtering job, but perhaps it will be good enough (and not significantly reduce the air flow). I'll try this out and report back with results next week.
Finally finished all of the backlogged wiring changes. From here on out, I really don't want to do any more. The biggest change was redoing my clock generation to use digital delay devices to properly shape my clocks. Looks good now, and I've cranked the main clock up to 3.75 Mhz. I was able to get things running at 4 Mhz, but it was a little unstable when running user-mode programs. I suspect my new critical path is in memory reference when copying memory between system and user memory, and I'm not going to try to shorten it. I'm inclined to leave the clock at 3.75 Mhz (unless I can find an oscillator in my parts bin that I can divide to something in the 3.75 to 4.0 range).
New schematic set with these change is now uploaded.
Since early this year I've had a list of wiring changes to apply to Magic-1, but I've been hesitant to do them. Magic-1 has been extremely stable (with the exception of a low tolerance for higher temperatures), but with all of the thousands of physical wiring connections I think I'm at the point that I want to avoid pulling boards and putting mechanical stress on anything for fear it will cause problems.
Anyway, yesterday evening I went ahead and took care of a few dozen changes - mostly "clean-up" to remove unused wires and chips left over from previous changes. I still have one last group of changes to make, and then I don't expect I'll make any further hardware modifications. The last change group is to clean up the system clock and I/O clock generation circuitry using digital delay devices rather than the hack I'm doing now. The purpose of that change group is to allow me to go ahead and permanently set the clock back up to 4 Mhz (I've been running at 3 lately).
Assuming that it all works as I expect, I'll finally move on to the Minix port. Although there's been a recent release of a new version of Minix (3.0), I've decided to port a previous version (2.x). To do this, I'll first need to clean up my toolchain, starting with writing a new assembler capable to generating relocatable objects (rather than the absolute assembler I have now).
Found a little time today to try out the new (eBay new...) logic analyzer. My old one was an HP 1630A, which served me well until a problem developed somewhere in its state acquisition system. The new one is of the same family, with more timing channels *and* a dual-channel digitizing oscilloscope - an HP 1631D.
Anyway, very cool toy - and after an hour of probing around I found a problem which is quite consistent with my observed failures when running at 4.0 Mhz (bungled serial I/O). In short, I'm getting a glitch on the read enable signal used by the uart's control registers. It happens frequently at 4 Mhz, but rarely (if ever) at 3.5 Mhz. Here's a smoking gun picture:
Click the picture to enlarge and get readable labels. The top line is the read enable signal, OE. When low, it causes data to be read from a UART register. It should only be read when I want it read because reads of the UART have side effects. In other words, a glitch here is a bad thing. To keep this signal from asserting at the wrong time, I have a special clock signal, IOCLK, which you can see in the second line. This signal is supposed to stay high until all address lines and key control signals have settled down. The key control signal in this instance is RW, shown in the bottom two lines. The first version is from the "analog" oscilloscope input, and the second is a copy from the digital timing input. The little blip in the center of the screen is RW bouncing around at a microcode instruction boundary before it settles down again. Note that IOCLK is going low during that bounce. That's the cause of the OE glitch.
The solution is simple - just delay IOCLK 20 or 30 nanoseconds. Actually, that should have already been done. My design called for IOCLK to follow CLKS by 60ns and it appears to be only following by about 30. I was doing this delay by just looping the signal through a few useless gates, but guess I didn't use enough. This would be a good time to clean up my act and get a real digital delay device.
I'm hoping this will enable me to permanently set Magic-1's clock speed to 4 Mhz - my original design goal. Other than the dropped serial I/O, 4 Mhz seems to work fine.
Not much activity on the project lately. The machine drops some serial port input at 4 Mhz, but I haven't had a chance to track down what I believe the problem to be (timing - not speed paths, I think). I've since rolled Magic-1 back to 3.5 Mhz where the problem still exists - but happens much less frequently.
To track this one down, I need a logic analyzer - and mine died last year. However, I bought a replacement on eBay (HP 1631D) which should arrive tomorrow. I hope to track this problem down, fix it and get Magic-1 back to 4 Mhz. After that happens, I don't expect to do any more hardware changes. The next big project will be getting Minix up and running. There is a new version of Minix, 3.0, that was just released. However, I think I will need to use the previous version (2.04), because of Magic-1's relatively slow speed and limited per-process address space.
I was pleasantly surprised to find that Magic-1 has been quite solid at 4 Mhz. There is a consistent problem when running in the loader (which sets full wait states for everything) in that input characters are frequently dropped - but this problem doesn't show up when running normally. I hooked up the logic analyzer and confirmed my suspicion: IOCLK isn't being delayed sufficiently. It only follows CLKS by 20-30 nanoseconds, whereas it really should be delayed by 60 or so nanoseconds.
I've had quite a few problems with my original (too simple) clocking scheme and my subsequent attempts at fixing it. What I've got now is a bit of a hack. I am trying to generate some fairly complex waveforms now by using gate delays for timing. This has turned out to be a bit more trial and error than is ideal. I'm going to give it some thought, but my inclination is to revisit my clock generation design and build my clocks with the aid of a digital delay device. I've never used (or even seen) one of these, but my understanding is that these things have a series of taps that I can run signals through and delay them by precise time values. Such a device would make life much easier.
WooHoo! - Magic-1 is running at 4 Mhz! Of course, it's not entirely stable, and I doubt it will stay up for too long - but it's doing pretty well. I have noticed a few dropped input chars, so I may have a timing issue related to IOCLK. I'll break out the logic analyzer and measure when some key signals are arriving relative to one another. Should be quite doable to make it stable. Also, I still haven't added the extra decoupling caps on the new memory board, so that may be contributing to the slight flakiness.
Oh, Magic-1's new Dhrystone benchmark score is 506. To put that in perspective, here are some historical numbers:
The new memory board is almost up and running now. I found the big problem last night, and what's left is a small design error that I'll deal with tonight. The problem that was causing me so much trouble this past week turned out to be a short between a signal line and the power pin next to it. It was a tough problem to find - I did two full board beep-outs and everything looked good. However, I could see no possible reason for my failing behavior except for a wiring issue. So, I decided to do yet another beep-out - but this time after I verified each connection I would also tap power and ground to see if there were any hidden shorts. Luckily, I found one about an hour into the process. It was a bad wrap - I must have pulled the gun off the pin before it completely wrapped, and a near-invisible end of the wrapped wire reached out and was touching the pin next to it.
After finding that, I did a very close visual inspection of each pin with a powerful magnifying glass - something I probably should have done to start with.
With that fixed, the machine booted up just fine. However, when I tried running a user-mode program it seemed to hang with constant drive activity. I turned on kernel verbose messages and discovered that when it launched a user process it would successfully fault in a few pages, but then get into an endless page fault loop. The problem turned out to be a minor design error. When I take a page fault, the fault handler is passed the faulting virtual address, but it also needs to know whether that address was in code space or data space. This information is passed via a bit in the machine status word, and is captured at the time of the memory fault. The bit in question is CODE_PTB, and is also one of the signals I'm capturing to enable address generation a half-cycle early. The problem is that the value of CODE_PTB that is captured at fault time is now not necessarily the one used to generate the address. That is the new xCODE_PTB signal generated on the new memory board (see "Page Table" schematic sheet).
The fix is pretty simple - just have the MSW capture xCODE_PTB instead of CODE_PTB at fault time. The only messy thing is that xCODE_PTB is a signal local to the memory card, whereas the MSW lives on the register card. So, I'll have to send xCODE_PTB across the backplane. To do that, I'll need to free up a backplane signal wire by getting rid of one of the redundant +5 or gnd lines. I'll also make a small change on the old memory card so that it can still be used if I need to switch back for some reason.
I also need to add some more decoupling capacitors to the new memory board. When I did the soldering on the board, I only had 10 decoupling caps. I've since gotten a bunch more, and will put in a couple dozen for good measure. Then, finally, I can start pushing the clock to see if I can get close to my original 4 Mhz target.
Still working of bringing up the new memory board. It all works fine so long as I don't turn paging on - but as soon as I do Magic-1 locks up on the next memory reference. It believes the next reference is a not-present page fault, and then goes into an endless fault loop when it tries to dump the machine context on the kernel stack (which it also believe is neither present nor writeable). What I thought would be the obvious problems that could lead to this situation check out okay. I've had to do a bit of rework on this board, so my earlier beeping out(s) may be outdated. When I find a few hours I'll do yet another beep-out on the board. Meanwhile, I've gone back to the old board and put Magic-1 back online.
I stayed up way too late, but finished the rewiring. It was ugly. One of the big problems with rework on a densely-wired board is that the wire-wrap bit sometimes won't cleanly fit over a post because there are too many wires bundled up next to it. I had quite a few of those - and they were pretty tricky to get. My other big concern is that I am almost out of my special cut-strip-wrap wire. I probably only have ten feet or so left. Besides being expensive, it's also somewhat difficult to get. Last time I ordered it, there was a two-month delay before the order was filled.
I'll have to beep out the entire board again before I try a live test. Perhaps tommorow. Here's the kitchen table aftermath from tonight's work:
Ripping out wires is among my least favorite tasks - but I guess it could have been a lot worse. Most of the wires were top-level, so I only had to undo a few dozen secondary wires. Here's the pile:
All together, I had to pull just over 100 wires. It should take me about two evening work sessions to replace them in the proper location, and then another session to beep out the board again. Meanwhile, I've put Magic-1 back online with the old memory card.
Arrghh! Is it possible to be this stupid?
I spent many hours last week wire-wrapping the new memory card, and the last couple of days beeping it out and desk-checking the design. It all looked good. Then, not unexpectedly, it failed when installed in Magic-1. However, it failed in some really strange ways - ways it shouldn't possibly fail. Signals were all weird - it seemed like half the backplane was shorting out.
And in fact, it was.
It's been nearly two years since I laid out the first Magic-1 board, and I forgot a small detail. The way I was using the 96-pin wire-wrap tags on both the backplane and the cards meant that I had to pick one that would have the rows "A", "B" and "C" reversed from the labelling. I chose to have the backplane match the wire-wrap tag labels, and reverse the board labels.
I forgot this when building the new memory card.
Here's the new card:
The rows of the two 96-pin backplane connectors were wired backwards. Row A was wired as row C, and row C was wired as row A. This will require a lot of work to fix - it isn't just a matter of removing single wires. Each wire I remove could result in having to pull two additional wires.
After surviving a week-long stress test, I took Magic-1 offline this evening. It handled more than 11 million IP packets while serving telnet sessions - all at 3.15 Mhz. Next up is work on the new memory card. I finished wire-wrapping today, and will be debugging for a while. It is a significant change - and requires a rewrite of the microcode (not difficult changes, just a lot of them).
I've continued work on the memory subsystem redesign. Here's a write-up of my current thoughts. I think I'm going to do it, though before I commit I want to do some timing measurements to make sure I won't just be uncovering another uncompressable speed path. I've fiddled a bit with compressing the memory path time by using fast F parts. Magic-1 runs for a short period at 3.4 Mhz, and then consistently crashes with a stack overflow. I haven't proven it, but this seems consistent to me with failure along the memory path - my guess is that it things are going bad on a timer interrupt, which causes a switch from user to supervisor mode (and thus does a page table mode switch). I've rolled back to 3.15 Mhz for now. Also, I went ahead and modified the "Factor" program in Magic-1's application menu to use long (32-bit) integers, rather than 16-bit ones.
Did a quick experiment tonight and confirmed that my SRAM to MDR (read) path is my critical speed path. What I did was replace most of the LS devices in that path with F parts and then tried cranking up the clock. Magic-1 failed at 3.6 Mhz, but appears to be running just fine at 3.4 Mhz. I'll let it go like this for a day or two - but this is good news. Further, I've checked my bring-up diary from last year (when I first attempted to find out how fast Magic-1 could go), and if my notes are correct I've got a lot of headroom in the MDR to Z-bit path. So, at the moment it looks like I'll probably go ahead with the plan to completely redesign my page table and address generation mechanism. I may get Magic-1 to 4 Mhz yet.
Very busy time at work last week, but finally found some time to fiddle with Magic-1. First, some photos from last weekend's Vintage Computer Festival. I didn't take nearly enough pictures, which is too bad. Lots of great stuff. Among the most memorable thing was the panel discussion featuring some members of the old Homebrew Computer Club from which so much great stuff originated. The room was packed, and because I was among the throng of folks gathered around Steve Wozniak in the hallway before the panel started I ended up standing in the back of the room. I wish I'd gotten a picture of him admiring Ken Sumrall's fantastically cool/nerdy nixie tube wristwatch. Here's the picture I did take:
From left to right under the screen are Michael Holley, Steve Wozniak, Allan Baum, Lee Felenstein and Bob Lash.
Magic-1's assigned display area was tucked into a corner, but lots of folks came by to check it out. I was surprised at the number who were already aware of the project from this website. Thanks to all who dropped by.
Note the banner taped to the bottom of the table. I printed that out on my Model 20 teletype machine. There were lots of great exhibits, including a Kenback-1 (the first personal computer), a great display of SWTPC gear, a history of single board computers, etc. The crowd-pleaser, though, was a fully functional portion of a Babbage difference engine constructed out of Mechano (Erector set) parts:
Beside the talks, panels, displays and vendor areas, it was also great having the event in the computer history museum. The musuem has recently completed its renovation of a PDP-1, complete with Spacewar! My friend Ken Sumrall (who is entirely to blame for me actually doing the Magic-1 project), was on the renovation team. He repaired the electro-mechanical Soroban typewriter used as an I/O device on the PDP-1, and was on hand to give VCF visitors demos of the PDP-1. Lots of folks got to play Spacewar!
This weekend I also did some fairly major rework of Magic-1's clock generation circuitry. In the ramp-up to VCF, I tripped over a problem with the hack I had earlier put in to handle slow I/O devices. Some background: when I originally envisioned Magic-1's virtual memory system, I intended to support wait states on a page-by-page basis. In short, fast SRAM would need no wait states, but slow EPROM and I/O devices could have a bit set in their page table entries which would cause Magic-1 to insert some extra cycles for memory accesses. However, when I built the machine, I never implemented it properly. Instead, I just threw in a hack by which I could programmatically drop the main clock speed in half. I would do this in the device drivers.
Well, that's a pretty kludgey hack. Not that I mind hacks, but I have a personal "rule" of sorts that if I ever get bitten by a hack more than a few time I need to go back in and fix it properly. So, that's what I did this weekend. It turned out to be more difficult than I anticipated. As is typical for me (a software guy), I focused on the logic of generating the now somewhat complicated clocks. Turning wait states on, off, resetting properly and correctly handling my front panel "stop clock" switch made this change a bit tricky - but from a logic point of view it only took the addition of a couple of new flip-flops and an AND gate. However, I didn't think enough (at all?) about what the propogation delays of these changes would mean. In short, my circuit didn't work in the real world because the new flip-flops caused enough of a delay that I missed the arrival of the _WAIT signal from the page table. I finally got things to work by delaying a troublesome edge about 30 nanoseconds by routing it through a series of spare inverters and AND gates that were on the board unused. It would have been better, I suppose, to use a proper digital delay device, but I just don't have any on hand. I don't think my basic design was all that bad - getting the timing right on this stuff is just tricky.
The other thing I realized last week was that can see a way to shave between 30 and 40 nanoseconds out of my memory path. This is a pretty huge improvement, and given that my memory path is currently the limiting factor in my clock speed, it could mean a sigificant speed-up for Magic-1. However, the change would essentially require me to completely rebuild the memory board and rewrite the microcode (just fairly minor microcode changes). If I decide to do this, I won't alter the existing memory board, but will build a new memory board using one of the spare wire-wrap boards I've got. The main issue for me now is determining how much speed-up it would really buy be. My memory path is my current critical path, but earlier the "alu-op to condition code" path was my limiting path. I cleaned that one up by using fast 74F parts, but I don't know how fast it got it (because I ran into the memory critical path). It may be that if I fix the memory problem, I'll just run into the old ciritical path, which I don't think can be easily improved. I think if I decide that making this change will move Magic-1 to 3.5 Mhz or higher (from the current 3.0), I'll do it. Otherwise, I'll probably just leave things the way they are. My early calculations, though, suggest that I could get the memory path running fast enough to support 4 Mhz - my original target. Sometime this week I'll try to calculate (and measure) how much headroom I've got in the "alu-op to condition code" path.
Had a great time at VCF 8.0 this weekend at the Computer History Museum in Mountain View, CA. I had the chance to show off Magic-1 and talk with a lot of folks who thought I was completely nuts - but in a good way. Lots of good speakers and real homebrew computer pioneers - I walked around in awe of the folks who showed up. Great exhibits - particularly a functional portion of a Babbage difference engine built out of Mechano (Erector set) parts.
I'm tired now (two days on my feet), and will post some pictures and details later - but have to mention that the weekend was capped off with Magic-1 getting named "Best of Show". It was a good weekend.
Whew! Big relief here - I found a nasty bug that's been tormenting me for weeks. And, unsurprisingly, it was a self-inflicted error. In short, I've been trying to polish up Magic-1's software stack in preparation for the VCF showing this weekend. But, I started getting I/O hangs when I made what should have been completely unrelated, tiny changes. I managed to get one version of the OS which would either hang or not hang depending on the order of a simple load instruction and a NOP. NOP then load - no hang. Load then NOP - hang. Crazy bug - the load was completely vanilla, nothing special at all. And, the hang was an I/O hang. The system was still responding to interrupts, but seemed to drop all serial data.
At this point, I'd convinced myself it was a hardware problem - I thought that perhaps some odd combination of bits changing state between the address of the load instruciton and the target of the load was somehow causing a spurious "read" signal on the UART's data register. I decided to break out the logic analyzer.
This was my first time since Magic-1 was put into the new case, and it worked great. I designed the layout of the front panel and cage such that you could unbolt the front panel and lay it down to give easy access to backplane signals. Anyway, my first test was the verify that my special "IOCLOCK" signal was getting done right. I did this only as a baseline before looking at the signals I was really interested in, but was surprised to find that signal flatlined. IOCLOCK is supposed to mimic the system clock, but shifted a half-phase. It is a bit of a hack - I added it after the fact to work around a design error I made when wiring up the UARTS. It only oscillates when Magic-1 is running at half-speed, and there is a special flip-flop that controls it. The important thing here is that this signal is what prevents spurious reads of the UART data register. Without it, incoming charactors will be ocassionally lost.
I'll shorten the story: I first suspected a broken wire, bent pin or other hardware problem. Didn't find one - all connections looked good. Just to be safe, I then replaced the ICs involved in it's generation. No luck - same behavior. Then, it finally dawned on me. A while back I was mucking around in the runtime assembly code "cleaning" things up. I remember touching the low-level clock speed-changing routine. That was a month or more ago, but sure enough - I broke the routine during my cleanup. I fixed the routine and things got much better. I still have a few repairs to make (hacks and workaround added during the search for the bug), but things seem solid again. I dropped the clock back to 2 Mhz during the bug hunt, and I'll try cranking it back up to 3 Mhz sometime before the weekend.
Lots of little tune-ups in the os/monitor to get ready for this weekend's outing at the Vintage Computer Festival. Things are looking pretty solid. The web server has processed several million packets during last week's stress testing, and I've tuned up the non-blocking I/O routines in the kernel to minimize the number of dropped packets when higher priority processes are running. Finally, using a walkthough, I ran the 350-point colossal cave adventure to completion. Brings back memories - especially when run on the green-screen dumb terminal that Ken Sumrall lent me to use at VCF.
I'm going to stop messing with the machine until after VCF, but will continue stressing the background web server.
The stress testing is going reasonably well, though it did show up a problem in the web server. Magic-1 successfully handled 500,000 packets but then the web server got confused and refused all new TCP connections. It handled pings during this time just fine. Some bug in my uIP port, no doubt. At some point this week I'll add some debugging code to try to shed some light on the problem. In other news, I ported the 350-point version of Original Adventure. This one is very close to the old FORTAN version, and is closer to the version I played back on my TRS80 Model-1 years ago. The other version of Adventure I ported earlier is the one that shipped with Minix, and was a modified 550-point game. The 350-point one takes quite a long time to start up because it has to parse a data file containing messages and item names, and I've had to disable save/restore functionality for the time being, but things seem to be working well.
For the next few days I'll be subjecting Magic-1 to some stress testing. I've run into a few cases in which the web server has hung, and although I've put in fixes for those cases I suspect there are a few more problems dealing with blocking processes. So, I'd expect web pages served by Magic-1 to be even slower than usual for a while. The telnet sessions should appear to be relatively unaffected, though, as it and all spawned processes will run at a higher priority than the web server. If anyone notices a problem, let me know.
Quite interesting time on the debugging front lately. I've been struggling a bit getting pre-emptive multi-tasking working reliably. In particular, I was running into those awful bugs that seem to move around. Magic-1 would fly off into the weeds at odd times, and what diagnostic messages I got were nonsensical. It took more than a week to find the first bug: an improper "optimization" in my LCC compiler retargeting that left an unbalanced stack on some flavors of leaf procedures. The tricky part there was when I enabled debugging kprintf()s, the affected leaf procedure stopped being a leaf and the problem disappeared.
The second problem had similar characteristics. It only showed up on a recompile of Original Adventure, and caused very strange death. Control flow was clearly going badly wrong. And, like the first problem, almost any code I added for debugging purposed caused the bug to disappear. I finally figured out that the added code mattered only in the way it shifted the existing code around. By spending an afternoon doing a binary search with a block of added NOPs, I was finally able to create passing and failing versions that differed by only the position of one instruction. It was a simple "CALL" instruction, but had the interesting property that it spanned a page boundary in the failing case [a CALL is 3 bytes long - one for the CALL opcode and 2 for the pc-relative target displacement. In this case, the opcode was on one page and the displacement bytes were on the following page]. Once I saw that, the problem was clear.
When a Magic-1 instruction faults, it is supposed to roll back the machine state to exactly as it was before the fault occurred. Given the small number of internal temporary registers that Magic-1 has, this was quite a challenge - but I thought I had covered all the cases. I didn't. What was happening in my bug case was that the page containing the opcode was in memory, while the page containing the rest of the instruction was on disk. During instruction execution, the CALL microcode materialized the return address, pushed it on the stack, and then attempted to fetch the branch displacement bytes. However, this page was still on disk and generated a page fault. The fault microcode restored the PC to before the call, but left the modified SP alone. After the faulting page was read from disk, the CALL instruction was restarted from the beginning - only SP now pointed 1 word down on the stack. After that, chaos reigned supreme.
The fix was fairly easy - a rewrite of the CALL microcode. I was particularly pleased that I was able to do a new CALL sequence in the same number of microinstructions.
For the moment, I'm doing something of a code freeze in preparation for Magic-1's visit to the Vintage Computer Festival in Mountain View, Nov. 4 and 5. The state of the world is that multi-tasking seems to be solid now. Magic-1 can finally serve web pages and run a telnet session at the same time. When you telnet in, you can see the current state of the running tasks by typing Ctl-G. My scheduling algorithm is overly simplistic, and will result in some dropped IP packets from time to time, but overall I'm very pleased. The blinky-light pattern is especially nice now. My "idle" background task consists of a single HALT instruction. When all other processes are blocked on I/O or the timer, control goes to the idle task - which "halts" the machine. When an interrupt arrives signaling I/O completion or timer expiration the HALT is interrupted. The interrupt is then serviced, which may result in a blocked process being moved to ready. If so, that process (having a higher priority than the idle task) will resume execution. There's a great blinky light pattern when this happens.
WooHoo! Magic-1 is multi-tasking. I got it working last night with two background tasks - an idle loop and the web server. Getting this working was one of those (all too rare) software development moments in which the code got smaller as the new feature progressed. Still a bit more work to do, though. I left things running when I went to bed last night, but the machine crashed sometime during the night. I'm not surprised - and I expect I've got interrupt window problems. Once I track that bug down, I'll also need to do some tuning. My "ping" response time degraded by 50% due to all of the new overhead added to handle task stacks and pre-emption. Should be fairly easily to speed up, though.
Lots of progress today on the pre-emptive multi-tasking front. The foundation work is almost complete. Running processes use a save state structure and all of the nasty code dealing with switching the kernel sp between the true kernel stack and the process save-state structure on interrupts and systems calls is working. I could probably push through and get it working in an hour or two, but I've decided to do a bit of restructuring of the process/task launch code. Most of it is done in assembly code now, but I think I can move quite a bit of it up into C code.
I've had Magic-1 offline most of the weekend. I'm opening it up again now.
10/1/2005 [evening update]
I managed to find a few hours today, and the update of the monitor/OS to use non-blocking interrupt-driven serial output ended up being very easy. I designed the code to do this originally, but after getting things working via polling last year, I never bothered to finish it. It only took a dozen or so lines of code, and it seems to work fine. The only problem I ran into turned out to be temperature sensitivity. At one point in the afternoon, I started getting odd failures while running the web server. I assumed it was the code changes I was making, but after reverting to a previous version and hitting the same problem, I tried running again at half speed (1.5 Mhz). The problem went away. I then removed the back panel from Magic-1 and added an external fan. Correct behavior at 3.0 Mhz followed.
So, sometime in the next couple of days I need to fiddle with Magic-1's cooling fans. I have 2 (3 counting the one in the power supply), but I previously made them run quite slowly in order to make Magic-1 quieter. Guess I've got to crank things up a bit.
Not sure when I'll get a chance to work more on the multi-tasking upgrade. Perhaps tomorrow.
Magic-1 continues to get heavy use from telnet visitors. I keep a simple command counter, and it shows that some 4,200 commands have been executed by visitors over the last 10 days. The down side of this is that since the machine is getting so much telnet usage, I've not been using it as a web server (because it can be either a telnet server or web server - but not both at the same time).
Anyway, I think it's time to address that shortcoming. The simple monitor/OS I've written for Magic-1 is almost capable of pre-emptive multi-tasking. I stopped just short during the implementation because my plan was (and is) to port Minix. I figured it would be a waste of effort to fully support timesharing in my temporary OS. However, it will be a while before I get to the Minix port. Partly this is because I've been busy elsewhere, but also because there is a big blip of activity in the Minix community right now and I'm waiting until it becomes stable again before I begin my port.
So, I've begun to investigate what I need to do to support at least limited multi-tasking. My minimum requirement is the ability to run the web server in the background while Magic-1 is serving telnet sessions. I don't believe I'll have much difficulty with task switching & scheduling - the code is almost there right now (just need to add a more flexible save-state structure and a few lists). The bigger effort is making my I/O non-blocking.
I effectively have four I/O paths: serial character in, serial character out, IDE block in and IDE block out. Of these "serial in" is interrupt driven, while the rest are polled. My initial plan is to just leave the IDE block operations polled, but interruptible (because the web server only uses serial I/O). I'll need to make serial out interrupt-driven, and then add a "suspend" system call. The way serial in works right now is that when data arrives through the UART an interrupt is generated. The serial port interrupt handler loads the new character and stores it in an OS buffer. Serial input system calls return characters from the buffer (not the UART directly). The code here is already non-blocking (if no char is available, 0 is returned). All I will have to do to complete this is to have the libc.a wrapper code for a "blocking" serial input invoke "suspend()" if it wants a new char and none is available, and then support task resumption after the next serial input interrupt is serviced.
Serial output will take a bit more effort. I already have an output buffer similar to the input buffer (but am not using it). What I'll need to do is change the output code to have two portions - filing the buffer, and copying chars from the buffer to the UART. The 2nd half of that code will done as an output interrupt handler (the UART is capable of generating an interrupt when it is ready for the next group of chars). The other change is that the code which copies the output string to the output buffer must return to the caller even if it has not copied all requested chars. For example, if the caller wants to output 100 chars and there is only space in the buffer for 20, the code must copy 20 and report to the caller how many it copied. The caller should then note how much of the request was completed, and then "suspend()" to be reawakened after the output interrupt handler is next run.
This all seems pretty simple, but I'm expecting this to take me a while to get right. Debugging problems with interrupts and timing is always tricky. I think I'll first make serial out interrupt-driven, then rework the process/task save state structures and finally add the process ready/blocked/run lists and suspend()/switch.
Not much activity on the Magic-1 (or Magic-16) front lately. What free time I've had has been consumed with projects around the house (clean out garage, minor house maintenance, etc.). I hope to resume with the Magic-16 project soon. Meanwhile, Magic-1 has been humming along rock solid. Over the last few days this website had a huge traffic spike - something like 6,000 visitors from South Korea in two days, and I think most of them also have been telnetting into Magic-1. It has been in constant use for the last several days, and more than 100 new guestbook messages have been left. From the comments, it appears Magic-1 was mentioned in a newspaper article.
I continue to be amazed at how solid the machine is. Except for neighborhood power outages and my occasional fiddling, the machine has only crashed one in the last few months - and that was because of overheating (I left a dehumidifier on in the bedroom with the door closed for 12 hours and the room got > 90 degrees.
Oh, one other bit of news. I will be showing off Magic-1 at the Vintage Computer Festival in Mountain View, CA in early November. More on that later.
The Summer edition of ExtremeTech Magazine with the interview & pictures of Magic-1 came out today. Looks good - go buy a copy (or two). Eagle-eyed readers may notice what appears to be a minor mistake in the article, but rest assured that it's no mistake. It's Karma. Back in my newspaper days I was notorious for misspelling names in the news articles I wrote. Today I repay some of that Karmic debt, as the ExtremeTech article refers to me as "Bill Buxbee" instead of "Bill Buzbee".
It's a nice article, though, and occupies the page facing in inside rear cover. Note also the small picture in the table of contents on page 3. My manager at work, on seeing a larger version, broke out laughing. Monica calls it my "alpha nerd" pose. Hey, I think it should have been the cover shot!
I've been continuing my exploration of Verilog and FPGAs, and have decided on my next step. To increase my fluency in Verilog, I've started writing a little program which will generate simple state machines using structural Verilog given regular-expression input specifications. More details on this later. Meanwhile, watch for the next issue of Ziff-Davis' ExtremeTech Magazine (should be on the newsstands Aug. 16). Magic-1 and I will be featured.
Oops - I just realized that when I switched hosting services a couple of months ago I failed to properly set up email forwarding. Instead of forwarding email from firstname.lastname@example.org to my real email address, it was sent to a bit bucket. I respond to most all email, so if you've sent me a message and got no response in the last two or three months it probably means I didn't see it. Should be fixed now. Sorry 'bout that.
Haven't had a chance to do much lately - busy with the paying job & summer household projects. I did, though, start getting up to speed on Verilog. Read Samir Palnitkar's "Verilog HDL" and have created a few trivial modules on the FPGA development board ("Hello World" and a Fibonacci generator). Fun stuff - looking forward playing with soft CPUs.
I'll probably start by getting an existing one running, then modify it and finally write the Verilog for Magic-16. The key feature of the Opal-Kelly XEM3001 that I'll be using is it's USB interface. The grand plan is to write simulations for main memory, the system clock, peripherals and a debugging interface on the PC, and then use the Opal-Kelly libraries to communicate with the development board. As debugging and development progresses, I can move more of the functionality into the FPGA (and also support a co-simulation type of testing).
Fiddled with the Magic-16 ISA a bit. I really like shift-and-add instructions - you can do elegant multiplications by small constants. So, I added a set. In truth, though, they consume more of the opcode space than they're probably worth, so I may scale back later. Right now, the biggest weakness in the ISA is the lack of decent bit twiddling instructions. It would be nice to have bit extract and deposit, along with a shift-double. However, about the only way I could fit them in would be to make them variable with the shift/size/bit position stored in a count control register. I don't think I want to go down that road.
A very cool FPGA development board, the Opal-Kelly XEM3001, was waiting for me when I came home from work last night. This little gizmo features a Xilinx Spartan-3 FPGA and a slick USB interface. It's finally time for me to start playing with FPGA. I've sketched out the basics of a CPU design to put in it. More on that later.
Well, it looks like Magic-1 is getting famous. Look for issue #4 (the summer issue) of Extreme Tech magazine. It will hit the newsstands on Aug. 16, and will include an interview w/ picture of me and Magic-1. Interestingly, this was all set up prior to the Slashdotting. I think getting a magazine interview finally impressed my wife, though the teen-age daughter continues to swear she'll die of embarrassment if any of her friends discover just how big a geek I am.
In other news, I finally got tired of typing my home DSL line's static IP in order to reach Magic-1. So, I've registered "magic-1.org" and used the freebie DNS servers at zoneedit.com to resolve to 22.214.171.124. Now, all you have to do to reach Magic-1 is go to http://www.magic-1.org, or "telnet www.magic-1.org" depending on what Magic-1 is doing.
Lots of new comments on Magic-1's guestbook. I left it doing telnet sessions all weekend and most of Monday. See them here.
This weekend I've tried to make Magic-1 available for telnet sessions, and have had many visitors. I think I'll try to make this a fairly regular occurrence. I'll keep Magic-1 running as a web server during the week, and open it up for telnet sessions on the weekend (except, of course, when I'm developing new software for it).
One of the things the telnet session allows is the ability to sign a guestbook. More than 100 people have, and it's getting a little long to display. So, I've decided to create a new page on this website and transfer the comments from Magic-1 there whenever they start getting too big.
Here's the Guestbook page - check it out.
This morning I transferred the web site to a new hosting service, and have been very pleased so far. Thanks to all of the great emails and comments. I particularly wanted to call out a pointer that Mikael P. sent me - the Netcraft site report for Magic-1 for 6/7/2005. My little homebrew computer ranked 170,399 on the whole damn internet, just edging out www.bustyandsexy.net (which came in at 170,400). I'd make a joke here about Magic-1 being better than sex, but I can't think of any wording which wouldn't get me into some sort of trouble.
Web site is still down, and my service request to the hosting service has gone unanswered. Decided to mirror temporarily to a subdomain on the Buzbee family site.
Decided to go ahead and bring down the web server in order to recompile it with 32-bit counters. Traffic is starting to significantly drop off, but remains pretty strong.
Well, this was certainly an exciting day - just as Monica and I were getting the kids ready for school, we appeared to lose internet access in the house. Then, I walked back to the bedroom and noticed that Magic-1's serial port interrupt LED was practically solid on. I guessed correctly - we'd been Slashdotted. Slashdot is an extremely popular Nerd website, and one my regular daily reads. I thought that some day they might take an interest in Magic-1, and apparently my getting Adam Dunkels' uIP to serve pages outside of my firewall the night before got the notice.
There was a front-page mention of Magic-1, along with links both to the main website as well as a direct link to Magic-1 (on my home static DSL line). Thousands of bored slashdotters were simultaneously trying to talk to Magic-1, saturating my poor DSL line. Interestingly, though, Magic-1 did just fine. Unlike me, it's smart enough to only bite off what it can chew. It accepted connections when it could, and just denied those it couldn't.
My home DLS line, though, was another matter. Like most, it has a much higher incoming bandwidth (1.5Mbits per second) than outgoing (128Kbits per second). Many requests in, few out. Now I know what a denial of service attack feels like. Because we had to get the kids to school, I didn't have time to do anything fancy. First, I just pulled the plug on the DLS connection, and then I reprogrammed the router to ignore packets destined for Magic-1, and then reconnected the DSL line. That done, Monica could do her email and I headed off to work.
Besides the heavy traffic to Magic-1, there apparently was an awesome storm of traffic to this website. This website was not designed to survive heavy traffic - it's filled with enormous pictures, documents and videos. My hosting service was forced to pull the plug within a few minutes of the link appearing on Slashdot. As I'm writing this (now 6/8/2005), the site is still down.
I read all of the Slashdot comments, and although most of them were pretty lame, it was nice to see that some folks appreciate this the kind of weirdness that led to Magic-1.
When I finally got home, I screwed up and accidentally killed the uIP web server before I could recover the stats on how well it performed before I turn off the DSL connection. I'm a bit bummed about that. However, I went ahead and put it back online and released the router block by about 6 p.m. It was fantastic to watch - Magic-1 just churned away and ended up serving up 25,000 pages by morning. I'd only used 16-bit counters, so I don't know how many packets flew through - the counters rolled over at least once.
Fun day - and thanks to all who sent me email (though I fear that I may have lost some email during the time in which the website (and mail forwarding) was shut down.
The good news is that Adam Dunkels' uIP TCP/IP stack is running like a champ on Magic-1. It's serving web pages and responding pings and displaying dynamic data on the TCP/IP stack as well as Magic-1's internal state.
The bad news is that I'm a hopeless idiot when it comes to network configuration. All of the above things work - but only behind my firewall. I've been struggling for two days now, and seem no closer to being able to get Magic-1 to respond to requests from the outside my firewall.
Any hints would be appreciated....
What I have is a wireless router/firewall (D-Link DI-713P) connected to my home DSL line. It's wired directly to my wife's computer, and talks wirelessly to my laptop and a wireless bridge on the other side of the house. The wireless bridge is connected to a hub, which in turn connects to my Linux box. My Linux box is connected to Magic-1 via a null-modem serial cable.
For reference, my local IPs are:
On the Linux box, I issue the following commands:
and on the D-713P router I set port forwarding to pass along all port 80 packets to 192.168.0.76. What I had hoped was that http requests sent to my static DLS line (126.96.36.199) would be forwarded to Magic-1's IP of 192.168.0.76. However, it just doesn't happen. Inside my firewall every computer in the house can get pages from 192.168.0.76, but outside requests to 188.8.131.52 fail. I know that the DI-713P's port forwarding does work in at least some cases. I set up apache on another computer in the house and had the DI-713P forward http requests to it's local IP (192.168.0.99). From outside my firewall, http requests to 184.108.40.206 worked just fine - forwarding on the 192.168.0.99. Something must be different when the forward target is the other end of a SLIP connection, I guess.
Anybody know how I should be setting things up? Email me at email@example.com.
Despite having declared the project "complete", I've been working furiously on it - at least on the software side. First, I can report that the crashing problem that I had right after last week's road trip has not returned. Reseating all the chips appears to have done the trick.
Now, what I've been doing on the software front is working on making Magic-1 a web server. I'm using Adam Dunkels' uIP package - a great little TCP/IP stack for small machines. I've been struggling with a series of tough and subtle problems, but am on the verge of complete success. So far Magic-1 is responding to pings and will properly serve web pages when requested by the "wget" utility. However, there is something wrong with the TCP/IP conversation when it comes to using the Internet Explorer and Firefox browsers. We negotiate a successful connection, but Magic-1 times out waiting for a GET request. Have to do some more probing there.
The physical setup has Magic-1's auxiliary serial port connected to the serial port of my Linux machine (the one with the Transmeta Crusoe CPU) using SLIP over a null-modem cable. The Linux box forwards packets out to the rest of the world, and my router is configured to pass packets designated for my home DSL static IP address (220.127.116.11, port 80) to Magic-1's local network IP address.
Took Magic-1 on a road trip last night - showed it off at the Monday night meeting of computer collectors at Vito's Pizza. It performed like a champ. Unfortunately, since I've had it home there have been two crashes - the first ones in months. This morning I pulled all of the boards and reseated all the chips. Hope that fixes things. Anyway, Magic-1 is back online.
Ken Sumrall gave me a 40 MB Kittyhawk drive, and lent me another to test. This led me to clean up and improve the IDE drivers in my os/monitor. I don't have as much info on the Kittyhawk as I'd like, and Googling hasn't helped all that much. So, I added a Kittyhawk page to this site to record what I discover. What I'm particularly interested in is info regarding how to set these drives as master or slave. So far I've only been able to use them as masters.