I have no idea why the networking in Basilisk II keeps stalling

And it is quite frustrating.  The most I can do is about 100MB worth of AppleTalk traffic, or 1.5GB of TCP/IP then the receive function EtherReadPacket just stop being called, and then the whole thing stalls out.

I don’t really ‘like’ my solution, but it does work.  I went ahead and chained the EtherInterrupt function to the 60Hz timer to ensure it’ll fire, and it seems to be working. The good thing is now I’m getting ~200K/sec using pcap or SLiRP.  So things are faster!

Then after scanning the changelog, I found this interrupt patch, and it doubled my throughput on the network to over 400K/sec!

427K/sec via SLiRP

427K/sec via SLiRP

So now I can copy about 350MB worth of data in about 5-7 minutes, and it doesn’t stall out.

357MB worth of AppleTalk

357MB worth of AppleTalk

I can now copy hundreds of MB worth of stuff from one AT server to another.

What is also surprising is that by using Internet Explorer 4.0.1 for MacOS, I get speeds of around 1.0Mb/sec(with as high as 1.6!)

Internet Explorer 4.0.1 screaming along

Internet Explorer 4.0.1 screaming along

I know IE has always had a bum rap, but it really is a better legacy browser on MacOS.

I also merged the scsi driver’s buffer with BasiliskII’s buffer so the scatter/gather can now handle the absurd requests of 4MB++ worth of reads in one swoop.

So I started to look at the SCSI passthru on Basilisk

And personally I’ve never seen the appeal for such things, but apparently for the world of emulation accessing physical media is a big deal. Of course what I didn’t think about was rescuing old machines by re-installing the OS under emulation, or copy protection.

The first thing I looked for was a GPL project that has SCSI disk support and isn’t too complicated.  The Previous project sure fits that bill, scsi.c is not even a thousand likes, and even better it works!

The only two major hurdles I ran into is that the Mac is sending a page request of 0x30 which as far as I can find is not listed anywhere else.  I ended up just making one up as a reply to see if it mattered.

The other is that it’s scatter/gather based, so when it’s going to read or write several contiguous sectors, it’ll blast down up to 256kb worth of data to be read or written to.  The ability to know that a large operation was in progress was already in Previous, it just wasn’t set to loop.  I guess the NeXT isn’t as aggressive, or it’s SG operations are better contained in the SCSI controller.

The final hurdle was in the Apple partitioning software.  I’ve been down this road a long while ago.  But the disktools from A/UX 3.0.1 doesn’t care about vendors and will thankfully format anything.

SCSI disk files

SCSI disk files

So not as exciting as talking to a real SCSI disk, but it’s safer.  I suspect that accessing a raw NT device name ought to work  I can test that on VMWare, but the trick is finding something that can read HFS and prove it’s a good exercise.

Another ‘feature’ I put back in is the ability to disable the math coprocessor on the 68040.  It feels more stable to rely on Apple’s old emulation code, but maybe that’s me.

As always files for this are on sourceforge.

PCem

PCem v9

PCem v9

From the main page:

PCem v9 released. Changes from v8.1 :

  • New machines – IBM PCjr
  • New graphics cards – Diamond Stealth 3D 2000 (S3 ViRGE/325), S3 ViRGE/DX
  • New sound cards – Innovation SSI-2001 (using ReSID-FP)
  • CPU fixes – Windows NT now works, OS/2 2.0+ works better
  • Fixed issue with port 3DA when in blanking, DOS 6.2/V now works
  • Re-written PIT emulation
  • IRQs 8-15 now handled correctly, Civilization no longer hangs
  • Fixed vertical axis on Amstrad mouse
  • Serial fixes – fixes mouse issues on Win 3.x and OS/2
  • New Windows keyboard code – should work better with international keyboards
  • Changes to keyboard emulation – should fix stuck keys
  • Some CD-ROM fixes
  • Joystick emulation
  • Preliminary Linux port

Thanks to HalfMinute, SA1988 and Battler for contributions towards this release.

Very excellent!

Announcing Cockatrice III

Well I was shuffling files back and forth into Shoebill, and with the advent of Ethernet support, I decided I wanted to build an AppleTalk network.  This endeavor seems to have taken a life of it’s own.

So, the first thing I did was tear into minivmac, as I figured it would be the easiest to modify, as ‘mini’ is in it’s name.  But it’s more geared to LocalTalk.  From it’s readme:

It does this by converting the LocalTalk packets between SDLC frames in the virtual machine to LocalTalk Over Ethernet (LTOE) packets.  These LTOE packets will be sent out the host machines Ethernet interface and will reach any other machine on the LAN.  LTOE packets are not routable and not recognized by EtherTalk devices.

Which is pretty creative, but I want to talk to A/UX, Windows NT and Cisco routers.  So this isn’t going to work out for me.

The next other ‘big’ names in Macintosh emulation are Basilisk II and SheepShaver.  Both of which are from Christian Bauer which is a sizable download (or so I thought) and has a very confusing release versions for Windows. So I went ahead and tried BasiliskII, which only does some native networking via a TUN/TAP & bridge solution (which is really popular solution for plenty of UNIX based stuff), which personally I don’t really care for.  The Windows version does support SLiRP, but for some strange and annoying reason it always crashes when I try to download anything big.  As a matter of fact, the Windows version crashes, a lot!

While digging around for various builds of Basilisk II, I found the defunct sourceforge page, which is thankfully still up.  And there I found the 0.8 and 0.9 release source code, which weighs in at a tiny 350kb in size.  This is something I could probably dive into.  So I went ahead and tried to build it on a Debian 7 x86 VM.  And much to my surprise, after altering configure to accept GCC 4.7, and forcing it to turn X11 on (I don’t know why it kept failing to detect it), I was able to build a binary in no time.  Even better, it worked!

So the first few goals were simple, I wanted to take 0.8 and remove it’s dependency on X11,and make it use SDL 1.2.  Why not SDL 2.0?  Well 2.0 is more about 3d space, and even to render a flat framebuffer it uses streaming textures.  Which is too heavy for me, so I’m sticking with 1.2.  I took a bunch of code from SDLQuake, and after a while of bashing it around, I was able to open a window, and capture some ouput from the framebuffer.  With even more bashing around I got it to work correctly.  I did make some small tweaks though, it only supports 8bit depth.  But I’m interested in networking, so 256 colours is fine by me.  Now that i could see what I was doing, I was able to then re-compile on OS X, and I was greeted with the Mac Boot screen.  The harder part was Windows, as the system code written by Lauri Pesonen who did an excellent job of porting BasiliskII to Windows, but to say their code took 100% advantage of the Win32 API would be an understatement.. And I wanted something more pure to being SDL so I really couldn’t use much of that code.  And what code I could find it was for far later versions.  However with enough pushing I did finally get BasiliskII to boot up on Windows.  I was once more again bitten by the fact that open on Windows defaults to being in ASCII mode.

The next thing to add was SDL input for the keyboard and mouse.  And at this point googling around for an example of an input loop for SDL that is appropriate for an emulator I stumbled uppon the fact that there already was a SDL support built into the more current version of Basilisk II.  But for some strange reason I kept going ahead, and incorporated some of the code into my 0.8 branch.  And then I could finally send some keystrokes, move the mouse, and click on things!  Things were looking up!

While looking at the SDL code, I did see they also have audio support, so I went ahead and borrowed the skeleton framework from there, although the initialization didn’t work at all as BasiliskII had drifted in how it hooked into the native sound support.  So I once more again turned to SDLQuake, and I was able to initialize sound, and Even get QuickTime to play the old Quadra quicktime video, which was the first QuickTime thing I’d ever seen, back when they were still making Quadras.

So now with video and sound in place, it was finally time to tackle the networking.  At first this seemed quite easy to do, and using SIMH for inspiration I was able to quickly replace the tun/tap code with some pcap code to open the interface, send packets, and receive packets.  One more again I started on Linux, made it build on OS X, although my MacBook air doesn’t have anything I can really inject packets into so I don’t know if it actually works.  The bigger test for me was on Windows with a GNS3 network, and with a few more minor changes I was happily sending AppleTalk to both Shoebill and Windows NT.

The next thing I wanted to tackle was SLiRP support.  Ironically to bring SLiRP to Shoebill I used the SLiRP from the github of Basilisk II.  At this point I figured this would be very simple, and I could wrap up later that day.  It ended up taking me three days.  Once more again my build would crash all the time, just like the later Basilisk II builds.  Using Internet Explorer 4.0.1 would seemingly crash the whole system within seconds with faults in SLiRP’s slirp_select_fill, and slirp_select_poll functions.  Now if you don’t call these functions SLiRP doesn’t process it’s TCP state and you end up with barely functioning UDP to only SLiRP which isn’t great beyond DHCP and DNS.  First I tried semaphores which only made things worse as the nature of Basilisk II’s threaded nature just made the requests stack up deadlocking within seconds.  I tried a mutex, timed mutexes and various other locking methods insdide of SLiRP and Basilisk II to no end.  Netscape would kind of work, but IE would crash the whole thing out after a few pages. Then a better solution hit me as I was playing with the system clock on the Windows build.  There is a 60Hz timer that calls a 1Hz timer once every 60 ticks.  What if I had the clock drive SLiRP?  And to my amazement not only did that work, but it worked great until I hit another problem that I had with Shoebill (that needs to be fixed now that I found away around it here).  There is a static buffer that passes data between SLiRP’s callback when it is going to send a packet to BasiliskII and when Basilisk II then feeds the packet to MacOS.  With enough traffic it will overwrite part of itself as they are on two different threads.  Once more again I tried semaphores, which of course is the wrong tool here as if something is stacking waiting for it to unstack is just crazy, and more mutexes.  The mutexes kind of worked but performance was horrible, as in 1992 dialup speed horrible.  And I didn’t want to simulate a 1992 internet experience 100%

So the obvious solution as a queue.  I took a simple queue implementation, added the ability to peek, changed it to accept a packet structure and I was set.  Now I only needed a mutex when I queued items, and dequeued them.  But I could hold 100 packets easily.

So with all that in place I can finally download files greater than 10MB, and even with Internet Explorer!

Download

Download

So the next was to make Pcap dynamically loaded, which for C++ is a bit of fun with __cdecl, GetProcAddress and all that fun.  But I had it working after a bit so now if the user doesn’t have WinPcap installed they don’t get an error message, and I don’t have to maintain two builds.  Nobody likes doing that kind of stuff.  Ever.

Multitasking.. Kind of.

Multitasking.. Kind of.

There is still plenty of things broken afterall I’m using an ancient version of Basilisk to base this off of. I’ve also removed a bunch of features as I wanted to make this more of a ‘core’ product with again a focus on networking.

Will this interest the majority of people? Probably not.  But for anyone who wants to actually download a file this may be somewhat useful.

Where to go from here?

Well there is still a lot of OS specific stuff in the code that I want to convert to SDL.  I’d like to build from a 100% more generic code tree rather than having private files here and there.  The CPU optimization programs that re-read GCC’s assembly output don’t do anything.  I want to try it through an older version of GCC and see if there is any difference in speed.  I also recently received the source code to vc5opti.cpp and I’d like to try that to see if it speeds up the Windows Visual C++ based build.  Long term I’d love to patch in the UAE CPU code from the newer versions that have a far more solid 68030/68881 and 68040 emulation.  The price of standing on so many tall shoulders is that when I fall off I don’t know if the CPU exceptions I see are faults in the CPU emulation, Basilisk II or just plain crashes in MacOS which was certainly not the most stablest thing once you mixed in multimedia and networking.  It was par with Windows 3.1, which honestly both of them were ‘saved’ with help from the older generation, ala BSD Unix for MacOS, and the VMS team for Windows.

So after all this I’m ready to release some binaries, and code.  Although the last thing I wanted to do is add more confusion by calling this Basillisk II v0.8.SOMETHING … A quick google search on Basilisk gave me this:

As for some reason I actually never did look up what a Basilisk was.  So seeing that this project is basically the same thing I chose Cockatrice.

The Cockatrice III source forge page is here, Windows binaries, Mac OS X binaries, and source code here.

There are plenty of bugs, and plenty of things not working, but it works well enough to do things, and that is a credit to everyone who worked on Basilisk II before me.

So Microsoft still has the MacOS Outlook for download

I was kind of surprised.  Even more so that I could get it working to my Exchange 5.5 server.

outlook on macos 8

Outlook on MacOS 8.0

Unless you have AppleTalk enabled on your server, you’ll need to setup your TCP/IP, and that also means you have to be able to resovle the exchange server by name.

If you want to use a hosts file, be sure to set the user mode to advanced in the control panel, and then setup a hosts file in the special format that looks more like a DNS zone record.

exchange.superglobalmegacorp.com A 172.18.8.50
exchange CNAME exchange.superglobalmegacorp.com
bbs CNAME bbs.superglobalmegacorp.com

Then save it somewhere like System/Control panels, point the TCP/IP panel to it, and that should do it.

For anyone who wants to try to connect to their Exchange server, you can find the client here, on their ancient ftp server.  And I suppose you could also try the one from Exchange 4.0SP2, but I didn’t bother, since this one is so new, like 1999!

 

You ever get that feeling you’re going nowhere

And no matter what you are do, this program simply will not run?  You think it’s something platform specific since building it is such a major pain with so many moving parts that you are bound to screw something up.

CRASH!

CRASH!

And then, while looking at the call stack, something looks familiar: I’m crashing at the same place on two different platforms with two different compilers.

It isn’t much, but even consistency lets you know you’ve at least got that much going.

Did you know that GCC used to support Windows NT ?

I’m not talking about MingGW, Cygwin or anything else like that, but rather when support for Windows NT first appeared, it had to be compiled with Microsoft Visual C++ 1.0 of all things.

It was back in the 2.6 days of GCC, right before the stagnation that led to EGCS.

`i[345]86-*-winnt3.5'
     This version requires a GAS that has not yet been released.  Until
     it is, you can get a prebuilt binary version via anonymous ftp from
     `cs.washington.edu:pub/gnat' or `cs.nyu.edu:pub/gnat'. You must
     also use the Microsoft header files from the Windows NT 3.5 SDK.
     Find these on the CDROM in the `/mstools/h' directory dated
     9/4/94.  You must use a fixed version of Microsoft linker made
     especially for NT 3.5, which is also is available on the NT 3.5
     SDK CDROM.  If you do not have this linker, can you also use the
     linker from Visual C/C++ 1.0 or 2.0.

     Installing GNU CC for NT builds a wrapper linker, called `ld.exe',
     which mimics the behaviour of Unix `ld' in the specification of
     libraries (`-L' and `-l').  `ld.exe' looks for both Unix and
     Microsoft named libraries.  For example, if you specify `-lfoo',
     `ld.exe' will look first for `libfoo.a' and then for `foo.lib'.

     You may install GNU CC for Windows NT in one of two ways,
     depending on whether or not you have a Unix-like shell and various
     Unix-like utilities.

       1. If you do not have a Unix-like shell and few Unix-like
          utilities, you will use a DOS style batch script called
          `configure.bat'.  Invoke it as `configure winnt' from an
          MSDOS console window or from the program manager dialog box.
          `configure.bat' assumes you have already installed and have
          in your path a Unix-like `sed' program which is used to
          create a working `Makefile' from `Makefile.in'.

          `Makefile' uses the Microsoft Nmake program maintenance
          utility and the Visual C/C++ V8.00 compiler to build GNU CC.
          You need only have the utilities `sed' and `touch' to use
          this installation method, which only automatically builds the
          compiler itself.  You must then examine what `fixinc.winnt'
          does, edit the header files by hand and build `libgcc.a'
          manually.

       2. The second type of installation assumes you are running a
          Unix-like shell, have a complete suite of Unix-like utilities
          in your path, and have a previous version of GNU CC already
          installed, either through building it via the above
          installation method or acquiring a pre-built binary.  In this
          case, use the `configure' script in the normal fashion.

Well that is quite the tall order!  I can’t believe I somehow managed to build it back then, and out of curiosity I managed to build it again.  And as you can see you need a ‘beta’ release of the GAS assembler, which is kind of impossible to find now, but it was part of the GNAT project, which used to distribute binary builds of the GCC core back in the day.  Luckily I have a version stuffed away, although it’s from GCC 2.8.1, not as ancient as it would have when GCC 2.6 was a new thing.   But it at lest assembles enough for my simple Hello World experiment.

To build GCC you need a working SED, which surprisingly CL 8.0 from the Win32s SDK, and it runs fine on Windows 7 x64 of all things.  I had to mess with some of the files, and substitute the linker & headder files from Visual C++ 2.0 but much to my amazement not only can I build GCC along with libgcc.a, it also builds incredibly fast.  On my machine I can compile GCC in about 5 seconds.  I remember in 1995 building this on Windows 95 (I was crazy) and it taking HOURS and HOURS.

So one nice thing about these binaries is that you don’t need any external DLL’s  Even the import section of a simple hello world exe is tiny:

GetVersion GetEnvironmentStrings GetCommandLineA ExitProcess RtlUnwind UnhandledExceptionFilter GetModuleFileNameA GetFileType GetStdHandle GetStartupInfoA VirtualFree VirtualAlloc WriteFile WideCharToMultiByte GetLastError FlushFileBuffers SetFilePointer SetStdHandle CloseHandle KERNEL32.dll

Which implies that it only needs KERNEL32 to function.

I don’t know if it’s of any use to anyone else, but HERE is my dump of all the source and tools I used to build GCC.

Saving RISC GCC from the dumpster

Well it seems that the old GNAT stuff from NYU.EDU is no more.

D:\gnat>ftp cs.nyu.edu
Connected to cs.nyu.edu.
220---------- Welcome to Pure-FTPd [privsep] [TLS] ----------
220-Local time is now 23:26. Server port: 21.
220-Only anonymous FTP is allowed here
220-IPv6 connections are also welcome on this server.
220 You will be disconnected after 15 minutes of inactivity.
User (cs.nyu.edu:(none)): anonymous
230-Your bandwidth usage is restricted
230 Anonymous user logged in
ftp> cd pub/gnat
250 OK. Current directory is /pub/gnat
ftp> ls
200 PORT command successful
150 Connecting to port 54057
README
c++-interface.ps.gz
papers
private
226 4 matches total
ftp: 46 bytes received in 0.00Seconds 46000.00Kbytes/sec.
ftp>

For those unfamiliar they provided binary builds of GCC for various platforms, including a bunch of RISC machines. It would be a great loss for anyone trying to bootstrap GCC on one of these platforms with missing parts.

Luckily I have saved them, and put them back online!

*Dec Alpha OSF4
*HP HPPA HPUX 10.20
*i386 Solaris 2.6
*SUN Sparc Solaris 2.5.1
*IBM PowerPC AIX 4.1

For anyone interested, this is how I used GNAT to turn Solaris 2.4 into something usable.

Its that time of the year again

provider issues….

Dear Customer,

In the past month we sent out a survey to our customers requesting feedback on the services you receive from us and in particular the location. We carried out this survey due to the situation we have with our current provider in Germany. We have been fighting long and hard in an attempt to rectify long standing issues with them with regards to stability and reliability. Instead of improving, the situation has become worse to the point where servers are randomly powered down because a DC technician has knocked a power cable out and hardware is failing with no replacements being actioned despite our requests. More recently, we’ve had issues with them working on their network and ‘forgetting’ to plug servers back in.

As you can imagine, this kind of service impacts greatly on you, our customers, and it’s not something we can continue to accept. Despite our best efforts a resolution with them has been unattainable so we have had to make the decision to move customers with immediate effect.

It’s really not their fault, so time to backup again, and get ready for the shuffle.

Maybe it’s time to move to CloudFlare.