More fun with the System16, kinda, sorta?

So, Ive been playing around with emulators, and for some reason I think it’d be awesome to have a real one.  So I check ebay, and yeah there is a few, Altered Beast, Shinobi, and even an Outrun, and a couple of Hang Ons!  Wow this is so cool, then I check the prices, and shipping and yeah it’s REAL expensive, REAL quick.  And even back when I did own an Altered Beast board, I never got it hooked up as it was ‘too hard’.

So, I’m about to give up on the whole thing, then I spot this Altered Beast board, for sale for a mere €50!  And the shipping isn’t too insane either!  But looking at the PCB board in the picture, and I can tell something is not quite right:

€50 board!

€50 board!

Now for those who don’t know, this clearly is not a System16 board.  However it certainly does have a 68000, and z80 processor!  Could this be some 2nd tier manufacturing job? Or perhaps it’s one of these infamous bootleg boards?

For comparison, here is a real SEGA System16 board

A real System16 board

A real System16 board

As you can see, they really look nothing alike.  Also the other give away is that the far cheaper €50 board is JAMMA compatible.  All the old SEGA boards are not.

What the heck is JAMMA anyways?  You see that edge connector?  That is where you would plug in the power, coin catchers, the player buttons, and the speakers to.  Even in the old days, recycling cabinets was a thing, and having modular boards was a ‘good thing’.  But SEGA didn’t want you to swap out their boards with anyone elses, so they used their own system.  But it’s just a wiring thing, there is nothing digitial locked down, no encryption either (look at HDMI!).  So you can use an adapter, to interface from SEGA to JAMMA.

Anyways, I went ahead and placed the order.

Now doing some more research, and the monitors used in 1980’s arcades were RGB+Sync driven.  Which are ancient, and of course, HEAVY. But a little bit of searching led me to the to the GBS 8200 v4.0.

GBS 8200 v40

GBS 8200 v4.0

AKA known as the “GBS8200 CGA/EGA/YUV/RGB To VGA Arcade Game Video Converter”.  Well this certainly looks perfect!  I mean from the description alone, it’ll do what I want.  Even better they make them a few KM from here, and I could get one for ~ $20 USD.  Perfect.

Next up is the power, I decided to get a “JAMMA” power supply.  A bunch of searching, and this one was the cheapest one I could find, and again shipping wasn’t too bad, but not great either.  The supply was again around $20 USD, but shipping was $15. OUCH.

MD-9916A JAMMA switching power supply

MD-9916A JAMMA switching power supply

I figured having the ability to screw in would be a ‘good thing’.

Naturally, I need the cables to wire this mess together, so I ordered a “JAMMA Cabinet Wire Wiring Harness Loom” for about $15 USD.  Naturally mine is all in Chinese since I went cheap.  But it’s OK, I have a multi meter so I can test continuity.

Finally I saw a QANBA N1 arcade style joystick in a local mall for $230 HKD.  That is less than HALF the price of the ones I see online in the USA, Europe, or Canada.  So at least that is nice.  Now with all the parts, I just have to wait for the board to arrive.  And wait, and wait.  Nothing updated on ebay, then suddenly I check a few days later, as it’s been two weeks by this point, and it turns out that it’s been sitting in the post office in Hong Kong for a week!  If only they let me know…  SF Express, and FedEx have come without issues.  Oh well, now I have the board!

I can now finally flip it over to reveal:

Graphics board

Graphics board

It’s all 74L TTL logic chips, EEPROMS, and some PALs as well.  There are NO custom SEGA chips at all.  If anything this is what is inside of the SEGA ASIC’s on the System16 board.  Whoa.

Ok, so this is certainly a bootleg board.  A quick search of MAME shows that they have a Datsu ROMset, so maybe this is one?  Nothing on the boards say Datsu, however it does say ALTER/S, and it shows being QA’d on 11/11/88.

Now it’s time to cable this thing up!

But first JAMMA boards are typically key’d so you cant put the adapter in backwards.  There is no key on this board, so I need to check the voltages to make sure I don’t flip it backwards.

JAMMA Standard Pinout

** Solder Side Parts Side **




GND A 1 GND
GND B 2 GND
+5v C 3 +5v
+5v D 4 +5v
-5v E 5 -5v
+12v F 6 +12v
Key, No Pin H 7 Key, No Pin
Coin Counter 2 J 8 Coin Counter 1
Coin Lockout K 9 Coin Lockout
Speaker (-) L 10 Speaker (+)
NC M 11 NC
Video Analog Green N 12 Video Analog Red
Video Composite Sync P 13 Video Analog Blue
Service Switch R 14 Video Ground
Tilt/Slam S 15 Test
Coin B T 16 Coin A
Player 2 Start U 17 Player 1 Start
Player 2 X-Dir Player 2 Up V 18 Player 1 Up Player 1 X-Dir
Player 2 Y-Dir Player 2 Down W 19 Player 1 Down Player 1 Y-Dir
Player 2 X-Clk Player 2 Left X 20 Player 1 Left Player 1 X-Clk
Player 2 Y-Clk Player 2 Right Y 21 Player 1 Right Player 1 Y-Clk
Player 2 Button 1 Z 22 Player 1 Button 1
Player 2 Button 2 a 23 Player 1 Button 2
Player 2 Button 3 b 24 Player 1 Button 3
1 Player 2 Button 4 NC c 25 NC Player 1 Button 4 1
1 Player 2 Button 5 NC d 26 NC Player 1 Button 5 1
2 Player 2 Button 6 GND e 27 GND Player 1 Button 6 2
GND f 28 GND

This is the standard pinnout of a JAMMA harness. Importantly you can see it’s Ground than +5v.  So looking at the 68000 processor to check it’s pinnout:

D4 1 64 D5
D3 2 63 D6
D2 3 62 D7
D1 4 61 D8
D0 5 60 D9
AS 6 59 D10
UDS 7 58 D11
LDS 8 57 D12
R/W 9 56 D13
DTACK 10 55 D14
BG 11 54 D15
BGACK 12 53 GND
BR 13 52 A23
VCC 14 51 A22
CLK 15 50 A21
GND 16 49 VCC
HALT 17 48 A20
Reset 18 47 A19
VMA 19 46 A18
E 20 45 A17
VPA 21 44 A16
BERR 22 43 A15
IPL2 23 42 A14
IPL1 24 41 A13
IPL0 25 40 A12
FC2 26 39 A11
FC1 27 38 A10
FC0 28 37 A9
A1 29 36 A8
A2 30 35 A7
A3 31 34 A6
A4 32 33 A5
 You can see it’s power input is on pin 14.  Likewise, the ground is on pin 53.  Also looking at the edge connector, you can see the two pairs of pins, which correspond to the double ground, and double +5v.
Connecting the harness

Connecting the harness

From there, it was a matter of connecting up the power supply, adding in the power to the video board, connecting the RGBS connector, and powering it up.  It was very cool to get a glimpse of Altered Beast!

Something is wrong

Something is wrong

And hello, it is a Datsu board.  I’ve tried to google about these boards, and all that I could find out is that they seemed to be popular in Italy.  They may have been made in Korea.  There was another variation called ‘Mutant Warrior/Super Warrior‘.  There was some posts about it in an Italian game forum mameitalia.net, and arcadeitalia.net . Google translate works fine enough to read, but they were in smaller places that couldn’t afford mainstream games, so enter the bootlegs.  And this makes sense, as the board I got was from rural France.

I maybe had a picture for 20 seconds, it was frozen, then the screen went black.  I power cycled, to nothing.  I tried it again to a green screen.  And again to a green screen.  At this point I think it’s died.  I let it rest for a few minutes, and try again.  Nothing.  I leave it powered up, and feel the processor, and it’s warm.  It’s doing something, so I think.  So I start to play with the video board, and as I change resolutions, I get an image!.. then it disappears.  Power cycling, and changing resolutions occasionally gives me an image.  I look more closely at the CPU board, and notice that it has 4 standoffs placed on each corner.  There is nothing in the middle, and over the past 28 years the board is sagging.

In order to fix the sag, I decouple the two boards, and spread them out.  I try it again, and it doesn’t show me anything. Eventually I play with all the video board settings, and manually set it to the RGBS input, and then the image stays!  The board is running.  I tweek some of the settings, and the pink goes away, and now it looks correct!

20160430_110924

LOGO

And even the intro animation is OK

Looks good

Looks good

OK, now it’s time to turn it off, and wire up the joystick.

The first step is to remove the joystick ball, and on the QANBA N1 you first flip it upside down, and remove the little cover.

Remove the cover

Remove the cover

to reveal the screwdriver slot to let you hold the stick in place as you unscrew the ball.

slot

slot

Now it pops off, and it’s really easy to remove the USB interface cables, and drag in the JAMMA cables.  Again use a tester to tone out what goes where. DO NOT FOLLOW MY COLORING GUIDE.  I’m pretty sure there is no colour standard, so just because mine is like this, yours will 99.9999% not be.  The only common thing is that each of these buttons needs a ground.

Joystick wired up

Joystick wired up

My harness has a common ground for P1 and P2, so I just tapped up the end and tucked it in the joystick body.  Now with wired up, I can put the joystick back together, and play!

And that is when I could finally see that something was wrong.  I was doing pretty well, then in the 2nd level I saw this weird thing:

An actual wall of text

An actual wall of text

The sprites are working fine, and the gameplay continues.  But eventually the wall of text effect went from the background to the foreground obscuring game play.

foreground tile corruption

foreground tile corruption

So no doubt something is bad on the board.  I need to get it looked at, and see about first dumping and checking the EEPROMS.  Next the RAM on the graphics board, may be suspect as well.  I think the CPU is fine since it runs OK, I’m just unable to really see pass the wall.

For the heck of it, I went and got some powered speakers, and hooked them up:

And it sounds so different from the SEGA version.  An inspection of the board shows that there is now YM2151, but rather a pair of YM2203’s and an OKI M5205 for the speech synths.

In retrospect, I probably should have gone with something like the arcade supergun.  I didn’t know it was a thing unfortunately.  My solution is more “traditional” , but it works.

GCC 6.1 released

From gnu:

https://gcc.gnu.org/ml/gcc-announce/2016/msg00000.html

After slightly more than a year since last major GCC release, we are proud
to announce new major GCC release, 6.1.

GCC 6.1 is a major release containing substantial new
functionality not available in GCC 5.x or previous GCC releases.

The C++ frontend now defaults to C++14 standard instead of C++98 it has
been defaulting to previously, for compiling older C++ code that might
require either explicitly compiling with selected older C++ standards,
or might require some code adjustment, see
<a href="http://gcc.gnu.org/gcc-6/porting_to.html" rel="nofollow">http://gcc.gnu.org/gcc-6/porting_to.html</a> for details.  The experimental
C++17 support has been enhanced in this release.

This releases features various improvements in the emitted diagnostics,
including improved locations, location ranges, suggestions for misspelled
identifiers, option names etc., fix-it hints and a couple of new warnings
have been added.

The OpenMP 4.5 specification is fully supported in this new release, the
compiler can be configured for OpenMP offloading to Intel XeonPhi Knights
Landing and AMD HSAIL.  The OpenACC 2.0a specification support has been
much improved, with offloading to NVidia PTX.

The optimizers have been improved, with improvements appearing in all of
intra-procedural optimizations, inter-procedural optimizations,
link time optimizations and various target backends.

See

  <a href="https://gcc.gnu.org/gcc-6/changes.html" rel="nofollow">https://gcc.gnu.org/gcc-6/changes.html</a>

for more information about changes in GCC 6.1.

This release is available from the FTP servers listed here:

 <a href="http://www.gnu.org/order/ftp.html" rel="nofollow">http://www.gnu.org/order/ftp.html</a>

The release is in gcc/gcc-6.1.0/ subdirectory.

If you encounter difficulties using GCC 6.1, please do not contact me
directly.  Instead, please visit <a href="http://gcc.gnu.org/" rel="nofollow">http://gcc.gnu.org</a> for information
about getting help.

Driving a leading free software project such as GNU Compiler Collection
would not be possible without support from its many contributors.
Not to only mention its developers but especially its regular testers
and users which contribute to its high quality.  The list of individuals
is too large to thank individually!

UAE’s 68000 core actually was once in MAME

I was kind of surprised to find it.

While I was looking for System16 stuff, I found the first version of MAME to include the UAE 68000 core starting in release MAME 28, although System16 emulation itself didn’t appear until MAME 33b3, but not playable until MAME 33b4.

So what does it mean?  Well at the time the UAE core was the way to go.  However from looking at the MAME source, the UAE core that they were using from System16 was already generated, while UAE still included the build68k program to parse the tables, and generate the 68000.  Instead they were editing the outputted C.  UAE wasn’t GPL until version 0.7(something), 0.7.6 for sure, so I don’t know why they weren’t using it from the source.

Eventually starting in MAME 35b2, the core was replaced with MUSASHI , so Among their reasons for dumping the early UAE CPU core was this laundry list:

  • New 68000 C core. For testing purposes, this is also being used in the DOS
    version instead of the asm core. [Karl Stenerud]
    Differences:

1. Faster. This code is, barring ram fetch time, almost twice as fast as the
existing C core in MAME. I’ve done extensive speed profiling on both
engines. The only problem now is the slow memory access in MAME due to
bankswitching et al.

2. Emulation more correct. I found many bugs in the MAME engine (and many,
many more in mine for that matter) when I pitted them head-to-head.
I have run random instructions from each opcode class at least 10 million
times, comparing the resultant CPU states, and have left it running random
instructions for 1 billion iterations. In every case, I have adhered to
the specs defined in M68000PM/AD REV. 1.

3. Disassembler is correct. The current M68000 disassembler in mame has a
tendency to disassemble instructions that have an invalid EA mode.

4. Cycle counting is 99.9% correct. The only instructions which don’t have
correct cycle counts are divs, divu, muls, mulu, and they’re not worth
counting correctly. (I’m not about to waste emulation time counting 0-1 and
1-0 sequences).

5. > 32 bit friendly. I’ve taken care to ensure maximum portability without
sacrificing speed. The result is conditional compiling dependant on your
architecture. I’ve also implemented and tested a compatible solution for
architectures that lack 8, 16, or 32 bit signed storage types.

6. The code is carefully laid out to be readable.

Also in MAME 35b4 added in was emulation of the NEC uPD7759 chip for speech, fleshing out the System16 emulation.

To compile these ancient versions, and inbetween I was using my Candadian cross DJGPP GCC 4.12 Win32 cross compiler.  For Allegro I’ve always found it builds far easier using GCC 2.7.2.1, a vintage compiler from back in the day I could just run in DOSBox.

Alien Syndrome

Alien Syndrome

Obviously with today’s machines, these ancient versions of MAME run fine on DOSBox!  It’s really amazing in the scope of emulators running emulators.

I found the source code to UAE 0.4

Kickstart in colour!

Kickstart in colour!

Wow what a change from UAE 0.1!  We now have colour, mouse and keyboard input, so we can finally interact with the machine.  Behind the scenes the biggest change of course was the ‘Heroic effort’ of rewriting UAE from C++ into C.  It certainly made reading the code much more easier as nothing is implicit, like it is in C++.

From the changelog between versions 0.3 and 0.4:

960203 filesys.c, action_read(): Slightly more efficient code (translate Amiga
address to real pointer).
Moved some common code in the generate_* functions in gencpu.c to a
separate function.
960202 Added an experimental fast disk option. Currently turned off by
default (it's not such a big win).
Attached sprite fixes (overlapping att. sprites looked bad, Katakis).
Add sleep(1) before resetting the console to text mode when using
SVGAlib: this might fix some screen corruption problems.
Add sprite/playfield priority checking to the most important case
(single playfield, no HAM).
In filesys.c, do_find(): open() returns -1 on error, not zero.
Return ERROR_OBJECT_WRONG_TYPE if do_find() is called for a directory
(fixes Champions of Krynn harddisk installation).
960201 Don't abort if sound driver not present, just set produce_sound to 0.
New files keybuf.c and keybuf.h to record keypresses in the right
order and without losing any. In cia.c, force 15 scanlines between
keypresses, just to be sure.
unixfs.device <em>does</em> work with Kick 1.3: just don't trust what Kick 1.3
sends in the startup packet. For now, disable more than one mount per
command line.
Started integrating Ernesto's new Mac sources.
Remove superfluous includes from some files.
960131 Added Ed's unixfs.device (great stuff).
Adding ULONGs to pointers is a bad idea on the Alpha if the ULONG value
really is signed. Add some casts to LONG in (pc_p + src) expressions
in genpu.c.
If DMACON is written and copper DMA is enabled, do a COPJMP1 at once.
Helps the "Interference" demo.
960129 More SGI fixes from Ed. Bugfixes and transdisk improvements from Marcus
Sundberg.
Remove EXTRA_DEFINES from Makefile. Breaks some systems.
Move common sprite code from pfield_doline() and pfield_doline_slow()
to new function pfield_sprite(). The same sprite may appear more than
once on the same line, so don't shift out the bits of sprdata[] and
sprdatb[] while displaying it (Turrican I).
In xwin.c and svga.c, barf if LINUX_SVGALIB doesn't match the file
being compiled.
Make all .o files depend on config.h in the Makefile.
No need to exit if sound driver unavailable, but -S given.
Small debugger fix: Missing space in output.
Fix for the sprite logic: Specifically, use a state variable indicating
whether the sprite has been restarted after a VSYNC. Fixes most
Turrican problems.
960124 Added Denis Sablic's patch for sound run-time option.
Added Ed Hanway's patch for better Makefile, X mouse cursor blanking
and more SGI compilation fixes.
960123 Include options.h everywhere.
Handle 8 bit GrayScale visuals like PseudoColor.
Remove C++ leftovers from joystick code.
960122 When using the joystick driver, the button test must come after
handle_events() in vsync_handler().
960118 Removed all the remaining C++ comments. Changed all inline keywords to
<strong>inline</strong>. Define <strong>inline</strong> if not using gcc.
Make proper prototypes for everything. Compile with maximum warnings +
-ansi + -pedantic.
Remove CIA_cycle(), obsolete.
Reimplemented the STOP optimization in newcpu.c. Removed DualCPU
support in CPU emulator.
Real nasty bug in pfield_doline() fixed: sprxpos could be evaluated as
negative, with not-so-amusing results. (Need to rewrite this in
Oberon to get array bounds checking :-)
960117 Heroic effort: Rewrote the thing in C. This might help fix some
problems with users being unable to compile it.
Fixed a problem in hsync_handler(): Only call flush_line() for lines
in the display window, i.e. when we did a prepare_line() before.
Better code for relative branches: Don't use setpc(getpc()+x) calls,
increment regs.pc_p instead.
960116 Reimplemented the function to load the Kickstart ROM. Use stdio instead
of fstreams since this apparently does not work on the Mac. Detect 256K
Kickstarts. Detect corrupt ROM images (calculate checksum).
Added Ernesto Corvi's Mac port. Changed it around a bit, so it
probably won't compile.
960115 Reinstate config.h options for X screen depth, so that DrawPixel() can
be inlined in custom.cc for speed. xlinebuffer is now incremented in
each call to DrawPixel() (for both X and SVGAlib) to get rid of some
address calculations.
960114 Fixed X generic pixel drawing routine for SHM.
Still trying to fix the harddisk emulation.
uae.device no longer breaks the debugger (can step through uae.device
functions now)
Bugs affecting performance: SPCFLAG_STOP never got reset, and DSKLEN()
would set SPCFLAG_DISK even if DMA was being turned off.
Made slow memory a run-time option.
Defer interrupts by one CPU instruction to give programs a chance to
read INTREQR ("Seeing is Believing" and "Substance" demos)
Added ScrollLock hack for X, too.
960113 SVGAlib version compiles again. Fixed SVGAlib mouse bug.
Fixed SHM bug: Maximum scanline is 313, not 312.
Sometimes, disk.cc missed a side change and would read the wrong data.
Fixed. Apparently, this was the worst compatibility problem.
Implemented trace mode.
960112 Changed layout of class amigamemory a little so that gcc can generate
better addressing modes.
Finally wrote functions in gencpu to generate MOVEMs.
960109 Integrated Ed Hanway's patches for better X support and run-time
configuration of some options.
Got rid of the direct VGA memory access. (Need to do this differently).
Changed the method of drawing lines: custom.cc now tells the graphics
code the line number and whether it needs to be doubleed before drawing
it.
Added Andre Beck's MIT-SHM patch.
Remove warnings for newcpu.cc.
960108 Fixed exceptions in op_illg(): Need to decrement PC.
960107 Added an "uae.device" resident module at 0xF00000. This emulates a hard
disk (fixed size 8MB for now).
960106 Moved some common code from pfield_doline() and pfield_doline_slow() to
a separate function. This fixes a potential HAM bug (two static vars
for the same value).
Sound support for Linux. Works only with graphics off and the CPU
slowed down.
Better SVGAlib keyboard support.
960105 Added AvailMem(), AllocMem(), AllocAbs() and FreeMem() dummies.
The Hardwired demo times the multiplication instructions and prints
"This demo don't like Axel" if they are too fast. Apparently, Axel has
a 68040. Added a WANT_SLOW_MULTIPLY option to config.h.
Fixed the fast blitter emulation (seems to work now).
960104 Fixed all the ChangeLog entries from 95 that said 96 (oops??!)
pfield_may_need_update() should check whether bitplane DMA is on.
Added ersatz.cc and ersatz.h. The purpose of these files is to
implement one or two Kickstart functions that are commonly called from
bootblocks. This should help support some games and demos that only use
the Kickstart as an initial track loader. So far, it's only good enough
for one program.
951223 More intelligent event handling in the CPU emulator. Slightly faster.
951222 Optimize CPU emulation by inlining cctrue(). Also, the real PC no
longer needs to be incremented each instruction. The real PC value
now has to be fetched by m68k_getpc().
Added direct screen access for SVGAlib, but it didn't help much. I'll
probably remove it again.
The gencpu executable is 2M smaller if it allocates memory
dynamically.
951216 custom_bput() enhanced a little. Now remembers the value that was
written in the other half of the register.
Apparently, the USEx bits of BLTCON0 are ignored in line draw mode.
(Silents-Demo)

At this point it really does work.  However a machine of 2016 compared to 1996 is just too fast.  As a result it is once more again unusable.  But it makes sense that code from this era would be built to run as fast as possible, however when it really can run fast, watch out!

I found this code while trying to find other older versions and found a post about uae-0.4.hqx, as the hqx suffix denotes that this was the Macintosh port, which thankfully included all the source, and it looks like it pretty much left the source to UAE intact.

It didn’t take much to modify the xwin.c module into a suitable module for SDL, and I was able to get it running on Linux, and with a simple re-compile onto Windows. I did amputate the filesystem sharing code.  I could fix it I guess, but considering the insane speed of 0.4, it really doesn’t matter.  If you want to test it, simply copy a 512KB kickstart to “kick.rom” and copy an ADF diskette image to df0.adf, and start uae.  Unlike 0.1 this will start right away.

Approaching Aster

It is really far too fast to actually play, just tapping enter after launching is enough to propel you into space in Frontier for example.  And  as you can see from the egg shape of Aster, older versions of UAE use a 1:1 pixel emulation which stretches, and distorts objects.  And it doesn’t correctly detect the screen margins.  I guess if it were 1996 it would be worth the time for something like SDL 2.0 where you can close the primary screen, and create another matching the needed resolution on the fly.

For anyone who cares to try my modified version of UAE-0.4 I’ve placed it on sourceforge.

If anyone has any old versions of UAE kicking around, especially any of the 0.5 releases I’d love to know.  Every old version I’ve found is here.

Fun with regex substitutions in Apache

Continuing from my previous post, I was now able to access my AltaVista server, however from a web browser I was unable to actually view any of the documents remotely.

In the pages though I did get the MS-DOS path to the usenet article in question:

Now how do I turn that into a URL?

Well as it turns out mod_rewrite does support regex, which in turn can do variable re-ordering!

After a bit of googling I found this page on stackoverflow, on how to convert a date between UK/US formats:

s/(\d{4})-(\d{2})-(\d{2})/$1-$3-$2/

Simple, right?  So what is going on here?  The parenthesis define a variable set, and on the substitution part you can recall them with $1, $2 , $3 etc.  So using this recipe I could take something like this:

u:\b227\comp\sys\laptops\3080

and convert it into the following:

http://debian7/usenet/b227/comp/sys/laptops/3080

The code for this would look something like this:

Substitute "s|&gt;u:.([a-z]{1,}[0-9]{3,})\\\([0-9a-z]{1,})\\\([0-9]{1,})|---&gt;&lt;a href="\"http://debian7/usenet/$1/$2/$3\""]Click for article|"

Although for some reason it’s embedding the URL’s even though I specified code formatting.

Now all I had to do was install IIS 4.0 off the Option Pack CD-ROM, onto my Windows NT 4.0 workstation, and create a virtual directory of /usenet which then pointed to the U: drive where AltaVista did it’s indexing.

So to this point that gives me a config file much like this:

ServerAdmin webmaster@localhost
DocumentRoot /var/www
SSLProxyEngine On
ProxyPass "/altavista/" "https://10.12.0.16"
ProxyPassReverse "/altavista/" "https://10.12.0.16/"
ProxyRequests Off
RewriteEngine On

SetOutputFilter INFLATE;SUBSTITUTE;DEFLATE
AddOutputFilterByType SUBSTITUTE text/html
#clean up urls
Substitute "s|127.0.0.1:6688|debian7/altavista|n"
Substitute "s|file:///C:\Program Files\DIGITAL\AltaVista Search\My Computer\images\|http://debian7/images/|n"
#protect the page
Substitute "s|launch=app||n"
Substitute "s|?pg=config&amp;what=init|?pg=h|n"
#fix title
Substitute "s|&lt;IMG src=\"http://debian7/images/av_personal.gif\" alt=\"[AltaVista] \"  BORDER=0 ALIGN=middle HEIGHT=72 VSPACE=0 HSPACE=0&gt;|&lt;a href=\"http://debian7/altavista\"&gt;&lt;IMG src=\"http://debian7/images/av_personal.gif\" alt=\"[AltaVista] \"  BORDER=0 ALIGN=middle HEIGHT=72 VSPACE=0 HSPACE=0&gt;<strong>|---&gt;|n"
Substitute "s|</strong>u:.([a-z]{1,}[0-9]{3,})\\\([0-9a-z]{1,})\\\([0-9a-z]{1,})\\\([0-9a-z]{1,})\\\([0-9a-z]{1,})\\\([0-9a-z]{1,})\\\([0-9]{1,})|---&gt;&lt;a href=\"http://debian7/usenet/$1/$2/$3/$4/$5/$6/$7\"&gt;Click for article|"
Substitute "s|&gt;u:.([a-z]{1,}[0-9]{3,})\\\([0-9a-z]{1,})\\\([0-9a-z]{1,})\\\([0-9a-z]{1,})\\\([0-9a-z]{1,})\\\([0-9]{1,})|---&gt;&lt;a href=\"http://debian7/usenet/$1/$2/$3/$4/$5/$6\"&gt;Click for article|"
Substitute "s|&gt;u:.([a-z]{1,}[0-9]{3,})\\\([0-9a-z]{1,})\\\([0-9a-z]{1,})\\\([0-9a-z]{1,})\\\([0-9]{1,})|---&gt;&lt;a href=\"http://debian7/usenet/$1/$2/$3/$4/$5\"&gt;Click for article|"
Substitute "s|&gt;u:.([a-z]{1,}[0-9]{3,})\\\([0-9a-z]{1,})\\\([0-9a-z]{1,})\\\([0-9]{1,})|---&gt;&lt;a href=\"http://debian7/usenet/$1/$2/$3/$4\"&gt;Click for article|"
Substitute "s|&gt;u:.([a-z]{1,}[0-9]{3,})\\\([0-9a-z]{1,})\\\([0-9]{1,})|---&gt;&lt;a href=\"http://debian7/usenet/$1/$2/$3\"&gt;Click for article|"
# Need links for the u:\news097f1\b120\comp\society\futures\1122
Substitute "s|&gt;u:.(news[0-9]{3,}f[0-9])\\\([b0-9]{1,})\\\([a-z]{1,})\\\([a-z]{1,})\\\([a-z]{1,})\\\([a-z]{1,})\\\([a-z]{1,})\\\([0-9]{1,})|---&gt;&lt;a href=\"http://debian7/usenet/$1/$2/$3/$4/$5/$6/$7/$8\"&gt;Click for article|"
Substitute "s|&gt;u:.(news[0-9]{3,}f[0-9])\\\([b0-9]{1,})\\\([a-z]{1,})\\\([a-z]{1,})\\\([a-z]{1,})\\\([a-z]{1,})\\\([0-9]{1,})|---&gt;&lt;a href=\"http://debian7/usenet/$1/$2/$3/$4/$5/$6/$7\"&gt;Click for article|"
Substitute "s|&gt;u:.(news[0-9]{3,}f[0-9])\\\([b0-9]{1,})\\\([a-z]{1,})\\\([a-z]{1,})\\\([a-z]{1,})\\\([0-9]{1,})|---&gt;&lt;a href=\"http://debian7/usenet/$1/$2/$3/$4/$5/$6\"&gt;Click for article|"
Substitute "s|&gt;u:.(news[0-9]{3,}f[0-9])\\\([b0-9]{1,})\\\([a-z]{1,})\\\([a-z]{1,})\\\([0-9]{1,})|---&gt;&lt;a href=\"http://debian7/usenet/$1/$2/$3/$4/$5\"&gt;Click for article|"
# Need links for  u:\news002f1\b1\fa.poli-sci\8
Substitute "s|&gt;u:.(news[0-9]{3,}f[0-9])\\\([b0-9]{1,})\\\([a-z\.\-]{1,})\\\([0-9]{1,})|---&gt;&lt;a href=\"http://debian7/usenet/$1/$2/$3/$4\"&gt;Click for article|"

&lt;Location /usenet/&gt;
    ProxyPass  http://10.12.0.16/usenet/
    RewriteEngine On
    SetOutputFilter INFLATE;SUBSTITUTE;DEFLATE
    AddOutputFilterByType SUBSTITUTE text/html
&lt;/Location&gt;

bla bla rest of the 000-default crap....

Simple right?

Searching for AltaVista

Searching for AltaVista

So now I get a nicely formatted page, I can click the mountain icon, and I jump back to home, and I can click on the articles and, because I have no extensions or MIME types to intercept it’ll just download them to my PC.  I guess I need to go through them all, convert them from UNIX format to MS-DOS, and stick a .txt extension on every single one of them.

I’m still thinking this thing is far too rickety to put on the internet, but we’ll see.

Fun with Apache, (mod_proxy, mod_rewrite), stunnel, And AltaVista Personal search

As you may remember from my prior attempt at using Altavista Search I ran out of space, and found out it only serves pages on 127.0.0.1:6688 and is pretty much hardcoded to do so.  It’s a “fine” hyrid java 1.01 application, with the bulk of it being java.  I finally got around to setting up a VM, and unpacking all of the utzoo archives, and indexing them.  I should have done something about the IO because this took too long (KVM).

SIXTEEN HOURS!!!

SIXTEEN HOURS!!!

So to cheat the system, I installed stunnel as a simple https to http proxy, which let me access my search VM anywhere.  However it still embedded 127.0.0.1 in all the pages.

via stunnel

via stunnel

Enter an Apache reverse proxy to talk to stunnel to talk to AltaVista search!

First to enable a few modules:

a2enmod substitute
a2enmod proxy
a2enmod ssl
a2enmod proxy_http
a2enmod rewrite

And adding this into the config:

SSLProxyEngine On
ProxyPass “/altavista/” “https://10.12.0.16”
ProxyPassReverse “/altavista/” “https://10.12.0.16/”
ProxyRequests Off
RewriteEngine On
SetOutputFilter INFLATE;SUBSTITUTE;DEFLATE
AddOutputFilterByType SUBSTITUTE text/html
Substitute “s/1997/2016/ni”
Substitute “s/97/16/ni”
Substitute “s|127.0.0.1:6688|debian7/altavista|n”
Substitute “s|file:///C:\Program Files\DIGITAL\AltaVista Search\My Computer\images\|http://debian7/images/|n”
Substitute “s|launch=app||n”
Substitute “s|<a href=http://debian7/altavista/?pg=q&what=0&fmt=d|<!—|n”
Substitute “s|><strong>|—>|n”
Substitute “s|</strong></a>||n”
Substitute “s|>u:\|->u:\|n”

This let me redirect all of those requests into a VM called debian7 on the /altavista path.  I also copied the images to the apache server, and now I get something that looks correct!

Apache in the mix!

Apache in the mix!

I cut the results short… But here is a search of something simple:

About 16598 documents match your query.

About 16598 documents match your query.

I also killed all the ‘working URL’s that simply open a desktop application on the index ‘server’.  Naturally it was a personal service, but as a server this isn’t any good.  As such you can’t click on any search results now.  I need something else to figure out how to take the result blocks like “u:\b128\comp\databases\2852” and turn them into URL’s.

Also, as much as I want to re-index I would be best to cut off the headers, or most of them so the preview lines make sense.  Xref, Path, even From & Newsgroups don’t interest me.

I hate to leave it as ‘good enough’ but if anyone has a solution…. I’ll be glad to make this wonderful resource available!

Old man yells at cloud

Simpsons  “The Old Man and the Key,”[1] Episode 13, Season 13

Simpsons “The Old Man and the Key,”[1] Episode 13, Season 13

Kind of funny that it’s gone from a joke, to being prophetic the old man yelling at the cloud.

Good luck to Intel’s former 11,000.  No doubt kids and their ‘cloud apps’ are taking away from the old fashioned server in a closet days of the 80’s,90s and 00’s.

There are signs of slowdown everwhere.

Empty retail space, in previously busy Causeway bay, Burger King went bust (except for the airport!)  Even the Tsim Sha Tsui area is facing a hit.  As expected, unemployment is going up.

I know local retailers where I rent blame the internet for taking sales, but the truth is that bad times are ahead… So be careful in 2016!