One Byte to CIDR: Teaching a 1 MB DECserver 90TL about Supernets

Summary

The problem. My two DECserver 90TL terminal servers, both with 1 MB of memory, live on a /22 network (in this post, 192.168.20.0 to 192.168.23.255). The only software a 1 MB unit can run, V1.1C, lets you store the mask 255.255.252.0, shows it back to you in LIST INTERNET, and then at every boot quietly runs with 255.255.255.0 instead. The firmware predates CIDR in spirit: it will not use a mask shorter than the address’s “natural” class mask, and 192.168.x.x is a class C address, so /24 is the shortest it allows.

The secondary problem. I first tried to get Claude to help with this, using Opus 5.5 on medium. It figured out what we should do (patch the firmware), where to get the firmware, how to do it (I had Ghidra already installed), etc. Then it kept getting blocked no matter what I did with cyber alerts. It wouldn’t even write a handoff document! I had to completely end that long running session that I was doing other VMS retrocomputing stuff on, as it was totally poisoned. I mean, Claude was totally useless, as has so often happened with my retrocomputing efforts. (And Anthropic doesn’t seem to do anything with the feedback I keep leaving about this.) Good thing I have Strata and Qwen on my 2xR9700 192G desktop.

The fix. A one-byte patch to the 1 MB units’ software image, MNENG1.SYS V1.1C. At boot, the firmware ORs the class C mask into the stored mask. Changing one displacement byte makes it OR the stored mask into itself instead, which does nothing, so the stored mask is used as is. Three more bytes change the banner from V1.1C to V1.1D, so you can see which image a unit is running. Both lab units now boot the patched image and run at /22.

(Units upgraded to 4 MB can run DEC’s newer DNAS software instead, which should not need this patch; see memory.)

Downloads, with SHA-256 sums in SHA256SUMS.TXT:

• MNENG1P.SYS (593,408 bytes): the patched V1.1D image. Transfer it in binary and serve it as MNENG1.SYS; see putting it into service.
• patch-manifest.tsv and mkpatch.py: the four byte edits, and the script that applies them to DEC’s stock MNENG1.SYS (kit DS9C011.ZIP), if you’d rather build the image yourself.
• README.TXT, which says how to install it on an OpenVMS load host and how to roll back, and SHA256SUMS.TXT.

file offset 0x41C45:  2C -> 34     move.l $2C(a4),d0  ->  move.l $34(a4),d0

The rest of this post is the long version: what a DECserver 90TL is, how it loads its software over MOP every time it boots, what is inside the firmware, how we found the clamp, and the traps we hit putting the new image into service. The work was done in one evening by a local LLM running as a coding agent, with me at the keyboard and near the hardware; the last section covers that setup.

The DECserver 90TL

The DECserver 90TL is a small Ethernet terminal server that Digital sold from about 1992. The installation guide on the V1.1C kit is dated January 1992; the last software release, V1.1C, came out in June 1997, and its banner says “Copyright 1996”. It connects up to eight serial terminals or printers to hosts on the Ethernet, using LAT (DEC’s Local Area Transport) for VAX and Alpha systems and Telnet for anything that speaks TCP/IP. It can also act as a SLIP router for small machines on its serial ports.

What the unit has, from DEC’s Software Product Description (SPD) and our own units:

That last-but-one row is what makes this project safe. The ROM only runs self-tests and then asks the network for its software, using MOP or BOOTP/TFTP. If a patched image is bad, the unit fails to boot, you put the original file back on the load host, and you power-cycle it. There is nothing on the unit to brick.

Our units are both part number DSRVE-M. They reached the lab with other people’s configurations in NVRAM and were factory reset. A VT420 on one of them reaches the lab’s OpenVMS machines over LAT.

Memory: 1 MB, and the 4 MB upgrade

Both of our units have 1 MB of memory; the ROM says so at power-up (Available Memory 1 Mbyte). The memory is a single 72-pin SIMM, and the 90TL can be upgraded to 4 MB. DEC’s upgrade is option 4A-MS40-WB, module part number 20-41141-05: a 1M×36, 80 ns, 72-pin SIMM. Those numbers come from a 2003 post on the port-vax mailing list, which also reports that generic 4 MB SIMMs did not work. I couldn’t find one for sale anywhere online.

The memory decides which software a unit can run, because the whole image has to fit in RAM:

• 1 MB: only MNENG1.SYS, the 90TL’s own software line, which ends at V1.1C. This is the image that clamps the mask, and the one this post patches.
• 4 MB: DEC’s later DECserver Network Access Software (DNAS), whose 90TL images (MNENG2.SYS, MNENG3.SYS) are 0.8 to 1.6 MB. The DNAS V2.4 readme shows a 90TL with 4 MB booting MNENG3 and says that with a 1 MB SIMM “there is only enough memory for the MNENG1 image”. The V2.4 release notes relax the mask rule: “For class C addresses, the subnet mask must be at least as long as 255.255.0.0”. So a 4 MB unit running DNAS V2.4 should be able to use a /22 without any patch. We haven’t been able to test that.

Everything that follows is about the 1 MB units and MNENG1.SYS.

Two databases, and a write-once identity

The DECserver keeps two copies of every setting:

The internet settings follow this pattern, but there’s a twist that turned out to matter: once the unit has an internet address, every SET INTERNET ADDRESS or SET INTERNET MASK fails with -358- Server internet address already set, even SET INTERNET ADDRESS NONE. The running IP identity can be set only once per boot. The online help confirms it in passing (“you should set the subnet mask before you set the internet address”). So the running mask can come only from the boot code reading NVRAM.

MOP: how the unit gets its software

MOP is DEC’s Maintenance Operation Protocol, the layer-2 protocol DEC machines use for downline loading, upline dumping and remote consoles. It runs directly on Ethernet; no IP is needed. A 90TL boots like this:

1. The ROM runs its self-tests and prints its banner on the console port at 9600 baud (FW V2.0.0 HW 3.0, Available Memory 1 Mbyte).
2. It multicasts a MOP Request Program to the dump/load assistance address AB-00-00-01-00-00, naming the software it wants: MNENG1.
3. A load host that knows the unit’s Ethernet address volunteers. The unit accepts, and the load host sends the image as a series of memory-load messages, each acknowledged, finishing with the transfer address.
4. The unit jumps into the image in RAM. Our 593,408-byte image takes about 22 seconds to load and start.

If the software crashes, it can do the same in reverse and upline dump its memory to the load host.

Our load host is a Compaq XP1000 (an Alpha workstation) running OpenVMS Alpha V8.4. On OpenVMS, MOP loading is done by LANCP (the LAN Control Program) and its server process, LAN$ACP, without DECnet. Each DECserver gets an entry in the LANCP node database:

$ MCR LANCP
LANCP> SET  NODE unit2/ADDRESS=08-00-2B-xx-xx-xx/FILE=MOM$LOAD:MNENG1.SYS/BOOT_TYPE=OTHER
LANCP> DEFINE NODE unit2/ADDRESS=08-00-2B-xx-xx-xx/FILE=MOM$LOAD:MNENG1.SYS/BOOT_TYPE=OTHER

SET changes the running database and DEFINE the permanent one, just as on the DECserver. BOOT_TYPE=OTHER means a plain system image rather than a VMS satellite boot. The image files live in SYS$SYSROOT:[MOM$SYSTEM], which is also known as MOM$LOAD and LAN$DLL. Each load is logged in SYS$STARTUP:LAN$ACP.LOG.

That log taught us the most important thing about deploying a patched image. More on that below.

The problem in the lab

Call the lab network 192.168.20.0/22, with the router at 192.168.21.1 and the two DECservers at 192.168.23.50 and .51. (These are example addresses; the real ones are different but have the same shape.) With the firmware clamping the mask to /24, each unit believed its network was only 192.168.23.0/24:

• The router at 192.168.21.1 was off its subnet, so it couldn’t be a gateway: DEFINE INTERNET GATEWAY 192.168.21.1 gave -353- Invalid gateway address, gateway not on our internet subnet.
• Without a gateway, the units could reach only 192.168.23.x. That covered the VAXen and the load host, but not the rest of the network or the Internet.

Storing the mask in a different order made no difference. We tried that, with reloads, before suspecting the firmware.

A same-day workaround came first: I added a second address, 192.168.23.1/22, to the router’s LAN interface. That address is inside each unit’s /24, so CHANGE INTERNET GATEWAY 192.168.23.1 worked, and pings to 192.168.21.1 and 8.8.8.8 succeeded for the first time. But the units still thought three quarters of their own LAN was somewhere else. The real fix had to be in the firmware.

Is there newer software for a 1 MB unit? No. V1.1C is the last MNENG1 release, and DNAS needs the 4 MB upgrade, which I couldn’t find (see memory). An mneng1.sys on the DNAS V2.0 CD turned out to be V1.1A, older than ours.

What is in MNENG1.SYS

The image is 593,408 bytes: 1,159 fixed 512-byte records, which is the format VMS wants for a MOP load file.

• Record 0 (bytes 0x000–0x1FF) is a load header. It has a product word and the magic number BB7AD57D at offset 4. The same magic appears in other DEC terminal server images of the same family. Nearby is the string NuMunchkin V5.4 ROM to DRAM, apparently the name of DEC’s internal build tool.
• Record 1 (offset 0x200) is a boot stub. It saves the stack pointer, then executes movec vbr,a0.

That movec vbr settled the CPU. The agent’s first guess had been MIPS, because DEC’s later terminal servers used MIPS processors. But MOVEC with the vector base register exists only on the 68010 and later, and the rest of the code disassembles cleanly as 68020 code.

07800200  move.l  SP,($078071f8).l
07800206  movec   VBR,A0
0780020a  move.l  A0,($078071fc).l
07800210  move.w  #$3f,D0
07800214  lea     ($7805238).l,A0

The absolute addresses in the stub gave the memory map: the image runs at virtual address $07800000 plus the file offset. The stub copies a table, fills a workspace, and jumps to the main entry point at $0780535C.

The rest is what you’d expect: the online help text (the entire command grammar), the error messages, a BSD-derived IP stack, LAT, and drivers near the end of the image.

Finding the clamp

Step 1: measure it from the outside

Before disassembling anything, we measured exactly what the boot code does to different stored masks. With DEFINE INTERNET MASK you can store a mask without changing the running one. The agent wrote a small Python driver for the unit’s Telnet console and a script that, for each mask, stores it, reboots unit #2, and reads SHOW INTERNET:

The /25 row is the clue. A mask longer than the class mask survives boot and works end to end. (It even correctly dropped the gateway, because 192.168.23.1 is outside 192.168.23.128/25. The gateway came back when /24 was restored, since only the running copy had been dropped.) So the IP stack can run with any mask. One decision at boot throws away the bits that are shorter than the class mask. In every row,

running mask = stored mask OR natural class mask

That is exactly RFC 950’s rule from 1985: a subnet mask may extend the network part of an address, never shorten it. DEC’s SPD for the 90TL lists “Subnet Addressing (RFC 950)” among its features. Supernetting came with CIDR in 1993, and nobody went back to the 90TL to add it. So this isn’t a bug fix; it is a backport of a feature that never shipped.

The -358 behaviour and DEFINE together pin the location. The SET validator never gets a chance to run, and DEFINE only writes NVRAM. The OR has to be in the boot code that brings up the interface from the stored parameters.

Step 2: Ghidra, and some dead ends

The agent imported the image into Ghidra 12.1.2 in headless mode (68000:BE:32:MC68020, loaded at $07800000), wrote Java scripts to grow the function list from code-pointer tables (about 3,800 functions in the end), and exported a full listing of about 107,000 instructions to search with grep. Capstone in a short Python script handled quick looks.

It found the textbook classful-mask routine almost at once, at $078414FE:

uint classful_mask(uint addr) {            /* FUN_078414fe */
  if ((addr & 0x80000000) == 0)          return 0xff000000;  /* class A */
  if ((addr & 0xc0000000) == 0x80000000) return 0xffff0000;  /* class B */
  if ((addr & 0xe0000000) == 0xc0000000) return 0xffffff00;  /* class C */
  return 0;
}

Next to it is a routing-table walk: a linked list headed by a global at $0789A714, with each entry holding a network at +0x30, a mask at +0x34, and a next pointer at +0x40. The unit’s own help confirmed that gateway entries carry their own network and mask (DEFINE INTERNET GATEWAY a.b.c.d NETWORK n.n.n.n MASK m.m.m.m), so the routing code was not the problem.

But the classful routine has only one caller, a helper that computes “the network part of an address”. The boot code must compute the class mask inline. These leads did not work:

• Following error messages. The strings are easy to find: Invalid subnet mask at file offset 0x29131, the -353- text at 0x2915D, and Internet checksum failure, parameters not loaded at 0x29304 (this one comes from the NVRAM loader). No 32-bit pointer to any of them exists in the image. Messages are reached through a table of message codes and offsets, walked by index from the Local -%03d- formatter. Decoding that table was slow, and it was put aside.
• Diffing V1.1A against V1.1C. The two versions have the same layout almost everywhere, but the whole IP module (about 18.5 KB) was rewritten between them, so the diff showed one big changed block and pointed nowhere.

Step 3: grep for the constants

What worked was simpler. The boot code must put the class masks $FF000000, $FFFF0000 and $FFFFFF00 into instructions as immediate values. A grep of the full listing for those constants found a few clusters. One was an IP-address parser and one was the routing module. The third was the interface setup code, and the clamp was there:

07841c06  btst.l  #31,D5                 ; D5 = our IP address
07841c0a  bne.b   $07841c16
07841c0c  move.l  #$ff000000,($2c,A4)    ; class A: natural mask /8
07841c14  bra.b   $07841c38
07841c16  move.l  D5,D0
07841c18  andi.l  #$c0000000,D0
07841c1e  cmpi.l  #$80000000,D0
07841c24  bne.b   $07841c30
07841c26  move.l  #$ffff0000,($2c,A4)    ; class B: /16
07841c2e  bra.b   $07841c38
07841c30  move.l  #$ffffff00,($2c,A4)    ; otherwise: /24
07841c38  move.l  D5,D0
07841c3a  and.l   ($2c,A4),D0
07841c3e  move.l  D0,($28,A4)            ; +$28 = classful network
07841c42  move.l  ($2c,A4),D0            ; D0 = natural mask      20 2C 00 2C
07841c46  or.l    D0,($34,A4)            ; stored |= natural      81 AC 00 34   <- the clamp
07841c4a  move.l  D5,D0
07841c4c  and.l   ($34,A4),D0
07841c50  move.l  D0,($30,A4)            ; +$30 = operational network

The interface structure pointed to by A4 holds the classful network at +$28, the natural mask at +$2C, the operational network at +$30, and the operational mask at +$34. When this code runs, +$34 holds the mask loaded from NVRAM. The or.l at $07841C46 is the clamp:

stored /22  FF FF FC 00
natural /24 FF FF FF 00
OR          FF FF FF 00   = /24

A stored /25 ORed with /24 is still /25, which is why longer masks always survived. The routines that read the mask later (the “is this address mine?” test, the routing walk) all use +$30 and +$34, which is why the /25 test worked end to end. Nothing else in the stack cares about classes.

The patch

The options were:

• Replace the or.l with two NOPs (4 bytes). This works.
• Remove or change the class branches. This would break +$28 and the broadcast address that’s computed from +$2C later, so no.
• Add a check (e.g. “only OR if the stored mask is zero”). It doesn’t fit without moving code.
• Change one byte. The move.l ($2C,A4),D0 just before the OR loads the natural mask into D0. Its last byte is the displacement $2C. Change it to $34, and the instruction loads the stored mask instead. The OR becomes stored |= stored, which changes nothing. Same length, same instruction, one byte.

We chose the one-byte change:

file offset 0x41C45 (VA $07841C45):  2C -> 34
07841c42  20 34 00 2C    move.l ($34,A4),D0
07841c46  81 AC 00 34    or.l   D0,($34,A4)      ; now a no-op

Because nothing changes length, no addresses move, and the MOP load format doesn’t care. There was no image checksum to fix either: the firmware checks its NVRAM parameter block, and the patched image loaded and ran.

I asked for one more change: make the banner say V1.1D, so that you can tell at a glance whether a unit loaded the patched image. That’s three more bytes, C (43) to D (44), at the three places the version string appears. The whole patch is a manifest:

# patch-manifest.tsv
41c45   2c  34   # move.l $2c(a4),d0  ->  move.l $34(a4),d0
088808  43  44   # V1.1C -> V1.1D
088850  43  44   # V1.1C -> V1.1D
0889ce  43  44   # V1.1C -> V1.1D

mkpatch.py checks the original byte at each site before changing it, refuses any edit that would change the length, and prints the SHA-256 of the result:

$ python3 mkpatch.py MNENG1.SYS patch-manifest.tsv MNENG1P.SYS
patched @41c45: 2c -> 34
patched @088808: 43 -> 44
patched @088850: 43 -> 44
patched @0889ce: 43 -> 44
4 site(s); bytes differing vs stock: 4 (0x41c45, 0x88808, 0x88850, 0x889ce)
wrote MNENG1P.SYS  (593408 B)  sha256 25d9a2042540bce666439de37e8cee098abb0c359b6823357462a9bcdaa8571e

One new hazard. On the stock firmware, DEFINE INTERNET MASK NONE (a zero mask) was harmless: zero OR the class mask is the class mask. On the patched firmware, a zero mask stays zero, and the unit believes the whole IPv4 Internet is on its local network. Don’t do that. If you do, the serial console and the MOP console don’t depend on IP, so you can still DEFINE INTERNET MASK 255.255.255.0 and reboot.

Putting it into service

The file has to look right to VMS

A MOP load file on VMS must be a sequential file of fixed 512-byte records. Binary FTP to the VMS host, followed by a VMS COPY into the load directory, produces that; ANALYZE/RMS_FILE then reports no errors. Two other routes caused trouble:

• FTP to a bracketed path doesn’t change directory. Sending to [MOM$SYSTEM]MNENG1P.SYS creates a file in the login directory whose name contains the brackets (^[MOM$SYSTEM^]MNENG1P.SYS). Send a plain file name, then COPY it on the VMS side.
• rcp to VMS writes a file VMS doesn’t trust. I suggested using our RSH/RCP tunnel instead of FTP. The bytes arrived intact (the SHA-256 matched), but ANALYZE/RMS_FILE reported “Unrecoverable error in structure of file”. That file wasn’t used. Copying from VMS with rcp is fine, and that’s how the final files were verified (below).

The first two reboots still said V1.1C

The plan was to keep unit #1 on the stock image and point only unit #2 at the new file, with LANCP SET NODE unit2/FILE=MOM$LOAD:MNENG1P.SYS, then DEFINE NODE as well. LANCP accepted both, and SHOW NODE showed the new file. Unit #2 rebooted and said V1.1C both times.

LAN$ACP.LOG showed why:

Requested file: LAN$DLL:MNENG1.SYS

The unit’s MOP request names the software it wants, and LAN$ACP served that name. In our tests the per-node /FILE and /ROOT settings made no difference. So deploying means swapping files in the load directory: rename the stock image to MNENG1.STK, and copy the patched image to MNENG1.SYS:

$ RENAME SYS$SYSROOT:[MOM$SYSTEM]MNENG1.SYS;1 MNENG1.STK
$ COPY   SYS$SYSROOT:[MOM$SYSTEM]MNENG1P.SYS;1 SYS$SYSROOT:[MOM$SYSTEM]MNENG1.SYS
$ SEARCH/EXACT MOM$LOAD:MNENG1.SYS "Communications Server V1.1D"

The catch is that every 90TL that loads from this host gets the new image at its next boot. Unit #1 was not rebooted until I decided to adopt the patch.

Rolling back is the reverse rename and a reboot. Nothing is stored on the unit.

It works

With the stored mask set to /22, unit #2 rebooted (SET PRIVILEGED, INITIALIZE DELAY 0, about 25 seconds) and came up like this:

DECserver 90TL Communications Server V1.1D (BL46-13) - LAT V5.1
Lab DECserver 90TL #2

(c) Copyright 1996, Digital Equipment Corporation - All Rights Reserved

Local> SHOW INTERNET
Internet Address:  192.168.23.51
Subnet Mask:       255.255.252.0

Local> SHOW INTERNET GATEWAY
Gateway:  192.168.23.1     Network:  any

Local> PING 8.8.8.8
 Pinging...
Local -411- PING successful: host reachable

Pings to the router alias, to 192.168.21.1, to 8.8.8.8 and to unit #1 all succeeded, LAT announcements stayed on, and the Telnet console worked. A second reboot gave the same results.

Then I rebooted unit #1 myself. It came up V1.1D with its stored /24 unchanged, gateway present, pings fine. The patch changes what the stored mask means, not what is stored, so a patched unit at /24 behaves exactly like the stock one.

As a final check, the agent copied both files back off the VMS host over RCP and hashed them in transit and locally:

• the stock MNENG1.SYS V1.1C (from DEC’s kit DS9C011.ZIP, June 1997), kept as MNENG1.STK;
• the patched MNENG1.SYS V1.1D, the file actually served.

Both matched the files we built, and rebuilding from the manifest gives the same hash. The sums are in SHA256SUMS.TXT and README.TXT.

Still to do: a full LAT regression from a terminal on a serial port. Everything we could test from the console and the network passed.

Who did the work: a local model in OpenCode

All of the analysis, tooling, testing and documentation above was done by an AI coding agent running entirely on my own workstation:

(The session started on Strata 0.2.8. It was upgraded to 0.2.11 that evening, before the firmware work began.)

Some numbers from the OpenCode session: it ran from 6:08 PM to 11:37 PM. The firmware work started at 8:46 PM; the first commit was at 8:54 PM; the /22 was running on unit #2 at 11:06 PM. The session used about 1.1 million input tokens and 307,000 output tokens, with 48 million tokens read from the prompt cache, and two context compactions.

How we divided the work:

• Me: the goal; the router change; power cycling and rebooting units; answering questions about the lab; the V1.1D banner idea; suggesting RCP; deciding when unit #1 adopted the patch.
• The agent: everything else. It found the CPU and memory map, wrote the Ghidra scripts, the console driver and the test scripts, ran the mask experiments, found and patched the clamp, did the VMS-side deployment and rollback planning, wrote the change records for the lab’s machines, kept everything in Git, and wrote an operations manual.

Some notes on how it went:

• It changed its mind when the evidence said so. MIPS became 68020 after one instruction. “Ghidra coverage is shallow” became “3,464 functions, my earlier reading was wrong”. “Per-node /FILE works” became “LANCP serves the requested file name” after the V1.1C banner and the log.
• It kept a written record. At my request it wrote the session state to its notes in Git before compacting its context, so nothing was lost when its context was compressed.
• The console driver was the turning point for the hardware work. The general-purpose Telnet MCP tool kept stalling at the DECserver’s # prompt. The cause: the tool refused Telnet option negotiation, and the DECserver then ignored keystrokes. The agent’s own driver accepts the options, sends a bare CR, retries the first access (the unit ignores it while the session settles), and never sends a password except at a real Password> prompt, because the unit allows only three failures.
• There were hiccups. Twice the model emitted a malformed tool call, and once the Strata server was restarted under it (connection refused); each time “please proceed” picked it up again. Once a thinking block said “since the content is in Japanese, I’ll translate…” when nothing was in Japanese. When I asked, it said it was a stray fragment and nothing had been done with it, which matched the transcript.

An earlier attempt with a hosted assistant ended when it treated reverse engineering this firmware as a possible cyber attack. This is a 30-year-old terminal server that I own, on my own network, running firmware from DEC’s freely distributed kit, so I tried again with a local model.

Files

The downloads, again, with SHA-256 sums in SHA256SUMS.TXT:

• MNENG1P.SYS: the patched V1.1D image for 1 MB units. Serve it as MNENG1.SYS.
• patch-manifest.tsv and mkpatch.py: build it yourself from DEC’s MNENG1.SYS V1.1C (kit DS9C011.ZIP); README.TXT has the stock file’s SHA-256 to check it against first.
• README.TXT: installing on an OpenVMS load host, rolling back, and the warning about a mask of NONE.

If you have a 1 MB DECserver 90TL on a network wider than its class mask, this should let it join the rest of the network.