I see this pattern in the projects I tackle in my professional life:
I enter into a new field – whether a new computer type, a new programming language, a new technology; I learn it; then I write about it and explain it to others; then I find its limits and try to push through those limits.
It began early.
In the 1980s I worked on several projects involving the venerable PDP-11 mini-computer. The system had two hard drives: one for system software, one for our applications. The operating system and the running app fit within a 64KByte memory space, yet it controlled the entire semiconductor test system.
I joined the team, went to the courses, programmed the applications. The programs worked well, but of course people wanted them to do more: “Run faster”, “Add more functionality”, “Create more reports”, etc. Yet of course we did not have control over the O/S, so how to meet these demands?
The first challenge: how to make the programs run faster? The goal was to improve throughput, since we were characterizing prototype silicon. The test programs ran to do shmoo plots over voltage, timing and temperature.
Fortunately for me, as a guard against future needs, IBM had purchased a license to the source code for the entire operating system and programming language interpreter. It was all written in assembly language. “No problem,” I cried! I enrolled in, then attended several weeks of DEC programming classes, in addition to having taken every class offered by the test equipment maker. When I got back to the office, I opened up the fan-fold code listings, and dug in.
A couple of months later, I had written a library of functions and subroutines which could be invoked by the test program. They replaced the interpreted code used previously, and – since that functionality was inside the shmoo loops – the test times improved dramatically! Check that one off the list.
The second challenge: how to fit larger and larger programs into the 32KBytes left over after the language interpreter and O/S used the other space? The goal was again to reduce test times: instead of sending temporary results to a file on disk, or loading and re-loading a series of test programs, the entire job needed to fit in memory.
This tester supported up to four concurrent users. Only one of them could access the test hardware; the other three worked in background – programming stations with their own share of physical memory on the PDP/11. I thought, “Why can’t I use their memory?”
I dug in again. I learned how the language interpreter handled memory partitions, and eventually pushed the program’s memory arrays into additional RAM (allowing the test results to be collected in RAM); then pried open the other three partitions and loaded test programs there – looping around from one to the other, all in RAM.
Not satisfied with that, I wrote a bit more code and allowed functions, subroutines and ‘overlays’ of program code to reside in RAM outside the 32KByte limit. The other test engineers could write segments of their jobs, and those would load once from disk and then be invoked repeatedly from a ‘main’ program, all the while managing memory.
It was quite a coup, since even the manufacturer of the test equipment had told me a year earlier that “There is no way to make room for larger test programs.” Hah!
Continuing the theme of successes, see Ambidextrous.