By M Carling and Simson L. Garfinkel

When NeXT quit the hardware business, common wisdom held that Intel machines would provide everything that Motorola systems provided – maybe more. Now that it is possible to run NEXTSTEP 3.1 and NEXTSTEP Developer on PCs from a wide range of makers, we can put that notion to the test.

What’s the real-world performance of a ‘486 or Pentium PC running NEXTSTEP? Which components and subsystems workbest? What’s the price range for a fully rigged developer system? What is the level of component integration? What kind of support do PC vendors provide for NEXTSTEP users?

These are the questions that NEXTSTEP customers are asking as they decide which PC model to adopt as a development platform. To get the answers, NeTWORLD put out the call for evalation units of the 17 developer systems listed in the June release of NeXT’s hardware-compatibility guide, as well as to several other manufacturers who are not yet listed but are targeting the NEXTSTEP market.

The first thing we learned is that few of the advertised NEXTSTEP PC suppliers are fully up to speed in servicing the market. While 14 PC suppliers responded to our request for test systems (Compaq provided us with both a ‘486 and Pentium-based computer, for a total of 15 systems), most were still uncertain about the level of their support for NEXTSTEP, final configurations, channels of distribution, and in some cases, pricing. Therefore, we are not yet able to provide finished evaluations of each product, complete with cube ratings.

Nevertheless, in putting these systems through their paces, we were able to reach important general conclusions that should be helpful to the first wave of developers faced with choosing systems.

This article includes our benchmark results, which readers can use to form their own opinions. Begininning with the next issue, we will provide rated reviewsof individual machines that are readyf or prime time.

To be included in this review, systems had to have 32 MB of RAM (those with more were limited to 32 MB for the benchmarks) and a 340 MB or larger hard disk. Once the systems were in-house, we installed NEXTSTEP 3.1 and NEXTSTEP Developer on each – not always an easy process – and then ran our benchmarks (see the sidebar, "Installation Blues"). Note that we include benchmark results for only 12 systems. The systems from Advance 2000 and Digital Equipment, as well as the ‘486 box from Compaq, could not be tested reliably, for teh reasons explained in the sidebar.

The mass-market PC world lives by two criteria: price and product differentiation. Cheap computers are almost indistinguishable; open any Sunday paper and you’ll see dozens of companies trying to sell off-brand or no-name computers on price alone. Be prepared to spend a little more money and you’ll soon find competition moving to the performance, expandability, and support areas.

While all these factors are imporant, the top priority for most developers is performance. Even with today’s top-of-the-line systems, developers spend a lot of time waiting. Speed up the system, and the developer becomes more productive.

The performance of a NEXTSTEP machine is determined by a variety of factors, including the CPU type, graphics subsystem, hard-disk drive, disk controller, and expansion-bus design. Our tests look at the variability in each of these factors.

We did not test Ethernet performance, since it depends almost entirely on the computer’s bus design and isn’t critical to most NEXTSTEP developers, whose entire development systems are normally resident on the computer’s hard disk. We also didn’t look at support for sound or serial ports, since both are limited under NEXTSTEP 3.1.

Here’s what we found:

CPU: With two exceptions, all of the machines we reviewed (see the chart "NeXTWORLD benchmarks") use Intel’s i486 DX2/66 microprocessor, which runs at 66 MHz and has an external-bus interface of 33 MHz. Not surprisingly, the CPU performance among those PCs with DX2/66 processors varied by less than ten percent. This variance can be attributed to secondary RAM caches, which ranged from 0KB to 256KB, and to motherboard design.

The Compaq Deskpro 5/60M contains Intel’s new Pentium processor. Running under NEXTSTEP, the Pentium chip made teh Compaq about 50 percent faster than teh ‘486-based systems we tested; however, other limitations prevented its real-world performance from improving correspondingly. Dell’s 450DE/2 DGX, on the other hand, was equipped with a DX2/50, the slowest CPU of those tested.

We are intrigued by the performance claims for Advance 2000’s 80MHz system, which pushes the external clock speed on a standard DX2/66 chip from 33MHz to 40MHz. Unfortunately, an installation problem prevented us from fully testing this prototype system.

As expected, all of the systems were faster on raw CPU performance than a NeXTstation Turbo Color, and the news on that score will only get better. By year-end, Intel is expected to release its clock-tripling chip, the DX3, which will sport an external speed of 33 MHz and an internal speed of 99 MHz. The DX3 will offer a noticeable performance improvement over other ‘486 chips but will not be as fast as the Pentium. Sometime in the first half of next year, Intel will introduce a ‘486/Pentium hybrid, the P24T, and Pentiums running at 100 MHz. The P24T is designed to fit into a socket that exists on the motherboard of most of the machines we tested and will offer near-Pentium performance. For now, most NEXTSTEP developers will likely be satisfied with the compilation performance of the DX2/66.

Bus: Having a fast bus in your system speeds communication between the CPU and any cards that are plugged into the bus (see the sidebar, "Take the Bus"). For example, if your video card plugs into the bus (as is the case with most of the systems we tested), having a fast bus will speed video performance. A high-speed disk interface plugged into a fast bus will likewise speed compiling.

The imporance of the bus diminishes with integrated systems, such as the NEC and Intel GX, which put the video and disk interface directly onto teh CPU board.

Graphics: Having a fast graphics subsystem makes windows drag and scroll perceptibly faster. While fast graphics won’t speed your compiles, they do make the system more enjoyable to use. Since NEXTSTEP treats the graphics subsystem as a dumb frame buffer, graphics performance depends largely on how fast the computer’s CPU can transfer data to, and within, the system’s video memory.

We found that local-bus video systems offered teh best graphics performance. The zippiest graphics were to be found on the Dell DGX (thanks to Dell’s JAWS video subsystem), followed by systems from eCesys, Epson, and Lucky-Goldstar that are equipped with Chips & Technologies’ Wingine video chip. PCs using teh ATI Mach-32 chip (AST, Continental, Data General, NCR, and Intel) were consistently slower, because a bug in the ATI chip causes wait states when writing directly to the frame buffer. The next revision of the ATI chip should be significantly faster.

All three Wingine-based PCs were jittery: Vertical lines oscillated left and right for no apparent reason. This sideways jitter made teh systems somewhat annyoying to use, despite the Wingine’s faster graphics performance. We were unable to test NeXT’s new driver for newer Epson Progressions (capable of displaying teh 16-bit color at 1120 by 832 pixels) that reportedly will eliminate this motion.

On the other hand, Compaq’s Qvision board provided the most stable video image of the machines tested. It was also the only PC that NEXTSTEP 3.1 supported at the 1280-by-1024 resolution (in 8-bit grayscale mode only). Unfortunately, the Qvision’s EISA-implementation helped make it the slowest performer of the bunch.

We were able to run the NEC Express/II in grayscale mode only, because NEXTSTEP 3.1 does not drive the computer’s ET 4000 graphics subsystem in color.

With the notable exception of Dell’s JAWS technology, these systems provided slower graphics performance at lower resolution than teh comparison NeXTstation Turbo Color. We look forward to running NEXTSTEP in 16-bit-color mode at 1280-by1024 resolution on some of the latest high-performance video adapters. Drivers for these adapters should be available later this year.

Disk: Since NEXTSTEP uses the hard disk for virtual-memory swap space, having a fast hard-disk drive and disk interface will do more than make your programs load faster; it will speed up the performance of your entire system.

The fastest SCSI interface cards we tested were DPT’s 2012 and 2022 and BusLogic’s BT747 EISA cards. Unfortunately, DPT’s cards appear to have compatibility problems with certain CD-ROM drives.

DPT’s SCSI adapter is unique among the SCSI cards we tested in allowing the user to add from 0.5 MB to 16.5 MB of additional RAM, which is used exclusively as a disk cache. We tested the DPT in the Data General, both with and without an additional 4.5MB of cache memory. Not surprisigly, we found that adding the cache RAM dramatically improved the overall performance as measured by the Compile benchmark. (Due to cache overhead, raw disk performance actually decreased with the additional RAM.)

Many of the PCs equipped with Adaptec’s ISA-based 154X-series cards achieved different disk-transfer rates, even with the same SCSI hard disks. The explanation lies with a jumper switch on the Adaptec interface board that sets the board’s DMA (direct memory access) transfer speed; in testing these PCs, some manufacturers set the speed to the maximum rate at which their system can reliably operate, giving it higher performance.

We also compared the performance of the Adaptec 1540C in the Advanced Logic Research machine with its synchronous SCSI turned both on and off. We found synchronous reads to be nearly 20 percent slower than asynchronous reads; overall performance of our disk-intensive Webster benchmark decreased by five percent in the synchronous mode.

The fastest disk performance we found was from the NEC Express/II, which was equipped with the integrated DPT 2022 SCSI controller.

None of the systems tested included local-bus SCSI controllers, and none are supported under NEXTSTEP 3.1. When drivers become available for local-bus SCSI adapters, we expect them to be at least as fast as the EISA controllers. NEXTSTEP 3.1 also doesn’t include a driver for the Adaptec 6260 SCSI interface card (provided by Compaq and integrated on the motherboard of the Intel GX) or its successor, the 6360. A 6260/6360 driver is being developed.

Lastly, we found some PCs to suffer SCSI errors due to long, circuitously routed ribbon cables of poor quality. The biggest offenders in this category were Compaq and Dell.

Monitor quality varied more than any other factor among the PCs we tested. We found that Compaq’s Qvision-2000 and NEC’s 5FG were actually better than NeXT’s 17-inch color MegaPixel Display; some monitors, like Data General’s, were about as good; and others, like Dell’s Ultra Scan and Lucky-Goldstar’s Logisys, were quite unpleasant. If monitor quality is of the highest importance, consider choosing a system without a monitor and buying it separately. Ikegamis are the best monitors we’ve seen for running NEXTSTEP.

If you’re used to the sleek styling and solid construction of NeXT’s black hardware, you won’t like what the Intel world has to offer. We found the PC cases to be poorly designed; they were difficult to open and close and were held together by too many screws.

The machines with the best construction were Data General’s ‘486 DX2/66 LE2 and Intel’s GX. Both computers were put together with precision and obvious attention to workmanship. Data General’s was the only computer we tested that could be opened without special tools. It had no screws; like a chinese puzzle, though, it took us nearly half an hour to figure out how to take it apart.

None of the systems were equipped with flopy-disk drives that could notify the operating system when a floppy disk was inserted or eject a floppy under program control.

Comparing prices for NEXTSTEP PCs is difficult at this time, since many vendors supplied only estimated prives for our configurations. Some models are subject to substantial discounting while others are not. Nevertheless, the range of prices in our chart – from less than $5000 to more than $8500 for Compaq’s Pentium – is well below NeXT’s former pricing for a comparably equipped NeXTstation Turbo Color. We were very surprised to find that the NCR sytem was the least expensive in our group. Among the best price/performance values in our comparison were the systems from Lucky-Goldstar, eCesys, Continental, Pioneer, and NEC. Strong performers at a higher price point included Dell and Data General.

For many buyers, factors like the preloading of NEXTSTEP, the support staff’s level of knowledge, and the lengeth of the warranty will be as important as either price or performance. We did not attempt to judge these factors for this article, but they will be a part of our rating criteria in future issues.

All of these PCs represent compromises in dollars, features, and performance. Nevertheless, we were struck by how similar all of them really are. Each machine had different advantages, owing to the manufacturer’s choice of monitor, bus, SCSI interface, and video adapter. But an ATI graphics adapter on one system performed similarly to an ATI adapter on another; the same is true for CPU and SCSI choices.

Our two real-world benchmarks, Webster and Compile, confirm that most of the machines are faster than a NeXTstation Turbo Color for typical tasks. Leaving aside the Pentium, the fastest compile times were achieved by the Continental, NEC, NCR, and Dell systems. The machines fastest at running the Webster benchmark were the Data General, Compaq, and NEC models.

Being fans of black hardware, we were attracted to Intel’s GX, a tightly integrated system that looks remarkably like a white NeXTstation. The GX features an integrated SCSI and ATI video interface on its system board, as well as built-in 16-bit sound (sound and SCSI are not yet supported). Our chief complaint was that the GX’s integrated SCSI interface lacks a connector for an internal drive. The NEC Express/II also integrates the SCSI and video controllers on the motherboard. Integrated designs are easier to configure and may cost less, but they could limit future expandability.

While Dell’s 450DE/2 DGX offers the best graphics performance, its high price and disappointing CPU performance might cause developers to shy away from purchasing teh system. We hope to see a Pentium PC with JAWS graphics. On the other hand, Epson’s Progression might make a great developer’s workstation – but its ISA bus results in slow disk accesss.

Considering the hype that has accompanied the introduction of Intel’s Pentium chip, we were suprised by the poor performance of Compaq’s Pentium machine. While this computer’s CPU had the fastest MIPS rating, its IDE disk subsystem slowed it down when compiling. Its video performance was likewise compromised, thanks to Compaq’s Qvision board. It shows that having a fifth-generation Intel processor isn’t enough to guarantee a killer system: All aspects of the computer must be fine-tuned to achieve high performance.

A few of these machines have not been approved by NeXT to run NEXTSTEP. Unauthorized systems will probably work, as long as NEXTSTEP drivers are available for the SCSI adapter adn the video interface is installed. There are, however, hundres of ‘486 motherboards on the market, and reports from the field indicate that some of them are not compatible. Buyer beware.

So how does Intel hardware stack up against the Motoroloa standard? As a group, today’s NEXTSTEP-compatible PCs are slightly faster and considerably cheaper than NeXT’s late hardware. On the other hand, they are not as well-designed or easy-to-use. There is, as yet, no one system that is ideally suited for NEXTSTEP development. If you need white hardware for development projects today, we recommend purchasing a system with both EISA and VESA local buses: Such systems offer the greatest promise for upgrading during hte coming years.

Installing NEXTSTEP on an Intel computer can be a trying experience. Every "industry-standard" PC is different, and NEXTSTEP’s installation program isn’t yet fully cognizant of the peculiarities. Before starting, therefore, be sure to read NeXT’s release notes, to see if there are any special procedures that you need to follow for the computer or any of the add-on cards you might have.

We found NEXTSTEP easy to install on computers with the Adaptec 1542B and 1542C SCSI cards. We had a difficult time installing NEXTSTEP on system equipped with DPT’s 2012 SCSI interface, due to numerous options in jumper settings, BIOS configurations, and EISA-configuration utility settings. We didn’t have any problem with the NEC Express/II, which had DPT’s newer 2022 controller integrated on the motherboard. Because we didn’t have a 2022 card available, we were unable to determine whether this was due to integration or to improvements of the 2022 over the 2012.

We were unable to install NEXTSTEP 3.1 on the Advance 2000 (which arrived running a prerelease version of NEXTSTEP) because the system we received did not include an EISA-configuration utility disk. The Compaq 486 wouldn’t run at all, but that was probably a problem with our individual unit. The DEC arrived underconfigured and requiring a video driver that is not yet available. We had to benchmark a different NCR than the one supplied by the vendor because the one we received inexplicably would not boot with NEXTSTEP – even for the manager of NeXT’s Software QA lab, who had personally signed off on the compatibility of that model.

Once you’ve got NEXTSTEP installed and the machine reboots, you’ll want to run the NEXTSTEP Configure application to tell NEXTSTEP which brand of SCSI interface card, video adapter, network interface, serial ports, and sound card you happen to have, and which interrupts are currently being used. Determining where they should be mapped in the computer’s memory space rarely requires user intervention. If you create a conflict, for example, by assigning both the floppy disk and SCSI controller to the same interrupt; Configure will alert you to the conflict and give you a chance to correct it.

It’s possible by using Configure to misconfigure your system so that it will no longer boot. If you do this, don’t panic. Instead, press your computer’s reset button, and at the NeXT boot prompt type config=Default. This will cause NEXTSTEP to boot using its built-in SCSI drivers and teh standard low-resolution (640x480) VGA device driver. You can then log in as root and try again with the Configure application.

Considering all of the difficulties with installing NEXTSTEP, we offer a simple suggestion: If you can possibly get NEXTSTEP preinstalled on your ‘486 computer, do it!

IBM’s original PC had a bus that was 8 bits wide and ran at 4MHz. The PC/AT expanded the bus to 16 bits wide and upped its speed to 6MHz. Today, we call the 16 bit PC bus industry-standard architecture (ISA), and its speed has crept up to 8MHz.

ISA has two significant shortcomings that limit the performance of an ISA NEXTSTEP system. One is the data bus: Only 16 bits wide, it transfers data at about 16MB/sec. If you plug a video adapter into the ISA bus, NEXTSTEP won’t give you anything but 2 bit gray. Also, ISA’s address bus is only 24 bits wide. Since 24 bits can only address 16MB of memory, an ISA SCSI controller that reads data from a disk destined for memory above the lower 16MB must write it to a buffer in the lower 16MB; the CPU must then copy the data to its destination. This is called double buffering.

EISA (Extended Industry Standard Architecture) runs at 8MHz, and both its data and address buses are 32 bits wide, eliminating the need for double buffering and giving it a maximum bandwidth of 32 MB/sec. EISA provides ample performance for SCSI and Ethernet but still leaves much to be desired for graphics performance. Hence, local bus.

Local bus refers to a bus that is connected directly to the processor, similar to the computer’s memory bus, an idea originally pioneered by Dell. Other PC vendors followed suit, developing a host of proprietary and incompatible implementations.

To standardize the local-bus arena, the Video Electronics Standards Association developed the VESA Local Bus, often referred to as VLB or simply VESA. Like EISA, VLB is 32 bits wide. Local bus, however, runs at the processor’s external speed: A ‘486 DX2/66 has a local bus running at 33 MHz, giving it a throughput of 132 MB/sec.

With the exception of Compaq and Dell, most vendors have dropped their proprietary local-bus implemenations in favor of VESA. (All teh machines using C&T’s Wingine chip set also have propritary local-bus implementations, since Wingine is inherently incompatible with VESA.)

Competing with teh VESA Local Bus is Intel’s Peripheral Component Interconnect, or PCI. Unlike local bus, PCI is independent of the computer’s CPU. Thus, a PCI card can work with, say, Intel’s Pentium or Motorola’s PowerPC. The advantage that VESA holds over PCI is price. It is expected that VESA will remain dominant in ‘486 machines and that PCI will become dominant in Pentium machines.

NEXTSTEP works just fine with ISA, EISA, VESA, and PCI systems. NEXTSTEP won’t work with MicroChannel Architecture (MCA) computers, such as IBM’s PS/2 series.