Tests

First of all, we'll describe specifics of this article. In fact, this is not quite testing, because tested processors sort of do not exist in reality (except for one -- the original AMD Phenom II X4 940 Black Edition with all cores unlocked). So it does not make sense to choose between these processors, as there is no real choice here. So we'll focus on the global issue: what tendencies in modern software illustrate our today's test results, is it possible to detect these tendencies and digest them. A single comparative diagram in one software group will hardly suffice here, so we'll sometimes publish two diagrams in software groups and a table with detailed test results.

The first diagram is a traditional one. It contains scores of each "processor" calculated according to our test method. But today we decided to published them in the form of graphs, not bars, so that you could see how rapidly (or reluctantly) performance grows with the increased number of cores. The second diagram is a graph, where each curve stands for performance gains from added processor cores for each application in a given software group. We start with a single-core system, which performance is taken for 100% (so all curves start from the same point). This diagram allows to track behavior of separate programs, which may be very useful in the scope of our article. And finally, a table with test results (for each application separately). Starting from the "2 cores" column, it includes not only absolute test results, but also percentage values. What are they? They reflect performance gains of a given system relative to the previous one. It's very important: relative to the previous one, not to the original single-core system. Thus, 12% in the 3-core column means that the triple-core system is faster by 12% than the dual-core system in a given application. The same result for the dual-core system would mean its performance advantage over the single-core system.

By the way, you'd better refresh arithmetics in your memory not to be surprised by obvious things: only a dual-core system can (only in the ideal case*) be faster than the single-core system by 100%. The triple-core system cannot be faster than the dual-core system by more than 50% even in the ideal case. The quad-core system may be faster than the triple-core system by no more than 33%.

Besides, we traditionally publish a link to a Microsoft Excel spreadsheet for curious readers, which contains all test results in the most detailed form. Besides, it includes two additional tabs "Compare #1" and "Compare #2" to facilitate their analysis. Just like the tables in the article, they compare the four systems percentagewise. The difference is simple: Compare #1 provides percentage values that are calculated just like in tables from the article -- relative to the previous system. Compare #2 compares all systems with the reference single-core system (like on the second diagram, only in the text form).

3D visualization

* The up arrow (↑) marks tests, where the highest results are the best, the down arrow (↓) marks tests, where the best results are the lowest.

We've always suspected how well 3D programs can use multi-core systems for interactive operations, not just for rendering. For this very reason we've singled out 3D visualization into a separate group of tests in our new test procedure. The first diagram does not look optimistic at all: upgrade from one to four cores generally yields measly 17% of performance gain. But maybe some applications are doing better than the others? Let's take a look at the second diagram and the table.

Maya gets the highest performance gain from multi-core systems: it rather optimistically (+26%) responds to the second core, almost just as well to the third core (another +22%), and only the fourth core adds only 1%. Lightwave and Pro/ENGINEER manage to use the second core for visualization, but much worse and only the second core -- they practically don't respond to the third core. We've marked two results with red, when the system actually slowed down with more cores. But don't worry, a difference of 1% may actually be a measurement error. However, let's keep it in mind just in case: it happens in both cases, when we increase the number of cores from two to three.

3D rendering

This diagram is much more optimistic: increasing the number of cores from one to two gives a 116% performance gain (!); and if the number of cores is increased from one to four -- 400% (that is the maximum possible theoretical ideal scenario). However, the second diagram and the table hold a big surprise in store: in two cases out of three, the second core increases rendering speed much higher than by 100%! That is it exceeds the theoretical limit. If the difference had been at least 5-6% -- we could have written it off to a measurement error (doubtful, but possible). However, it cannot possibly be 23% and 37%. So what happened?

We can actually conjecture many possible reasons. After all, the theory operates with "sterile" conditions, while our software runs under a certain operating system, which includes hundreds of background processes. As I wrote this article, I even started a new topic in iXBT.com for our readers to post their guesses. Lots of assumptions were gathered, but we've singled out two most plausible ones: effect of the increased shared cache with two cores unlocked (L2-Cache in the AMD Phenom II is "owned" by the core, so it's disabled together with its core), and poor parallel processing in software (that is, single-core systems slow down the rendering engine). By the way, if the second assumption is true, it's very sad. On the other hand, it's a funny tendency: it looks like software developers have already forgotten* about single-core systems, because multi-core processors dominate the market -- hence such strange effects.

* -- Of course, we can look at it from the conspiracy theory, assume evil design and clandestine collusion with CPU manufacturers... but it's too much in our point of view.

The diagram also shows that Lightwave render engine features the best optimizations for parallel execution: it always demonstrates almost maximum performance gains. Maya (MentalRay) has the most stable engine -- its performance curve is almost a straight line, no hump in the transition area from one to two cores.

Scientific and engineering analysis

As you may remember (you might want to check the test procedure, if you don't), this section includes results of various computational tests. This group is divided into engineering (SolidWorks, Pro/ENGINEER, UGS NX) and scientific (MAPLE, Mathematica, MATLAB) software. You can see that there is no accord even within these groups -- MAPLE from the scientific group actually drops performance by 8%, when the second core is enabled (That transition! There is always something wrong with it!) What concerns the engineering group, SolidWorks and Pro/ENGINEER with their +17 from the second core coexist with the totally indifferent UGS NX (+3% to the second core and no response to the third core).

The third core show even a worse situation: in fact, only MATLAB can use it more or less efficiently (don't forget that +12% for the third core in MAPLE accompany -8% for two cores). The fourth core is not used efficiently even by MATLAB -- but this program at least tries to do it. But the other programs fail to do it, Pro/ENGINEER even suffers from a little performance drop with four cores.

A similar situation was in the 3D Visualization group: only a couple of programs can effectively use several cores. No more. Besides, as in case with 3D Visualization, there is one well-parallelized program, which looks great on the common poor background. In this case it's MATLAB. That's why the overall performance gain between the extreme points (1 core and 4 cores) is slightly higher than with 3D Visualization -- 24%. However, those users, who want to buy a quad-core processor, will hardly be optimistic about this "high" efficiency.

Write a comment below. No registration needed!