[H-GEN] Which is better?
Russell Stuart
russell at stuart.id.au
Tue Apr 27 23:30:28 EDT 2004
On Wed, 2004-04-28 at 11:10, Adrian Sutton wrote:
> It leaves a heck of a lot more than memory allocation. Firstly, if the
> GUI is entirely done by Windows then the memory allocation isn't a
> factor because it's all handled by the same code that is used if you
> write an application in C.
Actually no. All the Windows objects (windows / components) have their
equivalents in C#. Ditto for the windows events.
> Secondly, the overhead of converting types
> from one language to another incurs a major performance hit.
You sound very sure of this. I take it you have looked at the internals
of MS's C#, or perhaps mono? Personally, I work on the PNet source. In
PNet there does happen to be a performance hit for PInvoke stuff, but
not a big one for the native libraries. I imagine that is the same for
MS's stuff as well - but who knows?
> The reason benchmarks are so useless with Java and the reason your
> results are completely meaningless is because the behavior of the
> garbage collector is completely uncontrolled. In your tests you are
> creating a whole bunch of objects but not controlling when or if the
> garbage collector will run. Thus, at any point in any of the tests the
> garbage collector may kick in either for a partial sweep or a full
> sweep which will most likely dwarf the time taken actually running the
> test. Further to this, the JIT may kick in at any time and completely
> change the actual code that's being run. Also consider the fact that
> the first test to run must load the relevant classes, however this does
> not affect the other tests because they are already loaded. Finally
> add in the fact that System.currentTimeMillis() is notoriously
> inaccurate.
Well as it happens I did re-order the tests a few times as I wrote the
code. There was no material difference in the times. The relative
positions of all tests remained the same, regardless of the order. I
can only conclude either the JIT or GC didn't run, or if they did then
they didn't effect the result.
Still, I doubt you will believe me. You have the code - re-order it
yourself and see if you get different results. And as for the class
loader - well you didn't look too closely at the test program before
responding did you?
BTW, System.currentTimeMillis() can inaccurate if you are trying to
measure milliseconds. It gets hit by the GC, amoung other things. But
as it happens, most of my tests ran for seconds, not milliseconds.
System.currentTimeMillis() is more than accurate enough for that.
> The actual times you've received show no significant difference between
> any of array tests.
Well, perhaps. There isn't a huge difference, I'll grant you. But it
is repeatable. For now, under 1.4.1, I'll assume the AJ, and Harry
versions are faster.
By the way, what is your basis for this claim? You ran the test
yourself perhaps, and and noted the differences between runs? Maybe you
are using some other method to measure the significance. Can you tell
us what it is?
> The format tests is really a test of doing a whole
> lot of string parsing compared to doing no string parsing (surprise,
> doing no string parsing is faster).
No its not surprising that moving the string parsing takes a while.
That is why a programmer should move it out of the loop. That was the
point I was trying to make, if you recall.
> That leaves the Dimension tests.
> Now, if you were an optimizing compiler and came across the code:
>
> > for (int i = 0; i < 10000; i += 1)
> > continue;
> > }
>
> Would you run it? No, this code will most likely be completely removed
> once the JIT kicks in on that piece of code. Thus, you're comparing
> creating 10000 Dimension objects to doing nothing. The result is no
> surprise.
No not at all. Not to you, not to me, not to any programmer. So why,
when I suggest a program should move it out of the loop as a matter of
course as he writes the code, do you jump and and down and say NO, NO -
you are wrong - java's memory allocation is so cheap you don't have to
do that.
By the way, the code was executed 10^8 times, and the JIT didn't remove
it.
> Worst of all, the dim unoptimized test must first initialize the entire
> AWT libraries before it can even begin, which would explain the entire
> difference in that test.
Again, you didn't look at the code, did you? That is not the case.
> In your specific example, on your specific platform, with your specific
> amount of RAM and with your specific CPU speed and architecture. The
> fact that Java 1.4.2 adds further optimization strategies to the JIT
> should not go unnoticed either. As mentioned above however, it most
> likely means that the garbage collector didn't kick in during Harry's
> test but left those objects to be cleaned up during a different test.
Again, you bring up a number of reasonable sounding things (JIT, GC),
but do you any evidence that they actually occurred? Or is this all
hand waving on your part? I am, for my part, am fairly convinced what
you are describing didn't happen under 1.4.1.
Your point about different JIT's is valid. Things may well be
different. I was curious enough to find out. I altered the tests to
reduce the run time, and to try and reduce any gc interference between
run times, I also perform the tests twice in the same run. This time I
used 1.5.0 beta 1. These are the results I got:
dim unoptimised: 32010
dim optimised: 2090
format unoptimised: 31923
format optimised: 7454
Array unoptimised: 20450
Array harry: 20537
Array aj: 20399
Array david1: 20484
Array david2: 20951
Array david3: 20395
===
dim unoptimised: 31630
dim optimised: 2081
format unoptimised: 31317
format optimised: 7509
Array unoptimised: 20537
Array harry: 20403
Array aj: 20404
Array david1: 20542
Array david2: 20390
Array david3: 20417
Under 1.5.0, there appears to be no significant difference between the
array implementations. Oddly, they all appear to be slower than 1.4.1.
Well, I guess it is a beta.
There also appears to be no significant difference between run 1 and run
2. This is yet more evidence against the JIT and GC effecting the
results.
> This most definitely does harm readability. Most programmers will
> expect that the iteration of an array will occur incrementally, not
> decrementally. By reversing the order of the iteration you break that
> expectation and make the programmer stop and think about exactly what
> the code does as they no longer recognise the pattern for a for loop
> iterating over an array. Again, there is You're talking about a speed
> up of 0.1 of a second even if you consider the measurements above to be
> valid.
So you claim. I personally don't see any difference. But in this case
beauty really is in the eye of the beholder, and being an older
programmer who lived when CPU cycles were scarce - I have seen a lot of
code like this. Perhaps you haven't.
> There is no difference in the times. Further to this, the point at
> which Java performs it's optimizations is undefined. There is no way
> to tell if it would have optimized it further or not. Assuming that
> compilers and JITs don't optimize code is just plain idiocy as it's
> clear that there are so many people employed (with Sun in this case) to
> implement these optimization routines.
This sounds a lot like what you have written above. Lots of
authoritative sounding phrases. But since you offer no way of checking
on them I would have to take your word for it.
========== Main.java ====================
package allocdemo;
public class Main {
interface Test {
public void test();
}
public static void main(String[] args) {
runTests();
runTests();
}
static void runTests() {
runTest("dim unoptimised", dim_unopt);
runTest("dim optimised", dim_opt);
runTest("format unoptimised", format_unopt);
runTest("format optimised", format_opt);
runTest("Array unoptimised", array_unopt);
runTest("Array harry", array_harry);
runTest("Array aj", array_aj);
runTest("Array david1", array_david1);
runTest("Array david2", array_david2);
runTest("Array david3", array_david3);
}
static void runTest(String description, Test test) {
System.gc();
long start = System.currentTimeMillis();
System.out.print(description + ": ");
System.out.flush();
for (int i = 0; i < 10; i += 1)
test.test();
for (int i = 0; i < 100*1000; i += 1)
test.test();
long end = System.currentTimeMillis();
System.out.println(end - start);
System.gc();
}
static Test array_unopt = new Test() {
public byte[] array = new byte[100000];
public void test() {
for (int i = 0; i < array.length; i += 1)
continue;
}
};
static Test array_harry = new Test() {
public byte[] array = new byte[100000];
public void test() {
int l = array.length;
for (int i = 0; i < l; i += 1)
continue;
}
};
static Test array_aj = new Test() {
public byte[] array = new byte[100000];
public void test() {
for (int i = array.length - 1; i >= 0; i -= 1)
continue;
}
};
static Test array_david1 = new Test() {
public byte[] array = new byte[100000];
public void test() {
for (int i = array.length - 1; i >= 0; --i)
continue;
}
};
static Test array_david2 = new Test() {
public byte[] array = new byte[100000];
public void test() {
for (int i = array.length - 1; i >= 0; i--)
continue;
}
};
static Test array_david3 = new Test() {
public byte[] array = new byte[100000];
public void test() {
for (int i = array.length - 1; i >= 0; i -= 1)
continue;
}
};
static Test dim_unopt = new Test() {
public void test() {
for (int i = 0; i < 10*1000; i += 1) {
java.awt.Dimension d = new java.awt.Dimension();
}
}
};
static Test dim_opt = new Test() {
public void test() {
java.awt.Dimension d = new java.awt.Dimension();
for (int i = 0; i < 10000; i += 1)
continue;
}
};
static Test format_opt = new Test() {
public void test() {
java.util.Date d = new java.util.Date();
java.text.SimpleDateFormat myFormat =
new java.text.SimpleDateFormat("yyyy-MM-dd hh:mm:ss");
for (int i = 0; i < 10; i += 1)
myFormat.format(d);
}
};
static Test format_unopt = new Test() {
public void test() {
for (int i = 0; i < 10; i += 1) {
java.text.SimpleDateFormat myFormat =
new java.text.SimpleDateFormat("yyyy-MM-dd hh:mm:ss");
myFormat.format(new java.util.Date());
}
}
};
}
More information about the General
mailing list