I'm working now on issue #4923 and this requires some measurements of the time the automated headless tests take to run. Thanks to Linus' efforts in setting up a continuous integration server this isn't very difficult, I will simply take the latests 10 results from the revisions of the JUnit reports summary and compare them with the next 10 results after making my change in the model-mdr implementation.
So, here are the values and the total average before the change:
revision | Tests | Failures | Errors | Success rate | Time (s) |
---|---|---|---|---|---|
rev=4456 | 1115 | 1 | 0 | 99.91% | 3065.731 |
rev=4443 | 1115 | 1 | 0 | 99.91% | 2921.652 |
rev=4431 | 1115 | 1 | 0 | 99.91% | 2899.956 |
rev=4408 | 1117 | 2 | 0 | 99.82% | 2792.906 |
rev=4392 | 1116 | 3 | 0 | 99.73% | 2789.595 |
rev=4378 | 1116 | 0 | 0 | 100.00% | 2866.223 |
rev=4367 | 1116 | 0 | 0 | 100.00% | 2842.319 |
rev=4355 | 1117 | 0 | 0 | 100.00% | 2861.930 |
rev=4345 | 1117 | 0 | 0 | 100.00% | 2887.080 |
rev=4340 | 1101 | 0 | 0 | 100.00% | 2787.845 |
Average time: | 2871.524 |
revision | Tests | Failures | Errors | Success rate | Time (s) |
---|---|---|---|---|---|
4462 | 1115 | 1 | 0 | 99.91% | 2715.539 |
4448 | 1115 | 1 | 0 | 99.91% | 2665.635 |
4437 | 1115 | 1 | 0 | 99.91% | 2665.288 |
4400 | 1116 | 3 | 0 | 99.73% | 2526.352 |
4383 | 1116 | 0 | 0 | 100.00% | 2627.562 |
4372 | 1116 | 0 | 0 | 100.00% | 2596.418 |
4360 | 1117 | 0 | 0 | 100.00% | 2651.676 |
4350 | 1117 | 0 | 0 | 100.00% | 2654.227 |
4332 | 1101 | 0 | 0 | 100.00% | 2514.612 |
4318 | 1101 | 0 | 0 | 100.00% | 2509.551 |
Average time: | 2612.686 |
Tomorrow I'll make tests with ArgoUML running to check if the MDRModelImplementation
constructor is called during its execution and if not I'll commit my changes to MDR. Then, it is a matter of waiting 10 days for the veredict of the performance hit.
Update on 2007-12-19: added averages and results after changes.
revision | Tests | Failures | Errors | Success rate | Time (s) |
---|---|---|---|---|---|
4611 | 1116 | 0 | 0 | 100.00% | 2910.440 |
4594 | 1116 | 0 | 0 | 100.00% | 2917.587 |
4581 | 1116 | 0 | 0 | 100.00% | 2905.284 |
4570 | 1116 | 0 | 0 | 100.00% | 2903.603 |
4559 | 1116 | 0 | 0 | 100.00% | 2898.110 |
4546 | 1116 | 0 | 0 | 100.00% | 2909.500 |
4533 | 1115 | 0 | 0 | 100.00% | 2906.624 |
4521 | 1115 | 0 | 0 | 100.00% | 2904.351 |
4508 | 1115 | 0 | 0 | 100.00% | 2939.427 |
4496 | 1115 | 0 | 0 | 100.00% | 2903.854 |
Average time: | 2909.88 |
revision | Tests | Failures | Errors | Success rate | Time (s) |
---|---|---|---|---|---|
4586 | 1116 | 0 | 0 | 100.00% | 2665.922 |
4575 | 1116 | 0 | 0 | 100.00% | 2665.115 |
4564 | 1116 | 0 | 0 | 100.00% | 2660.793 |
4552 | 1116 | 0 | 0 | 100.00% | 2677.356 |
4538 | 1115 | 0 | 0 | 100.00% | 2668.918 |
4526 | 1115 | 0 | 0 | 100.00% | 2677.958 |
4514 | 1115 | 0 | 0 | 100.00% | 2699.183 |
4501 | 1115 | 0 | 0 | 100.00% | 2687.672 |
4488 | 1115 | 0 | 0 | 100.00% | 2630.221 |
4474 | 1115 | 1 | 0 | 99.91% | 2757.297 |
Average time: | 2679.04 |
So, a ~2.5% performance hit in Java 6 and a ~1.3% hit under Java 5.
No comments:
Post a Comment