Computing on a Budget
High Performance Computing on a Budget
A very common question in the comments and the mails we receive through our Internet page is about the computer system which one should get when starting simulations with liggghts or openFoam.
To be fair this is not as easy to answer as it may seem. First of all. This depends very much on you financial situation and on the problem you want to solve. While a person working for a major enterprise may can effort a cluster system out of numerous server-blades the phd candidate in a uni may has different limitations.
Benchmark
Nevertheless, we want to give an easy access point here, like we are trying to do with this complete project. Our goal is to lower the entry step into DEM and CFD simulation for industrial relevant problems. Of cause we could recommend now something what we think is appropriate, but our goal was to put a system together, with the best bang for the buck that is available in the moment. It’s basically High performance computing on a budget.
But before we can do that, we have to create a test simulation which we ca use to measure the actual performance. Because only if we have measured the performance, we can be sure that we are recommending something which will work fine even on bigger projects. The first question than will be what is a bigger project? I would say that a modern system should be capable to handle 1 million particles in liggghts in a 20 s simulation. With that system you can easily simulate a lot of industrial applications and also a lot of science problems.
Our performance benchmark contains a belt conveyor with a rotating pulley and a larger hopper which will hold the particles. The number of particles will increase until 1 million particles are filled into the hopper. The benchmark is available on our webpage engineerdo.com. If you like you can run it on your computer and post the results in the comments or sent them to us via mail.
As a disclaimer, we did not get paid by any company to do this test. What we recommend and the number we show are tested by us. All computers we use are our own or systems we have access to.
The Results
We tested this simulation on nine different computers from the last years. Most of the computers are workstation PCs with two Intel processor CPUs and a different number of cores. However, within our test batch we also included two single CPU consumer PCs. One of these will be our recommendation. Additionally we were able to run the simulation on a cluster with 4 CPUs. The results are shown in this diagram.
The slowest computer was an i7 6400K with 4 cores running 4 GHz. On position 8 to 4 we have a variety of Intel Xeon Server CPUs with mainly two CPUs build into one workstation. From 8 up to 24 CPUs were used during the simulation.
Position one is set by the 4 CPU cluster using 64 cores. The second position is another Workstation using two Intel Xeon e52687W v3. On position thee we have our recommended system if you want to do high performance computing on a budget. As you can see this system is fairly compatible using a 12 core state of the art AMD Ryzen 9 3600x. We used this system instead of the 16 core alterative from AMD because of its higher clock rate and the lower price.
Avoid Background Tasks!
If we have a closer look to the results, we see that the computational performance is changing during the runtime of the simulation. While the simulation starts with no particles Liggghts is performing a high number of steps per second. The number of particles increases until 1 million are inserted while the steps per second which can be computed are decreasing. Also, after 15 s all particles are inserted and the system comes to a steady state with less and less kinetic energy in it. This can be seen in the increasing rate of steps per second at around 17s in the diagram. During the last two seconds the particles are mainly at rest so the solutions algorithm has less to do, due to no changing contacts.
If we look to these two results in detail, we can see, that the 12 Core high clock-rate CPU performs significantly better for smaller particle numbers but with a higher count of particles in the system the differences are getting smaller. As a result, we can assume, that a higher number of cores in a system should be used if very large numbers of particles have to be simulated. A cluster computer will perform at best if the number of particles increases significantly above one million.
Another interesting fact we could find in the results, is a partly reduced computational efficiency during the simulation. The orange and the blue line show a reduction of the step rates by roughly 25% for some time during the runtime of the simulation. This could be a result of backgrounds of these computers. Another reason could be insufficient cooling. However, cooling issues would probably stay during the runtime and would not fluctuate.
AMD – The best BANG for the BUCK
The system we are recommending uses a beside the Ryzen 9 from AMD a X570 Mainbord chip, 32 Gb of DDR 3200 MHz Ram and a Geforce RTX 2060 graphics card. For main storage a 1 TB M.2 PCI Express SSD is built into the system. If you want build a similar system, we placed a couple links to the main components in the description below the video. With these components you know what you can expect from your further simulation machine.
This system should cost you roughly between 1200 and 1300 €. The power supply with a 630 W power output is probably the smallest in the commotion and should also run the simulation much more energy and coast efficient.
IF you order your own system and you want to compare it using our benchmark please make sure that you are not changing the script. The simulation runtime can probably be improved, if the cores are aligned manually. However, the results are not comparable anymore if done so. The benchmark result is written into a csv data-file which you can sent in with the processor description of your computer
Our Benchmark
You can download the performance benchmark of Liggghts here:
