Basement Supercomputing

Limulus Personal Cluster Questions

The following topics about our personal HPC cluster solutions are discussed as a series of questions and answers.


Is this a real cluster?
Yes, it is a real cluster. The basic system has four motherboards which are cluster nodes in a standard off the shelf case with a single power supply. The main motherboard is always powered and functions just like a workstation. The three compute nodes can be powered-on when needed. The software is identical to that running on large clusters.

What can I run on such a system?
Any software that is designed to run on a Linux cluster including both High Performance Computing (HPC) and Apache Hadoop applications.

Can I do real HPC work on such a system?
Yes. Considering that surveys have shown that around 40% of HPC users use less than 16 cores (over 50% use less than 32 cores), it should be a very usable system.

How fast is it?
The first Limulus version achieved 385.5 double precision CPU GFLOPS (52% of peak running HPL). Note, benchmarking results in HPC are very application specific. While a GPU might match the performance of a small cluster for some applications, it is not a general purpose computing device, and therefore is not as flexible as a CPU cluster.

What Does Limulus Mean?
Limulus is an acronym for LInux MULti-core Unified Supercomputer.

What is the difference between a Limulus and a 8/16-core workstation?
In terms of core count, there is no difference. In terms of price and performance there can be a big difference. A multi-core SMP system (such as a dual socket workstation or server motherboard) can provide many cores, but depending on workload, you may not be able to get effective use from all of the cores due to memory contention (See Benchmarking A Multi-Core Processor For HPC and Exercising Multi-core). In a cluster design, such as Limulus, each node has one processor socket with exclusive access to the local memory. In addition, unused nodes can be powered off when not in use.

In terms of price, comparable workstation solutions start at $34/GFLOPS, while a Limulus personal workstation cluster is $16/GFLOP.

Why don't I just use the cloud?
An effective use case for the cloud must include cost, data transfer, security, and flexibility. While cloud has its place in HPC and Hadoop processing, there is nothing like "owning the reset switch."

What is Lumulus' market?
There are several areas where a personal cluster can be useful (i.e where you own the reset switch)

  • System administrators - a cluster sandbox to try new things, test software packages
  • Software developers - a private software development environment
  • Academic projects - instructional hardware, student projects, learn to run real HPC/Hadoop codes
  • Cloud staging - stage and develop cloud HPC software before launching it to the cloud
  • Small scale production work - test ideas, run applications under your control
  • Small and medium-sized business HPC - explore how HPC can help manufacturing without a huge investment
  • Big Data/Hadoop - try and test big data and Apache Hadoop projects using up to 8 TB of SSD storage without the administration overhead and cost of a data center cluster


How many cores can you fit on one case?
Currently, a single Limulus system can provide at least 16 cores. As technology progresses we expect that number to increase.

What kind of processors do you use?
Our current designs include processors from Intel (Skylake). For the nodes we use low power (65 Watt) quad-core x86_64 processors.

What kind of motherboards do you use?
We can use almost any standard Micro-ATX motherboard. However, we prefer to test them before we recommend any specific motherboard. Geometry and component issues (i.e. Gigabit Ethernet chipset) make some boards more desirable than others.

How do you fit those extra motherboards in a standard case?
We designed custom parts to hold the motherboards, a small Ethernet switch, and some other parts. We tried to keep the cost of the custom parts as low as possible. We also took the time to create a clean design and keep the cabling neat.

Can I attach a keyboard and monitor to the node motherboards?
Yes. There is a front panel that provides video, USB, and a power switch for each motherboard.

Why don't you pack a bunch of 12-core processors into the case?
Limulus is designed with a desk-side heat/power/performance/noise envelope. An HPC server can pack in many cores because in center there is dedicated power, cooling, and a tolerance for fan noise. Have you ever run an HPC server (or two) next to your desk?

Why don't you pack a bunch of GPU processors into the case?
Using GPUs is a great solution if it fits your problem, but GPUs require more power and more heat must be removed from the case. Thus, these devices (as currently used in HPC) don't fit into the Limulus heat/power/performance/noise envelope.

Do you use dual socket motherboards?
No. Limulus is designed to use single socket Micro ATX motherboards. These offer a balance of expansion (RAM and PCIe slots), power and size.

Can I connect multiple cases?
Yes. A second Limulus case can be connected to the first one with a single cable.

How are the nodes connected?
Gigabit Ethernet (GbE). There is now a low-cost 10-GbE option.

Does each motherboard have a hard drive?
No, in the standard design only the main motherboard has persistent storage. The nodes run using a RAM disk and are stateless (We use the Warewulf Provisioning software). It is possible to connect drives to the worker nodes, however, all drives will remain powered up while nodes are off. The Limulus software will place these drives in standby mode and they will become locally available to the node when restarted.

Our Hadoop models have drives connected to each motherboard, however, the nodes still boot and run out of the RAM disk.

Can I add extra hard drives?
Yes, each system comes with an SSD for system software and two additional spinning drives for application use. There is an additional 2.5 inch bay and two open 3.5 inch drive bays that can be used to increase the total number of drives to six. Drives can be configured as stand-alone disks or as RAID sets. The Limulus Hadoop version has eight 2.5 bays (holding SSD disks) and two 3.5 inch bays for large spinning disks.

Can I add expansion cards to the node motherboards?
Yes, there is an optional bracket for adding one low profile PCIe card.

Can I add expansion cards to the main motherboard?
Yes, all slots are available, but long cards may not fit (e.g. huge video cards).

Can I add video cards to the node motherboards?
No, it would create too much heat to move with the current low noise fans.

How big is the power supply?
850 Watts

Does it have ECC memory?
The current hardware does not support ECC (Error Correcting Memory). The requirement for ECC depends on your needs. In our experience, and in our testing, we have found excellent results with "quality" memory (not the bargain priced brands). We have never had a problem with non-ECC memory in our personal cluster systems. We have run (self checking) codes for days without any issues. Of course if you are planing to run a three week parallel job with no check-pointing you may want to consider ECC memory and/or check-pointing.

Can it use 10 GbE or IB?
We support 10 GbE using a switchless design. Single byte latencies range from 8-10 μsec (using Netpipe). IB is not currently supported.


How much power does a Limulus use?
When running HPL (16 cores) we measure 320 Watts. Of course it also depends on what you put in the system (i.e. disk, video card, etc.)

How many standard wall plugs does it use?

Can I manually turn nodes off and on?

Can I automatically turn nodes off and on?
Yes, and you can even integrate this into the resource scheduler.

Does it create a lot of heat?
Like all electronic devices, it generates heat. Unlike a high-end servers, it would make a poor space heater. We use 65 Watt processors. Limulus will not create anymore heat than most older PCs.

How loud is it?
It is very quiet. The use of large fans helps reduce noise considerably. It can sit next to a desk in an office without any significant impact on the ambient noise environment (i.e., it will not disturb conversations, listening to music, or phone calls.


What software does it run?
Linux of course. It comes installed with HPC cluster stack (the same software that runs on bigger clusters). All base packages are open source and built on top of Scientific Linux 6.X. Other RPMS and SRPMS are freely available.

Will updated software be available?

Is software support available?
Yes, each system comes with 90 days of support. Extended support options are available.

Is it the same software that runs on big clusters?
Yes. Applications can be moved from/to other clusters with little or no effort.

Will there be other open source application software available?

Can I install my own software?
Yes, this is an open source platform, you control your destiny!

Can I run commercial software?
If your software can run on a large Linux cluster, it can probably run on a Limulus system. Currently, we use Scientific Linux 6.X, which is a community rebuild of Red Hat 6.X. Of course it all depends on the software as well as vendor policies.

Can I run Windows on it?
Probably, but we have not tried.


Does it come with hardware support?
Yes. One year return to depot support is included. You must pay shipping to the depot.

Does it come with software support?
Yes. Ninety days of software support is included. Extended support options are available.

Your Cart

The cart is empty