One of the biggest buzzwords in today’s technology infrastructure is Deep Learning (or Machine Learning or even AI) and it seems that everyone is jumping on the bandwagon. However, you could find that the GPUs doing the processing for your neural network are waiting on the data to get to them. This means that you are losing potential time for the processing that the neural net has to wait on that data.
Frequently this bottleneck happens in the bus on the motherboard as it can only move as fast as the slowest pathway while your AI computer, with it’s top-of-line graphics cards (and likely multiple graphics cards), just cruises through the data once it finally gets there.
Now don’t get me wrong, but if you want an AI machine at the price those machines run, you definitely want to make sure that you are tailoring the machine to be as fast as possible and not have any bottlenecks to ensure a consistent and steady flow of data.
There are a few different paths that CAN be taken, but you should know what you are looking for before you go dropping a five figure amount of cash for one of these machines (and yes, it can and very likely will be at least that high and some of these machines reach six figures).
With the bottleneck occurring between the graphics cards and the hard drive, we need to figure out what we can do to fix it. Hard drives today come in a variety of connection ports to connect the drive to your computer. These are the SATA connections that are frequently used for SATA drives and SATA connected SSD drives. There is also a newer option, the M.2 drives. So let’s take a look at each of these.
SATA SSD Hard Drives
The first one up is the SATA spindle drive (called that because it is made up of platters spinning on a spindle. You can get these in speeds of 5400, 7200, and 10k rpms. These drives must spin-up in order to access the data that they store.
That kind of delay can cause some headaches when working with an Neural network system and leave your GPUs waiting for their next byte to come around the corner.
The next option is an SSD drive. This drive sits ahead of the SATA option above because a solid state drive does not need to spin up to get its data. The throughput is also about five times faster than a SATA spindle drive.
The third option is go with an NVMe ( Non-Volatile Memory Express ) drive. Some NVMe drives look like RAM sticks while others are daughter cards that plug into an NVMe slot on the motherboard. NVMe drives are mostly solid state drives, whether it looks like a RAM memory stick or it actually looks like a hard drive and gets slotted via a daughter card. NVMe SSDs run about five times faster than a SATA SSD.
M2 SSD Hard Drive
With the above information in mind, the NVMe SSD provides the fastest path to the graphics cards, thereby opening up that bottleneck as much as possible and keeps data flowing to the graphics cards.
Here at BOXX, we take every aspect into consideration so you get the best performance from your machine learning needs.
Head over to our Deep Learning page to learn more and see what answers we have for your AI questions or fill out the form to request a consultation.