Building Enterprise-Grade HPC/AI Environments

Definition: Artificial Intelligence, Machine Learning, Deep Learning

Artificial Intelligence (AI) is part of computer science, which deals with the development of intelligent machines that work and react like humans. It is the great project for the formation of a non-human intelligence. Most important parts are:

• The main tasks of machine learning (traditional computer vision) are data preparation, feature engineering, model architecture and numerical optimization. Feature engineering takes up almost 80 percent of the preparatory work.
• Deep Learning, part of Machine Learning, is a collection of easy to train mathematical units that are organized in layers, working together to solve complicated tasks. New are the layered network architecture and a scalable training method. DL learns features directly from the data; explicit feature engineering is not required. It has achieved an extremely high degree of accuracy, surpasses human performance in the classification of images, never tires and delivers results in a fraction of the time.

How to build an Enterprise Deep Learning / HPC environment

Today, deep learning and high-performance computing tasks are performed on the same hardware platform with shared cluster management software and other shared or separately used software. Before we deal with the necessary hardware and software, we first want to deal with the various work processes. Before we deal with the hardware and software required for this, we first want to take a closer look at the different workflows.

AI Workflow and Sizing, it all starts with the data

The most important thing for Deep Learning are your data. The volume could become peta bytes of data. The more the better.
A typical AI/deep learning development Workflow:

The workflow is detailed as follows:

Sizing DL training is highly dependent on data size and model complexity. A single Tesla NVlink Server (e.g. DGX-1) can complete a training experiment on a wide variety of AI models in one day. For example, the autonomous vehicle software team at NVIDIA developing NVIDIA DriveNet uses a custom Resnet-18 backbone detection network with 960x480x3 image size and trains at 480 images per second on such servers allowing training of 120 epochs with 300k images in 21 hours. Internal experience at NVIDIA has shown that five developers collaborating on the development of one AI model provides the optimal development time. Each developer typically works on two models in parallel thus the infrastructure needs to support ten model training experiments within the desired TAT (turn-around-time). A DGX POD with nine DGX-1 systems can provide one day TAT for model training for a five-developer workgroup. During schedule-critical times, multi-node scaling can reduce turnaround time from one day to four hours using eight DGX-1 servers. Once in production, additional DGX-1 systems will be necessary to support on-going model refinement and regression testing.

HPC Workflow and Sizing, it all starts with the Data Model

The most iportant thing for HPC are your data and data model. The volume could become peta bytes of data.
A typical HPC development Workflow:

The workflow is detailed as follows:

What are the building blocks of an Enterprise Deep Learning / HPC environment?

Following the workflow, we need the following system modules:

1. Storage Systems
   1. Storage that is equally well suited for Deep Learning and HPC
      Because supercomputing is write-intensive and sequential in access and AI is read-intensive and randomly oriented in access, the storage systems we offer are designed to support both HPC and AI in the best possible way. This way you avoid double investments in hardware, storage software and training.
   2. Storage that is easily expandable by adding additional servers or JBODs (HDD) or JBOFs (NVMe/SAS/SATA SSD) during operation.
   3. Storage that supports fast operational data, archived data, and extremely fast data stores temporarily mounted on the local NVMe SSDs of converged computing and storage servers.
   4. Fault tolerant Storage, compensation the loss of Data, complete HDD or SSD or complete servers including all data on their RAID volumes - and that with commodity servers and shared nothing hardware.
   5. Further information can be found in the storage part of this paper or in our web offer.
   
   
2. GPU-Compute Server
   1. GPU-Computing that is equally well suited for Deep Learning and HPC
      Since DL models often require an extreme amount of memory, it is very important that the GPU cards have as much local memory as possible. Therefore, current V100 GPU cards have 32 GB HBM2 memory.
   2. HPC is extremely compute-intensive and requires state-of-the-art graphics processors. HPC is extremely computationally intensive and requires state-of-the-art graphics processors. The tensor cores of the V100 can also be used for HPC applications.
   3. Further information can be found in the GPU Computing part of this paper or in our web offer.
   
   
3. Cluster Network
   1. Deploy the most cost effective Network solutions with the most advanced interconnect technology
      The network connects everything together and enables communication between all servers, administrators and developers, turning components into a system.
   2. Today's DL/HPC applications are heavily dependent on high bandwidth and low latency connections. In most cases, cluster fabrics are equipped with 40/100 Gbps FDR/EDR InfiniBand or 40/100 Gbps Ethernet.
   3. Further information can be found in the GPU Computing part of this paper or in our web offer.
   
   
4. Cluster Management Software
   1. Cluster Management Software expanding your data center and unleashing the unlimited power of the cloud.
   2. HPC is extremely compute-intensive and requires state-of-the-art graphics processors. HPC is extremely computationally intensive and requires state-of-the-art graphics processors. The tensor cores of the V100 can also be used for HPC applications.
   
   
5. Optimized software stack for GPU server
   1. NVIDIA is using one optimized software stack for the whole Family.
   2. For non-DGX Server with V100 GPUs, sysGen will provide a similar software stack.
   3. Further information can be found in the Software Management tab of this page
   
   
6. Available Deep Learning Solutions and Frameworks
   1. There are already several viable solutions such as DIGITS from NVIDIA.
   2. Many different frameworks for different programming languages (C, C++, Python, Java, Scala, Matlab) are available for own applications like tensor flow, caffe, PyTorch, Theano and Deeplearning4y.
   3. Further information can be found in the internet.
   
   

sysGen GPU-Server,TAKE ADVANTAGE OF 20 YEARS HPC EXPERIENCE IN CLOSE CONNECTION WITH LEADING TECHNOLOGY PROVIDERS - you never walk alone

GPU ACCELERATED SERVERS FOR HPC/DL WORKLOADS FOR HIGHEST PERFORMANCE

sysGen GPU-accelerated servers offer the highest possible performance. Reduce the runtimes of your high-performance computing applications and reduce deep learning training and inference times with the NVIDIA Volta architecture. Get access to over 500 HPC applications and all NVIDIA TensorRT™ Deep Learning Frameworks. Watch the following video to see how Volta GPUs offer powerful new solutions to the biggest challenges of our time.

What to look out for

Deep Learning and HPC are extremely compute-intensive and require state-of-the-art graphics processors.
Since DL models often require an extreme amount of memory, it is very important that the GPU cards have as much local memory as possible. Therefore, current V100 GPU cards have 32 GB HBM2 memory.
In the vast majority of cases, model calculations are distributed over 4 to 16 GPU cards. A frequent retrieval of new data from the mass storage slows down corresponding calculations considerably. In many cases, the individual GPU cards can therefore exchange data directly with each other. If the data exchange takes place via the PCIe bus, the calculation slows down to a lesser extent. It is important that all GPU cards are connected via one single CPU. This is referred to as a single root complex. Up to 10 GPU cards can be connected via single root, but a reasonable number is 8 GPU cards. Direct NVLink connections between the GPU cards offer a strong improvement compared to a PCIe connection. A single NVIDIA Tesla® V100 GPU supports up to six NVLink connections and total bandwidth of 300 GB/sec - 10X the bandwidth of PCIe Gen 3.

Systems with highest performance and power density:

NVIDIA DGX-2 is the fastest server available today with superior technology and the best price/performance ratio. It reduces runtime and thus development time dramatically. The performance per square foot ratio is the best, ever seen. On top you will get regular software updates with optimized docker containers, keeping your systems up to date. Your clusters will be always up to date.

STORAGE FOR HPC/DL WORKLOADS, HIGHEST PERFORMANCE, RELIABILITY, FAULT TOLERANCE and EASY EXPANDABILITY

sysGen supports a bunch of leading File systems, but when it comes to enterprise clusters, we prefere the leading File systems BeeGFS or offer solutions from our partners Pure Storage and DDN Storage. Because supercomputing is write-intensive and sequential in access and AI is read-intensive and randomly oriented in access, these storage systems are designed to support both HPC and AI in the best possible way.

Why use BeeGFS?

BeeGFS transparently spreads user data across multiple servers. By increasing the number of servers and disks in the system, you can simply scale performance and capacity of the file system to the level that you need, seamlessly from small clusters up to enterprise-class systems with thousands of nodes.

Take a look what BeeGFS can do for you:

• Shorten the time to get an insight into your data and information on site and in the cloud

• Cluster storage systems for High Performance Computing (HPC) and Deep Learning (DL)

• Modern cluster storage must handle workloads for HPC and DL equally well

• HPC computing is write-intensive and sequential - AI is more read-intensive and randomized

• sysGen storage systems are designed to support both HPC and AI with maximum performance

• BeeGFS runs on various platforms, such as X86, OpenPower, ARM and more

Why use Pure Storage FlashBlade?

Pure Storage FlashBlade is an extremely high-performance, highly secure scale-out architecture for unstructured data.

At up to 2x faster than previous generation AFAs, and with up to 3PB effective in 6U, //X90 delivers maximum acceleration and consolidation for all your workloads. It’s your enterprise-in-a-box
Take a look what Pure Storage can do for you:

• Elastic performance that grows with data – up to 17 GB/s

• Always-fast, from small, metadata-heavy workloads to large, streaming files

• All-flash performance with no caching or tiering

• Petabytes of capacity

• 10s of billions of objects and files

• “Tuned for Everything” design, no manual optimizations required

• Scale-out everything instantly by simply adding blades

Why use DDN Storage A3I?

DDN A³I storage solutions are fully-optimized to accelerate machine learning and artificial intelligence (AI) applications, streamlining deep learning (DL) workflows for greater productivity.

Up to 7.5x more images with DDN Storage or 5.5x faster training time. Thats all possible because DDN A³I solutions are fully-optimized to deliver massive performance acceleration to applications. The DDN shared parallel architecture fully saturates GPUs and ensures all efforts go towards productive AI use.
Take a look what DDN can do for you:

• Streamlined Workflows

• FLEXIBLE SCALING

• DEEP LEARNING ACCELERATOR

• AUGMENTED DATA DISCOVERY

• FULLY-OPTIMIZED AND INTEGRATED

• FROM THE AT-SCALE EXPERTS

Software Management

System developers often underestimate the installation and maintenance effort for complex software systems. Loss of time until a system is ready for use and performance losses due to poor tuning cause high costs and may also delay the introduction of new products. Often the maintenance effort for adapting to the rapidly evolving software innovation is also widely underestimated. Therefore, we deliver our systems and clusters with pre-installed software on request. This means that our HPC and DL systems or clusters can be used directly.

For all systems with TESLA GPU cards we install the NVIDIA GPU Cloud Software for free. For NVIDIA DGX systems, we offer NVIDIA support contracts for one, two, or three years. For all other systems we offer a sysGen update service for the NVIDIA GPU Cloud Software. However, you can also carry out the updates yourself.

NVIDIA'S OPTIMIZED SOFTWARE STACK FOR THE ENTIRE DGX FAMILY

One software stack for the whole Family:

• Single, unified stack for deep learning frameworks
• Predictable execution across platforms
• The protocol of choice for next-gen storage technologies
• Pervasive reach Support for RDMA over Converged Ethernet (ROCE)

CLUSTER SYSTEMS FOR HPC AND DEEP LEARNING

For HPC and DL cluster systems for parallel multiuser operation, we offer the following solutions:

• sysGen Open Source HPC Cluster
• Bright Enterprise HPC and DL Cluster Manager (see below)

BRIGHT CLUSTER MANAGER EXPANDS YOUR DATA CENTER AND UNLEASHES THE UNLIMITED POWER OF THE CLOUD

High-Performance Computing (HPC) is used for scientific, technical and commercial tasks in the calculation, modeling and simulation of complex systems and the processing of large amounts of data. sysGen has been a successful solution provider for HPC clusters for more than 20 years and had supplied the most powerful HPC clusters with GPU coprocessors in the EMEA region. Now traditional HPC and Articial Intelligence problems are running in the Cluster environment and must be handled on the same HPC systems.

sysGen supports High Performance Cluster for nearly all kind of Problems:

• HPC Clusters for Scientific solutions
• HPC Clusters for Deep Learning solutions
• Clusters for Big Data solutions
• Clusters for Clouds
• Clusters that do all above in one

OUR COMPREHENSIVE CLUSTER PHILOSOPHY

Our software automates the process of building and managing Linux clusters in your data center and in the cloud:

• Deliver computing capacity fast
• Provision 10 to 10,000+ nodes from bare metal in minutes
• Repurpose servers to accommodate fluctuating workloads on the fly
• Extend your on-premises environment to AWS and Azure dynamically
• Automate provisioning, deployment, and management

More Information:

READ more: Deep learning and high-performance computing are converging, and the required infrastructure and cluster software are virtually identical for both applications. Take a look at our solutions pages and get an idea of the extreme performance of Tesla V100 solutions.
https://www.sysgen.de/nvidia-dgx-2-the-fastest-path-to-ai-scale-on-a-whole-new-level.html

READ more: You should pay special attention to the world's most powerful HPC/DL server DGX-2. The DGX-s has 16 V100 cards connected bi-directionally via 12 NVSwitches at 2.4 TB/s and work like a single virtual GPU with 512GB memory. Thus, complex tasks are solved at a fraction of the previous computing time.
https://www.sysgen.de/dgx-2.html

Turn Key Server and Cluster

We deliver all systems, whether single Deep Learning Systems or complete Cluster Systems with pre-installed software. This means that, no matter for which requirement (HPC / DL), each system can be used directly without delaying productivity. Usually we use open-source packages, but depending on Budgt and project size professional management software is also used.

More Information:

READ more: Deep learning and high-performance computing are converging, and the required infrastructure and cluster software are virtually identical for both applications. Take a look at our solutions pages and get an idea of the extreme performance of Tesla V100 solutions.
https://www.sysgen.de/nvidia-dgx-2-the-fastest-path-to-ai-scale-on-a-whole-new-level.html

READ more: You should pay special attention to the world's most powerful HPC/DL server DGX-2. The DGX-s has 16 V100 cards connected bi-directionally via 12 NVSwitches at 2.4 TB/s and work like a single virtual GPU with 512GB memory. Thus, complex tasks are solved at a fraction of the previous computing time.
https://www.sysgen.de/dgx-2.html