Posts on Nov 2017

Preferred Networks’ private supercomputer ranked first in the Japanese industrial supercomputers TOP 500 list

MN-1, a private supercomputer used exclusively by Preferred Networks, Inc. (PFN, Headquarters: Chiyoda-ku, Tokyo, President and CEO: Toru Nishikawa) has recorded a LINPACK  performance of about 1.39 PetaFLOPS 2.  As a result, MN-1 is ranked 12th in the world and 1st in Japan among industrial supercomputers in the TOP500 List (http://www.top500.org), which shows the most powerful supercomputers as of November 2017. When including supercomputers for research purposes, it is listed as 91st in the world and 13th in Japan.   

 

 

About PFN’s private supercomputer MN-1   

MN-1 is equipped with a Graphics Processing Unit (GPU) platform from NTT Communications Corporation (NTT Com, Headquarters: Chiyoda-ku, Tokyo, President and CEO: Tetsuya Shoji) and NTT PC Communications Incorporated (NTTPC, Headquarters: Minato-ku, Tokyo, President and CEO: Motoo Tanaka), and contains 1,024 of NVIDIA (R) ’s Tesla (R) multi-node P100 GPUs. It utilizes Mellanox’s Infiniband interconnect to make high-speed distributed deep learning possible using ChainerMN 4, a distributed deep learning package developed by PFN.    

Using MN-1, PFN will further accelerate its research and development activities in various fields that require a huge amount of computing resources such as transportation systems, manufacturing and bio/healthcare.

 

 

 

1.  A benchmark to compare practical operation speed of computers

2.  A unit measuring computer performance. Peta is 1,000 trillion (10 to the power of 15) and FLOPS is used to count floating-point operations per second. So, 1 PetaFLOPS means that a computer is capable of performing 1,000 trillion floating-point calculations per second.

3.  Preferred Networks launches one of Japan’s most powerful private sector supercomputers
https://www.preferred-networks.jp/en/news/pr20170920

4.  Preferred Networks achieved the world’s fastest training time in deep learning
https://www.preferred-networks.jp/en/news/pr20171110

 

About Preferred Networks, Inc.

Founded in March 2014 with the aim of promoting business utilization of deep learning technology focused on IoT, PFN advocates Edge Heavy Computing as a way to handle the enormous amounts of data generated by devices in a distributed and collaborative manner at the edge of the network, driving innovation in three priority business areas: transportation, manufacturing and bio/healthcare. PFN promotes advanced initiatives by collaborating with world leading organizations, such as Toyota Motor Corporation, FANUC CORPORATION, and the National Cancer Center. (https://www.preferred-networks.jp/en/)

 

*Chainer (R)  is the trademark or the registered trademark of Preferred Networks, Inc. in Japan and other countries. Other company names and product names written in this release are the trademarks or the registered trademarks of each company.

Preferred Networks achieved the world’s fastest training time in deep learning, completed training on ImageNet in 15 minutes,using the distributed learning package ChainerMN and a large-scale parallel computer

November 10, 2017, Tokyo – Preferred Networks, Inc. (PFN, Headquarters: Chiyoda-ku, Tokyo, President and CEO: Toru Nishikawa) has achieved the world’s fastest training time in deep learning by using its large-scale parallel computer MN-1 1.

With the size of training data and the number of parameters expanding for the sake of better accuracy of deep learning models, computation time is also increasing. It is not unusual to take several weeks to train a model. Getting multiple GPUs to link with one another for faster training is very important to reduce the time spent on trial and error and verification of new ideas, and produce results quickly.

On the other hand, it is generally known in parallel/distributed learning that the accuracy and learning rate of a model decrease gradually with increased GPUs, due to larger batch sizes and GPU communication overhead.

This time, we have improved learning algorithms and parallel performance to address these issues, and used one of Japan’s most powerful parallel computers with 1,024 of NVIDIA(R)’s Tesla(R) multi-node P100 GPUs and leverages Chainer’s distributed learning package ChainerMN 2 for training.

As a result, we completed training ResNet-50 3 for image classification on the ImageNet 4 dataset in 15 minutes, which is a significant improvement from the previously best known result 5.

The research paper on this achievement is available in the following URL under the title “Extremely Large Minibatch SGD: Training ResNet-50 on ImageNet in 15 Minutes”. (https://www.preferred-networks.jp/docs/imagenet_in_15min.pdf)

Based on this research result, PFN will further accelerate its research and development activities in the fields of transportation systems, manufacturing, and bio/healthcare, which require large-scale deep learning.

 

1 One of the most powerful private supercomputer in Japan, contains 1,024 of NVIDIA(R)’s Tesla(R) multi-node P100 GPUs.https://www.preferred-networks.jp/en/news/pr20170920

2 A package adding distributed learning functionality with multiple GPUs to the open source deep learning framework Chainer

3 A network frequently used in the field of image recognition

4 A dataset widely used for image classification

5 Training completed in 31 minutes using Intel(R) Xeon(R) Platinum 8160 x 1,600(Y. You et al. ImageNet Training in Minutes. CoRR,abs/1709.05011, 2017)

 

■ About the Chainer Open Source Deep Learning Framework

Chainer is a Python-based deep learning framework being developed mainly by PFN, which has unique features and powerful performance that allow for designing complex neural networks easily and intuitively, thanks to its “Define-by-Run” approach. Since it was open-sourced in June 2015, as one of the most popular frameworks, Chainer has attracted not only the academic community but also many industrial users who need a flexible framework to harness the power of deep learning in their research and real-world applications.

Chainer incorporates the results of the latest deep learning research. With additional packages such as ChainerMN (distributed learning), ChainerRL (reinforcement learning), ChainerCV (computer vision) and through the support of Chainer development partner companies, PFN aims to promote the most advanced research and development activities of researchers and practitioners in each field. (http://chainer.org/

■ About Preferred Networks, Inc.

Founded in March 2014 with the aim of promoting business utilization of deep learning technology focused on IoT, PFN advocates Edge Heavy Computing as a way to handle the enormous amounts of data generated by devices in a distributed and collaborative manner at the edge of the network, driving innovation in three priority business areas: transportation, manufacturing and bio/healthcare. PFN promotes advanced initiatives by collaborating with world leading organizations, such as Toyota Motor Corporation, Fanuc Corporation, and the National Cancer Center. (https://www.preferred-networks.jp/en/)

*Chainer(R) is the trademark or the registered trademark of Preferred Networks, Inc. in Japan and other countries.