Posts Taged cupy

Preferred Networks releases version 6 of both the open source deep learning framework Chainer and the general-purpose matrix calculation library CuPy

May 16, 2019, Tokyo Japan – Preferred Networks, Inc. (PFN, Head Office: Tokyo, President & CEO: Toru Nishikawa) has released Chainer(TM) v6 and CuPy(TM) v6, major updates of PFN’s open source deep learning framework and general-purpose matrix calculation library, respectively. The latest version will run as-is on most of the code used in previous versions.
Chainer was released as open source software in 2015 and is known as a pioneer of flexible and intuitive deep learning frameworks based on the Define-by-Run method. Chainer has since been supported by many users and is being actively developed.

ChainerX, a C++ implementation of automatic differentiation that has experimentally been integrated into the main Chainer distribution since the release of the v6 beta version, now supports more examples. The use of ChainerX can significantly reduce overhead on the framework side in both forward and backward propagations without losing much of Chainer’s flexibility and backward compatibility, resulting in increased performance. In addition, Chainer and ChainerX source code does not need to be changed to use new hardware on ChainerX if a third-party developer implements the support for the hardware as a plug-in.

 

Main features of Chainer v6 and CuPy v6 are:

  • Integration of ChainerX
    • Fast and more portable multi-dimensional arrays and the automatic differentiation backend have been added.
    • A compatibility layer has been implemented to allow for the use of ChainerX arrays in the same manner as NumPy and CuPy arrays, allowing automatic differentiation with low overhead in C++.
    • An integrated device API has been introduced. The unified interface can handle the specification of devices or inter-device transfer for a wide variety of backends such as NumPy, CuPy, iDeep, and ChainerX.
  • Enhanced support for training in mixed precision
    • Mixed 16, a new default data type, has been added. It is a mixed precision mode that realizes transparent training using operations in single and half precisions.
    • Dynamic scaling that detects and automatically adjusts overflow has been implemented in order to avoid underflow in mixed precision training.
  • Addition of a function and link test tool
    • A test tool that generates unit tests for forward and backward propagations as well as second order differentials with minimal code has been added.
  • CuPy arrays to support NumPy functions
    • NumPy’s experimental feature __array_function__ is supported now. CuPy arrays have been directly applied to many __array_function__ enabled Numpy functions.

 

 

PFN will continue improving Chainer performance and expanding the backend. It will contribute to improved performance in a wide range of use cases by making ChainerX easier to use as well as supporting more arithmetic operations.

Chainer has incorporated a number of development results from external contributors. PFN will continue to quickly adopt the results of the latest deep learning research and promote the development and popularization of Chainer in collaboration with supporting companies and the OSS community.

 

  • About the Chainer(TM) Open Source Deep Learning Framework

Chainer is a Python-based deep learning framework developed and provided by PFN, which has unique features and powerful performance that allow for designing complex neural networks easily and intuitively, thanks to its “Define-by-Run” approach. Since it was open-sourced in June 2015, as one of the most popular frameworks, Chainer has attracted not only the academic community but also many industrial users who need a flexible framework to harness the power of deep learning in their research and real-world applications.

Chainer quickly incorporates the results of the latest deep learning research. With additional packages such as ChainerRL (reinforcement learning), ChainerCV (computer vision), and Chainer Chemistry(a deep learning library for chemistry and biology)and through the support of Chainer development partner companies, PFN aims to promote the most advanced research and development activities of researchers and practitioners in each field.(http://chainer.org/

Preferred Networks releases version 5 of both the open source deep learning framework, Chainer and the general-purpose array calculation library, CuPy.

Preferred Networks, Inc. (PFN, President and CEO: Toru Nishikawa) has released Chainer(TM) v5 and CuPy(TM) v5, major updates of PFN’s open source deep learning framework and general-purpose array calculation library, respectively.

In this major upgrade after six months, Chainer has become easier to use after integrating with ChainerMN, which has been provided as a distributed deep learning package to Chainer. The latest v5 will run as-is on most of the code used in previous versions.

 

Main features of Chainer v5 and CuPy v5 are:

  • Integrated with the ChainerMN distributed deep learning package

・With ChainerMN incorporated in Chainer, fast distributed deep learning on multiple GPUs can be conducted more easily.

  • Support for data augmentation library NVIDIA(R)

・Chainer v5 performs faster data preprocessing by decoding and resizing of JPEG images on GPUs.

  • Support for FP16

・Changing to half-precision floating-point (FP16) format is possible with minimal code changes.

・Reduced memory consumption, which allows larger batch sizes.

・Further speed increases with the use of NVIDIA(R) Volta GPU Tensor Cores.

  • Latest Intel(R) Architecture compatibility

・Chainer v5 supports the latest version 2 of Chainer Backend for Intel(R) Architecture (previously, iDeep, which was added to Chainer v4) for faster training and inference on Intel(R) Processors.

  • High-speed computing and memory saving for static graphs

・Chainer v5 optimizes computation and memory usage by caching static graphs that do not change throughout training. This speeds up training by 20-60%.

  • Enhanced cooperation with Anaconda Numba and PyTorch, enabling the mutual exchange of parallel data

・Added ability to pass a CuPy array directly to a JIT-compiled function by Anaconda Numba.

・DLpack:Array data can be exchanged with PyTorch and other frameworks.

  • CuPy basic operations are 50% faster

・Performance of basic operations such as memory allocation and array initialization has improved.

 

Chainer and CuPy have incorporated a number of development results from external contributors. PFN will continue to quickly adopt the results of the latest deep learning research and promote the development and popularization of Chainer and CuPy in collaboration with supporting companies and the OSS community.

 

◆ About the Chainer(TM) Open Source Deep Learning Framework

Chainer is a Python-based deep learning framework developed and provided by PFN, which has unique features and powerful performance that allow for designing complex neural networks easily and intuitively, thanks to its “Define-by-Run” approach. Since it was open-sourced in June 2015, as one of the most popular frameworks, Chainer has attracted not only the academic community but also many industrial users who need a flexible framework to harness the power of deep learning in their research and real-world applications.

Chainer quickly incorporates the results of the latest deep learning research. With additional packages such as ChainerRL (reinforcement learning), ChainerCV (computer vision), and Chainer Chemistry(a deep learning library for chemistry and biology)and through the support of Chainer development partner companies, PFN aims to promote the most advanced research and development activities of researchers and practitioners in each field. (http://chainer.org/

Preferred Networks released open source deep learning framework Chainer v4 and general-purpose array calculation library CuPy v4.

Tokyo, Japan, April 17, 2018 — Preferred Networks, Inc. (PFN, Headquarters: Chiyoda-ku, Tokyo, President and CEO: Toru Nishikawa) has released v4 of Chainer™ and CuPy™, major updates of the open source deep learning framework and the general-purpose array calculation library, respectively.

This major upgrade to Chainer and CuPy incorporates the results of the latest deep learning research over the last six months. The newly released v4 is largely compatible with previous versions of Chainer.

 

Main features of Chainer and CuPy v4 include:

  • Additional functions for fast, memory-efficient training on NVIDIA(R) GPUs *1

Chainer now supports NVIDIA TensorCore to speed up convolutional operations. Loss scaling has also been implemented to alleviate the vanishing gradient problem when using half-precision floats.

  • Quick installation of CuPy

We have begun providing a binary package of CuPy to reduce the installation time from 10 minutes down to about 10 seconds.

  • Optimized for Intel(R) Architecture

An Intel Deep Learning Package (iDeep) *2 backend has been added to make training and inference on Intel CPUs faster. This delivers an 8.9-fold improvement of GoogLeNet (a neural network used for image recognition) inference speed on CPUs, according to our benchmark results*3.

  • More functions supporting second order differentiation

Enhanced support for second order differentiation, which was first introduced in v3, allows easier implementation of the latest networks and algorithms.

  • A new function to export results of training with Chainer in the Caffe format

A function to export Chainer’s computational procedure and learned weights in the Caffe format has been added as experimental. This makes it easier to use the results of training with Chainer even in an environment where Python cannot be executed. (Exporting into the ONNX format is also available via the onnx-chainer package.)

 

◆Chainer ReleaseNote: https://github.com/chainer/chainer/releases/tag/v4.0.0

◆Update Guide:https://docs.chainer.org/en/latest/upgrade.html

 

Chainer and CuPy have taken in a number of development results from external contributors. PFN will continue working with supporting companies and the OSS community to promote the development and popularization of Chainer and CuPy.

 

* 1:http://docs.nvidia.com/deeplearning/sdk/mixed-precision-training/index.html

* 2:NumPy-compatible library for performing general arithmetic operations in deep learning at a high speed on Intel CPUs https://github.com/intel/ideep

* 3:The results of comparison in time to process an image between when iDeep was enabled and disabled. Intel Math Kernel Library was enabled in both cases. Intel Xeon(R) CPU E5-2623 v3 was used.

 

About the Chainer Open Source Deep Learning Framework

Chainer is a Python-based deep learning framework developed mainly by PFN, which has unique features and powerful performance that allow for designing complex neural networks easily and intuitively, thanks to its “Define-by-Run” approach. Since it was open-sourced in June 2015, as one of the most popular frameworks, Chainer has attracted not only the academic community but also many industrial users who need a flexible framework to harness the power of deep learning in their research and real-world applications.

Chainer incorporates the results of the latest deep learning research. With additional packages such as ChainerMN (distributed learning), ChainerRL (reinforcement learning), ChainerCV (computer vision) and through the support of Chainer development partner companies, PFN aims to promote the most advanced research and development activities of researchers and practitioners in each field. (http://chainer.org/

Preferred Networks released open source deep learning framework Chainer v3 and NVIDIA GPU array calculation library CuPy v2

Preferred Networks, Inc. (PFN, Headquarters: Chiyoda-ku, Tokyo, President and CEO: Toru Nishikawa) has released Chainer v3, a major update of the open source deep learning framework Chainer(R), as well as NVIDIA(R) GPU array calculation library CuPy™ v2.

We release a major upgrade of Chainer every three months that quickly incorporates the results of the latest deep learning research. The newly released Chainer v3 will run without the need to change most of your code.

 

Main features of Chainer v3 and CuPy v2 include:

1.  Automatic differentiation of second and higher order derivatives

Chainer now supports automatic differentiation of second order and higher derivatives in many functions. This will enable users to easily implement deep learning methods that require second order differentiation as per equations written in papers.

 

2. Improved CuPy memory allocation

In many neural nets, memory efficiency when using GPUs will improve significantly, and reallocation of memory will be reduced in some cases, increasing speed.

 

3. Sparse matrix support has been added to CuPy

Large-scale graph analysis and natural language processing, which have previously been highly costly to implement on GPUs, can now be implemented more easily thanks to sparse matrix calculation being available on the GPU.

◆ Chainer ReleaseNote: https://github.com/chainer/chainer/releases/tag/v3.0.0

Chainer v3 has taken in a number of development results from external contributors as its previous versions did. PFN will continue working with supporting companies and the OSS community to promote the development and popularization of Chainer.

 

◆ About the Chainer Open Source Deep Learning Framework

Chainer is a Python-based deep learning framework developed by PFN, which has unique features and powerful performance that enables users to easily and intuitively design complex neural networks, thanks to its “Define-by-Run” approach. Since it was open-sourced in June 2015, as one of the most popular frameworks, Chainer has attracted not only the academic community but also many industrial users who need a flexible framework to harness the power of deep learning in their research and real-world applications.

Chainer incorporates the results of the latest deep learning research. With additional packages such as ChainerMN (distributed learning), ChainerRL (reinforcement learning), ChainerCV (computer vision) and through the support of Chainer development partner companies, PFN aims to promote the most advanced research and development activities of researchers and practitioners in each field. (http://chainer.org/)

*Chainer(R) and CuPyTM are the trademark or the registered trademark of Preferred Networks, Inc. in Japan and other countries.