PyTorch 1.8 Improves FFT Support
Written by Kay Ewbank   
Monday, 15 March 2021

PyTorch has been updated with improved support for FFTs, better distributed model training, new APIs, library updates, and support for ways to improve and scale your code for performance at both inference and training time.

PyTorch is an optimized tensor library for deep learning using GPUs and CPUs. It aims to offer a replacement for NumPy that makes use of the power of GPUs, while providing a deep learning research platform that provides maximum flexibility and speed.

pytorch

Possibly the most important improvement to the new version is the inclusion of the torch.fft module. This module implements the same functions as NumPy’s np.fft module, but with support for hardware acceleration and autograd. Alongside this comes improved support for fast Fourier transforms (FFT) in the Torch.fft library to add support for 2D FFT functions, alongside the ability to use new FFT operators in stft, added helper functions and a fuzzing benchmark.

Other API improvements include new Linear Algebra functions, added support for autograd for complex tensors, and updates to improve performance for calculating hessians and jacobians.

The developers say there have been "significant updates and improvements to distributed training". These updates improve NCCL reliability, add support for Pipeline parallelism and RPC profiling; and also add support for communication hooks adding gradient compression.

Support has been added for doing python to python functional transformations via torch.fx, adding the ability to set up transformations where you can feed in a Module instance and get a transformed Module instance out of it.

A wide range of functions have been updated, mainly to improve NumPy compatibility. The torch.linalg module, modeled after NumPy’s np.linalg module, brings NumPy-style support for common linear algebra operations including Cholesky decompositions, determinants, eigenvalues and many others. 

Alongside the new version, the team is also releasing major updates to PyTorch libraries including TorchCSPRNG, TorchVision, TorchText and TorchAudio.

pytorch

More Information

PyTorch Website

PyTorch On GitHub

Related Articles

PyTorch Adds New APIs

PyTorch Scholarship Challenge

PyTorch Developer Day Updates

PyTorch Adds TorchScript API

PyTorch 1.5 Updates C++ API 

 

To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on Twitter, Facebook or Linkedin.

Banner


Master Large Language Model Ops
20/03/2024

New technology brings with it more career opportunities. You may never have imagined becoming an LLMOps consultant,  but there's now a Coursera Specialization which provides preparation for this  [ ... ]



Angular and Wiz To Merge
27/03/2024

Two web development frameworks used at Google are merging. One, Angular is open source and widely known, while the other, Wiz, is an internal web framework developed and used by Google for some o [ ... ]


More News

raspberry pi books

 

Comments




or email your comment to: comments@i-programmer.info

Last Updated ( Thursday, 02 December 2021 )