While working on a project, I was using a lot PyTorch, it is an open source machine learning library based on the Torch library, used for applications such as computer vision and natural language processing, primarily developed by Facebook's AI Research lab. I saw that there was an issue in the documentation when comparing it to the result I got. As I learned to use more and more Pytorch, I thought it was a good idea to learn more about it's world and the people behind. It's a highly used application so I know I would learn a lot.
While working on a project, I was using a lot PyTorch, it is an open source machine learning library based on the Torch library, used for applications such as computer vision and natural language processing, primarily developed by Facebook's AI Research lab. I saw that there was an issue in the documentation when comparing it to the result I got. As I learned to use more and more Pytorch, I thought it was a good idea to learn more about it's world and the people behind. It's a highly used application so I know I would learn a lot.
# About the issue <span style='font-size:8pt;'>*(2021/09/19)*</span>
# About the issue <span style='font-size:8pt;'>*(2021/09/19)*</span>
Pour de la classification d’image en utilisant plusieurs gpus (2).
En deep learning par defaut les calculs sont faits avec des float32 (32 bits pour chaque nombre). Pytorch propose “AMP” (Automatic Mixed Precision) qui mixe les calculs entre float16 et float32 (Plus rapide, plus léger).
Pour plusieurs GPU tu utilises “DataParallel” de Pytorch qui permet de paralléliser les calculs sur plusieurs gpu.
AMP et DataParallel ensemble ne fonctionnent pas directement (selon la doc). Dans les faits pourtant si. Tu as donc regardé les issues reliées à ton problème et à trouver qu’en fait le code avait été mis à jour pour résoudre ce problème, mais pas la doc.
Quand j’ai voulu utiliser DataParallel avec AMP j’ai été voir dans la doc le fonctionnement et quand j’ai comparé mes logs avec les informations de la docs, je me suis rendu compte qu’elle n’était pas à jour.De plus, en creusant un peu, j’ai vu un des contributeur qui avait commenté le faite que le code avait bien été modifié mais que la doc était encore celle pour l’ancien code. Il fallait rajouter des lignes de codes pour que ça fonctionne dans la version précédente alors que le code actuel lui ne demande pas. J’ai donc modifié la doc pour qu’elle soit en accord avec ce que le code fait
In deep learning by default, the calculations are made with float32 (32 bits for each number). Pytorch offers “AMP” (Automatic Mixed Precision) which mixes calculations between float16 and float32 which makes calculations faster and lighter.
For several GPUs you use “DataParallel” from Pytorch which allows to parallelize the calculations on several gpu.
According to the documention you couldn't use AMP and DataParallel,together, but when you tried with the code it worked. So I looked at the issues related to the problem and found that in fact the code had been updated to resolve this problem, but not the documentation.
When I wanted to use DataParallel with AMP I went to see how it works in the doc and when I compared my logs with the information in the docs, I realized that it was not up to date. Moreover, while digging a little, i saw one of the contributors who had commented that the code had been modified but that the doc was still the one for the old code.In the previous version, it was necessary to add lines of code for it to work while the current code does not require it. So I modified the doc to be consistent with what the code is doing.
# About the process
# About the process
Begin the process of correcting the documention, I had to fork the project on my account. Once the project copied, I could modify the doc.
Once that was done I created a new branch, commited and push the request. The Pull request was then created citing the issues.
As it is a highly developped application, it is developped by Facebook, there are many check that are runned through and you have to sign a contributor license agreement (CLA)
To conclusion, working in Open Source is something that really opened my eyes on the possibility of working on something big without central entity. That was my first experience in the world of Open Source and it was a really good one.
I'm also suprised to see how fast a project can grow when there are multiple contributors ready to put effort in it. There are over three thousand pull request right now on pytorch and forty thousands already closed. Futhermore there are still more than five thousands issues still unreasolve. To have all that open for everyone to use and see under the hood is a jewel of modern technology.
