Document speed comparison

[Borda]

What does this PR do?

Extend documentation on how the Pl is running almost the same fast as vanilla PT
Follow-up from #1587

PR review

Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.

Did you have fun?

Make sure you had fun coding 🙃

[edenlightning]

• make sure to add a todo for memory parity to catch leaks

[Borda]

• make sure to add a todo for memory parity to catch leaks

memory parity will be separate PR as it has to be developed first... 🐰

[codecov]

Codecov Report

Merging #2072 (4a2b193) into master (1b599ff) will not change coverage.
The diff coverage is n/a.

@@          Coverage Diff           @@
##           master   #2072   +/-   ##
======================================
  Coverage      93%     93%           
======================================
  Files         134     134           
  Lines        9928    9928           
======================================
  Hits         9242    9242           
  Misses        686     686           

[Borda]

config with k80 for MNIST dataset training on 15 epochs

# run 1
pt_times: [78.08037253795192, 77.65994176699314, 77.16897976701148, 75.97896099602804, 76.10812993405852, 77.09142099390738, 76.38175202498678, 76.61454085202422, 76.64365514100064, 76.46872986794915, 76.58440757205244, 75.86799949593842, 75.37725775991566, 77.95835449907463, 78.78175812901463, 78.05797199800145, 77.72009757102933, 77.71631882700603, 78.06795359996613]
pl_times: [82.00195560394786, 82.00709507102147, 81.805350583978, 81.26589621405583, 80.73785846401006, 80.55167946091387, 81.38005885004532, 81.20296528504696, 80.27230604400393, 80.75083769799676, 80.93695296195801, 80.93231990898494, 80.48659746092744, 81.36925745301414, 80.96931200707331, 80.6680577499792, 80.6818772739498, 81.12940620095469, 82.45112453098409]
# run 2
pt_times: [72.35451863205526, 71.80189288198017, 71.9814427359961, 71.92960607400164, 71.40658942295704, 72.75710438506212, 73.50131551991217, 73.19857055298053, 73.15937736898195, 72.86832409596536, 72.34491985896602, 72.94093307398725, 74.36359175201505, 74.67142662103288, 74.2357223139843, 74.14241052698344, 74.0228865040699, 74.53661659103818, 74.28851658396889] 
pl_times: [75.84128665702883, 76.46811646199785, 75.47480550594628, 75.47422935103532, 74.94576230400708, 75.44481814303435, 75.91588587092701, 75.67014791606925, 75.5910986890085, 75.1219390759943, 76.10618040000554, 75.65077104105148, 74.04137368802913, 74.90914194402285, 77.16411820601206, 77.57090287993196, 76.66609343804885, 76.90365857002325, 76.51128543494269]
# run 3
pt_times: [82.37473194301128, 82.97512700094376, 83.08479252899997, 83.83531925699208, 83.09167189500295, 82.96716181200463, 83.35179279604927, 83.84988487605006, 83.43091613892466, 82.8106702209916, 83.24989012500737, 83.35911275492981, 83.28093389503192, 82.61112239805516, 83.16935679502785, 82.62005160597619, 81.47776250902098, 82.40548102708999, 83.22479931509588]
pl_times: [85.93682769290172, 85.97661709308159, 87.47603658295702, 87.49147263797931, 87.64337389892898, 87.64634780201595, 87.6974927430274, 87.16452221293002, 86.25435132498387, 86.32870114094112, 85.65250021102838, 86.81863707804587, 85.7048433390446, 87.24022658006288, 86.82864296494517, 86.9083100440912, 86.09307709196582, 85.65272393194027, 85.86923094291706]

[stale]

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs. If you need further help see our docs: https://pytorch-lightning.readthedocs.io/en/latest/CONTRIBUTING.html#pull-request or ask the assistance of a core contributor here or on Slack. Thank you for your contributions.

[Borda]

or see the docs live https://81855-178626720-gh.circle-artifacts.com/0/html/trainer.html#benchmark-with-vanilla-pytorch

[awaelchli]

I have the feeling this does not fit into the "best practices" section. But there are not many other options. Maybe @edenlightning has a good idea? :)

[awaelchli]

maybe @SeanNaren wants to include his ddp benchmarking results here too? :) I've seen some great charts somewhere

[Borda]

I have the feeling this does not fit into the "best practices" section. But there are not many other options. Maybe @edenlightning has a good idea? :)

I think we can move it anytime, I just need this as base for memory benchmark :]

maybe @SeanNaren wants to include his ddp benchmarking results here too? :) I've seen some great charts somewhere

that would be great, so another PR?