Tuesday, November 15, 2016

Parallelizing HTMLPage Downloaders

Threading a Python HTML down loader does not result in a significant performance improvement.

Use Python asyncio or Twister or Tulip instead.

https://pawelmhm.github.io/asyncio/python/aiohttp/2016/04/22/asyncio-aiohttp.html

http://krondo.com/an-introduction-to-asynchronous-programming-and-twisted/