I'm reopening #346 because it looks like it make sense, I'm also using tox in my CI and would love to have an official Python docker image with multiple versions packed in it.

At least 3.6 and 3.7, but I can imagine some people needing more.

(In fact I used the #253 issue as a feature þ and it looks like fixed in the 3.7 image)