You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@tvm.apache.org by GitBox <gi...@apache.org> on 2022/06/21 23:37:13 UTC
[GitHub] [tvm] guberti opened a new issue, #11811: [ci] [Flaky Test] ai.stanford.edu going down causes CI to break
guberti opened a new issue, #11811:
URL: https://github.com/apache/tvm/issues/11811
The tutorial `gallery/how_to/work_with_microtvm/micro_train.py` demonstrates how transfer learning can be used to train special models that fit on microcontrollers (and can be compiled and autotuned with TVM). As part of this, it downloads the Stanford Cars dataset from https://ai.stanford.edu/.
Unfortunately, https://ai.stanford.edu/ went down around 2:30 PST on Jun 21. This caused the link to download the dataset to go offline as well. Since every run of the CI rebuilds the documentation and runs this training, it caused the CI to start failing as well:
(from https://ci.tlcpack.ai/blue/organizations/jenkins/tvm/detail/main/3605/pipeline):
```
[2022-06-21T23:10:04.975Z] Unexpected failing examples:
[2022-06-21T23:10:04.975Z] /workspace/gallery/how_to/work_with_microtvm/micro_train.py failed leaving traceback:
[2022-06-21T23:10:04.975Z] Traceback (most recent call last):
[2022-06-21T23:10:04.975Z] File "/workspace/gallery/how_to/work_with_microtvm/micro_train.py", line 170, in <module>
[2022-06-21T23:10:04.975Z] "http://ai.stanford.edu/~jkrause/car196/cars_train.tgz", f"{FOLDER}/images/target.tgz"
[2022-06-21T23:10:04.975Z] File "/usr/lib/python3.7/urllib/request.py", line 247, in urlretrieve
[2022-06-21T23:10:04.975Z] with contextlib.closing(urlopen(url, data)) as fp:
[2022-06-21T23:10:04.975Z] File "/usr/lib/python3.7/urllib/request.py", line 222, in urlopen
[2022-06-21T23:10:04.975Z] return opener.open(url, data, timeout)
[2022-06-21T23:10:04.975Z] File "/usr/lib/python3.7/urllib/request.py", line 525, in open
[2022-06-21T23:10:04.975Z] response = self._open(req, data)
[2022-06-21T23:10:04.975Z] File "/usr/lib/python3.7/urllib/request.py", line 543, in _open
[2022-06-21T23:10:04.975Z] '_open', req)
[2022-06-21T23:10:04.975Z] File "/usr/lib/python3.7/urllib/request.py", line 503, in _call_chain
[2022-06-21T23:10:04.975Z] result = func(*args)
[2022-06-21T23:10:04.975Z] File "/usr/lib/python3.7/urllib/request.py", line 1352, in http_open
[2022-06-21T23:10:04.975Z] return self.do_open(http.client.HTTPConnection, req)
[2022-06-21T23:10:04.975Z] File "/usr/lib/python3.7/urllib/request.py", line 1326, in do_open
[2022-06-21T23:10:04.975Z] raise URLError(err)
[2022-06-21T23:10:04.975Z] urllib.error.URLError: <urlopen error [Errno 110] Connection timed out>
```
This dataset is 2GB in size, and cannot be easily mirrored on `tlcpack`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] nverke commented on issue #11811: [ci] [Flaky Test] ai.stanford.edu going down causes CI to break
Posted by GitBox <gi...@apache.org>.
nverke commented on issue #11811:
URL: https://github.com/apache/tvm/issues/11811#issuecomment-1163351063
Another instance here https://ci.tlcpack.ai/blue/organizations/jenkins/tvm/detail/main/3602/pipeline
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] driazati commented on issue #11811: [ci] [Flaky Test] ai.stanford.edu going down causes CI to break
Posted by GitBox <gi...@apache.org>.
driazati commented on issue #11811:
URL: https://github.com/apache/tvm/issues/11811#issuecomment-1162497789
We have this problem generally with all network dependencies (even Docker images sometimes flake on `docker pull`). I think the right fix is some combination of consolidating allowed remotes (e.g. only resources from GitHub and S3 can be downloaded) so that a random outage like this doesn't break CI and some kind of HTTP cache in between the tests and the internet
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] guberti commented on issue #11811: [ci] [Flaky Test] ai.stanford.edu going down causes CI to break
Posted by GitBox <gi...@apache.org>.
guberti commented on issue #11811:
URL: https://github.com/apache/tvm/issues/11811#issuecomment-1162493076
#11812 is a hacky fix that uses a mirror link to unblock the CI. It's definitely not a good long-term fix for this issue, so I'm very open to solution ideas.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] areusch closed issue #11811: [ci] [Flaky Test] ai.stanford.edu going down causes CI to break
Posted by GitBox <gi...@apache.org>.
areusch closed issue #11811: [ci] [Flaky Test] ai.stanford.edu going down causes CI to break
URL: https://github.com/apache/tvm/issues/11811
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [tvm] nverke commented on issue #11811: [ci] [Flaky Test] ai.stanford.edu going down causes CI to break
Posted by GitBox <gi...@apache.org>.
nverke commented on issue #11811:
URL: https://github.com/apache/tvm/issues/11811#issuecomment-1163374379
Another instance: https://ci.tlcpack.ai/blue/organizations/jenkins/tvm/detail/main/3603/pipeline
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@tvm.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org