[TD] Historical edited files and profiling heuristics #4590
Conversation
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎
|
facebook-github-bot
added
the
CLA Signed
clee2000
changed the title
torchci/scripts/utils.py
Outdated
| from typing import Any, Dict, List, Optional, Union | ||
| import requests | ||
|
|
||
| import rockset # type: ignore[import] |
There was a problem hiding this comment.
Since this is a generic utils file inside the torchci/scripts folder, I'd suggest being picky about what exactly goes inside here so that this file doesn't get too bloated over time.
General utils could be kept here, but the rockset and heuristic utils are specialized enough to warrant their own files.
I do like the idea of a the naming convention though. What if those files were all prefixed with "utils_", so it became utils_rockset.py and utils_heuristics.py?
There was a problem hiding this comment.
i moved the functions out to their own files, but they seem a bit short haha
I also named it with utils in front for the files that are new
| @@ -0,0 +1,61 @@ | |||
| from collections import defaultdict | |||
There was a problem hiding this comment.
What do you think about putting these td_ scripts inside a td subfolder instead?
There was a problem hiding this comment.
possibly a future thing, but also weird python module import problems make this annoying
| ) | ||
|
|
||
|
|
||
| def cache_json(func): |
Add a heuristic based on profiling data and a heuristic based on historical edited files.
Addes caching for rockset queries and other functions.