Show only one browser per engine in the default view #1519
Comments
|
@thejohnjansen @mustjab can you weigh in? My calculus has been roughly that if Edgium isn't on the front page, then Microsoft engineers might not know the runs exist and be less interested in looking at wpt.fyi. And that for similar reasons Microsoft would be less keen to keep the runs reliable and sponsor the extra Azure Pipelines VMs if the results aren't really visible. That being said, I do agree that 3 columns is the more useful view, and I often go in and remove Edge from some view when it's cross-engine interop issues I'm after. If we start getting reliable results for WebKitGTK or some mobile browsers we'll be forced to grapple with a very similar issue: how to make runs discoverable without them being the default view. One thing we could experiment with is the http://wpt.fyi/ front page to give people an overview that isn't just a redirect to /results/, and there you'd have room to feature the configurations available and maybe also plug some of the good stuff in /interop/ and /insights/. |
|
I'd like to explore the idea of a redesign of the landing page, rather than focus on three vs. four vs. 8 column layout. I'm in no way convinced that 3 is better than 4, for example. In fact, for my workflow (and that of Microsoft in general) the most interesting view is Chrome and Edge with the Diff showing. That is the view that we need more than anything. Granted, that's an implementer's view, not a web-developer's view, but I think the design should be focused on user scenarios. For web-developers, I think the most interesting views are those that show differences, not views that show everything. The places where all engines pass all tests (or fail all tests) are not very interesting at all. What do you think for a timeline for designing a new landing page? |
|
I'm open to the idea of redesigning the landing page!
Could you elaborate a bit more, Philip? I can't quite picture the "overview" off the top of my head. Also, John, do you have any thoughts for a redesigned landing page that would bypass the debate of default browser selection? |
|
Note that even redesigning the landing page doesn't address all of the issue. If I get a link like
This supports the fact that the default view already isn't useful if your primary use case is about differences between browsers running the same engine, so it doesn't make sense to include multiple copies of the same engine on the default page. |
I disagree. I think the default view should be ONLY a diff of the engines that we run with a single-click solution for removing engines we don't care about at that time. But as I say: I think this needs design work, not something that should be designed in the comments section. |
|
@jgraham you're right of course that there has to be some default set of browsers, if nothing else then to keep existing links working. Because we already have some configurations we don't show by default and will gain more (WebKitGTK, probably some Android browsers) soon we'll need to make those easily discoverable. @Hexcles I was thinking a page with no results table but rather a few different sections/cards/widgets to guide people towards the most useful and common views we have. Maybe one widget for stable desktop, one for stable mobile, and so on, and maybe there being able to select or unselect browsers before going to the view, like a trimmed down query builder. (You could imagine highlighting new features and such too.) If it is very simple to get to your favorite view from the landing page, very simple to select/unselect browsers, and we encode the chosen browsers in the URL even for our defaults, then I think most of the tension between what people are interested in goes away. |
I was typing up almost exactly this. Pretty much starting with the "Interoperability" tab so all the text on top, but in "Edit" mode with n browsers as choices to add to a view that then gets created when you click "Submit" |
|
Maybe we can design it in the comments. :-) |
|
To be clear the browsers we currently run are:
And by the end of year we hope to also be running at least:
A table with 14+ columns as the default view doesn't seem useful to anyone. So some decision has to be made about what to show by default and in each view in some future iteration of the UI. I fully agree that some UI to switch between common use cases is a good idea; a |
|
The only immediate thing we could do to resolve this bug is to drop Edge from the default product set and require using the "EDIT" button to add the column back. Some folks working on Edge say the results are useful, and deleting is quicker than adding, so I think that'd be a shame. While some view will always be the fewest clicks away and we do have to decide what that is, if we start encoding the selected products in the URLs, then I think there will be a lot tension around what's preselected in a widget on the wpt.fyi landing page. FWIW, the view I think should be the fewest clicks to reach is experimental Chrome+Firefox+Safari. This would be achieved by "experimental desktop" being the most highlighted view on the landing page, with Chrome, Edge, Firefox, Safari and WebKitGTK being toggle buttons, and Edge and WebKitGTK being deselected by default. Making the chosen state sticky for the front page seems like a good idea. @thejohnjansen do you think that would be acceptable, to require two clicks to get to a view that includes Edge Dev? @jgraham if we can get this done in Q4, would that be acceptable? |
Yes, I think that is acceptable as an interim solution, but I don't think that is the appropriate long-term default view we should aspire to. |
|
One thing that is worth mentioning is that for the purposes of attribution for metadata changes, we're creating a GitHub OAuth login flow (which bubbles down to a "User" concept). "Favorite queries" is a pretty simple concept to add once users are a thing, and would mean that we don't need to agree on a one-size-fits-all to the same degree. |
|
@Hexcles and I discussed this a bit today, and while designing in the comments worked above expectation, it's probably best to use the RFC process for this. I think it'd be easier to communicate some of the options with mockups, after tinkering a bit to see what feels OK. |
|
@stephenmcgruer can I assign this to you? |
|
Under the assumption that this maps to the (re?)design of the wpt.fyi landing page that you and I talked about, then yes. I can probably get a mock into an RFC this year, but otherwise won't have much time to spend on it. |
|
Note that this is currently leading to a much worse UX as web-platform-tests/wpt#27877 means we've no runs of Edge in weeks, so we're currently showing an old aligned run on WPT. |
|
I have been looking at this recently, due to the discussion at TPAC 2020 (https://www.w3.org/2020/10/21-testing-minutes.html#t02). To summarize, that discussion ended in us agreeing to remove Edge from the default view, but include a prominent link to restore it. However, it was noted that no Edge developers came to WPT's TPAC meeting, so we held off from making a complete decision. My next step is to present a prototype for this, and gather feedback. To be clear, the Ecosystem Infra team (who are nearly single-handedly responsible for contributing to and maintaining wpt.fyi) is struggling for bandwidth currently, so we also want to take this step to reduce our overhead. (E.g. so that we don't have to worry about issues like web-platform-tests/wpt#27877 impacting the wpt.fyi front-page). |
|
I deeply appreciate you deferring the decision given no one from MS was in the room. I'm not sure what happened with that; I had planned on attending but I think I lost track of several breakout meetings this year. While I completely understand the desire to simplify the default view, I still think that focusing on differences between the engines is the most important use case, and providing an interim solution with 3 columns feels like throw-away work. |
|
I appreciate your understanding here :). Let me try to reply to some specifics, and I will try to be truthful but not rude! (With apologies if I miss the mark.)
Both Boaz and yourself are correct; there's no fundamental reason to show Chrome instead of Edge. It's worth noting we're already making filtering decisions for the default view in wpt.fyi, both of browsers that share engines with existing defaults (WebKitGTK) and of browsers that don't (Servo). For any of those browsers one could ask 'why not show this one instead'. In a perfect world, we would resource the implementation of user-profile stored default browser sets. Then we could bikeshed the 'logged out default' as much as we wanted :D. Unfortunately we're not in that world - I don't have the resources to implement that. Instead, I'm doing what I can tactically to address both the concerns that other browser vendors have raised around 'multiple Chromiums on the homepage' and problems like web-platform-tests/wpt#27877 (which I also got complaints from Chrome devs about). If Edge (or anyone else!) are interested in resourcing work on wpt.fyi to bring real user profiles (we've got the groundwork done) and developing UI and backend system to track default browser selection for the user, that would be amazing! We'd be happy to provide guidance and knowledge sharing where we can. I would probably off-the-cuff estimate it as a 1-2 quarter project including ramp-up (it's all unfortunately a bit messy; the code health of wpt.fyi isn't fantastic to be blunt).
I'm not personally convinced of this. I think there's a reasonable ideological (not in a bad way!) argument to be made for this, but practically speaking wpt.fyi is primarily used by browser engineers who tend to care how their specific browser compares to other specific browsers (for the sake of interop). I'd be happy to review or discuss design changes along these lines, but I would be very focused on making sure we don't make this tool harder to use for browser engineers :).
I'm very hopeful to make it one click :). Here's the mockup, with the note that my goal is for the state of including Edge to be sticky on a given browser:
Hopefully can get this prototype actually working sometime soon! >_< |
I think you hit the mark well. I'm not sure what you may have written and then deleted, but the reply you left in tact doesn't sound rude to me at all. Thank you for that. I do wish my team had resources for this in the coming quarter, but we don't either and I completely understand working under constrained resources. I'm trying to get more allocation for this. |
After a lot of discussion, we have decided to remove Edge as a default product from wpt.fyi due to overlap in browser engine between Chrome and Edge. In a perfect world we would invest in per-user default products, but there is no resourcing for that currently. To try to minimize the disruption, we add a breadcrumb link back to easily let users get Edge back on the page. See #1519
|
Hi all; apologies for the delay here. I tried very hard to get a solution that cached the user's choice via cookies or some other mechanism, but I ultimately was unsuccessful due to the complexity of how wpt.fyi implements 'default product set' (it's spread nastily over the backend and frontend). Instead, I have a prototype PR for a one-click link to add MS Edge back to the default page. See https://smcgruer-remove-edge-d-dot-wptdashboard-staging.uk.r.appspot.com/results/?label=experimental&label=master&aligned (#2431). Feel free to play around with it and give me feedback. |
The default view currently shows two blink based browsers, one gecko browser and one webkit browser. This doesn't seem particularly helpful to anyone. For people trying to compare engines, having two copies of any particular engine is adding noise in the form of extra columns to skip over, rather than value. For people working on specific browsers embedding a given engine the most informative view will often be differences with other implementations using the same engine; that requires a non-default query anyway since the default view will show far too much in the way of similarities to be useful for spotting differences.
I think the same argument applies to a single engine running on multiple platforms i.e. once we have mobile results I think putting those on the default view will be largely unhelpful even though the difference between desktop and mobile can be larger than the difference between different browsers with the same engine on the same platform.
If you feel there needs to be a rule to follow on what's in the default view, I'd suggest either a) the browser with a specific engine for which the results are most reliable, or b) the higest marketshare browser with a specific engine in cases where there isn't a significant difference in reliability.
The text was updated successfully, but these errors were encountered: